> ## Documentation Index
> Fetch the complete documentation index at: https://docs.auxia.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Amazon S3 setup guide

> Connect files in Amazon S3 to Auxia step by step

Follow these steps to connect files in an S3 bucket to Auxia. Setup takes about **15 minutes**.

## Prerequisites

* Access to the [Auxia Console](https://console.auxia.io) with permission to create connections
* An S3 bucket holding your data, laid out as described in [File source requirements](/data-ingestion/concepts/file-sources)
* Permission to edit the bucket policy (and, if you choose your own role, to create IAM roles) in the AWS account that owns the bucket
* If the bucket uses **SSE-KMS** encryption: permission to edit the KMS key policy

## Setup instructions

<Steps>
  <Step title="Start a new connection">
    In the Auxia Console, go to **Configuration → Connections**, click **Add New Connection**, and choose **Amazon S3**. Enter your **bucket name** — 3 to 63 lowercase letters, numbers, dots, or hyphens, without `s3://`.

    The wizard first asks you to confirm four statements. They're the contract your sync relies on:

    | Statement | What it means |
    | - | - |
    | Files are Parquet, CSV, or newline-delimited JSON | CSV must have a header row and be comma-separated |
    | You know whether files land in date folders | Each day in its own folder (such as `dt=2026-10-02`), or everything in one folder |
    | Data is append-only | New data is written as new files, never by rewriting existing ones |
    | You can edit permissions | In the AWS account that owns the bucket |
  </Step>

  <Step title="Grant Auxia read access">
    Under **How Auxia gets access**, choose a mode, then click **Generate access command**. No keys are ever shared or stored — Auxia keeps only a reference to the access you grant, and you can revoke it at any time.

    <Warning>
      **Always copy the policy or commands from the wizard.** The role and identities in them are resolved from your Auxia project. Never type or edit them by hand — a mistyped identity can grant access to a different Auxia project.
    </Warning>

    <Tabs>
      <Tab title="Use Auxia's role (recommended)">
        The shortest path: **one policy on your bucket**, and nothing to create or paste back. Auxia creates a reader role specific to your Auxia project the first time you click **Generate access command**; it can read only this bucket.

        1. Copy the policy under **Add this policy to your bucket**. It has this shape:

           ```json theme={null}
           {
             "Version": "2012-10-17",
             "Statement": [
               {
                 "Effect": "Allow",
                 "Principal": { "AWS": "AUXIA_ROLE_ARN" },
                 "Action": ["s3:GetObject", "s3:ListBucket"],
                 "Resource": [
                   "arn:aws:s3:::your-bucket-name",
                   "arn:aws:s3:::your-bucket-name/*"
                 ]
               }
             ]
           }
           ```

        2. In the [AWS console](https://console.aws.amazon.com/s3/), open your bucket and go to **Permissions → Bucket policy → Edit**.

        3. Paste the policy and click **Save changes**.

        <Warning>
          A bucket has **one** policy. If yours already has one, **add the Auxia statement to its `Statement` list** — pasting over it removes everyone else's access. For the same reason, the wizard gives this as JSON to paste rather than a command: `aws s3api put-bucket-policy` replaces the whole policy.
        </Warning>
      </Tab>

      <Tab title="Use a role in your own AWS account">
        Choose this when your security team needs to own the role and be able to inspect or revoke it. You create a read-only role in your AWS account that Auxia uses. **No bucket policy is involved.**

        1. Run the two AWS CLI commands the wizard shows, in the account that owns the bucket:
           * **Create the role Auxia will assume** — its trust policy lets only your Auxia project assume it
           * **Give that role read access to your bucket**
        2. Copy the role's ARN (it looks like `arn:aws:iam::123456789012:role/your-role-name`) and paste it under **Paste the role's identifier back here**.

        Both commands are safe to run again — for example after fixing a typo. To connect a second bucket later, run the wizard again: it reuses the same role and adds a policy for the new bucket.
      </Tab>
    </Tabs>

    **If the bucket uses SSE-KMS encryption**, also allow the role to decrypt. Add this statement to your KMS key policy, using the role ARN from the wizard:

    ```json theme={null}
    {
      "Sid": "AllowAuxiaDecrypt",
      "Effect": "Allow",
      "Principal": { "AWS": "AUXIA_ROLE_ARN" },
      "Action": ["kms:Decrypt", "kms:DescribeKey"],
      "Resource": "arn:aws:kms:REGION:ACCOUNT_ID:key/KEY_ID"
    }
    ```

    SSE-S3 needs no extra steps. SSE-C isn't supported — re-encrypt the data with SSE-S3 or SSE-KMS.

    Then click **Check Access**. Auxia lists the bucket and confirms: *"Access confirmed. We listed your bucket and can read it, N entries visible."* Access is reported per permission, so if a check fails you'll see exactly which permission is missing.
  </Step>
</Steps>

## Configure your tables

<Steps>
  <Step title="Choose what to sync" stepNumber={3}>
    Browse your bucket and **tick the folders that are tables** — one folder per table. Folders can sit several levels deep, the way warehouse exports usually do; type a prefix to jump straight to one instead of opening down to it.

    <Warning>
      Select the folder for each table, not a parent folder that holds several tables. See [One folder per table](/data-ingestion/concepts/file-sources#one-folder-per-table).
    </Warning>

    | Folder status | Meaning |
    | - | - |
    | **Ready** | Auxia can read this folder |
    | **Issue detected** | Readable, but something needs attention — the row explains what |
    | **Incompatible** | Can't be ingested as it stands |

    A folder that's already set up is marked as ingested and can't be selected again. If one table's user ID comes from a **lookup table** (see the next step), select the lookup table's folder now too.

    Click **Continue with Mapping**.
  </Step>

  <Step title="Map" stepNumber={4}>
    Auxia reads the selected folders and works out what each table holds.

    **Check what was detected.** For each table, confirm the **file format**, the **date partitioning**, and the **column schema** Auxia read. This is the most important review in the wizard — compact date folders such as `dt=20261002` are the most likely to be misread. Each table also shows a confidence level:

    | Confidence | What to do |
    | - | - |
    | **High** — recognized schema | Review and continue |
    | **Medium** — "Looks like … please check" | Verify the table type and mapping |
    | **Low** — not a recognized schema | Map the columns yourself |

    **Map Primary User ID.** Pick the column that identifies a person across your tables. See [Understanding the primary user ID](/data-ingestion/concepts/map-your-data#understanding-the-primary-user-id).

    **Match Fields to Auxia Schema.** Map **user ID**, **event name**, and **event timestamp** on event tables. Attribute tables have no event name; map the user attributes and their update timestamp instead. If a table has no event-name column, type a name and Auxia applies it to every row. Resolve anything flagged — for example **Schema type needed** (say whether the table holds events or user attributes) or **No timestamp**.

    **Tables with no user ID.** Device, session, or order events often carry only a device or session key, with a separate lookup table mapping that key to the user. When the wizard asks how to resolve the user ID, point it at the lookup table on the shared key, and choose the column that resolves to the user ID.

    <Note>
      * The lookup table must be **selected in the same setup** as the events table that joins to it.
      * A lookup table's timestamp usually records when the mapping was made, not when an event happened. **Don't map it as the event timestamp.**
    </Note>
  </Step>

  <Step title="Filter rows (optional)" stepNumber={5}>
    Auxia ingests every row of every table unless you add a rule to leave some out. Build conditions on any column and group them with **AND** / **OR**.

    Your files are still copied in full: a filter decides what this source reads into Auxia, not what leaves your bucket.
  </Step>

  <Step title="Schedule" stepNumber={6}>
    File sources check for new data **every day**. Tell Auxia how your date folders are named so each run reads the right day:

    * **Partition folder** — the prefix before `=` in your folder names, such as `dt`. Prefix only: no `=`, `/`, or spaces.
    * **Date format** — read from your folder names where possible (*"Read from your folder names, such as …"*). If the names read both ways (for example `03-04-2026`), you must confirm the format before continuing. If Auxia can't read a pattern, pick the one that matches.

    If a table has no date folders, the whole folder is copied each run. [Supported date formats →](/data-ingestion/concepts/file-sources#supported-date-formats)
  </Step>

  <Step title="Review and start ingestion" stepNumber={7}>
    Check each table's **Table name in Auxia**, type, columns, schedule, and row filter. Table names may use letters, digits, and underscores, and must not start with a digit or repeat another table's name.

    Click **Start ingestion**. Auxia creates one configuration per table, sets up the copy, and **switches ingestion on** — this can take a few minutes. A day's data appears the morning after the first daily run. [Monitoring with Data Health →](/data-ingestion/concepts/monitoring)

    Tables that are already set up are skipped (nothing is created for them), and so is a table whose folder another table already reads — two tables over the same files would ingest everything twice.

    <Warning>
      If you see *"Your configurations were created but Auxia couldn't switch them on"*, **contact Auxia before trying again.** Retrying creates duplicate configurations.
    </Warning>
  </Step>
</Steps>

## Related articles

* [Amazon S3 overview](/data-ingestion/data-connections/amazon-s3)
* [Troubleshooting Amazon S3](/data-ingestion/data-connections/amazon-s3/troubleshooting)
* [File source requirements](/data-ingestion/concepts/file-sources)
* [Map your data](/data-ingestion/concepts/map-your-data)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.