> ## Documentation Index
> Fetch the complete documentation index at: https://docs.auxia.io/llms.txt
> Use this file to discover all available pages before exploring further.

# File source requirements

> How to lay out files in Amazon S3 or Google Cloud Storage so Auxia can read them

These requirements apply to every file-based connector: [Amazon S3](/data-ingestion/data-connections/amazon-s3) and [Google Cloud Storage](/data-ingestion/data-connections/google-cloud-storage). The setup wizard asks you to confirm them before you start — they're the contract a sync relies on.

## Supported file formats

The self-serve setup wizard reads these formats:

| Format | Requirements |
| - | - |
| **Parquet** (recommended) | The best choice for performance, schema handling, and size. |
| **CSV** | Must have a **header row** and be **comma-separated**. |
| **Newline-delimited JSON** | One JSON object per line, with a `.ndjson` or `.jsonl` extension. A file holding a single JSON array is **not** newline-delimited JSON and is refused. |

<Warning>
  **CSV must be comma-separated.** A semicolon-, tab-, or pipe-delimited file is refused at setup, and the wizard shows the header it read. Re-export with commas, or switch to Parquet.
</Warning>

**Avro, ORC, Delta Lake, and Iceberg** are supported when the connection is set up with the Auxia team rather than through the wizard. Contact your Auxia point of contact.

**Compression.** Whole-file compression such as `.csv.gz` or `.parquet.gz` is not supported, and neither are archives such as `.tar.gz`. Compression *inside* a Parquet file may also not work out of the box — confirm the codec with your Auxia contact before committing to a compressed format.

**Encryption.** Bucket-level server-side encryption (for example S3 SSE-S3 and SSE-KMS) is transparent to Auxia. Client-side encrypted files (such as PGP) and encrypted columns are supported only in limited cases — bring your specific case to your Auxia contact.

## One folder per table

Each **table** you want Auxia to ingest must live in **its own folder**. In the wizard you select the folders that are tables — not a parent folder that holds several of them.

```
s3://your-bucket/auxia-export/
├── events/              ← select this folder (one table)
│   ├── dt=2026-10-01/
│   └── dt=2026-10-02/
├── user_attributes/     ← and this one (another table)
│   └── dt=2026-10-02/
└── purchases/           ← and this one
```

<Warning>
  **Don't select the parent folder** (`auxia-export/` above). Auxia would treat every file underneath it as a single table, and the different datasets can't be separated afterwards.
</Warning>

All files within one table folder should share the same schema. A table's schema is read from its **newest** file, and new files may add columns over time.

Table folders can sit several levels deep in the bucket — for example under a warehouse export path. You can type a prefix in the wizard to jump straight to them.

## Date folders

Auxia reads date-partitioned folders efficiently: each run reads only the newest day's folder. Use a key=value folder name with the date:

```
s3://your-bucket/auxia-export/events/dt=2026-10-02/part-0001.parquet
```

The key (`dt` above) is the **partition folder prefix** you enter during setup — the prefix only, with no `=`, `/`, or spaces.

### Supported date formats

| Format | Example folder |
| - | - |
| ISO date | `dt=2026-10-02` |
| Compact date | `dt=20261002` |
| Day-first ISO | `dt=02-10-2026` |
| Day-first compact | `dt=02102026` |
| Nested ISO / compact | a month folder, then a day folder |
| Snapshot | no date folders — the whole folder is one snapshot |

<Warning>
  **Ambiguous dates are confirmed explicitly.** A folder named `03-04-2026` could be 3 April or 4 March. Choosing wrong doesn't fail — it silently reads the wrong day, every day — so the wizard blocks until you confirm the format.
</Warning>

Dates that appear only in **file names** (not folder names) aren't detected. Without date folders, the whole folder is copied as a snapshot on each run.

## Append-only files

* Once written, a file must never be overwritten, modified, or deleted.
* To add or correct data, write a **new file** with a new name.

## Access

You grant Auxia **read-only** access to the bucket. Auxia never writes to your storage. See each connector's setup guide for the exact grant:

<CardGroup cols={2}>
  <Card title="Amazon S3 setup guide" icon="aws" href="/data-ingestion/data-connections/amazon-s3/setup-guide" />

  <Card title="Google Cloud Storage setup guide" icon="google" href="/data-ingestion/data-connections/google-cloud-storage/setup-guide" />
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.