Home/Data Ingestion/Data Connections/Google BigQuery

Google BigQuery

Setting up a data connection via Google BigQuery

Overview

Auxia's integration leverages GCP's IAM access control to connect any relevant table for highly granular user targeting, feature creation, model training, and performance measurement.

We recommend sharing tables rather than views. Direct table access enables efficient incremental updates and cost reduction, while views often necessitate reprocessing data.

Before proceeding, review the Source Data Requirements to ensure your tables are ready to be connected with Auxia.

---

Setting Up a BigQuery Connection

The Auxia Console provides a guided wizard to connect your BigQuery data in three steps: Connect → Explore → Mapping.

Prerequisites

• Access to the Auxia Console with connection editor permissions

• A GCP project with BigQuery datasets you want to connect

• Permission to grant IAM roles in your GCP project. The specific permissions depend on the access level you choose:

  Access Level	Command	Required Permission	Typical Role
  Project Level	gcloud projects add-iam-policy-binding	resourcemanager.projects.setIamPolicy	roles/resourcemanager.projectIamAdmin
  Dataset Level	bq query 'GRANT ... ON SCHEMA'	bigquery.datasets.setIamPolicy	roles/bigquery.dataOwner or roles/bigquery.admin

Step 1: Connect

1. Navigate to Connections in the Auxia Console and click Add New Connection → BigQuery.

2. The wizard will display a service account email unique to your Auxia project. Copy this email — you'll need it to grant access.

3. Enter your GCP Project ID (e.g., my-analytics-project).

4. Choose your access level:

   Access Level	Description	When to use
   Project Level	Auxia can connect to all current and new datasets in your GCP project	Simpler setup; recommended for most teams
   Dataset Level	Auxia can only connect to the specific datasets you list	When you need fine-grained control over which datasets are connected

5. Grant IAM permissions using one of the methods below (CLI, GCP Console, or Cloud Shell), then return to the Auxia Console and click Validate Connection.

note

IAM propagation note: IAM changes can take 1–5 minutes to propagate. If validation fails immediately after granting permissions, wait a few minutes and try again.

Granting Access via CLI (gcloud / bq)

Project-level access — grants the service account read access to connect all BigQuery data in the project:

gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \
  --role="roles/bigquery.dataViewer" \
  --condition=None

Replace:

• YOUR_PROJECT_ID with your GCP project ID (e.g., my-analytics-project)
• SERVICE_ACCOUNT_EMAIL with the service account shown in the Auxia Console

Dataset-level access — grants the service account read access to specific datasets only. Run once per dataset:

bq query --use_legacy_sql=false '
GRANT `roles/bigquery.dataViewer`
ON SCHEMA `YOUR_PROJECT_ID.YOUR_DATASET_NAME`
TO "serviceAccount:SERVICE_ACCOUNT_EMAIL";
'

Replace:

• YOUR_PROJECT_ID with your GCP project ID
• YOUR_DATASET_NAME with the dataset ID (e.g., customer_360)
• SERVICE_ACCOUNT_EMAIL with the service account shown in the Auxia Console

tip

The Auxia Console auto-generates these commands with your project ID and service account pre-filled. You can copy them directly from the wizard.

Granting Access via GCP Console (BigQuery UI)

If you prefer using the GCP Console UI instead of CLI:

For project-level access:

1. Go to the Google Cloud Console.
2. Navigate to IAM & Admin → IAM.
3. Click Grant Access (or + Add at the top).
4. In the New principals field, paste the service account email from the Auxia Console.
5. In the Select a role dropdown, search for and select BigQuery Data Viewer (roles/bigquery.dataViewer).
6. Click Save.

For dataset-level access:

1. Go to the Google Cloud Console.
2. Navigate to BigQuery (via the left sidebar or search).
3. In the Explorer panel, find and click on the dataset you want to share.
4. Click Sharing → Permissions.
5. Click Add Principal.
6. In the New principals field, paste the service account email from the Auxia Console.
7. In the Select a role dropdown, search for and select BigQuery Data Viewer.
8. Click Save.
9. Repeat steps 3–8 for each additional dataset.

Granting Access via Google Cloud Shell

If you don't have gcloud installed locally, you can use Google Cloud Shell directly in your browser. The bq command-line tool is bundled with the Google Cloud SDK, so no separate installation is needed.

1. Open Google Cloud Shell.
2. Ensure you're in the correct project: gcloud config set project YOUR_PROJECT_ID.
3. Run the gcloud or bq commands shown above (same as the CLI section).

Step 2: Explore & Map

Once your connection is validated, the wizard moves to dataset exploration and schema mapping. These steps are shared across all data connectors.

See Getting Started for the full guide on exploring datasets and mapping your schema.

---

Troubleshooting & FAQ

Access & Permissions

Q: I granted access but validation keeps failing. A: IAM changes can take 1–5 minutes to propagate in GCP. Wait a few minutes and click Validate Connection again. If it still fails after 5 minutes:

• Verify the service account email is correct (copy it directly from the Auxia Console).
• Confirm you granted the BigQuery Data Viewer role (not BigQuery User or BigQuery Admin).
• For dataset-level access, ensure you granted access to each dataset individually.

Q: How do I check if the IAM binding was applied correctly? A: Run the following command to list all IAM bindings for your project:

gcloud projects get-iam-policy YOUR_PROJECT_ID \
  --flatten="bindings[].members" \
  --filter="bindings.members:serviceAccount:SERVICE_ACCOUNT_EMAIL" \
  --format="table(bindings.role)"

You should see roles/bigquery.dataViewer in the output. For dataset-level access:

bq show --format=prettyjson YOUR_PROJECT_ID:YOUR_DATASET_NAME | grep -A5 "access"

Q: I don't have permission to grant IAM roles. What should I do? A: For project-level access, you need the Project IAM Admin role (roles/resourcemanager.projectIamAdmin). For dataset-level access, you need roles/bigquery.dataOwner or roles/bigquery.admin on the relevant datasets. If you don't have project-level permissions, you can still grant access one dataset at a time with roles/bigquery.admin. Alternatively, contact your GCP administrator to either:

• Grant you the necessary role, or
• Run the gcloud / bq commands on your behalf (share the commands from the Auxia Console wizard).

General

Q: Is my data copied out of BigQuery? A: Yes — Auxia reads data from your BigQuery tables and copies it into Auxia's infrastructure for model training and feature creation. Activity data has a short retention window. During the mapping step, you can selectively choose which tables and columns to include, so only the data you explicitly map is copied.

For troubleshooting related to table validation, schema mapping, and ingestion, see the Getting Started FAQ.