Skip to main content
Home/Data Ingestion/Data Connections/Google BigQuery

Google BigQuery

Setting up data ingestion via Google BigQuery

Auxia's integration leverages GCP's IAM access control to pull in any relevant table for highly granular user targeting, feature creation, model training, and performance measurement. We recommend sharing tables instead of views. Direct table access lets us process incremental updates efficiently and keep costs down, whereas views require full dataset reprocessing each time, slowing workflows and driving up expenses.

Sharing a BigQuery Dataset with Auxia​

Time required: < 10 minutes

Step 1​

Login to the GCP console and navigate to the BigQuery.

step1

Step 2​

Navigate to the dataset that needs to be shared. You can use the search pane on the left side bar.

step2

Step 3​

Click "Sharing", and then "Permissions".

step3

Step 4​

Click "Add Principal".

step4

Step 5​

Add Auxia's service account with the "BigQuery Data Viewer" role and click "Save".

    bigquery-data-transfer@auxia-gcp.iam.gserviceaccount.com

step5

Once this step is completed, click save and a member of the Auxia team will reach out to you to proceed with next steps.

Ingesting user attributes​

To begin ingesting user attributes, please ensure you have completed the steps here to structure your attributes table into rows or columns. Once you have completed the steps above to share the relevant Bigquery dataset or specific table with Auxia's service account, please complete the following steps:

Step 1: Navigate to your table​

Step 2: Set 'enable_change_history=TRUE'​

To proceed with the ingestion process, each table that's being ingested needs to have enable_change_history=TRUE. You can do this by running the following command below.

Remember to swap 'your-table-name' with the name of your BigQuery table
    CREATE TABLE `your-table-name` (
    product STRING,
    inventory INT64)
OPTIONS(enable_change_history=TRUE);

This is necessary because Auxia uses either the 'CHANGES' table-valued function or the 'APPENDS' table-valued function to ingest the data incrementally for efficient and real-time feature creation.

Step 3: Validate the option list and contact Auxia​

Once you've validated the enable_change_history label is set to TRUE within the table, please notify your Auxia POC.