Skip to main content
Home/Data Ingestion/Data Connections/Firebase Event Tracking with BigQuery Integration

Firebase Event Tracking and BigQuery Integration

Firebase tracks various app events, such as user interactions, app lifecycle events, and custom-defined events, through Firebase Analytics. These events are stored within Firebase's real-time database or Firestore, and are subsequently exported to BigQuery for detailed analysis and reporting.

The integration with BigQuery allows for the storage of event data in a standardized schema that can be queried and analyzed.

Firebase events tracking​

1. Intra-Day Firebase Events Data​

  • Definition:

    Intra-day event data is exported from Firebase to BigQuery in near real-time. It captures event data as it is generated and writes it into a temporary events_intraday_* table.

  • Characteristics:

    • Data in these tables is not final and can be subject to change (e.g., late-arriving events or updates).
    • Useful for real-time monitoring and analysis of event trends within the same day.
    • Typically replaced or overwritten when the day-end export is completed.

2. Day-End Firebase Events Data​

  • Definition:

    Day-end event data is the finalized, immutable dataset for any day earlier than

    (today - 2 days), exported from Firebase to BigQuery. It is written into a new table named

    events_* with the date as the suffix (e.g., events_20250108 for January 8, 2025).

  • Characteristics:

    • Immutable: Once written, it cannot be modified, ensuring data integrity and reliability.
    • Aggregates all events that occurred on a particular day, including late-arriving or updated events.
    • Suitable for historical analysis and reporting.
    • Helps maintain a complete and consistent record of daily activity.

How Firebase Handles Exports​

  • Firebase automatically updates the intra-day tables in real-time and finalizes the data by writing it to day-end tables after the end of each day.
  • Partitioning:
    • Day-end tables are partitioned by event_date for efficient querying.
    • Intra-day tables do not typically follow a partitioning structure, as they are temporary.

Export to BigQuery​

Firebase Analytics automatically exports event data to BigQuery when integrated. The events are stored in BigQuery tables, and the schema adheres to a standardized format that allows for efficient querying and reporting. The schema includes key fields for both event-level data and user-level information.

Architecture of how Auxia integrates Firebase Data with Bigquery​

Firebase Architecture Diagram

Standard BigQuery Schema​

The following is the standard schema of how Firebase exports app event data into BigQuery:

Table ingestion guidelines​

Sharing tables rather than views​

We recommend sharing BigQuery tables instead of views, as it offers significant benefits in terms of cost-efficiency, real-time data access, and workflow effectiveness.

  1. Cost and Efficiency: Direct table access allows us to process incremental data, reducing costs and compute time. Views require processing the entire dataset, which is time-consuming and costly.
  2. Real-Time Data: Direct access ensures we can work with the most up-to-date data, enabling faster decision-making.
  3. Effectiveness: With tables, we can deliver accurate, timely recommendations and seamlessly integrate fresh personalization fields, enhancing user engagement.

In short, direct table access improves efficiency, real-time analysis, and personalization, leading to more effective collaboration.

Set 'enable_change_history=TRUE'​

Each table that's being ingested needs to have enable_change_history=TRUE. You can do this by running the following command below.

-- Remember to swap 'your-table-name' with the name of your BigQuery table
ALTER TABLE `your-table-name`
SET OPTIONS(enable_change_history=TRUE);

This is necessary because Auxia uses either the 'CHANGES' table-valued function or the 'APPENDS' table-valued function to ingest the data incrementally for efficient and real-time feature creation.

Guidelines for Sharing a BigQuery Dataset with Auxia​

For more information on how to share a BigQuery dataset with Auxia, refer to Auxia's Google BigQuery integration document.