Home/Data Ingestion/Data Connections/GCP PubSub/Events and User Attributes Ingestion via PubSub

Events and User Attributes ingestion via Pub/Sub

Guide for publishing user events and attributes using Pub/Sub

Overview

This document outlines how to publish user events and user attributes using the Auxia system on Pub/Sub. It covers the correct structure for textproto data and the resulting message from Pub/Sub.

Events:

Schema for User Events:

Please refer to the schema specified in the documentation for Event Transaction data. Use the exact schema provided at the following link: Auxia User Events Schema.

Textproto Format for Publishing Events:

To publish user events in the Auxia system, the following textproto format is used. The schema for the Event and LogEventsRequest messages is as per the documentation.

Sample textproto:

project_id: "1874"
user_id: "12345"
events {
  event_name: "shop_transaction"
  event_properties {
    key: "spent_current_ammount"
    value {
      long_value: 200
    }
  }
  event_properties {
    key: "shop_id"
    value {
      long_value: 123
    }
  }
  event_properties {
    key: "shop_item_ids"
    value {
      string_value: "abc, def, ghi"
    }
  }
  client_event_timestamp {
    seconds: 1705932201
    nanos: 915585000
  }
  server_received_timestamp {
    seconds: 1705927421
    nanos: 0
  }
}

Sample Message Pulled from Pub/Sub:

{
"ackId": "<>",
"message": {
	"attributes": {
	"data_type": "EventData"
	},
	"data": "CgQxODc0EgUxMjM0NRp3ChBzaG9wX3RyYW5zYWN0aW9uGiAKDXNob3BfaXRlbV9pZHMSDxoNYWJjLCBkZWYsIGdoaRoNCgdzaG9wX2lkEgIIexocChVzcGVudF9jdXJyZW50X2FtbW91bnQSAwjIASoMCOnPw7wGEOjvyrQDMgYIyZfDvAY=",
	"messageId": "<>",
	"publishTime": "2025-01-22T12:43:22.345Z"
	}
}

Decoding the Message from Pub/Sub:

After pulling the message from Pub/Sub, it needs to be decoded into a readable format. The data field in the message contains a base64-encoded string. This string must be decoded into a JSON structure that matches the schema of the LogEventsRequest.

Sample decoded message:

{
  "projectId": "1874",
  "userId": "12345",
  "events": [
    {
      "eventName": "shop_transaction",
      "eventProperties": {
        "spent_current_ammount": {
          "longValue": "200"
        },
        "shop_id": {
          "longValue": "123"
        },
        "shop_item_ids": {
          "stringValue": "abc, def, ghi"
        }
      },
      "clientEventTimestamp": "2025-01-22T12:43:21.915585Z",
      "serverReceivedTimestamp": "2025-01-22T10:43:21Z"
    }
  ]
}

User Attributes:

Schema for User Attributes:

Please refer to the documentation of User Attributes. Use the exact schema below:

syntax = "proto3";

import "google/protobuf/timestamp.proto";

message AuxiaUserAttribute {
  string user_id = 1;
  map<string, PropertyValue> user_properties = 2;
  google.protobuf.Timestamp update_timestamp = 3;
}

message PropertyValue {
  oneof property_value {
    int64 long_value = 1;
    double double_value = 2;
    string string_value = 3;
    bool boolean_value = 4;
    google.protobuf.Timestamp timestamp_value = 5;
  }
}

Textproto Format for Publishing User Attributes:

To publish user events in the Auxia system, the following textproto format is used. The schema for the UserProperties messages is as per the schema mentioned above.

Sample textproto:

user_id: "12345"
user_properties {
  key: "language"
  value {
    string_value: "en"
  }
}
user_properties {
  key: "last_access_country"
  value {
    string_value: "us"
  }
}
user_properties {
  key: "country"
  value {
    string_value: "ja"
  }
}
update_timestamp {
  seconds: 1705933702
  nanos: 693271000
}

Sample Message Pulled from Pub/Sub:

{
  "ackId": "<>",
  "message": {
    "attributes": {
      "data_type": "UserAttribute"
    },
    "data": "CgUxMjM0NRIPCgdjb3VudHJ5EgQaAmphEhsKE2xhc3RfYWNjZXNzX2NvdW50cnkSBBoCdXMSEAoIbGFuZ3VhZ2USBBoCZW4aDAjG28O8BhDY88nKAg==",
    "messageId": "<>",
    "publishTime": "2025-01-22T13:08:23.141Z"
  }
}

Decoding the Message from Pub/Sub:

After pulling the message from Pub/Sub, it needs to be decoded into a readable format. The data field in the message contains a base64-encoded string. This string must be decoded into a JSON structure that matches the schema of the AuxiaUserAttribute.

Sample decoded message:

{
  "userId": "12345",
  "userProperties": {
    "language": {
      "stringValue": "en"
    },
    "last_access_country": {
      "stringValue": "us"
    },
    "country": {
      "stringValue": "ja"
    }
  },
  "updateTimestamp": "2025-01-22T13:08:22.693271Z"
}