BigQuery
BigQuery is a cloud-based data warehousing and analytics platform developed by Google. It allows users to store, analyze, and query large datasets in a fast and scalable way using SQL-like queries.
You can read from BigQuery as follows, and write to any target supported by Striim. Typically, you will set up data pipelines that read from BigQuery in two phases—initial load, followed by continuous replication—as described in Building pipelines from BigQuery.
Typical use cases for reading data from BigQuery include:
Reverse ETL-based use cases (for example, updating predicted LTV or churn scores on customer Profiles) to read from BigQuery and write to operational systems (such as CRM, SCM, and OLTP databases).
Consolidation of data warehouses from departmental instances to a corporate instance.
BigQuery feature summary
Feature | Initial load with Database Reader | Continuous incremental replication with Incremental Batch Reader |
|---|---|---|
Security and governance | ||
connection profile | ✓ | ✓ |
Operations | ||
initial load | ✓ | |
initial schema creation | with supported targets | |
schema evolution | ||
DML operations replicable in target | INSERT (UPDATE handled as INSERT, DELETE ignored) | |
Building applications / programmability | ||
automated pipelines | ||
other wizards | initial load only | initial load only |
Flow Designer | ✓ | ✓ |
TQL | ✓ | ✓ |
Runtime | ||
event type of output stream | WAEvent | WAEvent |
network connection fault tolerance | by JDBC driver | by JDBC driver |
recovery | ✓ | ✓ |
parallel threads | ✓ | ✓ |
metrics & auditing | ✓ | ✓ |