Skip to main content

BigQuery

BigQuery is a cloud-based data warehousing and analytics platform developed by Google. It allows users to store, analyze, and query large datasets in a fast and scalable way using SQL-like queries.

Typical use cases for reading data from BigQuery include:

  • Reverse ETL-based use cases (for example, updating predicted LTV or churn scores on customer Profiles) to read from BigQuery and write to operational systems (such as CRM, SCM, and OLTP databases).

  • Consolidation of data warehouses from departmental instances to a corporate instance.

You can read from BigQuery as follows, and write to any target supported by Striim. Typically, you will set up data pipelines that read from BigQuery in two phases—initial load, followed by continuous replication—as explained in this concept article on Pipelines.

  • For initial load, you can use Database Reader to create a point-in-time copy of the existing source BigQuery dataset at the target.

  • After initial load has completed, you can use Incremental Batch Reader to read, at regular intervals, the new data created in the same BigQuery dataset after the initial load was started, and then writing this new source data to the target, allowing for continuous updates in near real-time.

Striim UI wizards

You can read from BigQuery using the "Initial load only" wizards in the Striim UI: