Skip to main content

Striim for BigQuery Documentation

Add the tables to table groups (optional)

During Live Sync, Striim uses table groups to parallelize writing to BigQuery to increase throughput, with each table group mapped internally to a separate BigQuery writer. The batch policy for each table group is the minimum feasible LEE (end-to-end latency) for tables in the group. We recommend the following when you create your table groups:

  • Place your sensitive tables into individual table groups. These tables may have high input change rates or low latency expectations. You can group tables with a few other tables that exhibit similar behavior or latency expectations.

  • Place all tables that do not have a critical dependency on latency into the Default table group. By default, Striim places all new tables in a pipeline into the Default table group.

Table groups are not used during Initial Sync.

Create table groups

  1. Click Create a new table group, enter a name for the group, optionally change the batch policy, and click Create.

  2. Select the Default group (or any other group, if you have already created one or more), select the tables you want to move to the new group, select Move to, and click the new group.

  3. Repeat the previous steps to add more groups, then click Next.