Skip to main content

Switching from initial load to continuous replication of BigQuery sources

Do the following to make continuous incremental replication pick up where initial load left off:

  1. After initial load completes, query the target to get the highest check column value.

  2. In Incremental Batch Reader, set the Start Position value to the next valid value after the highest. For example:

    • If the check column is of type integer and the highest check column value in the target is 100000, set Start Position to 100001.

    • If the check column is of type timestamp and its precision is milliseconds, set Start Position to one millisecond after the highest check column value in the target.