Skip to main content

Replicating Oracle data to Google BigQuery

Striim provides a template for creating applications that read from Oracle and write to BigQuery. See Creating an application using a template for details.

The following application will replicate data from all tables in MYSCHEMA in Oracle to the corresponding tables in mydataset in BIgQuery. The tables in BigQuery must exist when the application is started. All source and target tables must all have a UUID column. In the source tables, the UUID values must be unique identifiers. See the notes for the Mode property in BigQuery Writer) for additional details.Big Query Writer

CREATE SOURCE OracleCDCSource USING OracleReader ( 
  CommittedTransactions: false,
  Username: 'myuser',
  Password: 'mypass',
  ConnectionURL: '192.168.33.10:1521/XE',
  Tables: 'MYSCHEMA.%'
 ) 
OUTPUT TO DataStream;

CREATE TARGET BigQueryTarget USING BigQueryWriter(
  ServiceAccountKey: '<path>/<configuration file>.json',
  ProjectId: '<project ID>',
  Mode: 'MERGE',
  Tables: "MYSCHEMA.%,mydataset.% keycolumns(UUID)"
INPUT FROM DataStream;

See BigQuery Writer for details of the property values.