Striim 4.0.4 documentation

Table of Contents

Common source-target combinations

The following examples are just the most popular among Striim's customers. There are many other possibilities.

  • Database to database, for example, from MySQL, Oracle, or SQL Server to MariaDb, PostgreSQL, or Spanner in the cloud. See the Change Data Capture (CDC) Guide for a full list of supported sources and Writers overview for a full list of supported targets.

    The most common use for this kind of pipeline is to allow a gradual migration from on-premise to cloud. Applications built on top of the on-premise database can be gradually replaced with new applications built on the cloud database. Once all the legacy applications are replaced, the pipeline can be shut down and the on-premise database can be retired.

    In this model, updates, and delete operations on the the source tables are replicated to the target with no duplicates or missing data (that is, "exactly once processing or E1P"). This consistency is ensured even after events such as a server crash require restarting the application (see Recovering applications).

  • Database to data warehouse, for example, from Oracle, PostgreSQL, or SQL Server (on premise or in the cloud) to Google BigQuery, Amazon Redshift, Azure Synapse, or Snowflake. See Readers overview for a full list of supported sources and Writers overview for a full list of supported targets.

    The primary use for this kind of pipeline is to update data warehouses with new data in near real time rather than in periodic batches.

    Typically data warehouses retain all data so that business intelligence reports can be generated from historical data. Consequently, when rows are updated or deleted in the source tables, instead of overwriting the old data in the target Striim appends a record of the update or delete operation. Striim ensures that all data is replicated to the target, though after events such as a server crash require restarting the application there may be duplicates in the target (that is, "at least once processing" or A1P).