Striim 3.10.1 documentation

Oracle to Kafka Quick Guide

This document provides detailed instructions for creating a Striim application to capture change data from one or more tables in Oracle and write it to one or more partitions of a Kafka topic. This reference application uses the most common options:

  • reads Oracle change data using LogMiner (Striim also supports XStream Out)

  • runs KafkaWriter in sync mode with recovery enabled to ensure no lost or duplicate events after recovery from a cluster failure ("exactly-once processing"; see Setting KafkaWriter's mode property: sync versus async for other supported options)

  • writes the Oracle data to Kafka in Avro format (Striim also supports DSV, JASON, and XML)

This document assumes the following:

  • You have sysdba privileges in Oracle to perform the necessary configuration, and can access its host via ssh and scp with the privileges necessary to install and start the Forwarding Agent.

  • You have a Kafka instance and the privileges necessary to perform all the tasks discussed below.

To create the Striim application, you will perform the following tasks, as detailed in this document. It is important that you perform the steps in the order in which they are described here. We strongly recommend that you read the entire document before starting to follow the instructions.

  1. enable archivlog and supplemental log data in Oracle

  2. create an Oracle role and user for use by Striim

  3. if your Kafka instance uses SASL authentication or SSL encryption, configure Striim accordingly

  4. create an application using Striim's Oracle CDC to Kafka template

  5. modify the application to enable recovery

  6. optionally, modify the application to:

    • write to multiple Kafka partitions

    • increase throughput by using multiple instances of OracleReader or KafkaWriter

    • deploy the source on the Oracle host using the Forwarding Agent

We strongly recommend you read this entire Quick Guide before starting to follow any of the detailed instructions.