Skip to main content

Kafka Writer

Note

As part of upgrading to Striim 5.4, any Kafka Writer adapters in created or quiesced applications will be upgraded to the new versions of the adapters and corresponding connection profiles will be created. However, after upgrading you import TQL exported from Striim 5.2 or earlier, any Kafka Writer components will use the deprecated version Kafka 2.1 versions of the adapters. To use the new versions of the adapters, edit the TQL before importing to use the new properties, and create a connection profile with the appropriate properties.

Kafka Writer supports writing to one or more topics in a Kafka cluster in Apache Kafka, Amazon MSK, Confluent Cloud, or Confluent Platform versions 2.0.0 to 4.0.1.

Among other improvements, in this release, Kafka Writer:

  • supports writing different combinations of source table to Kafka topics:

    • single topic with one partition

    • single topic with multiple partitions

    • multiple topics with single partition each

    • multiple topics with multiple partitions

  • can write to existing topics or create them automatically if they do not exist.

  • supports SASL and mTLS based authentication methods as well SSL-based encryption for data security..

  • uses connection profiles for connecting to Kafka and the Confluent schema registry.

  • supports multiple message formats including Avro, DSV, JSON, and XML.

  • supports recovery with both E1P (exactly once processing) or A1P (at least once processing.

  • supports parallel writes to multiple topics for higher throughput.