Striim 4.0.4 documentation

Writing to multiple Kafka partitions

If the INPUT FROM stream is partitioned, events will be distributed among Kafka partitions based on the values in the input stream's PARTITION BY property. All events with the same value in the PARTITION BY field will be written to the same randomly selected partition. Striim will distribute the data evenly among the partitions to the extent allowed by the frequency of the various PARTITION BY field values (for example, if 80% of the events have the same value, then one of the Kafka partitions will contain at least 80% of the events). In the example above, to enable partitioning by city, you would revise the definition of TransformedDataStream as follows:

CREATE STREAM TransformedDataStream OF TransformedDataType PARTITION BY City;

To override this default behavior and send events to specific partitions based on the PARTITION BY field values, see Creating a custom Kafka partitioner.