Striim 4.0.4 documentation

Using send() functions

Open processors publish their results to downstream components using send() functions. There are three versions and each has effects on the way the platform handles the output, especially for recovery.

function

description

send(List<WAEvent> added, List<WAEvent> removed)

This method publishes data to a downstream component as a list of added and removed elements. It is typically used to send a single event as the lone element in the added list with an empty removed list. It may also be used when processing aggregations, indicating the events added to and removed from the aggregation.

The WAEvents in the lists have positions which indicate event ordering. The position is null when recovery is not in use but must have a valid value when using recovery. The position of each event must be strictly greater than the event which comes before it -- either before it in the batch, or before it in a previous batch.

send(ITaskEvent batch)

Use this method to publish data to a downstream component as a batch. This method is commonly used by customers familiar with the Striim batching interface (added, removed, and snapshot). The ITaskEvent parameter will be directly passed to the subscribing downstream components.

The ITaskEvent contains WAEvents having positions which indicate event ordering. The position is null when recovery is not in use but must have a valid value when using recovery. The position of each event must be strictly greater than the event which comes before it, either before it in the batch, or before it in a previous batch.

send(Object o);

Use this method to publish raw Object data to a downstream component. This method is commonly used when the Open Processor emits simple values and recovery is not used. The Object parameter will be packaged as the payload in a stream event and delivered to downstream subscribing components.

The Object will be assigned the batch position of the input batch. When a batch is delivered to the OP, the batch position is calculated and saved. Subsequent uses of this method will assign the previously calculated batch position to output events.

  • If recovery is not in use, the positions are null.

  • If each input batch comprises a single event, then the batch position is equal to that event position, which is unique, therefore output events will have unique positions which are fully compatible with recovery.

  • If each input batch comprises multiple events, but this method is called only once per batch, then the batch position will be applied once so it will be unique and fully compatible with recovery.

Warning

If the input batch comprises multiple events, and if this method is called multiple times while processing that plural batch, then each of the output events will have the same position, the input batch position. The events which share a position may not be detected by recovery, which can lead to a failure of exactly once processing.