Striim 3.9.7 documentation

Step 2: filter the data stream

CsvDataSource outputs the data to CsvStream, which is the input for the query CsvToPosData:

This CQ converts the comma-delimited fields from the source into typed fields in a stream that can be consumed by other Striim components. Here, "data" refers to the array mentioned above, and the number in brackets specifies a field from the array, counting from zero. Thus data[1] is MERCHANT ID, data[4] is DATETIME, data[7] is AUTH AMOUNT, and data[9] is ZIP.

TO_STRING, TO_DATEF, and TO_DOUBLE functions cast the fields as the types to be used in the Output to stream. The DATETIME field from the source is converted to both a dateTime value, used as the event timestamp by the application, and (via the DHOURS function) an integer hourValue, which is used to look up historical hourly averages from the HourlyAveLookup cache, discussed below.

The other six fields are discarded. Thus the first line of data from posdata.csv has at this point been reduced to five values:

• D6RJPwyuLXoLqQRQcOcouJ26KGxJSf6hgbu (merchantId)

• 20130312173210 (DateTime)

• 17 (hourValue)

• 2.20 (amount)

• 41363 (zip)

The CsvToPosDemo query outputs the processed data to PosDataStream:

PosDataStream assigns the five remaining fields the names and data types in the order listed above:

• PRIMARY ACCOUNT NUMBER to merchantID

• DATETIME to dateTime

• the DATETIME substring to hourValue

• AUTH AMOUNT to amount

• ZIP to zip