Skip to main content

Striim Cloud 4.1.0 documentation

Step 2: filter the data stream
csvstream.png

CsvDataSource outputs the data to CsvStream, which is the input for the query CsvToPosData:

cvstoposdata.png

This CQ converts the comma-delimited fields from the source into typed fields in a stream that can be consumed by other Striim components. Here, data refers to the array shown above. The number in brackets specifies a field from the array, counting from zero. Thus data[1] is MERCHANT ID, data[4] is DATETIME, data[7] is AUTH AMOUNT, and data[9] is ZIP.

TO_STRING, TO_DATEF, and TO_DOUBLE functions cast the fields as the types to be used in the Output to stream. The DATETIME field from the source is converted to both a dateTime value, used as the event timestamp by the application, and (via the DHOURS function) an integer hourValue, which is used to look up historical hourly averages from the HourlyAveLookup cache.

The other fields are discarded. Thus the first line of data from posdata.csv has at this point been reduced to the following values:

  • D6RJPwyuLXoLqQRQcOcouJ26KGxJSf6hgbu (merchantId)

  • 20130312173210 (DateTime)

  • 17 (hourValue)

  • 2.20 (amount)

  • 41363 (zip)

The CsvToPosDemo query outputs the processed data to PosDataStream:

posdatastream.png

PosDataStream assigns the remaining fields the names and data types in the order listed above:

  • PRIMARY ACCOUNT NUMBER to merchantID

  • DATETIME to dateTime

  • the DATETIME substring to hourValue

  • AUTH AMOUNT to amount

  • ZIP to zip