Striim 4.0.4 documentation

Table of Contents

Creating sources and caches using Source Preview

Source Preview is a graphical alternative to defining sources and caches using TQL. With it, you:

  1. browse regular or HDFS volumes accessible by the Striim server

  2. select the file you want

  3. select the appropriate parser (Apache, structured text, unstructured text, or XML)

  4. choose settings for the selected parser, previewing the effects on how the data is parsed

  5. generate a new application containing the source or cache, or add it to an existing application

For sources, Source Preview will also create:

  • a CQ to filter the raw data and convert the fields to Striim data types

  • a stream of type WAEvent linking the source and CQ

  • an output stream of a new type based on the parser settings you chose in Source Preview

Create a source

The following steps create a source from the sample data used by PosApp:

If you are not on the Striim home page, go there by clicking the Striim (striim_logo_inline.png) icon at the left side of the top menu.

home_page_403.png
  1. Select Source Preview > Samples > PosDataPreview.csv > Preview, .

  2. Check Use first line for column names and set columndelimiter to , (comma).

  3. PosApp uses only the MERCHANTID, DATETIME, AUTHAMOUNT, and ZIP columns, so uncheck the others.

  4. Set the data types for DATETIME to DateTime (check Unix Timestamp) and for AUTHAMOUNT to Double. Leave MERCHANTID and ZIP set to String.

    SourcePreviewColumns1.png
    SourcePreviewColumns2.png
  5. The data is now parsed correctly, the columns have been selected, and their names and data types have been set, so click Save.

  6. For Name enter PosSourceApp.

  7. If you are logged in as admin, for Namespace enter PosSourceNS. Otherwise, select your personal namespace. Then click Next.

  8. For Name enter PosSource, then click Save.

The new PosSourceApp application appears in the flow editor.

PosSourceApp403.png

At this point you could add additional components such as a window, CQ, and target to refine the application, or export it to TQL for use in manually coded applications.

Add a cache

The following steps will add a cache to the PosSourceApp application:

  1. Download USAddressesPreview.txt.

  2. Select user_menu_icon_403.png > My Files, click Select File next to No File Selected (not the one next to Cancel), navigate to and double-click USAddressesPreview.txt, and click Upload and Select

  3. Click striim_logo_inline.png to return to the Striim home page and select Source Preview > Browse, select USAddressesPreview.txt, and click Select File.

  4. Check Use first line for column names, set columndelimiter to \t (tab), set the data type for latVal and longVal to Double, and click Save.

  5. Select Use Existing, select PosSourceApp, and click Next.

  6. Select Create cache, for Name enter ZipCache, for Cache Key select Zip, leave Cache Refresh blank, and click Save.

Warning

If you save as a cache and deploy the application, the entire file will be loaded into memory.

Continue with Modifying an application using the Flow Designer.