Skip to main content

Building pipelines with ServiceNow Reader

You can read from ServiceNow using the ServiceNow Reader and write to any target supported by Striim. Typically, you will set up pipelines in two phases—initial load, followed by continuous incremental replication—as explained in Pipelines.

  • For initial load, use ServiceNow Reader in Initial Load mode to create a point-in-time copy of selected tables at the target.

  • After initial load has completed, start continuous replication by reading new or changed records created after the initial load began, then writing those changes to the target. For near-real-time replication of new source data, use the ServiceNow Reader in Incremental mode.

Pipelining options: You may build applications manually (Flow Designer or TQL). Where available, you can also use automated pipeline wizards to create separate initial-load and incremental applications and configure target schemas. Choose the approach that fits your operational requirements.

Manual build (general guidance):

  • Before performing initial load, identify or confirm a stable, monotonically increasing watermark for each table (for example, sys_updated_on for incremental mode).

  • Create target schema and tables as needed, or enable schema creation during initial load.

  • Perform initial load with Mode=InitialLoad and the required Start timestamp if you need a historical window.

  • Switch to continuous replication by setting Mode=IncrementalLoad and providing an appropriate starting timestamp (for example, the final watermark from initial load). Configure Polling interval and upsert semantics at the target.

Create a ServiceNow Reader application using the Flow Designer

This procedure outlines how to use Striim’s Flow Designer to build and configure data pipelines. Flow Designer enables you to visually create applications with minimal or no coding.

  1. Go to the Apps page and click Start from scratch.

  2. Provide the Name and Namespace for your app.

  3. In the component section, expand Sources, search for ServiceNow Reader, and select it.

  4. In the properties panel, enter connection and runtime properties (for example, Connection URL, authentication, Tables, Mode, Polling interval).

  5. Click Save. Drag processors/enrichers and a target to complete the pipeline. Deploy and start the application.

Create a ServiceNow Reader application using TQL

The following sample TQL uses the ServiceNow Reader to read from a table and output WAEvents.

CREATE APPLICATION ServiceNow_App;

CREATE SOURCE sn_source USING Global.ServiceNowReader (
  Mode: 'InitialLoad',
  PollingInterval: '120s',
  ClientSecret: '<client-secret>',
  Password: '<password>',
  Tables: 'incident;problem',
  Username: '<username>',
  ClientId: '<client-id>',
  ConnectionUrl: 'https://<instance>.service-now.com/',
  FetchSize: 10000,
  MaxConnections: 20,
  ThreadPoolCount: 10,
  MigrateSchema: true
)
OUTPUT TO sn_stream;

END APPLICATION ServiceNow_App;