Splunk Reader

Note

This adapter is in preview and is available on Striim Developer only. See Striim Developer for more information.

Splunk is a software platform used to search, monitor, and analyze machine-generated data. It primarily helps organizations gain real-time insights from logs, metrics, and other machine-generated information, enabling them to track system performance, security events, user activity, and more. Splunk provides a dashboard where users can visualize the generated data.

Striim’s Splunk Reader extracts and loads data from Splunk in real-time.

Feature summary

Feature		Supported?	Notes
Objects	Standard objects	✓
Objects	Custom objects	✓
Authentication	Basic authentication		Username and password
	OAuth authentication	✓	Manual configuration based
	Custom authentication methods		Not all methods may be supported
Operations	Automated mode	✓
	Initial load	✓
	Pull-based incremental load	✓
	Push-based incremental load
	Automated pipeline
Governance	Connection profile
	Sherlock AI
	Sentinel AI
Schema handling	Initial schema creation	✓	Works with supported targets
Schema handling	Schema evolution
Setup	Wizard template
	Flow Designer	✓
	Striim TQL	✓
Runtime	Resilience/recovery	✓	Supported only for Incremental load. Recovers the data based on the provided Incremental load marker.
	Parallel execution
	Metrics	✓	Standard metrics

Supported authentication method

The Splunk Reader supports connecting to Splunk using basic authentication (username and password).

Supported objects

The Splunk Reader supports the following objects:

AlertsInInternalServer
DataModels
Datasets
LookUpReport
SearchJobs
UploadedModel

Splunk Reader properties

Property	Type	Default value	Notes
Connection pool size	Integer	20	Specifies the maximum number of active connections.
Connection URL	String		The URL and port number for accessing Splunk Web. For example: `https://<hostname>:<portnumber>/`
Exclude tables	String		A list of tables excluded from read operations. Typically used to create a list of exceptions when the Tables property includes wildcards. Misconfiguration of the Tables and Exclude Tables properties can cause "Invalid table names" errors.
Incremental load marker	String		The incremental load marker is a unique incremental column in each object used for incremental load. When no marker is specified, tables are resynced at each polling interval. Specify the name of the column that contains the start position value. This column must meet the following criteria: It should have an integer or timestamp data type (for example, a creation timestamp or an employee ID). The values must be unique and continuously increasing to ensure proper incremental reading.
Migrate schema	Boolean	False	Only available in Initial Load or Automated mode. Set to `True` to enable initial schema migration, which propagates the object schema from the source to the target.
Mode	Select list: Automated mode Initial load Incremental load	Automated	Automated mode applies incremental updates to objects that support incremental load and performs full resyncs for objects that do not support incremental load.
Polling interval	Integer	5m	Specifies an interval as an integer followed by a unit. Supported units are days (`d`), hours (`h`), minutes (`m`), or seconds (`s`). The reader polls the source at the specified interval.
Start Position	String	%=-1	Value of the incremental load marker that defines the initial reading position.
Tables	String		A semicolon-delimited (;) list of objects to read from the source. Supports the `%` wildcard. Misconfiguration of the Tables and Exclude Tables properties can cause "Invalid table names" errors. Do not modify this property when recovery is enabled for the application.
Thread pool count	Integer	10	The number of parallel running threads. The default value of zero specifies single-threaded operation. When the value of the thread pool counter is higher than the connection pool size, large data ingestion operations can cause the app to halt. Since best performance is achieved when using one thread for each table being synced, increasing the size of the connection pool to match the number of threads in use is a performance best practice.
Username	String		The Splunk user’s account name.
Password	Password		The password for the Splunk user account.

Limitations

The following are limitations or notes for the Splunk Reader:

The Splunk Reader currently only supports basic authentication (username and password).
There is no Striim wizard support for Splunk Reader. Create your application using the Flow Designer.