Striim 4.1.0 documentation

Release notes

The following are the release notes for Striim and Striim Cloud 4.1.0.

Except for a few corrections to this documentation, release 4.1.0 is identical to Striim 4.0.6. Future updates will be numbered accordingly (for example, 4.1.0.1 rather than 4.0.6.1).

Requirements

See System requirements (this is not relevant to Striim Cloud).

Changes that may require modification of your TQL code, workflow, or environment

  • Handling of underscore characters in Tables and Excluded Tables properties: Underscore characters are now always literal underscores. In previous releases, they were sometimes treated as single-character wildcards, in which cases escaping them with a backslash ()\_) indicated a literal underscore. Also, it is no longer necessary to escape backslashes when Using non-default case and special characters in table identifiers.

  • Importing TQL may fail with missing property errors: In previous releases, any reader, parser, writer, or formatter properties that have default values could be omitted from TQL and on import they would be given those default values. In this release, some TQL that imported without error in previous releases may fail on import due when these properties are not specified. The error message will indicate what property you must add for import to work. This may also happen with sample code in the documentation.

  • Oracle Reader's XStream support has been deprecated.

  • Salesforce Reader : The sObject property has been removed in favor of the new Objects property. If you use the export-import method to upgrade from an earlier Striim release, the value of the sObject property will be copied to the Objects property.

  • Oracle Reader's DDL capture mode has been deprecated in favor of schema evolution: The DDL support documented in Including DDL operations in Oracle Reader output has been deprecated in this release, Oracle Reader's DDL Capture Mode property does not appear in the UI, and creating new applications using this deprecated feature is no longer supported.. Applications created in earlier releases that use this feature are still supported but schema evolution cannot be enabled for those applications. If you wish to migrate from the old feature to schema evolution, you must create a new application.

  • New Halt state & alerts: The new HALT status indicates that an application failure is due to an external cause, such as a source or target database being offline. If you have any alerts on application Crash, you should create additional alerts for Halt. For more in formation, see Sending alerts about servers and applications.

  • An in-place upgrade from Striim 3.x will delete all data persisted to Elasticsearch. To preserve this data, use the export-import method instead. See In-place upgrade.

  • BigQuery expects time-partitioned tables by default: By default, starting in 4.0, BigQuery expects target tables to be partitioned by ingestion time or DATE, DATETIME, or TIMESTAMP columns. See "Improving performance by partitioning BigQuery tables" in BigQuery Writer.

  • Oracle Reader transaction buffer defaults have changed: By default, Oracle Reader now automatically buffers transactions larger than 100MB to disk. See "Transaction Buffer Type" in Oracle Reader properties.

  • Kafka SASL configuration has changed: Starting in Striim 4.0, SASL properties are specified in the Kafka Config property of Kafka Reader and Kafka Writer. If you connect to a Kafka cluster that uses SASL, see Configuring Kafka for details.

  • Legacy SQL Server JDBC drivers: SQL Server 2008 requires an older version of Microsoft's JDBC driver that is not compatible with the most recent SQL Server versions. See Install the Microsoft JDBC driver for more information.Installing the Microsoft JDBC driver

Customer-reported issues fixed in this release

  • DEV-14600: GGTrailParser: "Unsupported record, skipping it" error

  • DEV-15431: OracleReader > KuduWriter: race condition error

  • DEV-16218: GTrailParser issue

  • DEV-17194: MySQL Reader issue when database timezone is different from Striim timezone

  • DEV-19271: Snowflake Writer in Windows: invalid UTF8 detected in string

  • DEV-21122: Database Reader / Oracle issue reading views

  • DEV-21503: BigQuery Writer: JobId sometimes missing from server log

  • DEV-21785: EmailAdapter issue with getting subject line from field values

  • DEV-22224: MS SQL Reader / Azure SQL DB: issue accessing replica that is not in the PRIMARY or SECONDARY role

  • DEV-22253: DSV Formatter useQuotes issue

  • DEV-22737: Flow Designer issue with CQ that includes user-defined function

  • DEV-22830: JMS Reader issue after dropping and recreating

  • DEV-23193: Avro Formatter and Parquet Formatter: special character $ support issue

  • DEV-23477: Mongo DB Reader codec error

  • DEV-23645: Kafka Reader logs unnecessary error messages

  • DEV-23718: GG File Reader: Kryo serialisation issue

  • DEV-23772: DatabaseWriter / MySQL Connection Retry Policy issue

  • DEV-23815: GG File Reader unexpected quiesce

  • DEV-24204: Dashboard query issue after deleting WActionStore

  • DEV-24862: Oracle Reader with 19c MissingFileException error

  • DEV-25315: MySQL Reader ArrayIndexOutOfBoundsExceptionCause error

  • DEV-25414: Flow Designer error after editing CQ

  • DEV-25799: GG Trail Reader issue with extracting ROWID from metadata

  • DEV-26026: email alerts about application crash / quiesce are delayed

  • DEV-26498: errors running Striim as a service on Windows

  • DEV-26567: GG Trail Reader missed one delete record

  • DEV-26750: Striim installed using RPM: StriimDiagUtility issue

  • DEV-26999: Striim installed using RPM: StriimDiagUtility issue

  • DEV-27012: Exception thrown by open processor not handled as expected

  • DEV-27074: SQL Server > Filewriter memory issue

  • DEV-27099: console issue

  • DEV-27106: putUserData function should not require so much memory

  • DEV-27133: custom Java function not available after using LOAD command

  • DEV-27144: MySQL Reader schema evolution issue

  • DEV-27160: GG Trail Reader schema evolution includes tables that are not specified in Tables property

  • DEV-27169: GG Trail Reader does not quiesce on DDL when CDDLAction is set to quiesce

  • DEV-27171: GG Trail Reader schema evolution issue

  • DEV-27241: Snowflake Writer: issues due to three-part table names not being specified

  • DEV-27291: false low disk space message

  • DEV-27388: Oracle Reader > MSJet: memory issue on 30-million-row insert

  • DEV-27389: Oracle Reader connection retry isue

  • DEV-27420: MSSQLReader error due to invalid startposition format

  • DEV-27518: Oracle Reader: OperationName UNSUPPORTED errors

  • DEV-27657: Snowflake Writer: issue with NUMBER(38,0) when number has 38 digits

  • DEV-27730: FileReader: file lineage issue when FileReader is running on Forwarding Agent

  • DEV-27885: GG Trail Parser / Reader: issue with very large transaction

  • DEV-28068: CPU usage higher after upgrading from 3.10.3 to 4.0.4

  • DEV-28240: MongoDBReader: error due to unexpected error in oplog

  • DEV-28247: MySQL Reader: incorrect time coversion

  • DEV-28264: Oracle Reader: recovery checkpoint issue after upgradingfrom 3.10.3.6 to 3.10.3.8A

  • DEV-28265: Azure Synapse Writer: must specify three-part table name when using wildcard with ColumnMap

  • DEV-28272: LEE report issue

  • DEV-28310: Azure Synapse Writer: issue with CLOB values when size > 8000

  • DEV-28333: Oracle Reader: recovery checkpoint issue

  • DEV-28379: OracleReader: issue with CLOB value update

  • DEV-28431: Snowflake Writer: identity column not incrementing

  • DEV-28452: MongoDB Reader: issue due to timeout by MongoDB

  • DEV-28460: MSSQL Reader: issue due to missing SQL Server role, need better error message

  • DEV-28467: SpannerBatchReader failed with DEADLINE exception

  • DEV-28493: JMS Reader: typo in doc

  • DEV-28509: MongoDB Reader: recovery issue

  • DEV-28516: high memory usage issue

  • DEV-28527: MySQL Reader: issue with special characters in table names

  • DEV-28535: MySQL Reader: SAVEPOINT issue

  • DEV-28541: Oracle Reader: database sessions created by Striim are not cleared as expected

  • DEV-28593: Spanner Writer: "transaction contains too many mutations" error

  • DEV-28708: GG Trail Reader missing trail sequence number error

  • DEV-28712: GG Trail Parser: issue when operation spans multiple records

  • DEV-28766: File Reader: header row read again after recovery

  • DEV-28777: Database Writer: vendor exception code for unique index violation appears in server log but not in UI

  • DEV-28778: Salesforce Reader reads objects older than StartTimestamp

  • DEV-28802: error in documentation for in-place upgrade

  • DEV-28921: GG Trail Reader: issue when operation spans multiple records

  • DEV-29174: Kafka Reader with JSON Parser: value with 20 digits exceeds upper bound of long data type

  • DEV-29291: false "Alert application is not running" error

  • DEV-29366: MariaDB Reader: misleading error message when binlog configuration is wrong

  • DEV-29476: Database Writer / SQL Server: issue with checkpoint table

  • DEV-29671: GG Trail Reader: ALTER TABLE not being handled by CDDL Process

  • DEV-29713: Spanner Batch Reader: issue with Augment Query Clause

Resolved issues

No previously reported known issues were fixed in this release.

Known issues

  • HTTPReader is missing the Parser menu in the UI (DEV-1854)

    Workaround: Use the console or import TQL.

  • WActionStore silently discards events with null value for event type key (DEV-5599)

  • Creating a source using HTTPReader and DSVParser fails with error (DEV-5916)

    Workaround: Have the sender include type=dsv in the URI. For example: curl -X POST "http://192.168.1.107:9090?type=dsv&charset=utf-8"...

  • Dashboard queries not dropped with the dashboard or overwritten on import (DEV-6068)

    When you drop a dashboard, its queries are not dropped. If you drop and re-import a dashboard, the queries in the JSON file do not overwrite those already in Striim. Workaround: drop the namespace.

  • SORTER objects do not appear in the UI (DEV-8142)

  • DatabaseWriter shows no error in UI when MySQL credentials are incorrect (DEV-8933)

    If your DatabaseWriter Username or Password values are correct, you will see no error in the UI but no data will be written to MySQL. You will see errors in webaction.server.log regarding DatabaseWriter containing Failure in Processing query and command denied to user.

  • DatabaseWriter needs separate checkpoint table for each node when deployed on multiple nodes (DEV-11305)

  • Import of custom Java function fails (DEV-17653)

    : IMPORT STATIC may fail. Workaround: use lowercase import static.

  • When DatabaseReader Tables property uses wildcard, views are also read (DEV-19903)

    Workaround: use Excluded Tables to exclude the views.

  • Stream names are case-sensitive in CREATE WINDOW (DEV-28456)

Third-party APIs, clients, and drivers used by readers and writers

  • Azure Event Hub Writer uses the azure-eventhubs API version 3.0.2.

  • Azure Synapse Writer uses the bundled SQL Server JDBC driver.

  • BigQuery Writer uses the google-cloud-bigquery client for Java API version 2.2.1.

  • Cassandra Cosmos DB Writer uses cassandra-jdbc-wrapper version 3.1.0.

  • Cassandra Writer uses cassandra-java-driver version 3.0.0.

  • Cloudera Hive Writer uses hive-jdbc version 1.1.0.

  • CosmosDB Reader, in initial load mode, uses Microsoft Azure Cosmos SDK for Azure Cosmos DB SQL API 4.29.0.

  • CosmosDB Writer uses documentdb-bulkexecutor version 2.3.0.

  • Derby: the internal Derby instance is version 10.9.1.0.

  • Elasticsearch: tthe internal Elasticsearch cluster is version 5.6.4.

  • GCS Writer uses the google-cloud-storage client API version 1.106.0.

  • Google PubSub Writer uses the google-cloud-pubsub client API version 1.110.0.

  • Hive Writer uses hive-jdbc version 3.1.2.

  • Hortonworks Hive Writer uses hive-jdbc 3.1.2.

  • The HP NonStop readers use OpenSSL 1.0.2n.

  • JMS Reader and JMS Writer use the JMS API 1.1.

  • Kafka: the internal Kafka cluster is version 0.11.0.1.

  • Kudu: the bundled Kudu Java client is version 1.13.0.

  • MapR: the bundled MapR client is version 5.1.

  • MongoDB: the bundled MongoDB Java driver is version 3.6.4.

  • Mongo Cosmos DB Reader uses mongo-driver-sync version 3.8.1.

  • MongoDB Reader and MongoDB Writer use mongodb-driver-core and mongodb-driver-sync version 3.8.1.

  • MySQL Reader uses mysql-binlog-connector-java version 0.21.0.

  • Oracle: the bundled Oracle JDBC driver is ojdbc8.jar.

  • PostgreSQL: the bundled PostgreSQL JDBC 4.2 driver is version 42.2.2. As of Striim 4.0.4 3 this has been updated to version 42.2.19.

  • S3 Reader uses the AWS Java SDK for Amazon S3 aws-java-sdk-s3 version 1.11.320.

  • Salesforce Reader uses the Force.com REST API version 42.0.0.

  • Snowflake Writer uses snowflake-jdbc version.3.12.5.

  • Spanner Writer uses the google-cloud-spanner client API version 1.28.0 and the bundled JDBC driver is google-cloud-spanner-jdbc version 1.1.0.

  • SQL Server: the bundled Microsoft SQL Server JDBC driver is version 7.2.2.