Striim 4.1.0 documentation

Mongo Cosmos DB Reader limitations

  • The change stream (see Change streams in Azure Cosmos DB’s API for MongoDB) does not capture timestamps for operations.

  • The change stream does not capture deletes. Use the "soft delete" approach to add an IsDeleted field with value True to target documents that have been deleted in the source (see Cosmos DB setup for Mongo Cosmos DB Reader and Mongo Cosmos DB Reader example output).

  • The change stream captures field-level updates as document replace operations, so the entire document will be read.

  • If multiple updates are made to a document in a short period of time, the change stream may consolidate them all into a single document update.

  • The order of operations is guaranteed to be preserved only for events with the same shard key in the change stream. The order of operations may not be preserved for events with different shard keys.

  • Multi-region writes are not supported.

  • Document _id fields must be unique across all shards. Otherwise you may encounter errors or data corruption.

  • Recovery (see Recovering applications) from the point at which the application stopped is not possible until the change stream has two resume tokens for each collection. Prior to that point, after the application restarts, Mongo Cosmos DB Reader will start reading from the latest document, resulting in a gap in the target from the time the application stopped until it was restarted. In other words, at-least once processing (A1P) is not guaranteed until after Mongo Cosmos DB Reader has been running for a few hours or days.

    To tell whether recovery would result in data loss, run the command.SHOW <namespace>.<application name> CHECKPOINT HISTORY. If the output includes any occurrences of ResumeToken[null], when the application is restarted Mongo Cosmos DB Reader will resume reading from the latest document. To avoid this, you may start from scratch with a new initial load. If you need advice or assistance in this situation, Contact Striim support.