Skip to main content

Striim Cloud 4.1.0 documentation

Cosmos DB setup for Mongo Cosmos DB Reader

SSL

By default, SSL is enabled in Cosmos DB API for Mongo DB. Mongo Cosmos DB Reader uses SSL to connect to Cosmos DB. Other encryption methods are not supported.

Server Side Retry

Server Side Retry is enabled by default for Cosmos DB API for Mongo DB 3.6 and later. Disabling it may result in rate-limiting errors. For more information, see Prevent rate-limiting errors for Azure Cosmos DB API for MongoDB operations.

Capturing deletes

Azure Cosmos DB API for MongoDB's change stream does not capture deletes. To work around this limitation:

  1. Using the MongoDB shell, create a _ts index on the collection with expireAfterSeconds set to -1. For example:

    db.mycollection.createIndex({"_ts":1}, {expireAfterSeconds: -1})

    The -1 value means Cosmos DB will not automatically delete any documents, but Mongo Cosmos DB Reader will be able to set the TTL for individual documents in order to delete them. For more information, see Expire data with Azure Cosmos DB's API for MongoDB.

  2. Set Mongo Cosmos DB Reader's Cosmos DB Config property to:

    {"Operations": {"SoftDelete": {"FieldName" : "IsDeleted","FieldValue" : "true"}}}

    This will add "IsDeleted":"true" to the output for deleted fields, as shown by the example in Mongo Cosmos DB Reader example output . That will tell Cosmos DB Writer or Mongo Cosmos DB Writer to delete the corresponding document in the Cosmos DB target. Mongo Cosmos DB Reader will also set the TTL of the deleted document to 5, so it will be deleted by Cosmos DB five seconds later.