Writers overview
The following is a summary of reader capabilities.
writer | input stream type(s) | supports replication | output(s) | DDL support | parallel threads | recovery3 |
|---|---|---|---|---|---|---|
user-defined | no | Avro, delimited text, JSON, XML | optional rollover on MySQL or Oracle DDL | no | A1P | |
user-defined | no | Avro, delimited text, JSON, XML | optional rollover on MySQL or Oracle DDL | no | A1P | |
user-defined | no | Avro, delimited text, JSON, XML | - | no | A1P | |
user-defined | no | Avro, delimited text, JSON, XML | - | no | AIP (default) or E1P3 4 | |
user-defined, WAEvent1 | yes | SQL Data Warehouse table(s) | - | no | A1P | |
user-defined, WAEvent1 | yes | BigQuery table(s)2 | - | no | A1P | |
user-defined, WAEvent1 | insert only | Hive table(s)2 | - | no | A1P | |
user-defined, JSONNodeEvent, WAEvent1 | yes | CosmosDB documents | - | no | A1P | |
user-defined, WAEvent1 | yes | table(s)2 | replication Oracle to Oracle only | yes | E1P4 5 | |
user-defined | no | Avro, delimited text, JSON, XML | optional rollover on MySQL or Oracle DDL | no | A1P | |
user-defined | no | Avro, delimited text, JSON, XML | optional rollover on MySQL or Oracle DDL | yes | A1P | |
user-defined | no | Avro, delimited text, JSON, XML | - | no | A1P | |
user-defined, WAEvent1 | yes | Hazelcast map(s)2 | - | no | A1P | |
user-defined, WAEvent1 | yes | HBase table(s)** | - | no | A1P | |
user-defined | no | Avro, delimited text, JSON, XML | optional rollover on MySQL or Oracle DDL | no | A1P | |
user-defined | no | Avro, delimited text, JSON, XML | - | no | A1P | |
user-defined | no | Avro, delimited text, JSON, XML | - | no | E1P4 | |
user-defined, WAEvent1 | yes (insert-only in some cases) | Hive table(s)2 | - | no | E1P (when using MERGE) or A1P | |
user-defined, WAEvent1 | yes (insert-only in some cases) | Hive table(s)2 | - | no | E1P (when using MERGE) or A1P | |
user-defined | no | delimited text, JSON, XML | - | no | A1P | |
user-defined, WAEvent1 | requires schema registry, append only | Avro, delimited text, JSON, XML | can track schema evolution using schema registry | yes | E1P4 | |
user-defined | no | Avro, delimited text, JSON, XML | - | no | E1P4 | |
user-defined, WAEvent* | yes | Kudu table(s)2 | - | no | A1P | |
user-defined | no | MapR DB table | - | no | A1P | |
user-defined | no | Avro, delimited text, JSON, XML | no | A1P | ||
user-defined, WAEvent1 | no | Avro, delimited text, JSON, XML | no | A1P | ||
user-defined | no | Avro, delimited text, JSON, XML | no | A1P | ||
user-defined, WAEvent1 | yes | Redshift table(s)2 | no | A1P | ||
user-defined | no | Avro, delimited text, JSON, XML | optional rollover on MySQL or Oracle DDL | yes | A1P | |
user-defined, WAEvent1 | yes (append-only or merge) | Snowflake table(s)2 | - | no | A1P | |
user-defined, WAEvent1 | yes | Spanner table(s)2 | - | no | E1P4 | |
any except Avro | n/a | log file or terminal | all input is written | no | A1P |
1. WAEvent must be the output of a Database Reader, Incremental Batch Reader, or SQL CDC source.
2. With an input stream of a user-defined type, output is to a single table or map. Output to multiple tables or maps requires source database metadata included in WAEvent.
3. A1P ("at-least once processing") means that after recovery there may be some duplicate events written to the target. E1P ("exactly once processing") means there will be no duplicate events.
4. If the source is WAEvent from Incremental Batch Reader, recovery is A1P.
5. When writing to Cosmos DB, recovery is A1P.
For more information, see:
Type for discussion of user-defined types
SQL CDC replication examples and Replicating MongoDB data to Azure CosmosDB
How update and delete operations are handled in writers for discussion of "insert only"
Formatters for discussion of Avro, delimited text, JSON, and XML outputs