Writers overview
The following is a summary of writer capabilities.
writer | input stream type(s) | supports replication | supports Database Reader auto-quiesce | output(s) | DDL support | parallel threads | recovery3 |
|---|---|---|---|---|---|---|---|
JSONNodeEvent, user-defined, WAEvent, XMLNodeEvent | no | no | Avro, delimited text, JSON, XML | optional rollover on schema evolution | no | A1P | |
JSONNodeEvent, ParquetEvent8, user-defined, WAEvent, XMLNodeEvent | no | yes | Avro, delimited text, JSON, XML | optional rollover on schema evolution | no | A1P | |
JSONNodeEvent, ParquetEvent8, user-defined, WAEvent, XMLNodeEvent | no | yes | Avro, delimited text, JSON, XML | - | no | A1P | |
JSONNodeEvent, user-defined, WAEvent, XMLNodeEvent | no | yes | Avro, delimited text, JSON, XML | - | no | AIP (default) or E1P3 4 | |
user-defined, WAEvent1 | yes | yes | Azure Synapse table(s) | schema evolution | yes | A1P | |
user-defined, WAEvent1 | yes | yes | BigQuery table(s)2 | schema evolution | yes | A1P | |
user-defined, WAEvent1 | yes | yes | Cosmos DB Cassandra API tables | - | yes | E1P4 5 | |
user-defined, WAEvent1 | insert only | no | Hive table(s)2 | - | no | A1P | |
user-defined, JSONNodeEvent, WAEvent1 | yes | yes | CosmosDB documents | - | yes | A1P7 | |
user-defined, WAEvent1 | yes | yes | JDBC to table(s) in a supported DBMS2 | schema evolution | yes | E1P4 6 | |
user-defined, WAEvent1 | yes | yes | Delta Lake tables in Databricks | schema evolution | yes | A1P | |
JSONNodeEvent, ParquetEvent8, user-defined, WAEvent, XMLNodeEvent | no | yes | Avro, delimited text, JSON, XML | optional rollover on schema evolution | no | A1P | |
Galera | see Database Writer, above | ||||||
JSONNodeEvent, ParquetEvent8, user-defined, WAEvent, XMLNodeEvent | no | yes | Avro, delimited text, JSON, XML | optional rollover on schema evolution | yes | A1P | |
JSONNodeEvent, user-defined, WAEvent, XMLNodeEvent | no | yes | Avro, delimited text, JSON, XML | - | no | A1P | |
user-defined, WAEvent1 | yes | no | Hazelcast map(s)2 | - | no | A1P | |
user-defined, WAEvent1 | yes | no | HBase table(s)** | - | yes | A1P | |
JSONNodeEvent, user-defined, WAEvent, XMLNodeEvent | no | no | Avro, delimited text, JSON, XML | optional rollover on schema evolution | no | A1P | |
user-defined, WAEvent1 | yes (insert-only in some cases) | yes | Hive table(s)2 | - | yes | E1P (when using MERGE) or A1P | |
user-defined, WAEvent1 | yes (insert-only in some cases) | no | Hive table(s)2 | - | no | E1P (when using MERGE) or A1P | |
HP NonStop SQL/MP & SQL/MX | see Database Writer, above | ||||||
JSONNodeEvent, user-defined, WAEvent, XMLNodeEvent | no | no | delimited text, JSON, XML | - | no | A1P | |
user-defined, JSONNodeEvent, WAEvent1 | requires schema registry, append only | yes | Avro, delimited text, JSON, XML | can track schema evolution using schema registry | yes | E1P4 | |
JSONNodeEvent, user-defined, WAEvent, XMLNodeEvent | no | yes | Avro, delimited text, JSON, XML | - | no | E1P4 | |
user-defined, WAEvent* | yes | yes | Kudu table(s)2 | - | yes | A1P | |
user-defined | no | no | MapR DB table | - | yes | A1P | |
JSONNodeEvent, user-defined, WAEvent, XMLNodeEvent | no | no | Avro, delimited text, JSON, XML | no | A1P | ||
JSONNodeEvent, user-defined, WAEvent, XMLNodeEvent | no | no | Avro, delimited text, JSON, XML | no | A1P | ||
MariaDB | see Database Writer, above | ||||||
MemSQL | see Database Writer, above | ||||||
JSONNodeEvent, user-defined, WAEvent1 | yes | yes | CosmosDB documents | no | A1P7 | ||
JSONNodeEvent, user-defined, WAEvent1 | yes | yes | MongoDB documents | - | yes | A1P | |
user-defined | no | no | Avro, delimited text, JSON, XML | no | A1P | ||
MySQL | see Database Writer, above | ||||||
Oracle Database | see Database Writer, above | ||||||
PostgreSQL | see Database Writer, above | ||||||
user-defined, WAEvent1 | yes | yes | Redshift table(s)2 | yes | A1P | ||
JSONNodeEvent, ParquetEvent8, user-defined, WAEvent, XMLNodeEvent | no | yes | Avro, delimited text, JSON, XML | optional rollover on schema evolution | yes | A1P | |
SAP HANA | see Database Writer, above | ||||||
user-defined, WAEvent1 | yes (append-only or merge) | yes | Snowflake table(s)2 | schema evolution | yes | A1P | |
user-defined, WAEvent1 | yes | yes | Spanner table(s)2 | schema evolution | yes | E1P4 | |
SQL Server | see Database Writer, above | ||||||
any except Avro | n/a | no | log file or terminal | all input is written | no | A1P | |
Footnotes 1. WAEvent must be the output of a Database Reader, Incremental Batch Reader, or SQL CDC source. 2. With an input stream of a user-defined type, output is to a single table or map. Output to multiple tables or maps requires source database metadata included in WAEvent. 3. A1P ("at-least once processing") means that after recovery there may be some duplicate events written to the target. E1P ("exactly once processing") means there will be no duplicate events. 4. If the source is WAEvent from Incremental Batch Reader, recovery is A1P. 5. Primary key updates to source rows cannot be replicated. 6. When writing to Cosmos DB, recovery is A1P. 7. Not supported when the writer's input stream is the output of Cosmos DB Reader or Mongo Cosmos DB Reader in incremental mode. 8. When the input stream is of type ParquetEvent, the writer must use Avro Formatter or Parquet Formatter. For more information, see:
| |||||||