Striim 3.9.8 documentation

Writers overview

The following is a summary of writer capabilities.

writer

input stream type(s)

supports replication

supports Database Reader auto-quiesce

output(s)

DDL support

parallel threads

recovery3

ADLS Gen1 Writer

user-defined

no

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

ADLS Gen2 Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

Azure Blob Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

-

no

A1P

Azure Event Hub WriterAzure Event Hub Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

-

no

AIP (default) or E1P3 4

Azure SQL DWH (Data Warehouse) Writer

user-defined, WAEvent1

yes

yes

SQL Data Warehouse table(s)

-

yes

A1P

BigQuery WriterBigQuery Writer

user-defined, WAEvent1

yes

yes

BigQuery table(s)2

-

yes

A1P

Cassandra Cosmos DB Writer

user-defined, WAEvent1

yes

yes

Cosmos DB Cassandra API tables

-

yes

E1P4 5

Cloudera Hive Writer

user-defined, WAEvent1

insert only

no

Hive table(s)2

-

no

A1P

Cosmos DB WriterCosmos DB Writer

user-defined, JSONNodeEvent, WAEvent1

yes

yes

CosmosDB documents

-

yes

A1P

Database WriterDatabase Writer

user-defined, WAEvent1

yes

yes

table(s)2

replication Oracle to Oracle only

yes

E1P4 6

File Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

GCS Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

yes

A1P

Google PubSub Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

-

no

A1P

Hazelcast WriterHazelcast Writer

user-defined, WAEvent1

yes

no

Hazelcast map(s)2

-

no

A1P

HBase WriterHBase Writer

user-defined, WAEvent1

yes

no

HBase table(s)**

-

yes

A1P

HDFS Writer

user-defined

no

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

HDInsight HDFS Writer

user-defined

no

no

Avro, delimited text, JSON, XML

-

no

A1P

HDInsight Kafka Writer

user-defined

no

no

Avro, delimited text, JSON, XML

-

yes

E1P4

Hive WriterHive Writer

user-defined, WAEvent1

yes (insert-only in some cases)

yes

Hive table(s)2

-

yes

E1P (when using MERGE) or A1P

Hortonworks Hive Writer

user-defined, WAEvent1

yes (insert-only in some cases)

no

Hive table(s)2

-

no

E1P (when using MERGE) or A1P

JMS Writer

user-defined

no

no

delimited text, JSON, XML

-

no

A1P

Kafka WriterKafka Writer

user-defined, WAEvent1

requires schema registry, append only

yes

Avro, delimited text, JSON, XML

can track schema evolution using schema registry

yes

E1P4

Kinesis Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

-

no

E1P4

Kudu Writer

user-defined, WAEvent*

yes

yes

Kudu table(s)2

-

yes

A1P

MapR DB Writer

user-defined

no

no

MapR DB table

-

yes

A1P

MapR FS Writer

user-defined

no

no

Avro, delimited text, JSON, XML

no

A1P

MapR Stream Writer

user-defined, WAEvent1

no

no

Avro, delimited text, JSON, XML

no

A1P

MQTT Writer

user-defined

no

no

Avro, delimited text, JSON, XML

no

A1P

Redshift WriterRedshift Writer

user-defined, WAEvent1

yes

yes

Redshift table(s)2

yes

A1P

S3 Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

yes

A1P

Snowflake WriterSnowflake Writer

user-defined, WAEvent1

yes (append-only or merge)

yes

Snowflake table(s)2

-

yes

A1P

Spanner Writer

user-defined, WAEvent1

yes

yes

Spanner table(s)2

-

yes

E1P4

SysOut

any except Avro

n/a

no

log file or terminal

all input is written

no

A1P

Footnotes

1. WAEvent must be the output of a Database Reader, Incremental Batch Reader, or SQL CDC source.

2. With an input stream of a user-defined type, output is to a single table or map. Output to multiple tables or maps requires source database metadata included in WAEvent.

3. A1P ("at-least once processing") means that after recovery there may be some duplicate events written to the target. E1P ("exactly once processing") means there will be no duplicate events.

4. If the source is WAEvent from Incremental Batch Reader, recovery is A1P.

5. Primary key updates to source rows cannot be replicated.

6. When writing to Cosmos DB, recovery is A1P.

For more information, see: