Striim 3.10.2 documentation

Writers overview

The following is a summary of writer capabilities.

writer

input stream type(s)

supports replication

supports Database Reader auto-quiesce

output(s)

DDL support

parallel threads

recovery3

supports LEE

ADLS Gen1 Writer

user-defined

no

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

yes

ADLS Gen2 Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

yes

Azure Blob Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

-

no

A1P

yes

Azure Event Hub WriterAzure Event Hub Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

-

no

AIP (default) or E1P3 4

yes

Azure Synapse Writer

user-defined, WAEvent1

yes

yes

Azure Synapse table(s)

-

yes

A1P

yes

BigQuery WriterBigQuery Writer

user-defined, WAEvent1

yes

yes

BigQuery table(s)2

-

yes

A1P

yes

Cassandra Cosmos DB Writer

user-defined, WAEvent1

yes

yes

Cosmos DB Cassandra API tables

-

yes

E1P4 5

yes

Cloudera Hive Writer

user-defined, WAEvent1

insert only

no

Hive table(s)2

-

no

A1P

yes

Cosmos DB WriterCosmos DB Writer

user-defined, JSONNodeEvent, WAEvent1

yes

yes

CosmosDB documents

-

yes

A1P

yes

Database WriterDatabase Writer

user-defined, WAEvent1

yes

yes

table(s)2

replication Oracle to Oracle only

yes

E1P4 6

yes

File Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

no

GCS Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

yes

A1P

yes

Google PubSub Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

-

no

A1P

yes

Hazelcast WriterHazelcast Writer

user-defined, WAEvent1

yes

no

Hazelcast map(s)2

-

no

A1P

yes

HBase WriterHBase Writer

user-defined, WAEvent1

yes

no

HBase table(s)**

-

yes

A1P

no

HDFS Writer

user-defined

no

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

yes

Hive WriterHive Writer

user-defined, WAEvent1

yes (insert-only in some cases)

yes

Hive table(s)2

-

yes

E1P (when using MERGE) or A1P

yes

Hortonworks Hive Writer

user-defined, WAEvent1

yes (insert-only in some cases)

no

Hive table(s)2

-

no

E1P (when using MERGE) or A1P

no

JMS Writer

user-defined

no

no

delimited text, JSON, XML

-

no

A1P

yes

Kafka WriterKafka Writer

user-defined, WAEvent1

requires schema registry, append only

yes

Avro, delimited text, JSON, XML

can track schema evolution using schema registry

yes

E1P4

yes

Kinesis Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

-

no

E1P4

no

Kudu Writer

user-defined, WAEvent*

yes

yes

Kudu table(s)2

-

yes

A1P

yes

MapR DB Writer

user-defined

no

no

MapR DB table

-

yes

A1P

no

MapR FS Writer

user-defined

no

no

Avro, delimited text, JSON, XML

no

A1P

no

MapR Stream Writer

user-defined, WAEvent1

no

no

Avro, delimited text, JSON, XML

no

A1P

yes

MongoDB Writer

JSONNodeEvent, user-defined, WAEvent1

yes

yes

MongoDB documents

-

yes

A1P

yes

MQTT Writer

user-defined

no

no

Avro, delimited text, JSON, XML

no

A1P

no

Redshift WriterRedshift Writer

user-defined, WAEvent1

yes

yes

Redshift table(s)2

yes

A1P

yet

S3 Writer

user-defined

no

yes

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

yes

A1P

yet

Snowflake WriterSnowflake Writer

user-defined, WAEvent1

yes (append-only or merge)

yes

Snowflake table(s)2

-

yes

A1P

yet

Spanner Writer

user-defined, WAEvent1

yes

yes

Spanner table(s)2

-

yes

E1P4

yew

SysOut

any except Avro

n/a

no

log file or terminal

all input is written

no

A1P

yes

Footnotes

1. WAEvent must be the output of a Database Reader, Incremental Batch Reader, or SQL CDC source.

2. With an input stream of a user-defined type, output is to a single table or map. Output to multiple tables or maps requires source database metadata included in WAEvent.

3. A1P ("at-least once processing") means that after recovery there may be some duplicate events written to the target. E1P ("exactly once processing") means there will be no duplicate events.

4. If the source is WAEvent from Incremental Batch Reader, recovery is A1P.

5. Primary key updates to source rows cannot be replicated.

6. When writing to Cosmos DB, recovery is A1P.

For more information, see: