Striim 3.9.6 documentation

Writers overview

The following is a summary of reader capabilities.

writer

input stream type(s)

supports replication

output(s)

DDL support

parallel threads

recovery3

ADLS Gen1 Writer

user-defined

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

ADLS Gen2 Writer

user-defined

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

Azure Blob Writer

user-defined

no

Avro, delimited text, JSON, XML

-

no

A1P

Azure Event Hub Writer

user-defined

no

Avro, delimited text, JSON, XML

-

no

AIP (default) or E1P3 4

Azure SQL DWH Writer

user-defined, WAEvent1

yes

SQL Data Warehouse table(s)

-

no

A1P

Big Query Writer

user-defined, WAEvent1

yes

BigQuery table(s)2

-

no

A1P

Cloudera Hive Writer

user-defined, WAEvent1

insert only

Hive table(s)2

-

no

A1P

Cosmos DB Writer

user-defined, JSONNodeEvent, WAEvent1

yes

CosmosDB documents

-

no

A1P

Database Writer

user-defined, WAEvent1

yes

table(s)2

replication Oracle to Oracle only

yes

E1P4 5

File Writer

user-defined

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

GCS Writer

user-defined

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

yes

A1P

Google PubSub Writer

user-defined

no

Avro, delimited text, JSON, XML

-

no

A1P

Hazelcast Writer

user-defined, WAEvent1

yes

Hazelcast map(s)2

-

no

A1P

HBase Writer

user-defined, WAEvent1

yes

HBase table(s)**

-

no

A1P

HDFS Writer

user-defined

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

no

A1P

HDInsight HDFS Writer

user-defined

no

Avro, delimited text, JSON, XML

-

no

A1P

HDInsight Kafka Writer

user-defined

no

Avro, delimited text, JSON, XML

-

no

E1P4

Hive Writer

user-defined, WAEvent1

yes (insert-only in some cases)

Hive table(s)2

-

no

E1P (when using MERGE) or A1P

Hortonworks Hive Writer

user-defined, WAEvent1

yes (insert-only in some cases)

Hive table(s)2

-

no

E1P (when using MERGE) or A1P

JMS Writer

user-defined

no

delimited text, JSON, XML

-

no

A1P

Kafka Writer

user-defined, WAEvent1

requires schema registry, append only

Avro, delimited text, JSON, XML

can track schema evolution using schema registry

yes

E1P4

Kinesis Writer

user-defined

no

Avro, delimited text, JSON, XML

-

no

E1P4

Kudu Writer

user-defined, WAEvent*

yes

Kudu table(s)2

-

no

A1P

MapR DB Writer

user-defined

no

MapR DB table

-

no

A1P

MapR FS Writer

user-defined

no

Avro, delimited text, JSON, XML

no

A1P

MapR Stream Writer

user-defined, WAEvent1

no

Avro, delimited text, JSON, XML

no

A1P

MQTT Writer

user-defined

no

Avro, delimited text, JSON, XML

no

A1P

Redshift Writer

user-defined, WAEvent1

yes

Redshift table(s)2

no

A1P

S3 Writer

user-defined

no

Avro, delimited text, JSON, XML

optional rollover on MySQL or Oracle DDL

yes

A1P

Snowflake Writer

user-defined, WAEvent1

yes (append-only or merge)

Snowflake table(s)2

-

no

A1P

Spanner Writer

user-defined, WAEvent1

yes

Spanner table(s)2

-

no

E1P4

SysOut

any except Avro

n/a

log file or terminal

all input is written

no

A1P

1. WAEvent must be the output of a Database Reader, Incremental Batch Reader, or SQL CDC source.

2. With an input stream of a user-defined type, output is to a single table or map. Output to multiple tables or maps requires source database metadata included in WAEvent.

3. A1P ("at-least once processing") means that after recovery there may be some duplicate events written to the target. E1P ("exactly once processing") means there will be no duplicate events.

4. If the source is WAEvent from Incremental Batch Reader, recovery is A1P.

5. When writing to Cosmos DB, recovery is A1P.

For more information, see: