Skip to main content

Readers overview

The following is a summary of reader capabilities. For more information, see:

Readers summary table

reader

input(s)

output stream type(s)

supports replication

recoverable

Cosmos DB Reader (see Azure Cosmos DB using Core (SQL) API)

Initial Load mode: Cosmos DB documents using Microsoft Azure Cosmos SDK for Azure CosmosDB SQL API

Incremental mode: Cosmos DB documents using Cosmos DB's change feed

JSONNodeEvent

requires upsert support in writer, so only to Cosmos DB Writer and MongoDB Writer

yes, but see Cosmos DB Reader limitations

Database ReaderDatabase Reader

JDBC from a supported DBMS (see Database Reader)Database Reader

WAEvent

insert only

if output is persisted to a Kafka stream (or use Incremental Batch Reader instead)

File Reader

Apache access log, Avro, binary, delimited text, free-form text (using RegEx), GoldenGate trail file, JSON, name-value pairs, Parquet, XML

AvroEvent (when input is Avro), user-defined JSON (when input is JSON), ParquetEvent (when input is Parquet) or WAEvent

for GoldenGate only

yes

GCS Reader

Apache access log, Avro, binary, delimited text, free-form text (using RegEx), JSON, name-value pairs, Parquet, XML

JSONNodeEvent, ParquetEvent, user-defined, WAEvent, XMLNodeEvent

no

A1P

HDFS Reader

Apache access log, binary, delimited text, free-form text (using RegEx), JSON, name-value pairs, Parquet, XML

user-defined JSON (when input is JSON), ParquetEvent (when input is Parquet) or WAEvent

no

yes

HP NonStop Enscribe, SQL/MP, and SQL/MX Readers

NonStop TMF audit trail

WAEvent

yes

yes

HTTP Reader

Apache access log, binary, free-form text (using RegEx), JSON, name-value pairs, XML

WAEvent or user-defined JSON (when input is JSON)

no

if output is persisted to a Kafka stream

Incremental Batch ReaderIncremental Batch Reader

JDBC from same sources as Database Reader

WAEvent

insert only

yes

JMS ReaderJMS Reader

Apache access log, delimited text, free-form text (using RegEx), JSON, name-value pairs, XML

WAEvent or user-defined JSON (when input is JSON)

no

yes

JMX Reader

Java Management Extensions (JMX)

JSONNodeEvent

no

yes

Kafka ReaderKafka Reader

Apache access log, Avro, delimited text, free-form text (using RegEx), JSON, name-value pairs, XML

WAEvent, AvroEvent (when input is Avro), or user-defined JSON (when input is JSON)

no

yes

MapR FS Reader

Apache access log, binary, delimited text, free-form text (using RegEx), JSON, name-value pairs, XML

WAEvent or user-defined JSON (when input is JSON)

no

yes

MariaDB Reader (see MariaDB / SkySQL)

MariaDB Galera Cluster binary log (binlog)

WAEvent

yes

yes

Mongo Cosmos DB Reader (see Azure Cosmos DB using Cosmos DB API for MongoDB)

Initial Load mode: Azure Cosmos DB documents using mongo-driver-sync

Incremental mode: Cosmos DB documents using Azure Cosmos DB API for MongoDB's change stream

JSONNodeEvent

insert and delete only

requires upsert support in writer, so only to Cosmos DB Writer and MongoDB Writer

see Mongo Cosmos DB Reader limitations

MongoDB Reader (see MongoDB)MongoDB

MongoDB replica set operations log (oplog.rs)

JSONNodeEvent

yes

yes

MQTT Reader

Avro, delimited text, JSON, name-value pairs, XML

WAEvent, AvroEvent (when input is Avro), or user-defined JSON (when input is JSON)

no

yes

MS SQL Reader / MSJet (see SQL Server)SQL Server

SQL Server transaction log

WAEvent

yes

yes

MultiFile Reader

Apache access log, Avro, binary, delimited text, free-form text (using RegEx), JSON, name-value pairs, XML

WAEvent, AvroEvent (when input is Avro), or user-defined JSON (when input is JSON)

no

if output is persisted to a Kafka stream

MySQL Reader (see MySQL)MySQL / MariaDB

MySQL binary log (binlog)

WAEvent

yes

yes

OPCUA Reader

an OPC-UA server

OPCUA Data Change Event

no

yes

OJet (see Oracle Database)Oracle Database

Oracle logs

WAEvent

yes

yes

Oracle Reader (see Oracle Database)Oracle Database

Oracle logs

WAEvent

yes

yes

PostgreSQL Reader (see PostgreSQL)PostgreSQL

PostgreSQL logical replication slot

WAEvent

yes

yes

S3 Reader

Apache access log, Avro, binary, delimited text, free-form text (using RegEx), JSON, name-value pairs, Parquet, XML

AvroEvent (when input is Avro), user-defined JSON (when input is JSON) ParquetEvent (when input is Parquet), or WAEvent

no

yes

Salesforce Pardot Reader

Force.com REST API

WAEvent

yes

yes

Salesforce ReaderSalesforce Reader

Force.com REST API

WAEvent

yes

yes

Salesforce Platform Event Reader

Salesforce platform event message subscription

WAEvent

insert only

yes

Salesforce Push Topic Reader

Salesforce Streaming API

WAEvent

yes

yes

ServiceNow Reader

ServiceNow tables

WAEvent

insert and update only

yes

SQL Server

see MS SQL Reader / MS Jet, above

TCP Reader

Apache access log, binary, delimited text, free-form text (using RegEx), JSON, name-value pairs, XML

WAEvent or user-defined JSON (when input is JSON)

no

if output is persisted to a Kafka stream

Teradata

see Database Reader, above

UDP Reader

Apache access log, binary, collectd, delimited text, free-form text (using RegEx), JSON, Kafka stream, name-value pairs, NetFlow v5 or v9, SNMP, XML

WAEvent, CollectdEvent (when input is collectd), or user-defined JSON (when input is JSON)

no

if output is persisted to a Kafka stream

Windows Event Log Reader

Windows Application, Security, or System event log

WindowsLogEvent

no

yes