Streaming Data Integration
August 31, 2023
Striim and Confluent complement each other and enable you to:
- Write streaming SQL Queries on Kafka (3x faster than KSQL)
- Stream data from databases with high-performance, ACID-compliant Change Data Capture
- Build self-healing data pipelines
Fortune 500 companies power their cloud initiatives with Striim
Companies Choose Striim for Scalable, Real-Time Replication
“Our legacy analytics platform used to take an hour per customer data load and weeks for each new deployment. With Striim we are able to transfer operational data to Snowflake in near realtime.”
Prashant Soral,
CTO at Inspyrus
“We chose Striim as it provides continuous access to the data in our MySQL database without impacting its performance, and without taking the data out of the Google Cloud environment.”
Paolo Giangiacomo,
Systems Integration Manager, HomeServe
Striim + Confluent: Feature-by-Feature
Cloud-Scale Architecture
Both Striim and Confluent scale horizontally on in-memory compute with high availability and failover for maximum uptime.
Striim Alone
Confluent Alone
Striim + Confluent
Enables Hybrid-Cloud Use Cases
Both Striim and Confluent can be deployed on-premise and in the cloud.
Striim Alone
Confluent Alone
Striim + Confluent
Support for High Consistency Database Transactions
Striim is designed to maintain transaction integrity (i.e., ACID properties) during real-time data movement so the target database remains consistent with the source.
Striim Alone
Confluent Alone
Striim + Confluent
High-Performance, E1P Change Data Capture (CDC) from Databases
Striim supports high-performance, E1P log-based CDC for many popular databases including: Oracle, PostgreSQL, MongoDB, MySQL, HPE Nonstop, and SQL Server. E1P, high performance CDC is required for mission-critical workloads.
For CDC use cases, Confluent offers Debezium connectors for various databases (which does not support high-performance E1P CDC.)
Striim Alone
Confluent Alone
Striim + Confluent
Fully Managed Kafka in the Cloud
Confluent offers fully managed Kafka clusters across the cloud.
Striim only manages streaming data in Kafka and Confluent deployments
Striim Alone
Confluent Alone
Striim + Confluent
Real-time Transformations + Analysis
Striim is a modern, real-time ELT solution. Striim users can build detailed in-flight transformations, data masking, filtering logic using high-speed SQL queries. Striim scales horizontally with in-memory compute for high performance transformations.
Confluent also allows real-time SQL-based transformations and analysis.
Striim Alone
Confluent Alone
Striim + Confluent
Real-Time Data Visualization Dashboards
Striim offers real-time dashboards visualizing end-to-end data delivery from source to target. Striim matches source and target transactions and alerts users to missing transactions, making it easy to catch issues as they happen. Striim offers data delivery and latency SLAs. Customers see end-to-end latency under 2 seconds.
Striim Alone
Confluent Alone
Striim + Confluent
Real-Time Data Enrichment
Striim supports data enrichment and normalization using in-memory key-value stores for historic data. This allows you to enrich raw, real-time data with historical aggregates and lookup data.
Confluent also supports data enrichment.
Striim Alone
Confluent Alone
Striim + Confluent
Custom Alerts
Striim allows custom alerts on data delivery SLAs, data loss, and user-defined rules.
Confluent allows users to configure alerts for 4 types of triggers, but doesn’t offer custom alerts.
Striim Alone
Confluent Alone
Striim + Confluent
Automated Corrective Actions
Striim allows users to create workflows to perform corrective actions. In the case of errors, or failures, you can also automate workflows to perform corrective actions. By tapping into error or status streams you can trigger compensating data flows to start, or perform other actions to remediate problems.
Striim Alone
Confluent Alone
Striim + Confluent
Cloud Partnerships
Both Striim and Confluent offer cloud partnerships with AWS, Microsoft, and Google.
Striim Alone
Confluent Alone
Striim + Confluent
Data Types: Structured,
Semi-Structured, and Unstructured Data
Striim Alone
Confluent Alone
Striim + Confluent
Hundreds of Data Sources and Targets
Striim offers connectors to enterprise databases, log files, messaging systems (including Kafka as both a source and target), Hadoop, cloud applications, and IoT devices in real time.
Confluent also offers a wide variety of connectors.
Striim Alone
Confluent Alone
Striim + Confluent
Striim offers a modern data platform that's both powerful and easy to use
Create data flows with a few clicks
Select from hundreds of templates to simplify building your data flows. A step-by-step wizard will lead you through the process of connecting to your source and target to create a data flow application. You can also create custom data flows from scratch.
Process your data with real-time SQL queries
Your data flow defines how to collect, process, and deliver data. The simplest data flow just has a source, a stream, and a target. In many cases you will need to perform some processing on your data. Striim enables you to set up continuous SQL queries optimized for streaming, real-time data.
Monitor your data flow in real-time
Our built-in dashboards and monitoring enable you to see the state of your data flows in real-time and easily identify any bottlenecks. Striim can also validate that your data has been delivered and provide visibility into the end-to-end lag. This level of visibility is essential for mission-critical systems that may have SLAs regarding how current the data is.
You can also drill down on any of the components in a data flow to see detailed statistics that include read/write rate, lag, latency, CPU usage, and many other metrics. This detailed information can help identify any bottlenecks, and aids in tuning data flows for maximum performance and minimal latency.
Set alerts and create workflows to take corrective action
Striim allows you to define SQL-based custom alerts so you can stay informed about the status and performance of your data flows.
In the case of errors, or failures, you can also automate workflows to perform corrective actions. By tapping into error or status streams you can trigger compensating data flows to start, or perform other actions to remediate problems.
Striim gives us a single source of truth across domains and speeds our time to market delivering a cohesive experience across different systems.
Neel Chinta, IT Manager at Macy's
Sources
Microsoft SQL Server
MongoDB
MySQL
Oracle
PostgreSQL
Salesforce
Targets
Azure Synapse Analytics
Databricks
Google BigQuery
Kafka
PostgreSQL
Snowflake
Helpful how-to guides
We've prepared a few articles to help you get started
Streaming Data Integration
August 29, 2023
