Change Data Capture for Databases
Make the Switch to Efficient, Real-Time Data Synchronization
Your business transactions captured in relational databases are critical input to understand the state of your business operations. Batch ETL operations cannot support fast operational decision making or continuous data synchronization needs for hybrid cloud architectures. The Striim platform offers a low-impact, real-time change data capture (CDC) feature to integrate, analyze, and visualize the database changes. It enables a smart data architecture for high-volume data environments by moving and processing only the changed data.
Why Change Data Capture with Striim
Striim is the only end-to-end platform to provide ingestion and processing of real-time change data in a single platform that also includes streaming analytics and visualization. Moving change data continuously as new database transactions occur enables you to analyze up-to-date information and respond to time-sensitive issues immediately. By using Striim’s log-based CDC capabilities you can minimize overhead on the source systems, extending hardware lifetime, and ensure timely data processing without facing batch window limitations. Moving only the change data, and throughout the day, allows better network bandwidth utilization and drives communication cost savings.
Striim has built-in monitoring to validate that the database transactions have been applied successfully to the target, which provides additional assurance for data consistency especially during migration use cases.
Popular CDC Use Cases
Businesses use Striim’s CDC capabilities to feed real-time data to their big data lakes and enterprise messaging systems, such as Kafka, for timely operational decision making. They also migrate from on-premises databases to cloud environments without downtime, and keep cloud-based analytics environments up-to-date with on-premises databases using CDC.
Striim’s CDC feature enables zero data loss for any replicated environment by monitoring the replication process and detecting any latency and consistency issues. The CDC feature, together with Striim’s streaming analytics capabilities, can compare the sources and targets at the transaction level continuously as transactions are replicated.
Operational Intelligence
Use real-time change data from transactional systems for operational decision making
Cloud Integration and Migration
Move data between on-premises and cloud systems continuously and without downtime
Replication Monitoring
Monitor any replicated environment to validate database consistency for zero data loss
How CDC with Striim Works
Striim makes it easy to access, structure, and organize change data from enterprise databases. For log-based CDC, after performing initial load, Striim reads new database transactions – including inserts, updates, and deletes – from source databases’ transaction or redo logs without impacting the database workload. During the data movement, Striim maintains ACID (atomicity, consistency, isolation, durability) properties of database transactions to provide reliable data movement.
Striim filters, transforms, aggregates, and enriches change data while in motion. It can perform streaming analytics and visualization if desired before delivering change data to various targets including cloud, big data, databases, files and messaging systems such as Kafka, with sub-second latency. Striim has built-in monitoring mechanism to track key metrics for database transactions, such as volume and lag, and to verify source and target data consistency.
Transaction Integrity
Maintains ACID properties during the data movement
Pre-Processing
Filters, aggregates, transforms, and enriches before delivering to the target system
Continuous Monitoring
Tracks CDC operations’ key metrics and verifies database consistency with built-in validation
User Interface Walkthrough
- Generate Streams
- Define Data Flow
- Visualize Insights
- Monitor Solutions
