STRIIM BLOG

5 Advantages of Real-Time ETL for Snowflake

Irem Radzik
December 19, 2018 · 2 minute read

If you have Snowflake or are considering it, now is the time to think about your ETL for Snowflake. This blog post describes the advantages of real-time ETL and how it increases the value gained from Snowflake implementations.Striim - ETL for Snowflake

With instant elasticity, high-performance, and secure data sharing across multiple clouds, Snowflake has become highly in-demand for its cloud-based data warehouse offering. As organizations adopt Snowflake for business-critical workloads, they also need to look for a modern data integration approach. A streaming ETL for Snowflake approach loads data to Snowflake from diverse sources – such as transactional databases, security systems’ logs, and IoT sensors/devices – in real time, while simultaneously meeting scalability, latency, security, and reliability requirements.

Striim offers an out-of-the-box adapter for Snowflake to stream real-time data from enterprise databases (using low-impact change data capture), log files from security devices and other systems, IoT sensors and devices, messaging systems, and Hadoop solutions, and provide in-flight transformation capabilities. With built-in HA, security, reliability, and scalability features, Striim makes it easy for Snowflake customers to seamlessly adopt a modern, enterprise data warehouse that uses up-to-date data and supports operational decision making.

There are several advantages of using a streaming data integration solution such as Striim to enable real-time ETL for Snowflake. Here are the Top 5:

  1. Using Striim’s real-time data synchronization capabilities, businesses have the option to implement a phased migration to Snowflake from existing on-prem or cloud-based data warehouses. As such, there is no downtime for the legacy environment, and risks are minimized by allowing for extensive testing of the new Snowflake environment.
  2. When combined with low-impact CDC, real-time ETL turns enterprise databases into a streaming source of critical business transactions, enabling richer business insights.  In addition to log files, sensors, and messaging systems, Striim continuously ingests real-time data from cloud-based or on-premises data warehouses and databases such as Oracle, Oracle Exadata, Teradata, Netezza, Amazon Redshift, SQL Server, HPE NonStop, MongoDB, and MySQL.
  3. Real-time ETL enables low-latency data for time-sensitive analytics use cases, such as detecting and predicting security threats instantaneously, enabling location-based marketing etc. that provide significant operational value to the business.
  4. Before loading the data to Snowflake with sub-second latency, Striim allows users to perform in-line transformations, including denormalization, filtering, enrichment and masking, using a SQL-based language. In-flight data processing reduces the time needed for data preparation as it delivers the data in a consumable form.
  5. In-flight transformation also enables a simplified and scalable data architecture that has many related benefits including:
    • Minimizing ETL workloads by performing transformations while data is in motion
    • Optimizing data storage by filtering out unnecessary data
    • Enabling end-to-end recoverability and full resiliency without needing to handle many different components and network hops across these components
    • Supporting compliance with privacy-related regulations by enabling data masking before delivery

Striim runs on both AWS and Azure, and enables easy deployment and fast time-to-market for real-time ETL to Snowflake with its wizards-based, drag-and-drop UI.

I invite you to learn more about how Striim supports Real-Time ETL for Snowflake by visiting our Striim for Snowflake solution page, schedule a demo with a Striim technologist, or download the Striim platform to get started!