STRIIM BLOG

Load Data to Snowflake in Real Time

Irem Radzik, Ryan Siss
January 9, 2019 · 2 minute read

The ability to load data to Snowflake in real time greatly expands the operational use cases for this cloud-based data warehouse. Built for the cloud, Snowflake delivers a level of efficiency and flexibility that traditional data warehouses are unable to match. By adding new sources and loading data to Snowflake in a streaming fashion to gain operational intelligence, Snowflake users can gain an even bigger return on their investment.Load Data to Snowflake in Real TimeThe Striim platform enables companies to load data to Snowflake in real time with minimal impact from data warehouses, databases, log files from security devices and other systems, sensors, messaging systems, and Hadoop solutions, with in-flight transformations. Striim enables companies to stream enterprise data from on-premises and cloud-based sources to Snowflake in real time, with built-in scalability, security, and reliability. This minimizes risks in migrating to Snowflake and supports time-sensitive, operational decision making.

Load Data to Snowflake from Diverse Sources in Real Time

The Striim platform continuously ingests real-time data from a variety of sources out-of-the-box – including databases, data warehouses, security, and other systems’ log files, sensors, cloud applications, messaging systems, and Hadoop solutions – both on-premises or in the cloud. For enterprise databases and data warehouses such as Oracle, Oracle Exadata, Teradata, Redshift, SQL Server, MySQL, HPE NonStop, Amazon RDS, and MongoDB, the platform offers non-intrusive change data capture (CDC) to minimize the impact on source systems. Striim also supports major data formats, including JSON, XML, AVRO, delimited, binary, free text, and change records.

In-Flight Data Processing Using SQL

Through SQL-based continuous queries, the Striim platform filters, aggregates, transforms, joins, masks, and enriches multiple streams of real-time data in-memory to rapidly prepare the data for different downstream users before delivering to Snowflake. By delivering the data in a consumable format for advanced analytics applications, Striim further accelerates time-to-insight with Snowflake solutions.

Striim also comes with built-in monitoring capabilities. The platform enables users to continuously monitor the health of the data pipelines via real-time dashboards and alerts

Striim offers several benefits to Snowflake users, including the ability to:

  • Load data to Snowflake in real time with minimal impact on sources
  • Continuously ingest data from legacy data warehouses, databases, logs, messaging, sensors, and more
  • Simplify data architectures with in-flight data processing, without extensive coding
  • Use low-latency data in Snowflake for operational decision making

Moreover, Striim is the only streaming data integration platform for Snowflake that includes:

  • Real-time data integration from a wide variety of enterprise data sources
  • Designed for high-volume, high velocity data
  • Non-intrusive CDC from databases with event guarantees
  • Built-in security, scalability, and reliability
  • In-flight enrichment via built-in cache
  • Quick to deploy via SQL-based queries and wizards-based UI

To learn more about how to load data to Snowflake in real time, please visit our Striim for Snowflake product page, schedule a demo with a Striim technologist, or download a free trial of the Striim platform and try it for yourself.