Why Hybrid Cloud Needs Streaming Integration

2 Minute Read

What is hybrid cloud? What are the data considerations? And why is streaming integration an essential part of this modern data architecture?

Gartner defines a hybrid cloud service as a cloud computing service that is composed of some combination of private, public and community cloud services, from different service providers. What this means is that, if you combine services from multiple clouds, or from a cloud with on-premises or co-located services, you are doing hybrid cloud.

The conversation about hybrid cloud is often related to applications, virtual machines, micro-services and how these things are scaled or moved between clouds, or between on-premises and cloud. However, a bigger consideration is the data.

Almost all hybrid cloud use cases require movement of data in addition to movement and scalability of services. Spinning up additional service instances in a cloud to rapidly and elastically scale is useless if these services can’t access the data they need.

I’ll give you a few examples.

Imagine you run a retail website from your private data center, but occasionally you need additional scale, like on Cyber Monday. You can spin up additional web servers in the cloud on-demand, but they need access to your product database. Since this is on-prem, there are latency and security issues with the cloud talking to it, so you need read-only instances in the cloud. How do you create these cloud databases, and ensure the product catalog and inventory are always up-to-date?

Another common use case is using cloud services for elastically scalable analytics. In this case, the data originates in existing systems on-prem and needs to be moved to cloud storage, databases, or a data warehouse for analytics. If you want the analytics to always run against the most current data, you need a way not only to transfer the initial on-prem data to the cloud, but also to ensure any updates are continuously applied. You may also want to filter, obfuscate, or encrypt some of the data for security or privacy purposes.

The situation gets even more complex if data originates on one cloud, or SaaS application, and needs to be collected and transformed before being delivered to another. You need integration software that can work with on-premises and cloud sources, collect the data continuously, process and format it, before delivering to other cloud endpoints.

Striim’s streaming integration software is perfectly positioned to enable hybrid cloud use cases. We support the continuous real-time collection of data from on-premises and cloud databases, files, and messaging systems. Striim can process and format the data in-memory before delivering to Azure, AWS, or Google storage, databases, messaging systems, and data warehouses. What’s more, multiple endpoints can be targeted in a single data flow, ensuring your data gets to where you need it, when you need it.

Please visit our Hybrid Cloud Solution page to learn more. Ready to get started with Striim? Download a fully functional evaluation copy, or via Striim’s integration offerings in the Azure Marketplace.


Streaming Data Integration

Declarative, Fully Managed Data Streaming Pipelines

Data pipelines can be tricky business —failed jobs, re-syncs, out-of-memory errors, complex jar dependencies, making them not only messy but often disastrously unreliable. Data teams