USE CASE
Breaking the Streaming and Batch Silos
No more separation of streaming and batch pipelines. Unified ingestion and transformation in a single system.
Simplify Data Pipelines in One System
Unify stream and batch ingestion and processing pipelines in one architecture. Stream and process data at low latency where your historical data is. Ingest data easily with pre-built streaming connectors via Snowflake Openflow. Use SQL to process streaming data for many use cases with Dynamic Tables.
Optimize Cost Without Wasted Compute
Streaming ingest for rowsets is as much as 50% cheaper than file ingestion at the same volume. Dynamic Tables help you avoid wasted compute by providing performance guidance with incremental or full refresh for more efficient transformations.
Take Advantage of the AI Data Cloud
Since streaming capabilities are deeply integrated with the AI Data Cloud, you can still enjoy the security and governance capabilities you’ve come to rely on through Snowflake Horizon.
High throughput, low latency streaming data
Snowflake Openflow directly connects to streaming sources, including Apache Kafka† and Amazon Kinesis†, plus Kafka Sink*, so streaming data can flow into Snowflake and back to streaming systems.
Thanks to the new Snowpipe Streaming integration with Openflow†, streaming ingestion now delivers 10GB/s throughput with five-second-to-query latency and inline transformation.
Adjust Latency with a Single Parameter Change
With Dynamic Tables, you can use SQL or Python to declaratively define data transformations. Snowflake will manage the dependencies and automatically materialize results based on your freshness targets. Dynamic Tables only operate on data that has changed since the last refresh, making high data volumes and complex pipelines simpler and more cost-efficient.
Easily adapt to evolving business needs by making a batch pipeline into a streaming pipeline — with a single latency parameter change.
Bring Streaming to Open Lakehouse
Snowflake’s streaming capabilities work with Apache Iceberg format to help you build an open lakehouse architecture easily with versatile processing options.
Snowflake Openflow persists data in Apache Iceberg format and supports Apache Polaris-based catalogs. Then build low latency, declarative processing with Dynamic Tables for both Snowflake managed and unmanaged* Apache Iceberg Tables.
“Snowflake is helping us fulfill our vision to centralize data and is quickly becoming our enterprise data platform across the city.”
Rohit Gupta
Chief Technology Officer, Department of Technology
City and County of San Francisco
- 50% faster data refresh operations with 18x more data
- 300x faster data refresh in Tableau dashboards
“With a great partner in Snowflake, I don’t feel like we’re going to be left behind by the AI world. It’s moving so quickly, but I know that there’s a really strong foundation there that we can jump into and operate on.”
David Webb
Data Architect, Travelpass
- 65% cost savings by switching from their previous platform
- 350% improved efficiency to deliver data to business units
Developer Resources
Follow along with Snowflake's Quickstart Tutorials or join an upcoming webinar to get up and running with Snowflake Dynamic Tables.
