📄️ Basics
SQLFlow is a high-performance stream processing engine that simplifies building data pipelines. SQLFlow pipelines are built using SQL. Think of SQLFLow as a lightweight, modern Flink. SQLFLow embeds DuckDB and Apache Arrow for high performance stream processing. SQLFlow can process 10's of thousands of events per second using only 100's of MiB of memory.
📄️ Configuration
The heart of SQLFlow is the pipeline configuration file. Each configuration file specifies: