Best Open Source data engineering Libraries
A curated list of the most popular GitHub repositories tagged with data engineering. Select any project to visualize its architecture and dive into the codebase using RepoMind's AI engine.
#1apache/superset
Apache Superset is a Data Visualization and Data Exploration Platform
#2GokuMohandas/Made-With-ML
Learn how to develop, deploy and iterate on production-grade ML applications.
#3apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
#4DataTalksClub/data-engineering-zoomcamp
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. The next cohort starts in January 2026. Join the course here 👇🏼
#5eugeneyan/applied-ml
📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
#6Avaiga/taipy
Turns Data and AI algorithms into production-ready web applications in no time.
#7argoproj/argo-workflows
Workflow Engine for Kubernetes
#8dagster-io/dagster
An orchestration platform for the development, production, and observation of data assets.
#9andkret/Cookbook
The Data Engineering Cookbook
#10xonsh/xonsh
🐚 Python-powered shell. Full-featured, cross-platform and AI-friendly.
#11risingwavelabs/risingwave
Event streaming platform for agents, apps, and analytics. Continuously ingest, transform, and serve event data in real time, at scale.
#12mage-ai/mage-ai
🧙 Build, run, and manage data pipelines for integrating and transforming data.
#13redpanda-data/connect
Fancy stream processing made operationally mundane
#14growthbook/growthbook
Open Source Feature Flags, Experimentation, and Product Analytics
#15Zipstack/unstract
LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
#16weifuwan/seatunnel-web
Making Apache SeaTunnel easier to use.
#17estuary/flow
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
#18AltimateAI/altimate-code
Open-source agentic data engineering harness for dbt, SQL, and cloud warehouses. 100+ tools, 10 warehouses, AI-powered.
#19animetanver-commits/fabric-unified-data-blueprint
Microsoft Fabric Unified Data Foundation with Databricks & Purview 2026
#20benseverndev-oss/goldenmatch
Zero-config entity resolution & record linkage. The zero-tuning Fellegi-Sunter path beats hand-tuned Splink head-to-head and scales from a CSV to a verified 100M-row dedupe in 9.2 min. Fuzzy/exact/probabilistic + PPRL + LLM + identity graph. Python + edge-safe TypeScript (WASM), SQL-native in Postgres & DuckDB, MCP/REST + dbt/Airflow.