Best Open Source data engineering Libraries
A curated list of the most popular GitHub repositories tagged with data engineering. Select any project to visualize its architecture and dive into the codebase using RepoMind's AI engine.
#1apache/superset
Apache Superset is a Data Visualization and Data Exploration Platform
#2GokuMohandas/Made-With-ML
Learn how to develop, deploy and iterate on production-grade ML applications.
#3apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
#4DataTalksClub/data-engineering-zoomcamp
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. The next cohort starts in January 2026. Join the course here 👇🏼
#5eugeneyan/applied-ml
📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
#6Avaiga/taipy
Turns Data and AI algorithms into production-ready web applications in no time.
#7argoproj/argo-workflows
Workflow Engine for Kubernetes
#8dagster-io/dagster
An orchestration platform for the development, production, and observation of data assets.
#9andkret/Cookbook
The Data Engineering Cookbook
#10semantica-agi/semantica
Graph-Native Infrastructure for Context and Accountable AI Systems
#11xonsh/xonsh
🐚 Python-powered shell. Full-featured, cross-platform and AI-friendly.
#12risingwavelabs/risingwave
Event streaming platform for agents, apps, and analytics. Continuously ingest, transform, and serve event data in real time, at scale.
#13mage-ai/mage-ai
🧙 Build, run, and manage data pipelines for integrating and transforming data.
#14redpanda-data/connect
Fancy stream processing made operationally mundane
#15lakehq/sail
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
#16Zleap-AI/SAG
A new SOTA for RAG — an original retrieval architecture and an open-source knowledge base for humans and agents.
#17DataRecce/recce
The data-validation toolkit for enhanced dbt (data build tool) PR review
#18snowflakedb/snowpark-python
Snowflake Snowpark Python API
#19Hebbian-Robotics/hflow
Open source SDK for building multimodal data-quality, processing, enrichment, and curation pipelines for robotics and Physical AI.
#20animetanver-commits/fabric-unified-data-blueprint
Microsoft Fabric Unified Data Foundation with Databricks & Purview 2026