Apache Airflow
FreeIndustry-standard workflow orchestration for data and ML pipelines.
Tool Info
Overview
Apache Airflow schedules and monitors batch workflows as directed acyclic graphs.
It is the most widely deployed orchestrator in data engineering.
Common for RAG indexing jobs and ML batch pipelines.
Features
- DAG-based workflows
- Rich operator ecosystem
- Scheduling
- Managed providers
Pricing
Pros
- Industry standard
- Huge operator library
- Proven at scale
Best For
When NOT to Use
- Operational overhead
- Not asset-centric like Dagster
Typical Users
Community Insights
Real implementation experiences shared by AI practitioners.
Loading practitioner experiences…
Related Tools
Alternatives
Tags
Related Guides
- ETL
Engineering guide to ETL and ELT — batch pipelines, watermarks, idempotent loads, SCD, orchestration, and production data movement.
- AI System Architecture
Reference architecture for modern enterprise AI systems, bringing together retrieval, knowledge graphs, AI agents, LLM gateways, caching, observability, security, and production deployment.
Stay Updated
Get the latest AI news, tools, and engineering guides delivered to your inbox.
Subscribe to Newsletter