Overview
Dagster is an orchestration platform and development framework for data assets. It emphasizes testability, observability, and a modular development workflow, helping teams define, run, and monitor data assets with robust lineage and tooling.
Key features
- Declarative assets and graph models for organizing data logic.
- Built-in observability, logging and lineage visualization.
- Large integration ecosystem and pluggable execution backends.
Use cases
- Data engineering and ML workflow orchestration at scale.
- Bringing data assets into standard software engineering practices.
- Centralized scheduling, monitoring, and governance of data tasks.
Technical notes
Dagster is Python-native, supports containerized deployments, and integrates with CI/CD pipelines and third-party tools to provide an engineering-first data platform experience.