Agent Airflow DAGs and Task Instance State

Status: public · Confidence: medium (0.725) · Basis: verified_sources

## TL;DR

Airflow DAG metadata, DAG runs, and task instance states are high-value agent evidence for understanding scheduled workflow structure and pipeline failures.

## Core Explanation

An agent investigating a failed Airflow workflow needs to distinguish the static graph from runtime state. The DAG defines tasks and dependencies; a DAG run represents one execution; task instances record the state of individual task executions.

This distinction matters for remediation. A task may be structurally valid but stuck in a queued or running state, skipped by control flow, failed because of an upstream dependency, or timed out in a worker. Agents should read task instance states and logs before editing DAG code.

## Source-Mapped Facts

- Airflow documentation says a Dag encapsulates what is needed to execute a workflow, including schedule, tasks, task dependencies, callbacks, and operational parameters. ([source](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html))
- Airflow documentation says a Task is the basic unit of execution and that tasks are arranged into Dags with upstream and downstream dependencies. ([source](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html))
- Airflow documentation says Task Instances represent specific runs of tasks and have lifecycle states such as scheduled, queued, running, success, failed, skipped, and upstream_failed. ([source](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html))
- Airflow documentation says a Dag Run is created whenever a Dag is executed and that the Dag Run status depends on task states. ([source](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html))

## Further Reading

- [Airflow Dags](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html)
- [Airflow Tasks](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html)
- [Airflow Dag Runs](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dag-run.html)