Airflow DAG processor
The built-in Airflow DAG processor system type — the import-error check that catches a DAG disappearing silently, the heartbeat that fires on absence, and why Airflow needs a type per process.
Airflow’s DAG processor parses your DAG files. It is the process nothing else can speak for, and the reason this type exists separately from Apache Airflow.
Key: airflow-dag-processor · applied automatically to services emitting airflow.dag_processing., airflow.dagbag_size or airflow.dag_processor_heartbeat metrics.
Why a type per process
Section titled “Why a type per process”Airflow is four processes and they emit almost disjoint metrics. A single type matching the bare airflow. prefix is offered to all four and then carries checks that can only fire on one of them: measured on a live cell, four of seven such checks were dead wherever the type was applied.
So the scheduler, the DAG processor and the triggerer each have their own type, and every check sits where its metric actually exists. Setup - the [metrics] block, giving each process its own service.name - is on the Airflow page and applies to all three.
The check that justifies the type
Section titled “The check that justifies the type”A DAG file that stops parsing stops running, and fails nowhere. No task errors, no failed run, nothing on the scheduler to alert on. The DAG simply is not there any more, and stays gone until somebody notices the schedule went quiet. It is the quietest way for scheduled work to disappear, and only this process sees it.
That is what airflow.dag_processing.import_errors reports, and why the check on it is Critical rather than a warning.
Starter checks
Section titled “Starter checks”| Check | Condition | Severity | Why it matters |
|---|---|---|---|
| DAG processor not running | airflow.dag_processor_heartbeat stops arriving | Critical | No file is reparsed: edits never take effect and a new DAG never appears. |
| DAG import errors | airflow.dag_processing.import_errors > 0 | Critical | See above - a DAG vanishes with nothing failing. |
| No DAGs loaded | airflow.dagbag_size < 1 | Critical | A bundle that failed to sync, or a path that moved. |
| DAG file not reparsed | airflow.dag_processing.last_run.seconds_ago > 600, split by file_name | Warning | Edits to that file are not live. |
| DAG parsing slow | airflow.dag_processing.total_parse_time > 30s | Warning | Delays every schedule. |
The heartbeat fires on the absence of its metric: a process that dies emits nothing left to threshold, so a plain comparison would go quiet exactly when it matters. It is guarded to only fire for a metric that has reported before.
Tuning notes
Section titled “Tuning notes”- No DAGs loaded needs your number. At
< 1it catches losing all of them. Set it to your DAG count and it catches losing some - at the cost of firing during a deliberate removal. - DAG file not reparsed depends on your parse interval. Ten minutes is generous against the 30-second default and wrong if you have raised it.
- DAG parsing slow is nearly always one file. A module-level API call or database query runs on every parse of every file. Thirty seconds is a placeholder; compare it to what your deployment normally does before trusting it.