CelleryExecutor does not respect AIRFLOW__OPERATORS__DEFAULT_QUEUE for SyncCallback

[elinger]

Under which category would you file this issue?

Providers

Apache Airflow version

3.3.1

What happened and how to reproduce it?

When I set a DAG deadline through a SyncCallback, the scheduler fails to execute the callback because my Airflow setup does not have a queue named default.

I configure the default queue with AIRFLOW__OPERATORS__DEFAULT_QUEUE, but the CeleryExecutor does not respect this setting when dispatching SyncCallbacks.

See also https://github.com/apache/airflow/blob/main/providers/celery/src/airflow/providers/celery/executors/celery_executor.py#L193

What you think should happen instead?

No response

Operating System

No response

Deployment

No response

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

No response

Official Helm Chart version

No response

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[ccoliu]

I'd like to work on this. The fix is to use [operators] default_queue for callbacks, the same way the Edge executor does.