Skip to content

Commit 560c2f8

Browse files
authored
docs: Added note on webserver workers (#685)
* added note on api workers * minor change * changelog * feedback review: swapped order
1 parent 138f333 commit 560c2f8

File tree

2 files changed

+28
-0
lines changed

2 files changed

+28
-0
lines changed

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66

77
- Add a flag to determine if database initialization steps should be executed ([#669]).
88
- Add new roles for dag-processor and triggerer processes ([#679]).
9+
- Added a note on webserver workers to the trouble-shooting section ([#685]).
910

1011
### Fixed
1112

@@ -22,6 +23,7 @@
2223
[#678]: https://github.com/stackabletech/airflow-operator/pull/678
2324
[#679]: https://github.com/stackabletech/airflow-operator/pull/679
2425
[#683]: https://github.com/stackabletech/airflow-operator/pull/683
26+
[#685]: https://github.com/stackabletech/airflow-operator/pull/685
2527

2628
## [25.7.0] - 2025-07-23
2729

docs/modules/airflow/pages/troubleshooting/index.adoc

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,29 @@ webservers: # Also add to other roles!
2525
----
2626

2727
See e.g. https://github.com/minio/minio/issues/20845[this MinIO issue] for details.
28+
29+
== Setting API Workers
30+
31+
In Airflow the webserver (called the API Server in Airflow 3.x+) can use multiple workers.
32+
This is determined by the environment variable `+AIRFLOW__API__WORKERS+` and is set by default to `4` in Airflow 2.x and `1` in Airflow 3.x+.
33+
The reason for this difference is that Airflow uses a backend library to manage child processes and in 3.x+ this library can cause child processes to be killed if a hard-coded startup timeout is exceeded.
34+
For most cases with Airflow 3.x+ a default of `1` should be sufficient, but if you run into performance issues and would like to add more workers, you can either modulate multiple worker processes at the level of webserver, keeping the default of a single worker per webserver:
35+
36+
[source,yaml]
37+
----
38+
webservers:
39+
roleGroups:
40+
default:
41+
replicas: 2 # add a replica (with a single worker)
42+
----
43+
44+
or change the environment variable using `envOverrides`:
45+
46+
[source,yaml]
47+
----
48+
webservers:
49+
envOverrides:
50+
AIRFLOW__API__WORKERS: 2 # something other than the default of 1
51+
----
52+
53+
TIP: Our strong recommendation is to increase the webserver replicas, with each webserver running a single worker, as this removes the risk of running into timeouts or memory issues.

0 commit comments

Comments
 (0)