Multi-GPU, only 1 image generates

[frenzybiscuit]

Expected Behavior

Under core parameters I set the images to 2. Then I run "Generate" on a flux workflow.

It says 2 generating/2 running but only one image is generating at a time.

If I select 1 image instead and then hit the generate button twice, it works.

I'm using a pip install and am on version 0.9.5.2

Actual Behavior

.

Steps to Reproduce

.

Debug Logs

2025-03-30 13:46:10.749 [Info] User local requested 2 images with model 'flux1-dev-fp8.safetensors'...
2025-03-30 13:46:10.749 [Debug] [BackendHandler] Backend request #2 for model flux1-dev-fp8.safetensors, maxWait=7.00:00:00.
2025-03-30 13:46:10.769 [Debug] [BackendHandler] Backend request #3 for model flux1-dev-fp8.safetensors, maxWait=7.00:00:00.
2025-03-30 13:46:12.772 [Debug] [BackendHandler] backend #1 will load a model: /mnt/portable/swarmui/SwarmUI/Models/unet/flux1-dev-fp8.safetensors, with 2 requests waiting for 2 seconds
2025-03-30 13:46:12.821 [Debug] [ComfyUI-1/STDERR] got prompt
2025-03-30 13:46:12.827 [Debug] [ComfyUI-1/STDERR] Using pytorch attention in VAE
2025-03-30 13:46:12.827 [Debug] [ComfyUI-1/STDERR] Using pytorch attention in VAE
2025-03-30 13:46:12.880 [Debug] [ComfyUI-1/STDERR] VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
2025-03-30 13:46:12.958 [Debug] [ComfyUI-1/STDERR] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
2025-03-30 13:46:13.372 [Debug] [ComfyUI-1/STDERR] clip missing: ['text_projection.weight']
2025-03-30 13:46:13.475 [Debug] [ComfyUI-1/STDERR] model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16
2025-03-30 13:46:13.475 [Debug] [ComfyUI-1/STDERR] model_type FLUX
2025-03-30 13:46:14.565 [Debug] [ComfyUI-1/STDERR] Prompt executed in 1.74 seconds
2025-03-30 13:46:14.681 [Debug] [BackendHandler] backend #1 loaded model, returning to pool
2025-03-30 13:46:14.773 [Debug] [BackendHandler] Backend request #2 found correct model on #1
2025-03-30 13:46:14.773 [Debug] [BackendHandler] Backend request #2 finished.
2025-03-30 13:46:14.774 [Debug] [BackendHandler] Backend request #3 found correct model on #1
2025-03-30 13:46:14.774 [Debug] [BackendHandler] Backend request #3 finished.
2025-03-30 13:46:14.774 [Debug] [ComfyUI-1/STDERR] got prompt
2025-03-30 13:46:14.789 [Debug] [ComfyUI-1/STDERR] got prompt
2025-03-30 13:46:15.535 [Debug] [ComfyUI-1/STDERR] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
2025-03-30 13:46:15.557 [Debug] [ComfyUI-1/STDERR] clip missing: ['text_projection.weight']
2025-03-30 13:46:16.410 [Debug] [ComfyUI-1/STDERR] Requested to load FluxClipModel_
2025-03-30 13:46:17.675 [Debug] [ComfyUI-1/STDERR] loaded completely 22639.675 9319.23095703125 True
2025-03-30 13:46:18.011 [Debug] [ComfyUI-1/STDERR] model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16
2025-03-30 13:46:18.011 [Debug] [ComfyUI-1/STDERR] model_type FLUX
2025-03-30 13:46:19.112 [Debug] [ComfyUI-1/STDERR] Requested to load Flux
2025-03-30 13:46:20.605 [Debug] [ComfyUI-1/STDERR] loaded completely 13433.870893188476 11350.067443847656 True
2025-03-30 13:46:20.614 [Debug] [ComfyUI-1/STDERR]
2025-03-30 13:46:23.396 [Debug] [ComfyUI-1/STDERR] 0%| | 0/25 [00:00<?, ?it/s]
2025-03-30 13:46:26.170 [Debug] [ComfyUI-1/STDERR] 4%|▍ | 1/25 [00:02<01:06, 2.78s/it]
2025-03-30 13:46:28.954 [Debug] [ComfyUI-1/STDERR] 8%|▊ | 2/25 [00:05<01:03, 2.78s/it]
2025-03-30 13:46:31.740 [Debug] [ComfyUI-1/STDERR] 12%|█▏ | 3/25 [00:08<01:01, 2.78s/it]
2025-03-30 13:46:34.527 [Debug] [ComfyUI-1/STDERR] 16%|█▌ | 4/25 [00:11<00:58, 2.78s/it]
2025-03-30 13:46:37.317 [Debug] [ComfyUI-1/STDERR] 20%|██ | 5/25 [00:13<00:55, 2.78s/it]
2025-03-30 13:46:40.108 [Debug] [ComfyUI-1/STDERR] 24%|██▍ | 6/25 [00:16<00:52, 2.79s/it]
2025-03-30 13:46:42.908 [Debug] [ComfyUI-1/STDERR] 28%|██▊ | 7/25 [00:19<00:50, 2.79s/it]
2025-03-30 13:46:45.714 [Debug] [ComfyUI-1/STDERR] 32%|███▏ | 8/25 [00:22<00:47, 2.79s/it]
2025-03-30 13:46:48.521 [Debug] [ComfyUI-1/STDERR] 36%|███▌ | 9/25 [00:25<00:44, 2.80s/it]
2025-03-30 13:46:51.332 [Debug] [ComfyUI-1/STDERR] 40%|████ | 10/25 [00:27<00:41, 2.80s/it]
2025-03-30 13:46:54.151 [Debug] [ComfyUI-1/STDERR] 44%|████▍ | 11/25 [00:30<00:39, 2.80s/it]
2025-03-30 13:46:56.974 [Debug] [ComfyUI-1/STDERR] 48%|████▊ | 12/25 [00:33<00:36, 2.81s/it]
2025-03-30 13:46:59.802 [Debug] [ComfyUI-1/STDERR] 52%|█████▏ | 13/25 [00:36<00:33, 2.81s/it]
2025-03-30 13:47:02.634 [Debug] [ComfyUI-1/STDERR] 56%|█████▌ | 14/25 [00:39<00:30, 2.82s/it]
2025-03-30 13:47:05.474 [Debug] [ComfyUI-1/STDERR] 60%|██████ | 15/25 [00:42<00:28, 2.82s/it]
2025-03-30 13:47:08.318 [Debug] [ComfyUI-1/STDERR] 64%|██████▍ | 16/25 [00:44<00:25, 2.83s/it]
2025-03-30 13:47:11.165 [Debug] [ComfyUI-1/STDERR] 68%|██████▊ | 17/25 [00:47<00:22, 2.83s/it]
2025-03-30 13:47:14.018 [Debug] [ComfyUI-1/STDERR] 72%|███████▏ | 18/25 [00:50<00:19, 2.84s/it]
2025-03-30 13:47:16.872 [Debug] [ComfyUI-1/STDERR] 76%|███████▌ | 19/25 [00:53<00:17, 2.84s/it]
2025-03-30 13:47:19.728 [Debug] [ComfyUI-1/STDERR] 80%|████████ | 20/25 [00:56<00:14, 2.85s/it]
2025-03-30 13:47:22.588 [Debug] [ComfyUI-1/STDERR] 84%|████████▍ | 21/25 [00:59<00:11, 2.85s/it]
2025-03-30 13:47:25.450 [Debug] [ComfyUI-1/STDERR] 88%|████████▊ | 22/25 [01:01<00:08, 2.85s/it]
2025-03-30 13:47:28.314 [Debug] [ComfyUI-1/STDERR] 92%|█████████▏| 23/25 [01:04<00:05, 2.85s/it]
2025-03-30 13:47:31.179 [Debug] [ComfyUI-1/STDERR] 96%|█████████▌| 24/25 [01:07<00:02, 2.86s/it]
2025-03-30 13:47:31.179 [Debug] [ComfyUI-1/STDERR] 100%|██████████| 25/25 [01:10<00:00, 2.86s/it]
2025-03-30 13:47:31.179 [Debug] [ComfyUI-1/STDERR] 100%|██████████| 25/25 [01:10<00:00, 2.82s/it]
2025-03-30 13:47:31.180 [Debug] [ComfyUI-1/STDERR] Requested to load AutoencodingEngine
2025-03-30 13:47:31.503 [Debug] [ComfyUI-1/STDERR] loaded completely 207.26940155029297 159.87335777282715 True
2025-03-30 13:47:31.859 [Debug] [ComfyUI-1/STDERR] Prompt executed in 77.08 seconds
2025-03-30 13:47:32.047 [Debug] [ComfyUI-1/STDERR]
2025-03-30 13:47:32.311 [Info] Generated an image in 4.04 sec (prep) and 77.23 sec (gen)
2025-03-30 13:47:34.913 [Debug] [ComfyUI-1/STDERR] 0%| | 0/25 [00:00<?, ?it/s]
2025-03-30 13:47:37.780 [Debug] [ComfyUI-1/STDERR] 4%|▍ | 1/25 [00:02<01:08, 2.87s/it]
2025-03-30 13:47:40.653 [Debug] [ComfyUI-1/STDERR] 8%|▊ | 2/25 [00:05<01:05, 2.87s/it]
2025-03-30 13:47:43.525 [Debug] [ComfyUI-1/STDERR] 12%|█▏ | 3/25 [00:08<01:03, 2.87s/it]
2025-03-30 13:47:46.401 [Debug] [ComfyUI-1/STDERR] 16%|█▌ | 4/25 [00:11<01:00, 2.87s/it]
2025-03-30 13:47:49.276 [Debug] [ComfyUI-1/STDERR] 20%|██ | 5/25 [00:14<00:57, 2.87s/it]
2025-03-30 13:47:52.152 [Debug] [ComfyUI-1/STDERR] 24%|██▍ | 6/25 [00:17<00:54, 2.87s/it]
2025-03-30 13:47:55.030 [Debug] [ComfyUI-1/STDERR] 28%|██▊ | 7/25 [00:20<00:51, 2.87s/it]
2025-03-30 13:47:57.908 [Debug] [ComfyUI-1/STDERR] 32%|███▏ | 8/25 [00:22<00:48, 2.88s/it]
2025-03-30 13:48:00.787 [Debug] [ComfyUI-1/STDERR] 36%|███▌ | 9/25 [00:25<00:46, 2.88s/it]
2025-03-30 13:48:03.666 [Debug] [ComfyUI-1/STDERR] 40%|████ | 10/25 [00:28<00:43, 2.88s/it]
2025-03-30 13:48:06.546 [Debug] [ComfyUI-1/STDERR] 44%|████▍ | 11/25 [00:31<00:40, 2.88s/it]
2025-03-30 13:48:09.427 [Debug] [ComfyUI-1/STDERR] 48%|████▊ | 12/25 [00:34<00:37, 2.88s/it]
2025-03-30 13:48:12.310 [Debug] [ComfyUI-1/STDERR] 52%|█████▏ | 13/25 [00:37<00:34, 2.88s/it]
2025-03-30 13:48:15.192 [Debug] [ComfyUI-1/STDERR] 56%|█████▌ | 14/25 [00:40<00:31, 2.88s/it]
2025-03-30 13:48:18.078 [Debug] [ComfyUI-1/STDERR] 60%|██████ | 15/25 [00:43<00:28, 2.88s/it]
2025-03-30 13:48:20.962 [Debug] [ComfyUI-1/STDERR] 64%|██████▍ | 16/25 [00:46<00:25, 2.88s/it]
2025-03-30 13:48:23.848 [Debug] [ComfyUI-1/STDERR] 68%|██████▊ | 17/25 [00:48<00:23, 2.88s/it]
2025-03-30 13:48:26.733 [Debug] [ComfyUI-1/STDERR] 72%|███████▏ | 18/25 [00:51<00:20, 2.88s/it]
2025-03-30 13:48:29.620 [Debug] [ComfyUI-1/STDERR] 76%|███████▌ | 19/25 [00:54<00:17, 2.88s/it]
2025-03-30 13:48:32.507 [Debug] [ComfyUI-1/STDERR] 80%|████████ | 20/25 [00:57<00:14, 2.89s/it]
2025-03-30 13:48:35.395 [Debug] [ComfyUI-1/STDERR] 84%|████████▍ | 21/25 [01:00<00:11, 2.89s/it]
2025-03-30 13:48:38.284 [Debug] [ComfyUI-1/STDERR] 88%|████████▊ | 22/25 [01:03<00:08, 2.89s/it]
2025-03-30 13:48:41.172 [Debug] [ComfyUI-1/STDERR] 92%|█████████▏| 23/25 [01:06<00:05, 2.89s/it]
2025-03-30 13:48:44.060 [Debug] [ComfyUI-1/STDERR] 96%|█████████▌| 24/25 [01:09<00:02, 2.89s/it]
2025-03-30 13:48:44.060 [Debug] [ComfyUI-1/STDERR] 100%|██████████| 25/25 [01:12<00:00, 2.89s/it]
2025-03-30 13:48:44.060 [Debug] [ComfyUI-1/STDERR] 100%|██████████| 25/25 [01:12<00:00, 2.88s/it]
2025-03-30 13:48:44.401 [Debug] [ComfyUI-1/STDERR] Prompt executed in 72.36 seconds
2025-03-30 13:48:44.808 [Info] Generated an image in 81.28 sec (prep) and 72.52 sec (gen)

Other

.

[mcmonkey4eva]

This is intended behavior.
For most users, it takes longer to load the model a second time than it does to just generate twice on one backend, so Swarm by default when you just have two gens will load the model once and queue both onto the first backend.

You can either:

• (A) hit Load Now on the model (under hamburger menu) to force load it on all backends, or
• (B) generate more than two images at once (so that Swarm will decide to use the next backend)

The default OverQueue value on a backend of 1 means that in addition to one image generating, it can queue up 1 more directly to the backend. You can tweak this value to configure how much a backend will get overused or not.

---

Side note ... flux dev fp8, 25 steps, default res I presume? on linux + rtx 3090? 2.88s/it is very slow. I'd expect closer to 1.2s/it on a base install, faster still with optimizations applied. If you have very high res that speed makes sense, but if that's a base (1024x1024 ish) res, something's slowing you down.

[frenzybiscuit]

Yeah, I power limit the GPUs to 250W.

You're correct, setting to more then two images does the trick. Sorry for the bug report!