Hv issue719 job manager threaded job start #736
Conversation
|
@soxofaan maybe good to rediscuss from this point (local unit tests are passing) |
|
by the way, I don't think you had to create a new PR, you just could have continued on #730 by pushing to its feature branch (we now have three open PRs for this feature I think, which gets a bit messy) |
|
worked on your initial comments @soxofaan currently checking why some unit tests are not passing |
indeed, that problem has been fixed on master by now |
|
One issue I still see now and want to resolve is that currently launching jobs (to queue them for start) is still tied in to the backend load. This means than when running at full capacity you will not already 'precreate jobs' which can instantly start running |
I'm not sure that pre-creating jobs instead of on the fly like we currently do will make that big of a difference as the time to create a job is usually negligible compared to the required time to start a job. Or do you experience different timings? |
|
@soxofaan ready for review |
|
Ran a small stress test for 30 short lived jobs (10 parallel jobs).
Total time (standard): 2728.00 seconds So the time between creating a job and running a job became 20% shorter. This does need to put in perspective that these gains are small vs the actual duration of entire openEO jobs... |
…ager-threaded-job-start
using `ruff format`, `ruff check --fix --select F401,I001` and `isort`
| while time.time() - start < timeout: | ||
| results = worker_pool.process_futures() | ||
| if results: | ||
| return results |
There was a problem hiding this comment.
this would only return the first batch of results, so you risk losing results from still ongoing tasks
There was a problem hiding this comment.
I actually think it makes sense to implement the wait feature as an option directly in process_futures
- split `Task` hierarchy for better separation of concerns - more tests for some basic classes - clean up unused fixtures from tests - DummyBackend: add setup_job_start_failure
| while time.time() - start < timeout: | ||
| results = worker_pool.process_futures() | ||
| if results: | ||
| return results |
There was a problem hiding this comment.
I actually think it makes sense to implement the wait feature as an option directly in process_futures
- support waiting directly in process_futures - switch to DummyBackend - use more dummy tasks TestJobManagerWorkerThreadPool to simplify setup
…github.com/Open-EO/openeo-python-client into hv_issue719-job-manager-threaded-job-start
…me in a threadsafe manner
…ager-threaded-job-start
|
FYI: I merged master in your feature branch |
No description provided.