green: Don't fall back to epoll() if possible #12186
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The green scheduler can optimize its runtime based on this by deciding to not go to sleep in epoll() if there is no active I/O and there is a task to be stolen. This is implemented for librustuv by keeping a count of the number of tasks which are currently homed. If a task is homed, and then performs a blocking I/O operation, the count will be nonzero while the task is blocked. The homing count is intentionally 0 when there are I/O handles, but no handles currently blocked. The reason for this is that epoll() would only be used to wake up the scheduler anyway. The crux of this change was to have a `HomingMissile` contain a mutable borrowed reference back to the `HomeHandle`. The rest of the change was just dealing with this fallout. This reference is used to decrement the homed handle count in a HomingMissile's destructor. Also note that the count maintained is not atomic because all of its increments/decrements/reads are all on the same I/O thread.
This is in preparation for running do_work in a loop while there are no active I/O handles. This changes the do_work and interpret_message_queue methods to return a triple where the last element is a boolean flag as to whether work was done or not. This commit preserves the same behavior as before, it simply re-structures the code in preparation for future work.
Currently, a scheduler will hit epoll() or kqueue() at the end of *every task*.
The reason is that the scheduler will context switch back to the scheduler task,
terminate the previous task, and then return from run_sched_once. In doing so,
the scheduler will poll for any active I/O.
This shows up painfully in benchmarks that have no I/O at all. For example, this
benchmark:
for _ in range(0, 1000000) {
spawn(proc() {});
}
In this benchmark, the scheduler is currently wasting a good chunk of its time
hitting epoll() when there's always active work to be done (run with
RUST_THREADS=1).
This patch uses the previous two commits to alter the scheduler's behavior to
only return from run_sched_once if no work could be found when trying really
really hard. If there is active I/O, this commit will perform the same as
before, falling back to epoll() to check for I/O completion (to not starve I/O
tasks).
In the benchmark above, I got the following numbers:
12.554s on today's master
3.861s with rust-lang#12172 applied
2.261s with both this and rust-lang#12172 applied
cc rust-lang#8341
bors
added a commit
that referenced
this pull request
Feb 14, 2014
Any single-threaded task benchmark will spend a good chunk of time in `kqueue()` on osx and `epoll()` on linux, and the reason for this is that each time a task is terminated it will hit the syscall. When a task terminates, it context switches back to the scheduler thread, and the scheduler thread falls out of `run_sched_once` whenever it figures out that it did some work. If we know that `epoll()` will return nothing, then we can continue to do work locally (only while there's work to be done). We must fall back to `epoll()` whenever there's active I/O in order to check whether it's ready or not, but without that (which is largely the case in benchmarks), we can prevent the costly syscall and can get a nice speedup. I've separated the commits into preparation for this change and then the change itself, the last commit message has more details.
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Jul 25, 2022
minor: Move inferred type completions
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Any single-threaded task benchmark will spend a good chunk of time in
kqueue()on osx andepoll()on linux, and the reason for this is that each time a task is terminated it will hit the syscall. When a task terminates, it context switches back to the scheduler thread, and the scheduler thread falls out ofrun_sched_oncewhenever it figures out that it did some work.If we know that
epoll()will return nothing, then we can continue to do work locally (only while there's work to be done). We must fall back toepoll()whenever there's active I/O in order to check whether it's ready or not, but without that (which is largely the case in benchmarks), we can prevent the costly syscall and can get a nice speedup.I've separated the commits into preparation for this change and then the change itself, the last commit message has more details.