std::rt: Fix the set up of the main thread so that it doesn't try to ste...

[brson]

...al work

This is causing really awful scheduler behavior where the main thread scheduler is
continually waking up, stealing work, discovering it can't actually run the work,
and sending it off to another scheduler.

No test cases because we don't have suitable instrumentation for it.

[brson]

This fixes a problem described on dev-servo

[sanxiyn]

CC #9831.

[alexcrichton]

The patch itself looks fine to me, but I'm a little confused about what's going on here. This seems to be indicative of a larger problem if we've got runnable tasks that are waking up schedulers they can't run on. I'm not really sure why a task wouldn't be run on a scheduler, and if it's targeted at a specific scheduler then it seems like we shouldn't be waking up other schedulers regardless.

Could you explain a bit more about what's happening? I'm not sure why the main scheduler is determining that it can't run this task, and I'm also not sure why the task is waking up the main scheduler if it can't be run on it.

[brson]

@alexcrichton

Schedulers run on a single thread. Within the runtime, some subset of the schedulers are work stealing schedulers, some are not. The work stealing schedulers are collaborating to schedule tasks across threads. The non-work stealing schedulers are currently used solely for 'single-threaded schedulers' in which one task is assigned it's own scheduler and it's own thread.

There are two reasons that a scheduler may not be able to run a task:

• If the task is 'homed', e.g. if it is tied to a single-thread scheduler or is doing I/O, then it can only run on the 'home' scheduler.
• If the scheduler has the 'run_anything' flag unset, i.e. it is tied to a specific homed task.

Tasks that are homed to a specific scheduler are sent directly to that scheduler for execution - they do not wake up any other schedulers. Tasks that are not homed but can't be run on the local scheduler are sent instead to a scheduler-designated 'friend' scheduler for reevaluation.

All schedulers share a SleeperList which contains a stack of work-stealing schedulers that are waiting for work. When a work-stealing scheduler discovers that there is no work to do it pushes it's handle to the list and idles. When a scheduler makes work available it pops a scheduler handle from the sleeper list and sends it a message to wake up and steal.

What was happening here is that the main thread scheduler, which is a single-threaded scheduler that runs the homed main task, was pushing itself to the sleeper list as if it were a work-stealing scheduler. It would get woken up to look for work, steal random, unhomed tasks, but because the main thread scheduler's 'run_anything' flag is false could not actually run the tasks, so it would send it to its friend scheduler.

Another way to phrase this change might be to make Scheduler not push to the SleeperList when the run_anything flag is false. This change as written though specifically makes the main thread scheduler's set up identical to those created with spawn(SingleThreaded).

[alexcrichton]

Thanks for the explanation! I think it's fine to have the identical setup, and I also think that it makes the most sense.