opal/event: use epoll by default on Linux #10930
Conversation
Under normal circumstances epoll and poll produce similar performance on Linux. When busy polling is enabled they do not. Testing with a TCP-based system shows a significan performance degredation when using poll with busy waiting enabled. This performance regression is not seen when using epoll. This PR adjusts the default value of opal_event_include to epoll on Linux only to fix the regression. Fixes open-mpi#10929 Signed-off-by: Nathan Hjelm <[email protected]>
hjelmn
force-pushed
the
use_epoll_on_linux_because_it_is_objectively_better
branch
from
fdc46d2 to
279f6b6
Compare
|
:bot:ibm:retest |
|
@hjelmn Will you cherry-pick to v5.0.x? |
|
Just curious: we have multiple projects initializing the common event library. If they call that init function with different arguments, which one gets used? I suspect it is the first caller that sets things, but is that a tad fragile (e.g., if someone changes the code init sequence so PMIx goes first - which I think we were doing at one point)? Are we in trouble if the second (or later) library actually depends on a particular setting? I'm wondering if we need to set some kind of global envar that the libraries can pick up to ensure they set things up as desired. |
|
@jsquyres Yes. The idea is to get this into v5.0.x. @rhc54 Each project should get its own event base if they are not using opal directly. These event bases will use whatever was selected for their project. It might be worth looking into whether PMIx should use epoll as well in this situation. I don't have a system large enough to test how launch performance scales with busy polling enabled. |
Under normal circumstances epoll and poll produce similar performance on Linux. When busy polling is enabled they do not. Testing with a TCP-based system shows a significan performance degredation when using poll with busy waiting enabled. This performance regression is not seen when using epoll. This PR adjusts the default value of opal_event_include to epoll on Linux only to fix the regression.
Fixes #10929
Signed-off-by: Nathan Hjelm [email protected]