coll/han: set as default except if processes are consecutive across nodes #10963
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bosilca
previously approved these changes
Oct 21, 2022
|
I'm confused; I thought the plan was to always switch to HAN (assuming multiple ranks / node) and then adjust tuned so that the delta wasn't really important? I'm also not a huge fan of linear == good mapping, given that AWS has very few instances with power of two cores. |
|
@devreal can you fix the conflict? Any thoughts on Brian's questions? |
|
@devreal ping ^ |
coll/han provides better latency than coll/tuned if processes are mapped to nodes nonconsecutively, e.g., using --rank-by node. In that case coll/han reduces the amount of cross-node traffic. Its benefits are less clear with linear process placements. We try to detect linear process placement and if found reduce the priority of coll/han to below coll/tuned. A new mca parameter coll_han_priority_penalty is used to control the adjustment (10 by default). Signed-off-by: Joseph Schuchart <[email protected]>
devreal
force-pushed
the
han_priority_penalty
branch
from
5b70a53 to
acb0984
Compare
|
Is it okay to call |
|
@bosilca and I are not sure that we really want this. I'll mark it as draft for now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
coll/hanprovides better latency thancoll/tunedin cases where processes are mapped to nodes nonconsecutively, e.g., using--rank-by node. In that casecoll/hanreduces the amount of cross-node traffic. Its benefits are less clear with linear consecutive process placements. We try to detect linear process placement and if found reduce the priority ofcoll/hanto belowcoll/tuned. A new mca parametercoll_han_priority_penaltyis used to control the adjustment (10 by default).This PR addresses #10347 for
coll/hanby increasing the default priority ofcoll/hanto 35 (coll/tunedstands at 30)Signed-off-by: Joseph Schuchart [email protected]