Skip to content

Commit 52e81ee

Browse files
committed
rmaps: fixed the ordering of mpirun target nodes
Fixed the desync of job-nodelists between mpirun and orted daemons. The issue was observed when using RSH launching because user can provide arbitrary order of nodes regarding HNP placement. The mpirun process propagate the daemon's nodelist order to nodes. The problem was that HNP itself is assembling the nodelist based on user provided order. As the result ranks assignment was calculated differently on orted and mpirun. Consider following example: * User launches mpirun on node cn2. * Hostlist is cn1,cn2,cn3,cn4; ppn=1 * mpirun is passing hostlist cn[2:2,1,3-4]@0(4) to orteds So as result mpirun will assing rank 0 on cn1 while orted will assign rank 0 on cn2 (because orted sees cn2 as the first element in the node list) Signed-off-by: Boris Karasev <[email protected]>
1 parent bc1d7ff commit 52e81ee

File tree

1 file changed

+7
-7
lines changed

1 file changed

+7
-7
lines changed

orte/mca/rmaps/base/rmaps_base_support_fns.c

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -253,13 +253,12 @@ int orte_rmaps_base_get_target_nodes(opal_list_t *allocated_nodes, orte_std_cntr
253253
/* find the nodes in our node array and assemble them
254254
* in daemon order if the vm was launched
255255
*/
256-
while (NULL != (item = opal_list_remove_first(&nodes))) {
257-
nptr = (orte_node_t*)item;
256+
for (i=0; i < orte_node_pool->size; i++) {
258257
nd = NULL;
259-
for (i=0; i < orte_node_pool->size; i++) {
260-
if (NULL == (node = (orte_node_t*)opal_pointer_array_get_item(orte_node_pool, i))) {
261-
continue;
262-
}
258+
if (NULL == (node = (orte_node_t*)opal_pointer_array_get_item(orte_node_pool, i))) {
259+
continue;
260+
}
261+
OPAL_LIST_FOREACH_SAFE(nptr, next, &nodes, orte_node_t) {
263262
if (0 != strcmp(node->name, nptr->name)) {
264263
OPAL_OUTPUT_VERBOSE((10, orte_rmaps_base_framework.framework_output,
265264
"NODE %s DOESNT MATCH NODE %s",
@@ -332,8 +331,9 @@ int orte_rmaps_base_get_target_nodes(opal_list_t *allocated_nodes, orte_std_cntr
332331
/* reset us back to the end for the next node */
333332
nd = (orte_node_t*)opal_list_get_last(allocated_nodes);
334333
}
334+
opal_list_remove_item(&nodes, (opal_list_item_t*)nptr);
335+
OBJ_RELEASE(nptr);
335336
}
336-
OBJ_RELEASE(nptr);
337337
}
338338
OBJ_DESTRUCT(&nodes);
339339
/* now prune for usage and compute total slots */

0 commit comments

Comments
 (0)