Longer types for use with MPI_MINLOC and MPI_MAXLOC

[jeffhammond]

Ticket 319

(was https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/319)

When MPI_MINLOC or MPI_MAXLOC are used with one-sided operations such as MPI_Accumulate, they are restricted to predefined data types. The 2-field structure types do not currently support location fields longer than int. This proposal would add the following predefined types.

MPI Datatype	C type
MPI_FLOAT_LONG	struct {float val; long loc;}
MPI_DOUBLE_LONG	struct {double val; long loc;}
MPI_QUAD_LONG	struct {__float128 val; long loc;} (optional, pending ticket #318)
MPI_FLOAT_LONG_LONG_INT	struct {float val; long long loc;} (optional)
MPI_DOUBLE_LONG_LONG_INT	struct {double val; long long loc;} (optional)
MPI_QUAD_LONG_LONG_INT	struct {__float128 val; long long loc;} (optional, pending ticket #318)
MPI_2LONG	struct {long val; long loc;}
MPI_2LONG_LONG_INT	struct {long long val; long long loc;} (optional)

MPI Datatype	Fortran type (optional)
MPI_2REAL16	pair of REAL*16
MPI_2INTEGER4	pair of INTEGER*4
MPI_2INTEGER8	pair of INTEGER*8

Ticket 342

(was https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/342)

MPI_FLOAT_INT, MPI_LONG_INT, MPI_DOUBLE_INT, MPI_SHORT_INT, MPI_2INT and MPI_LONG_DOUBLE_INT are an inadequate set of built-in datatypes for use with MPI_{MAX,MIN}LOC, especially since MPI_Accumulate does not permit user-defined reductions. There are very good use cases for MPI_2LONG, for example, in sparse graph work by the PETSc team (Jed Brown and Matt Knepley).

The set of types that are valid for use with MPI_{MAX,MIN}LOC should be expanded to include all pairs of built-in datatypes with one another. The reason is that the location (the second element in the pair) need not be a location in a computer science sense, i.e. an offset or address, but rather can be an arbitrary tag.

I see from Ticket 145 that this might cause issues with name uniqueness, hence it may be sensible to add "AND" between the two types when they are different. Thus MPI_LONG_AND_LONG_INT is distinguished from MPI_LONG_LONG_AND_INT as well as the scalar type MPI_LONG_LONG_INT.

If people don't like LOC being a non-integral type, at least let LOC be LONG in addition to INT for all existing pair types.

[ftillier]

Note that MPI_FLOAT_INT may not be any different than MPI_FLOAT_LONG on platforms where sizeof(int) == sizeof(long). Also, any reason you added the _INT suffix to the LONG_LONG versions and not to the LONG version? Shouldn't it just be MPI_FLOAT_LONG_LONG?

What about {long;long long}? MPI_FLOAT_LONG_LONG_LONG? Your proposal to add 'AND' would be especially welcome here.

[jeffhammond]

@ftillier I would much rather have a general pair type constructor that generates built-in types, along the lines of MPI_Type_create_f90_real , rather than create a big static list. Then we don't even have to care about naming these, since that will be the job of the user.

[hjelmn]

I opened an RFC to start the discussion of whether we should continue to maintain the MPI_MINLOC and MPI_MAXLOC operations as part of the MPI standard. See the linked RFC in the rmawg issue.

[jeffhammond]

@ehsantn has noted that int64_t and other types are not supported as values in M**LOC reductions. It's a minor issue if long int is 64-bits but a major one otherwise.

[jdinan]

Traditionally (that is, in reductions) LOC is an MPI process id, which is an int, and is populated by MPI. We would need to make sure that any new types continue to make sense for reductions. Or somehow isolate these new types for use only in the RMA interfaces.

[ehsantn]

Tuples are fundamental programming concepts and have uses beyond just process ids. For example, we use MPI_MINLOC for implementing Pandas idxmin operation when possible:

In [58]: S = pd.Series([11, 1, 3], index=[4, 3, 5])

In [59]: S.idxmin()
Out[59]: 3

In [60]: S = pd.Series([11, 1, 3], index=["A", "B", "C"])

In [61]: S.idxmin()
Out[61]: 'B'

[jprotze]

Traditionally (that is, in reductions) LOC is an MPI process id, which is an int, and is populated by MPI. We would need to make sure that any new types continue to make sense for reductions. Or somehow isolate these new types for use only in the RMA interfaces.

How does Example 6.19 fit into this tradition? The example is there at least since MPI-2.1 (E 5.19).
The formal definition of maxloc/minloc does not restrict the index to rank ids.
As the other examples point out, the application explicitly needs to set the index value to the local rank id in order to get the reduction you described.

[mahermanns]

I agree with @jprotze here. Example 6.19 to me clearly seems to indicate the index part to be used in a more general sense than just rank ids. Especially, as the application needs to explicitly set the rank value (and it's not populated by MPI itself).

I think using the rank seems just the most frequent use "in the wild" that it became synonymous with the index. 🤔

[wgropp]

The application example behind minloc and maxloc was determining the pivot row for LU factorization with partial pivoting. In that case, the "loc" value is the row of the distributed matrix, and the corresponding process rank can be determined from that. Easy, illustrative examples simply use process rank for loc, but the motivating application needed an int (and yes, an int - at the time, that was what the app needed).