can MPI_Offset and MPI_Count be 128b?

[jeffhammond]

Problem

We often say that MPI_Offset and MPI_Count can be 128b, because of exabyte filesystems, for example.

The standard says "offset values in one language may be passed directly to another language without conversion".

The Fortran standard does not guarantee that a type capable of supporting 128b exists, and there are compilers that do not support a compatible type.

MPI 4.0

For I/O there is a need to give the size, displacement, and offset into a file. These quantities can easily be larger than 32 bits which can be the default size of a Fortran integer. To overcome this, these quantities are declared to be INTEGER(KIND=MPI_OFFSET_KIND) in Fortran. In C one uses MPI_Offset. These types must have the same width and encode address values in the same manner such that offset values in one language may be passed directly to another language without conversion.

Fortran 2023 (draft)

16.10.2.14 INT8, INT16, INT32, and INT64
The values of these default integer scalar named constants shall be those of the kind type parameters that specify an INTEGER type whose storage size expressed in bits is 8, 16, 32, and 64 respectively. If, for any of these constants, the processor supports more than one kind of that size, it is processor dependent which kind value is provided. If the processor supports no kind of a particular size, that constant shall be equal to −2 if the processor supports a kind with larger size and −1 otherwise.

Proposal

It seems that we need an advice to implementers, to indicate that offset and count cannot be 128b without non-standard integer support from the associated Fortran compiler.

This is made more interesting in the context of a standard ABI, because we have to fix the C ABI without making assumptions about Fortran compiler behavior, which means sticking strictly to what the Fortran standard guarantees we can use, which is a signed 64-bit integer.

Personally, I would like us to explicitly give up on the idea that an exabyte file system requires MPI I/O to use 128b offsets, because nobody is going to allocate a single file larger than 16 exabytes, even if the underlying filesystem may be larger. MPI I/O can do the appropriate translation from MPI offsets to filesystem offsets.

Changes to the Text

Impact on Implementations

Impact on Users

References and Pull Requests