MPI_SEEK_END is limited to 6 MiB

[tukss]

Thank you for taking the time to submit an issue!

Background information

What version of Open MPI are you using? (e.g., v4.1.6, v5.0.1, git branch name and hash, etc.)

v5.0.5

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

It was built with spack on Fedora 41 (GCC 14.2.1).

If you are building/installing from a git clone, please copy-n-paste the output from git submodule status.

N/A

Please describe the system on which you are running

• Operating system/version: Fedora 41
• Computer hardware: amd64 architecture
• Network type: local only (single rank)

---

Details of the problem

When I open a file larger than 6 MiB using MPI-IO and try to seek relative to its end with MPI_File_seek and the MPI_SEEK_END option, the seek is relative to the 6 MiB mark instead. I think the reason is that MPI_SEEK_END is running the additional step of calling mca_io_ompio_file_get_eof_offset here. This function always returns at most the size set when creating the view, which seems to be unconditionally MCA_IO_DEFAULT_FILE_VIEW_SIZE (equal to 6 MiB).

It looks like the problematic behavior was introduced in #6867, which was fixing issue #6858, also related to MPI_SEEK_END.

I have a small Fortran test problem triggering the behavior. Just put any file larger than 6 MiB under the name testfile and run the program. I am seeking to 4 bytes before the end of the file, one time using MPI_SEEK_END and one time with MPI_SEEK_SET and getting the size of the file using MPI_File_get_size beforehand. With OpenMPI on an 8 MiB file I get this output:

 file size:               8388608
 position after MPI_SEEK_END:               6291452              6291452
 position after MPI_SEEK_SET:               8388604              8388604

Trying the same using MPICH gives me the expected output:

 file size:               8388608
 position after MPI_SEEK_END:               8388604              8388604
 position after MPI_SEEK_SET:               8388604              8388604

Reproduer:

program seek_end_reproducer
  use mpi

  integer :: fh, ierr
  integer(kind=MPI_OFFSET_KIND) :: pos, offset, byte_offset, file_size

  call mpi_init(ierr)
  
  call mpi_file_open(MPI_COMM_WORLD, 'testfile', MPI_MODE_RDONLY, MPI_INFO_NULL, fh, ierr)

  ! Setting the view has no effect.
  call mpi_file_set_view(fh, 0_MPI_OFFSET_KIND, MPI_BYTE, MPI_BYTE, "native", MPI_INFO_NULL, ierr)

  call mpi_file_get_size(fh, file_size, ierr)

  print*, 'file size: ', file_size

  pos = -4
  call mpi_file_seek(fh, pos, MPI_SEEK_END, ierr)

  call mpi_file_get_position(fh, offset, ierr)
  call mpi_file_get_byte_offset(fh, offset, byte_offset, ierr)

  print*, 'position after MPI_SEEK_END: ', offset, byte_offset

  call mpi_file_seek(fh, file_size + pos, MPI_SEEK_set, ierr)

  call mpi_file_get_position(fh, offset, ierr)
  call mpi_file_get_byte_offset(fh, offset, byte_offset, ierr)

  print*, 'position after MPI_SEEK_SET: ', offset, byte_offset
  
  call mpi_finalize(ierr)
end program seek_end_reproducer

[edgargabriel]

@tukss thank you for the bug report, I will take a look at it, but it might be a few days until I get to it.

[edgargabriel]

@tukss sorry it took so long to get to it, would you have a chance to confirm that PR #13007 fixes the issue for you as well?

I added also some tests based on your reproducer to our internal testsuite, hopefully this avoids breaking the issue later again.

[tukss]

@edgargabriel Thanks. Your PR fixes the problem for me.