Added an initial implementation of partitioned communication over persistant MPI calls. #8641

mdosanjh · 2021-03-17T20:02:37Z

Signed-off-by: Matthew Dosanjh [email protected]

This is an initial implementation of partitioned communication using persistent sends and persistent receives. This should be compliant with the current release candidate of the MPI-4.0 specification.

ibm-ompi · 2021-03-17T20:18:16Z

The IBM CI (GNU/Scale) build failed! Please review the log, linked below.

Gist: https://gist.github.com/03dace16a79e5958d962c52e752aee8d

ibm-ompi · 2021-03-17T20:21:02Z

The IBM CI (XL) build failed! Please review the log, linked below.

Gist: https://gist.github.com/6e3e4e117f7c2aefd199166acefc6be9

ibm-ompi · 2021-03-17T20:23:15Z

The IBM CI (PGI) build failed! Please review the log, linked below.

Gist: https://gist.github.com/f6049cb406966176744f98a0685e5655

ibm-ompi · 2021-03-18T18:05:10Z

The IBM CI (XL) build failed! Please review the log, linked below.

Gist: https://gist.github.com/3fc96a30c7741cd755a49eafcc7be691

ibm-ompi · 2021-03-18T18:49:02Z

The IBM CI (PGI) build failed! Please review the log, linked below.

Gist: https://gist.github.com/eb81ff3fe01960c95f32f78d509140d3

ibm-ompi · 2021-03-18T19:07:27Z

The IBM CI (GNU/Scale) build failed! Please review the log, linked below.

Gist: https://gist.github.com/54c78e4f404b5a0637d79fc9793b2ae0

ibm-ompi · 2021-03-18T19:55:33Z

The IBM CI (XL) build failed! Please review the log, linked below.

Gist: https://gist.github.com/7a7ba36c4d7585b1384612d9500095b2

ibm-ompi · 2021-03-18T20:32:14Z

The IBM CI (XL) build failed! Please review the log, linked below.

Gist: https://gist.github.com/1a6ea556e35bbe709dd715057608cb1b

ibm-ompi · 2021-03-18T21:09:53Z

The IBM CI (XL) build failed! Please review the log, linked below.

Gist: https://gist.github.com/fec21300f6fdbde084d421c53690c9c3

ibm-ompi · 2021-03-18T22:26:25Z

The IBM CI (XL) build failed! Please review the log, linked below.

Gist: https://gist.github.com/fbe93757a0df3d339c7e34f226c21ebd

mdosanjh · 2021-03-18T23:30:15Z

I'm having trouble parsing these errors from the checks (one seems to be happening in OB1?) does anyone have insight on to what I need to fix?

mdosanjh · 2021-03-19T04:03:25Z

bot:retest

ibm-ompi · 2021-03-19T04:20:49Z

The IBM CI (XL) build failed! Please review the log, linked below.

Gist: https://gist.github.com/08068d909e505586843f9b9497d51a1e

jsquyres · 2021-03-19T16:01:05Z

@mdosanjh Not sure why you're asking about ob1, the error is later:

Making all in mpi/fortran/use-mpi-f08/bindings
make[2]: Entering directory '/tmp/ompi-src/ompi/mpi/fortran/use-mpi-f08/bindings'
  PPFC     ompi-mpifh-bindings.lo
"mpi-f-interfaces-bind.h", line 331.28: 1513-230 (S) Procedure ompi_parrived_f was declared with the BIND(C) attribute.  Dummy argument flag of ompi_parrived_f must have interoperable type and type parameters.
** ompi_mpifh_bindings   === End of Compilation 1 ===
1501-511  Compilation failed for file ompi-mpifh-bindings.F90.

mdosanjh · 2021-03-19T17:21:20Z

@jsquyres Apologies, this is my first time interacting with the pull request check system and ob1 was where https://gist.github.com/08068d909e505586843f9b9497d51a1e stopped/truncated as far as I could tell, so I had assumed that was where the issue was. I have subsequently found a local system to replicate the error on and I believe I have fixed it. (I just noticed the truncation banner at the top of that page)

@mdosanjh Not sure why you're asking about ob1, the error is later:

Making all in mpi/fortran/use-mpi-f08/bindings
make[2]: Entering directory '/tmp/ompi-src/ompi/mpi/fortran/use-mpi-f08/bindings'
  PPFC     ompi-mpifh-bindings.lo
"mpi-f-interfaces-bind.h", line 331.28: 1513-230 (S) Procedure ompi_parrived_f was declared with the BIND(C) attribute.  Dummy argument flag of ompi_parrived_f must have interoperable type and type parameters.
** ompi_mpifh_bindings   === End of Compilation 1 ===
1501-511  Compilation failed for file ompi-mpifh-bindings.F90.

ompi/mca/part/base/part_base_frame.c

ompi/mca/part/base/part_base_precvreq.c

ompi/mca/part/base/part_base_psendreq.c

ompi/mca/part/persist/part_persist.c

ompi/mca/part/persist/part_persist.h

ompi/mca/part/persist/part_persist_sendreq.h

ompi/mpi/c/pready_list.c

ompi/mpi/fortran/mpif-h/pready_list_f.c

mdosanjh · 2021-03-22T21:39:01Z

bot:retest

jsquyres · 2021-03-22T22:03:29Z

@mdosanjh Please see https://github.com/open-mpi/ompi/wiki/PRJenkins for a listing of the bot commands. If possible, please be judicious with re-running CI, because our CI can get backed up with queued builds, etc.

jsquyres · 2021-03-22T22:03:52Z

Oops -- clicked the wrong button there; didn't mean to close this PR.

mdosanjh · 2021-03-22T22:57:34Z

@mdosanjh Please see https://github.com/open-mpi/ompi/wiki/PRJenkins for a listing of the bot commands. If possible, please be judicious with re-running CI, because our CI can get backed up with queued builds, etc.

Apologies, the build kept failing on ARMV8 with a GitHub check out error (with a reference a commit (other than mine) not being a tree). I'll try to be more specific with the bot commands in the future.

hppritcha · 2021-03-26T14:49:31Z

ompi/mpi/fortran/mpif-h/Makefile.am

@@ -340,6 +340,12 @@ lib@OMPI_LIBMPI_NAME@_mpifh_la_SOURCES += \
        iscatter_f.c \
        iscatterv_f.c \
        issend_f.c \
+	parrived_f.c \


please put in alphabetical order

hppritcha · 2021-03-26T14:50:39Z

ompi/mpi/fortran/mpif-h/profile/Makefile.am

@@ -254,6 +254,12 @@ linked_files = \
        piscatterv_f.c \
        pisend_f.c \
        pissend_f.c \
+        pparrived_f.c \


these too should be in alphabetical order in the source file list

hppritcha · 2021-03-26T14:51:06Z

ompi/mpi/fortran/mpif-h/prototypes_mpi.h

@@ -304,6 +304,12 @@ PN2(void, MPI_Irecv, mpi_irecv, MPI_IRECV, (char *buf, MPI_Fint *count, MPI_Fint
 PN2(void, MPI_Irsend, mpi_irsend, MPI_IRSEND, (char *buf, MPI_Fint *count, MPI_Fint *datatype, MPI_Fint *dest, MPI_Fint *tag, MPI_Fint *comm, MPI_Fint *request, MPI_Fint *ierr));
 PN2(void, MPI_Isend, mpi_isend, MPI_ISEND, (char *buf, MPI_Fint *count, MPI_Fint *datatype, MPI_Fint *dest, MPI_Fint *tag, MPI_Fint *comm, MPI_Fint *request, MPI_Fint *ierr));
 PN2(void, MPI_Issend, mpi_issend, MPI_ISSEND, (char *buf, MPI_Fint *count, MPI_Fint *datatype, MPI_Fint *dest, MPI_Fint *tag, MPI_Fint *comm, MPI_Fint *request, MPI_Fint *ierr));
+PN2(void, MPI_Parrived, mpi_parrived, MPI_PARRIVED, (MPI_Fint *request, MPI_Fint *partition, ompi_fortran_logical_t *flag, MPI_Fint *ierr));


alphabetical order

hppritcha · 2021-03-26T14:51:24Z

ompi/mpi/fortran/use-mpi-f08/Makefile.am

@@ -320,6 +321,12 @@ mpi_api_files = \
        iscatterv_f08.F90 \
        isend_f08.F90 \
        issend_f08.F90 \
+        psend_init_f08.F90 \


alphabetical order

hppritcha · 2021-03-26T14:53:46Z

ompi/mpi/fortran/use-mpi-ignore-tkr/pmpi-ignore-tkr-interfaces.h

@@ -165,6 +165,12 @@
 #define MPI_Iscatterv PMPI_Iscatterv
 #define MPI_Isend PMPI_Isend
 #define MPI_Issend PMPI_Issend
+#define MPI_Psend_init PMPI_Psend_init


alphabetical order

hppritcha · 2021-03-26T14:57:16Z

ompi/request/request_dbg.h

@@ -25,6 +25,7 @@ typedef enum {
    OMPI_REQUEST_GEN,      /**< MPI-2 generalized request */
    OMPI_REQUEST_WIN,      /**< MPI-2 one-sided request */
    OMPI_REQUEST_COLL,     /**< MPI-3 non-blocking collectives request */
+    OMPI_REQUEST_PART,     /**< MPI-4 partitioned communication request */


why was this added here rather than after OMPI_REQUEST_COMM?

@hppritcha Thank you for the review, I've fixed all of the alphabetical orderings and as per this comment, there was not a reason for the placement, so I have moved OMPI_REQUEST_PART to be after OMPI_REQUEST_COMM.

hppritcha · 2021-03-26T15:03:03Z

ompi/include/mpi.h.in

@@ -2310,6 +2321,17 @@ OMPI_DECLSPEC  int PMPI_Isend(const void *buf, int count, MPI_Datatype datatype,
                              int tag, MPI_Comm comm, MPI_Request *request);
 OMPI_DECLSPEC  int PMPI_Issend(const void *buf, int count, MPI_Datatype datatype, int dest,
                               int tag, MPI_Comm comm, MPI_Request *request);
+OMPI_DECLSPEC  int PMPI_Precv_init(const void* buf, int partitions, MPI_Count count,


hppritcha · 2021-03-26T15:03:06Z

ompi/include/mpi.h.in

@@ -1645,6 +1645,17 @@ OMPI_DECLSPEC  int MPI_Isend(const void *buf, int count, MPI_Datatype datatype,
                             int tag, MPI_Comm comm, MPI_Request *request);
 OMPI_DECLSPEC  int MPI_Issend(const void *buf, int count, MPI_Datatype datatype, int dest,
                              int tag, MPI_Comm comm, MPI_Request *request);
+OMPI_DECLSPEC  int MPI_Precv_init(const void* buf, int partitions, MPI_Count count,


could you please add these in alphabetical order?

ibm-ompi · 2021-03-27T00:02:41Z

The IBM CI (GNU/Scale) build failed! Please review the log, linked below.

Gist: https://gist.github.com/757b00059d0d4eb755171f50e177e95e

ibm-ompi · 2021-03-27T00:05:05Z

The IBM CI (XL) build failed! Please review the log, linked below.

Gist: https://gist.github.com/7f9273003d2846772017c4643f28d9c4

ibm-ompi · 2021-03-27T00:09:51Z

The IBM CI (PGI) build failed! Please review the log, linked below.

Gist: https://gist.github.com/945f245ddfc09f44492840d286a255ba

hppritcha · 2021-04-13T16:53:23Z

@mdosanjh could you rebase this PR? Looks like a simple conflict.

@hppritcha

Signed-off-by: Matthew Dosanjh <[email protected]> Addressed ordering issues as per review by @hppritcha fixed ordering in ompi/mpi/c makefiles

mdosanjh · 2021-04-14T04:26:44Z

@mdosanjh could you rebase this PR? Looks like a simple conflict.

@hppritcha It should be rebased now.

mdosanjh · 2021-04-14T20:33:04Z

@hjelmn I just wanted check to see if you've gotten a chance to re-review this since I incorporated your requested changes.

mdosanjh · 2021-04-15T17:29:31Z

@hjelmn I just wanted check to see if you've gotten a chance to re-review this since I incorporated your requested changes.

It was suggested that I put some sort of timeline here to say when I would 'dismiss' the review, specifically the 23rd. (If this is unreasonable, please let me know)

hppritcha · 2021-07-29T16:43:34Z

@regrant does this PR include all partitioned comm related items that went in to MPI 4.0?

regrant · 2021-07-29T16:46:54Z

@hppritcha This has all of the MPI 4.0 related content, so we're 4.0 compliant for partitioned communication with this merge.

mdosanjh force-pushed the persist-part branch from 4d4c5ae to 0c5d0ec Compare March 18, 2021 16:06

mdosanjh force-pushed the persist-part branch from 4907d6f to 62f32f3 Compare March 18, 2021 19:23

mdosanjh force-pushed the persist-part branch from 62f32f3 to efdf697 Compare March 18, 2021 20:11

mdosanjh force-pushed the persist-part branch from ef6bae1 to 70304a8 Compare March 18, 2021 20:46

mdosanjh force-pushed the persist-part branch 3 times, most recently from 68512cc to 8324fea Compare March 18, 2021 22:09

mdosanjh force-pushed the persist-part branch from a5767b9 to 54e4e70 Compare March 19, 2021 17:17

mdosanjh requested a review from hjelmn March 19, 2021 17:32

mdosanjh added the MPI-4.0 label Mar 19, 2021

mdosanjh self-assigned this Mar 19, 2021

hjelmn requested changes Mar 19, 2021

View reviewed changes

mdosanjh force-pushed the persist-part branch 3 times, most recently from 2d9bd52 to fadd3be Compare March 22, 2021 15:50

mdosanjh requested a review from hjelmn March 22, 2021 17:42

jsquyres closed this Mar 22, 2021

jsquyres reopened this Mar 22, 2021

mdosanjh requested a review from hppritcha March 26, 2021 13:46

hppritcha requested changes Mar 26, 2021

View reviewed changes

mdosanjh force-pushed the persist-part branch 3 times, most recently from 24b8bb9 to e0d646b Compare March 26, 2021 23:49

mdosanjh force-pushed the persist-part branch from e0d646b to f595956 Compare March 27, 2021 00:49

mdosanjh requested a review from hppritcha March 27, 2021 02:04

Added an initial implementation of partitioned communication.

d79e13d

Signed-off-by: Matthew Dosanjh <[email protected]> Addressed ordering issues as per review by @hppritcha fixed ordering in ompi/mpi/c makefiles

mdosanjh force-pushed the persist-part branch from f595956 to d79e13d Compare April 14, 2021 00:37

hppritcha approved these changes Apr 14, 2021

View reviewed changes

hjelmn approved these changes Apr 16, 2021

View reviewed changes

hppritcha merged commit 03ed56c into open-mpi:master Apr 20, 2021

mdosanjh deleted the persist-part branch April 20, 2021 18:01

jsquyres mentioned this pull request Oct 19, 2021

Is partitioned communication going to v5.0? #9554

Closed

Added an initial implementation of partitioned communication over persistant MPI calls. #8641

Added an initial implementation of partitioned communication over persistant MPI calls. #8641

Uh oh!

Conversation

mdosanjh commented Mar 17, 2021

Uh oh!

ibm-ompi commented Mar 17, 2021

Uh oh!

ibm-ompi commented Mar 17, 2021

Uh oh!

ibm-ompi commented Mar 17, 2021

Uh oh!

ibm-ompi commented Mar 18, 2021

Uh oh!

ibm-ompi commented Mar 18, 2021

Uh oh!

ibm-ompi commented Mar 18, 2021

Uh oh!

ibm-ompi commented Mar 18, 2021

Uh oh!

ibm-ompi commented Mar 18, 2021

Uh oh!

ibm-ompi commented Mar 18, 2021

Uh oh!

ibm-ompi commented Mar 18, 2021

Uh oh!

mdosanjh commented Mar 18, 2021

Uh oh!

mdosanjh commented Mar 19, 2021

Uh oh!

ibm-ompi commented Mar 19, 2021

Uh oh!

jsquyres commented Mar 19, 2021

Uh oh!

mdosanjh commented Mar 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mdosanjh commented Mar 22, 2021

Uh oh!

jsquyres commented Mar 22, 2021

Uh oh!

jsquyres commented Mar 22, 2021

Uh oh!

mdosanjh commented Mar 22, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ibm-ompi commented Mar 27, 2021

Uh oh!

ibm-ompi commented Mar 27, 2021

Uh oh!

ibm-ompi commented Mar 27, 2021

Uh oh!

hppritcha commented Apr 13, 2021

Uh oh!

mdosanjh commented Mar 19, 2021 •

edited

Loading