More on MPI

Barrier and Broadcast

• Barrier:
  MPI_Barrier(MPI_COMM_WORLD);
• Broadcast:
• Explanation of parameters the same as for the send and receive. The source parameter is the process from which the broadcast originates.
  MPI_Bcast(void *data, int size, MPI_Datatype type, int source, MPI_Comm com);

MPI Scatter and Gather

• Group functions that can manage the data distribution and collecting more easily.
• Scatter: divides a big array into a number of smaller parts equal to the number of processes and sends each process (including the source) a piece of the array in rank order.
• Gather: the opposite, receives data stored in small arrays from all the processes (including the source or root) and concatenates it in the receive array in rank order.

Prototypes

int MPI_Scatter ( void *sendbuf, int sendcnt, MPI_Datatype sendtype, void *recvbuf, int recvcnt, MPI_Datatype recvtype, int root, MPI_Comm comm ) ;

The value of the parameter sendcnt is the size of the data to be sent to each process and not the total size of the sendbuf array. If the type of the send and receive data is the same, then sendcnt and recvcnt should have the same value. The array sendbuf is only used by the root process.

int MPI_Gather ( void *sendbuf, int sendcnt, MPI_Datatype sendtype, void *recvbuf, int recvcnt, MPI_Datatype recvtype, int root, MPI_Comm comm ) ;

Same remark: the value of the parameter recvcnt is the size of the data received by the root from each process, and not the size of the total array recvbuf. The value of this parameter is in general equal to the value of the sendsize, if the type of the data is the same. The array recvbuf is only used by the root process.

Example Using Scatter

const int recvsize = 50;
int *sendbuf, recvbuf[recvsize];
int sendsize = nb_proc*recvsize;
sendbuf = new int[sendsize];
if (proc_id == 0)
  Generate_data(sendbuf, sendsize);
MPI_Scatter(sendbuf, sendsize, MPI_INT, recvbuf, recvsize, MPI_INT, 0, MPI_COMM_WORLD);
for (i=0; i<nb_proc; i++)
  Print_data(recvbuf, recvsize);

Example Using Gather

const int sendsize = 50;
int sendbuf[sendsize], *recvbuf;
int recvsize = nb_proc*sendsize; 
if (proc_id == 0)
  recvbuf = new int[recvsize];
for (i=0; i<nb_proc; i++)
  Generate_data(sendbuf, sendsize);
MPI_Gather(sendbuf, sendsize, MPI_INT, recvbuf, recvsize, MPI_INT, 0, MPI_COMM_WORLD);
if (proc_id == 0)
  Print_data(recvbuf, recvsize);

Receive Any Source

• A receive message in MPI can have the constant MPI_ANY_SOURCE in the place of the source process.
• This allows the process to receive a message form any process sending it.
• All the other restrictions (size, etc) still apply.
• The id of the source process can be taken from status.MPI_SOURCE after the message is received.
• An MPI_ANY_TAG is also available.

Non-Blocking Send and Receive

• Non-blocking communication:
  MPI_Isend(&data, count, datatype, dest, tag, comm, &request);
  MPI_Irecv(&data, count, datatype, dest, tag, comm, &request);
• The request variable must be of type MPI_Request.
• These calls initiate a send or receive operation without waiting for it to complete.

Solving the Send - Recv