Chris@19: Chris@19: Chris@19: An improved replacement for MPI_Alltoall - FFTW 3.3.4 Chris@19: Chris@19: Chris@19: Chris@19: Chris@19: Chris@19: Chris@19: Chris@19: Chris@19: Chris@19: Chris@19: Chris@19: Chris@19:
Chris@19: Chris@19: Chris@19:

Chris@19: Previous: Advanced distributed-transpose interface, Chris@19: Up: FFTW MPI Transposes Chris@19:


Chris@19:
Chris@19: Chris@19:

6.7.3 An improved replacement for MPI_Alltoall

Chris@19: Chris@19:

We close this section by noting that FFTW's MPI transpose routines can Chris@19: be thought of as a generalization for the MPI_Alltoall function Chris@19: (albeit only for floating-point types), and in some circumstances can Chris@19: function as an improved replacement. Chris@19: Chris@19: Chris@19:

MPI_Alltoall is defined by the MPI standard as: Chris@19: Chris@19:

     int MPI_Alltoall(void *sendbuf, int sendcount, MPI_Datatype sendtype,
Chris@19:                       void *recvbuf, int recvcnt, MPI_Datatype recvtype,
Chris@19:                       MPI_Comm comm);
Chris@19: 
Chris@19:

In particular, for double* arrays in and out, Chris@19: consider the call: Chris@19: Chris@19:

     MPI_Alltoall(in, howmany, MPI_DOUBLE, out, howmany MPI_DOUBLE, comm);
Chris@19: 
Chris@19:

This is completely equivalent to: Chris@19: Chris@19:

     MPI_Comm_size(comm, &P);
Chris@19:      plan = fftw_mpi_plan_many_transpose(P, P, howmany, 1, 1, in, out, comm, FFTW_ESTIMATE);
Chris@19:      fftw_execute(plan);
Chris@19:      fftw_destroy_plan(plan);
Chris@19: 
Chris@19:

That is, computing a P × P transpose on P processes, Chris@19: with a block size of 1, is just a standard all-to-all communication. Chris@19: Chris@19:

However, using the FFTW routine instead of MPI_Alltoall may Chris@19: have certain advantages. First of all, FFTW's routine can operate Chris@19: in-place (in == out) whereas MPI_Alltoall can only Chris@19: operate out-of-place. Chris@19: Chris@19: Chris@19:

Second, even for out-of-place plans, FFTW's routine may be faster, Chris@19: especially if you need to perform the all-to-all communication many Chris@19: times and can afford to use FFTW_MEASURE or Chris@19: FFTW_PATIENT. It should certainly be no slower, not including Chris@19: the time to create the plan, since one of the possible algorithms that Chris@19: FFTW uses for an out-of-place transpose is simply to call Chris@19: MPI_Alltoall. However, FFTW also considers several other Chris@19: possible algorithms that, depending on your MPI implementation and Chris@19: your hardware, may be faster. Chris@19: Chris@19: Chris@19: Chris@19: Chris@19: