FFTW 3.3.5: MPI Data Distribution Functions

As described above (see MPI Data Distribution), in order to cannam@127: allocate your arrays, before creating a plan, you must first cannam@127: call one of the following routines to determine the required cannam@127: allocation size and the portion of the array locally stored on a given cannam@127: process. The MPI_Comm communicator passed here must be cannam@127: equivalent to the communicator used below for plan creation. cannam@127:

The basic interface for multidimensional transforms consists of the cannam@127: functions: cannam@127:

cannam@127:

ptrdiff_t fftw_mpi_local_size_2d(ptrdiff_t n0, ptrdiff_t n1, MPI_Comm comm,
cannam@127:                                  ptrdiff_t *local_n0, ptrdiff_t *local_0_start);
cannam@127: ptrdiff_t fftw_mpi_local_size_3d(ptrdiff_t n0, ptrdiff_t n1, ptrdiff_t n2,
cannam@127:                                  MPI_Comm comm,
cannam@127:                                  ptrdiff_t *local_n0, ptrdiff_t *local_0_start);
cannam@127: ptrdiff_t fftw_mpi_local_size(int rnk, const ptrdiff_t *n, MPI_Comm comm,
cannam@127:                               ptrdiff_t *local_n0, ptrdiff_t *local_0_start);
cannam@127: 
cannam@127: ptrdiff_t fftw_mpi_local_size_2d_transposed(ptrdiff_t n0, ptrdiff_t n1, MPI_Comm comm,
cannam@127:                                             ptrdiff_t *local_n0, ptrdiff_t *local_0_start,
cannam@127:                                             ptrdiff_t *local_n1, ptrdiff_t *local_1_start);
cannam@127: ptrdiff_t fftw_mpi_local_size_3d_transposed(ptrdiff_t n0, ptrdiff_t n1, ptrdiff_t n2,
cannam@127:                                             MPI_Comm comm,
cannam@127:                                             ptrdiff_t *local_n0, ptrdiff_t *local_0_start,
cannam@127:                                             ptrdiff_t *local_n1, ptrdiff_t *local_1_start);
cannam@127: ptrdiff_t fftw_mpi_local_size_transposed(int rnk, const ptrdiff_t *n, MPI_Comm comm,
cannam@127:                                          ptrdiff_t *local_n0, ptrdiff_t *local_0_start,
cannam@127:                                          ptrdiff_t *local_n1, ptrdiff_t *local_1_start);
cannam@127:

These functions return the number of elements to allocate (complex cannam@127: numbers for DFT/r2c/c2r plans, real numbers for r2r plans), whereas cannam@127: the local_n0 and local_0_start return the portion cannam@127: (local_0_start to local_0_start + local_n0 - 1) of the cannam@127: first dimension of an n₀ × n₁ × n₂ × … × n_d-1 array that is stored on the local cannam@127: process. See Basic and advanced distribution interfaces. For cannam@127: FFTW_MPI_TRANSPOSED_OUT plans, the ‘_transposed’ variants cannam@127: are useful in order to also return the local portion of the first cannam@127: dimension in the n₁ × n₀ × n₂ ×…× n_d-1 transposed output. cannam@127: See Transposed distributions. cannam@127: The advanced interface for multidimensional transforms is: cannam@127:

cannam@127:

ptrdiff_t fftw_mpi_local_size_many(int rnk, const ptrdiff_t *n, ptrdiff_t howmany,
cannam@127:                                    ptrdiff_t block0, MPI_Comm comm,
cannam@127:                                    ptrdiff_t *local_n0, ptrdiff_t *local_0_start);
cannam@127: ptrdiff_t fftw_mpi_local_size_many_transposed(int rnk, const ptrdiff_t *n, ptrdiff_t howmany,
cannam@127:                                               ptrdiff_t block0, ptrdiff_t block1, MPI_Comm comm,
cannam@127:                                               ptrdiff_t *local_n0, ptrdiff_t *local_0_start,
cannam@127:                                               ptrdiff_t *local_n1, ptrdiff_t *local_1_start);
cannam@127:

These differ from the basic interface in only two ways. First, they cannam@127: allow you to specify block sizes block0 and block1 (the cannam@127: latter for the transposed output); you can pass cannam@127: FFTW_MPI_DEFAULT_BLOCK to use FFTW’s default block size as in cannam@127: the basic interface. Second, you can pass a howmany parameter, cannam@127: corresponding to the advanced planning interface below: this is for cannam@127: transforms of contiguous howmany-tuples of numbers cannam@127: (howmany = 1 in the basic interface). cannam@127:

The corresponding basic and advanced routines for one-dimensional cannam@127: transforms (currently only complex DFTs) are: cannam@127:

cannam@127:

ptrdiff_t fftw_mpi_local_size_1d(
cannam@127:              ptrdiff_t n0, MPI_Comm comm, int sign, unsigned flags,
cannam@127:              ptrdiff_t *local_ni, ptrdiff_t *local_i_start,
cannam@127:              ptrdiff_t *local_no, ptrdiff_t *local_o_start);
cannam@127: ptrdiff_t fftw_mpi_local_size_many_1d(
cannam@127:              ptrdiff_t n0, ptrdiff_t howmany,
cannam@127:              MPI_Comm comm, int sign, unsigned flags,
cannam@127:              ptrdiff_t *local_ni, ptrdiff_t *local_i_start,
cannam@127:              ptrdiff_t *local_no, ptrdiff_t *local_o_start);
cannam@127:

As above, the return value is the number of elements to allocate cannam@127: (complex numbers, for complex DFTs). The local_ni and cannam@127: local_i_start arguments return the portion cannam@127: (local_i_start to local_i_start + local_ni - 1) of the cannam@127: 1d array that is stored on this process for the transform cannam@127: input, and local_no and local_o_start are the cannam@127: corresponding quantities for the input. The sign cannam@127: (FFTW_FORWARD or FFTW_BACKWARD) and flags must cannam@127: match the arguments passed when creating a plan. Although the inputs cannam@127: and outputs have different data distributions in general, it is cannam@127: guaranteed that the output data distribution of an cannam@127: FFTW_FORWARD plan will match the input data distribution cannam@127: of an FFTW_BACKWARD plan and vice versa; similarly for the cannam@127: FFTW_MPI_SCRAMBLED_OUT and FFTW_MPI_SCRAMBLED_IN flags. cannam@127: See One-dimensional distributions. cannam@127:

6.12.4 MPI Data Distribution Functions