FFTW 3.3.8: MPI Data Distribution Functions

As described above (see MPI Data Distribution), in order to cannam@167: allocate your arrays, before creating a plan, you must first cannam@167: call one of the following routines to determine the required cannam@167: allocation size and the portion of the array locally stored on a given cannam@167: process. The MPI_Comm communicator passed here must be cannam@167: equivalent to the communicator used below for plan creation. cannam@167:

The basic interface for multidimensional transforms consists of the cannam@167: functions: cannam@167:

cannam@167:

ptrdiff_t fftw_mpi_local_size_2d(ptrdiff_t n0, ptrdiff_t n1, MPI_Comm comm,
cannam@167:                                  ptrdiff_t *local_n0, ptrdiff_t *local_0_start);
cannam@167: ptrdiff_t fftw_mpi_local_size_3d(ptrdiff_t n0, ptrdiff_t n1, ptrdiff_t n2,
cannam@167:                                  MPI_Comm comm,
cannam@167:                                  ptrdiff_t *local_n0, ptrdiff_t *local_0_start);
cannam@167: ptrdiff_t fftw_mpi_local_size(int rnk, const ptrdiff_t *n, MPI_Comm comm,
cannam@167:                               ptrdiff_t *local_n0, ptrdiff_t *local_0_start);
cannam@167: 
cannam@167: ptrdiff_t fftw_mpi_local_size_2d_transposed(ptrdiff_t n0, ptrdiff_t n1, MPI_Comm comm,
cannam@167:                                             ptrdiff_t *local_n0, ptrdiff_t *local_0_start,
cannam@167:                                             ptrdiff_t *local_n1, ptrdiff_t *local_1_start);
cannam@167: ptrdiff_t fftw_mpi_local_size_3d_transposed(ptrdiff_t n0, ptrdiff_t n1, ptrdiff_t n2,
cannam@167:                                             MPI_Comm comm,
cannam@167:                                             ptrdiff_t *local_n0, ptrdiff_t *local_0_start,
cannam@167:                                             ptrdiff_t *local_n1, ptrdiff_t *local_1_start);
cannam@167: ptrdiff_t fftw_mpi_local_size_transposed(int rnk, const ptrdiff_t *n, MPI_Comm comm,
cannam@167:                                          ptrdiff_t *local_n0, ptrdiff_t *local_0_start,
cannam@167:                                          ptrdiff_t *local_n1, ptrdiff_t *local_1_start);
cannam@167:

These functions return the number of elements to allocate (complex cannam@167: numbers for DFT/r2c/c2r plans, real numbers for r2r plans), whereas cannam@167: the local_n0 and local_0_start return the portion cannam@167: (local_0_start to local_0_start + local_n0 - 1) of the cannam@167: first dimension of an n₀ × n₁ × n₂ × … × n_d-1 cannam@167: array that is stored on the local cannam@167: process. See Basic and advanced distribution interfaces. For cannam@167: FFTW_MPI_TRANSPOSED_OUT plans, the ‘_transposed’ variants cannam@167: are useful in order to also return the local portion of the first cannam@167: dimension in the n₁ × n₀ × n₂ ×…× n_d-1 cannam@167: transposed output. cannam@167: See Transposed distributions. cannam@167: The advanced interface for multidimensional transforms is: cannam@167:

cannam@167:

ptrdiff_t fftw_mpi_local_size_many(int rnk, const ptrdiff_t *n, ptrdiff_t howmany,
cannam@167:                                    ptrdiff_t block0, MPI_Comm comm,
cannam@167:                                    ptrdiff_t *local_n0, ptrdiff_t *local_0_start);
cannam@167: ptrdiff_t fftw_mpi_local_size_many_transposed(int rnk, const ptrdiff_t *n, ptrdiff_t howmany,
cannam@167:                                               ptrdiff_t block0, ptrdiff_t block1, MPI_Comm comm,
cannam@167:                                               ptrdiff_t *local_n0, ptrdiff_t *local_0_start,
cannam@167:                                               ptrdiff_t *local_n1, ptrdiff_t *local_1_start);
cannam@167:

These differ from the basic interface in only two ways. First, they cannam@167: allow you to specify block sizes block0 and block1 (the cannam@167: latter for the transposed output); you can pass cannam@167: FFTW_MPI_DEFAULT_BLOCK to use FFTW’s default block size as in cannam@167: the basic interface. Second, you can pass a howmany parameter, cannam@167: corresponding to the advanced planning interface below: this is for cannam@167: transforms of contiguous howmany-tuples of numbers cannam@167: (howmany = 1 in the basic interface). cannam@167:

The corresponding basic and advanced routines for one-dimensional cannam@167: transforms (currently only complex DFTs) are: cannam@167:

cannam@167:

ptrdiff_t fftw_mpi_local_size_1d(
cannam@167:              ptrdiff_t n0, MPI_Comm comm, int sign, unsigned flags,
cannam@167:              ptrdiff_t *local_ni, ptrdiff_t *local_i_start,
cannam@167:              ptrdiff_t *local_no, ptrdiff_t *local_o_start);
cannam@167: ptrdiff_t fftw_mpi_local_size_many_1d(
cannam@167:              ptrdiff_t n0, ptrdiff_t howmany,
cannam@167:              MPI_Comm comm, int sign, unsigned flags,
cannam@167:              ptrdiff_t *local_ni, ptrdiff_t *local_i_start,
cannam@167:              ptrdiff_t *local_no, ptrdiff_t *local_o_start);
cannam@167:

As above, the return value is the number of elements to allocate cannam@167: (complex numbers, for complex DFTs). The local_ni and cannam@167: local_i_start arguments return the portion cannam@167: (local_i_start to local_i_start + local_ni - 1) of the cannam@167: 1d array that is stored on this process for the transform cannam@167: input, and local_no and local_o_start are the cannam@167: corresponding quantities for the input. The sign cannam@167: (FFTW_FORWARD or FFTW_BACKWARD) and flags must cannam@167: match the arguments passed when creating a plan. Although the inputs cannam@167: and outputs have different data distributions in general, it is cannam@167: guaranteed that the output data distribution of an cannam@167: FFTW_FORWARD plan will match the input data distribution cannam@167: of an FFTW_BACKWARD plan and vice versa; similarly for the cannam@167: FFTW_MPI_SCRAMBLED_OUT and FFTW_MPI_SCRAMBLED_IN flags. cannam@167: See One-dimensional distributions. cannam@167:

6.12.4 MPI Data Distribution Functions