cannam@95: cannam@95: cannam@95: Transposed distributions - FFTW 3.3.3 cannam@95: cannam@95: cannam@95: cannam@95: cannam@95: cannam@95: cannam@95: cannam@95: cannam@95: cannam@95: cannam@95: cannam@95: cannam@95: cannam@95:
cannam@95: cannam@95:

cannam@95: Next: , cannam@95: Previous: Load balancing, cannam@95: Up: MPI Data Distribution cannam@95:

cannam@95:
cannam@95: cannam@95:

6.4.3 Transposed distributions

cannam@95: cannam@95:

Internally, FFTW's MPI transform algorithms work by first computing cannam@95: transforms of the data local to each process, then by globally cannam@95: transposing the data in some fashion to redistribute the data cannam@95: among the processes, transforming the new data local to each process, cannam@95: and transposing back. For example, a two-dimensional n0 by cannam@95: n1 array, distributed across the n0 dimension, is cannam@95: transformd by: (i) transforming the n1 dimension, which are cannam@95: local to each process; (ii) transposing to an n1 by n0 cannam@95: array, distributed across the n1 dimension; (iii) transforming cannam@95: the n0 dimension, which is now local to each process; (iv) cannam@95: transposing back. cannam@95: cannam@95: cannam@95:

However, in many applications it is acceptable to compute a cannam@95: multidimensional DFT whose results are produced in transposed order cannam@95: (e.g., n1 by n0 in two dimensions). This provides a cannam@95: significant performance advantage, because it means that the final cannam@95: transposition step can be omitted. FFTW supports this optimization, cannam@95: which you specify by passing the flag FFTW_MPI_TRANSPOSED_OUT cannam@95: to the planner routines. To compute the inverse transform of cannam@95: transposed output, you specify FFTW_MPI_TRANSPOSED_IN to tell cannam@95: it that the input is transposed. In this section, we explain how to cannam@95: interpret the output format of such a transform. cannam@95: cannam@95: cannam@95:

Suppose you have are transforming multi-dimensional data with (at cannam@95: least two) dimensions n0 × n1 × n2 × … × nd-1. As always, it is distributed along cannam@95: the first dimension n0. Now, if we compute its DFT with the cannam@95: FFTW_MPI_TRANSPOSED_OUT flag, the resulting output data are stored cannam@95: with the first two dimensions transposed: n1 × n0 × n2 ×…× nd-1, cannam@95: distributed along the n1 dimension. Conversely, if we take the cannam@95: n1 × n0 × n2 ×…× nd-1 data and transform it with the cannam@95: FFTW_MPI_TRANSPOSED_IN flag, then the format goes back to the cannam@95: original n0 × n1 × n2 × … × nd-1 array. cannam@95: cannam@95:

There are two ways to find the portion of the transposed array that cannam@95: resides on the current process. First, you can simply call the cannam@95: appropriate ‘local_size’ function, passing n1 × n0 × n2 ×…× nd-1 (the cannam@95: transposed dimensions). This would mean calling the ‘local_size’ cannam@95: function twice, once for the transposed and once for the cannam@95: non-transposed dimensions. Alternatively, you can call one of the cannam@95: ‘local_size_transposed’ functions, which returns both the cannam@95: non-transposed and transposed data distribution from a single call. cannam@95: For example, for a 3d transform with transposed output (or input), you cannam@95: might call: cannam@95: cannam@95: 

     ptrdiff_t fftw_mpi_local_size_3d_transposed(
cannam@95:                      ptrdiff_t n0, ptrdiff_t n1, ptrdiff_t n2, MPI_Comm comm,
cannam@95:                      ptrdiff_t *local_n0, ptrdiff_t *local_0_start,
cannam@95:                      ptrdiff_t *local_n1, ptrdiff_t *local_1_start);
cannam@95:
cannam@95:

cannam@95: Here, local_n0 and local_0_start give the size and cannam@95: starting index of the n0 dimension for the cannam@95: non-transposed data, as in the previous sections. For cannam@95: transposed data (e.g. the output for cannam@95: FFTW_MPI_TRANSPOSED_OUT), local_n1 and cannam@95: local_1_start give the size and starting index of the n1 cannam@95: dimension, which is the first dimension of the transposed data cannam@95: (n1 by n0 by n2). cannam@95: cannam@95:

(Note that FFTW_MPI_TRANSPOSED_IN is completely equivalent to cannam@95: performing FFTW_MPI_TRANSPOSED_OUT and passing the first two cannam@95: dimensions to the planner in reverse order, or vice versa. If you cannam@95: pass both the FFTW_MPI_TRANSPOSED_IN and cannam@95: FFTW_MPI_TRANSPOSED_OUT flags, it is equivalent to swapping the cannam@95: first two dimensions passed to the planner and passing neither cannam@95: flag.) cannam@95: cannam@95: cannam@95: