annotate fft/fftw/fftw-3.3.4/doc/html/MPI-Data-Distribution.html @ 40:223f770b5341 kissfft-double tip

Try a double-precision kissfft
author Chris Cannam
date Wed, 07 Sep 2016 10:40:32 +0100
parents 26056e866c29
children
rev   line source
Chris@19 1 <html lang="en">
Chris@19 2 <head>
Chris@19 3 <title>MPI Data Distribution - FFTW 3.3.4</title>
Chris@19 4 <meta http-equiv="Content-Type" content="text/html">
Chris@19 5 <meta name="description" content="FFTW 3.3.4">
Chris@19 6 <meta name="generator" content="makeinfo 4.13">
Chris@19 7 <link title="Top" rel="start" href="index.html#Top">
Chris@19 8 <link rel="up" href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" title="Distributed-memory FFTW with MPI">
Chris@19 9 <link rel="prev" href="2d-MPI-example.html#g_t2d-MPI-example" title="2d MPI example">
Chris@19 10 <link rel="next" href="Multi_002ddimensional-MPI-DFTs-of-Real-Data.html#Multi_002ddimensional-MPI-DFTs-of-Real-Data" title="Multi-dimensional MPI DFTs of Real Data">
Chris@19 11 <link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage">
Chris@19 12 <!--
Chris@19 13 This manual is for FFTW
Chris@19 14 (version 3.3.4, 20 September 2013).
Chris@19 15
Chris@19 16 Copyright (C) 2003 Matteo Frigo.
Chris@19 17
Chris@19 18 Copyright (C) 2003 Massachusetts Institute of Technology.
Chris@19 19
Chris@19 20 Permission is granted to make and distribute verbatim copies of
Chris@19 21 this manual provided the copyright notice and this permission
Chris@19 22 notice are preserved on all copies.
Chris@19 23
Chris@19 24 Permission is granted to copy and distribute modified versions of
Chris@19 25 this manual under the conditions for verbatim copying, provided
Chris@19 26 that the entire resulting derived work is distributed under the
Chris@19 27 terms of a permission notice identical to this one.
Chris@19 28
Chris@19 29 Permission is granted to copy and distribute translations of this
Chris@19 30 manual into another language, under the above conditions for
Chris@19 31 modified versions, except that this permission notice may be
Chris@19 32 stated in a translation approved by the Free Software Foundation.
Chris@19 33 -->
Chris@19 34 <meta http-equiv="Content-Style-Type" content="text/css">
Chris@19 35 <style type="text/css"><!--
Chris@19 36 pre.display { font-family:inherit }
Chris@19 37 pre.format { font-family:inherit }
Chris@19 38 pre.smalldisplay { font-family:inherit; font-size:smaller }
Chris@19 39 pre.smallformat { font-family:inherit; font-size:smaller }
Chris@19 40 pre.smallexample { font-size:smaller }
Chris@19 41 pre.smalllisp { font-size:smaller }
Chris@19 42 span.sc { font-variant:small-caps }
Chris@19 43 span.roman { font-family:serif; font-weight:normal; }
Chris@19 44 span.sansserif { font-family:sans-serif; font-weight:normal; }
Chris@19 45 --></style>
Chris@19 46 </head>
Chris@19 47 <body>
Chris@19 48 <div class="node">
Chris@19 49 <a name="MPI-Data-Distribution"></a>
Chris@19 50 <p>
Chris@19 51 Next:&nbsp;<a rel="next" accesskey="n" href="Multi_002ddimensional-MPI-DFTs-of-Real-Data.html#Multi_002ddimensional-MPI-DFTs-of-Real-Data">Multi-dimensional MPI DFTs of Real Data</a>,
Chris@19 52 Previous:&nbsp;<a rel="previous" accesskey="p" href="2d-MPI-example.html#g_t2d-MPI-example">2d MPI example</a>,
Chris@19 53 Up:&nbsp;<a rel="up" accesskey="u" href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI">Distributed-memory FFTW with MPI</a>
Chris@19 54 <hr>
Chris@19 55 </div>
Chris@19 56
Chris@19 57 <h3 class="section">6.4 MPI Data Distribution</h3>
Chris@19 58
Chris@19 59 <p><a name="index-data-distribution-371"></a>
Chris@19 60 The most important concept to understand in using FFTW's MPI interface
Chris@19 61 is the data distribution. With a serial or multithreaded FFT, all of
Chris@19 62 the inputs and outputs are stored as a single contiguous chunk of
Chris@19 63 memory. With a distributed-memory FFT, the inputs and outputs are
Chris@19 64 broken into disjoint blocks, one per process.
Chris@19 65
Chris@19 66 <p>In particular, FFTW uses a <em>1d block distribution</em> of the data,
Chris@19 67 distributed along the <em>first dimension</em>. For example, if you
Chris@19 68 want to perform a 100&nbsp;&times;&nbsp;200 complex DFT, distributed over 4
Chris@19 69 processes, each process will get a 25&nbsp;&times;&nbsp;200 slice of the data.
Chris@19 70 That is, process 0 will get rows 0 through 24, process 1 will get rows
Chris@19 71 25 through 49, process 2 will get rows 50 through 74, and process 3
Chris@19 72 will get rows 75 through 99. If you take the same array but
Chris@19 73 distribute it over 3 processes, then it is not evenly divisible so the
Chris@19 74 different processes will have unequal chunks. FFTW's default choice
Chris@19 75 in this case is to assign 34 rows to processes 0 and 1, and 32 rows to
Chris@19 76 process 2.
Chris@19 77 <a name="index-block-distribution-372"></a>
Chris@19 78
Chris@19 79 <p>FFTW provides several &lsquo;<samp><span class="samp">fftw_mpi_local_size</span></samp>&rsquo; routines that you can
Chris@19 80 call to find out what portion of an array is stored on the current
Chris@19 81 process. In most cases, you should use the default block sizes picked
Chris@19 82 by FFTW, but it is also possible to specify your own block size. For
Chris@19 83 example, with a 100&nbsp;&times;&nbsp;200 array on three processes, you can
Chris@19 84 tell FFTW to use a block size of 40, which would assign 40 rows to
Chris@19 85 processes 0 and 1, and 20 rows to process 2. FFTW's default is to
Chris@19 86 divide the data equally among the processes if possible, and as best
Chris@19 87 it can otherwise. The rows are always assigned in &ldquo;rank order,&rdquo;
Chris@19 88 i.e. process 0 gets the first block of rows, then process 1, and so
Chris@19 89 on. (You can change this by using <code>MPI_Comm_split</code> to create a
Chris@19 90 new communicator with re-ordered processes.) However, you should
Chris@19 91 always call the &lsquo;<samp><span class="samp">fftw_mpi_local_size</span></samp>&rsquo; routines, if possible,
Chris@19 92 rather than trying to predict FFTW's distribution choices.
Chris@19 93
Chris@19 94 <p>In particular, it is critical that you allocate the storage size that
Chris@19 95 is returned by &lsquo;<samp><span class="samp">fftw_mpi_local_size</span></samp>&rsquo;, which is <em>not</em>
Chris@19 96 necessarily the size of the local slice of the array. The reason is
Chris@19 97 that intermediate steps of FFTW's algorithms involve transposing the
Chris@19 98 array and redistributing the data, so at these intermediate steps FFTW
Chris@19 99 may require more local storage space (albeit always proportional to
Chris@19 100 the total size divided by the number of processes). The
Chris@19 101 &lsquo;<samp><span class="samp">fftw_mpi_local_size</span></samp>&rsquo; functions know how much storage is required
Chris@19 102 for these intermediate steps and tell you the correct amount to
Chris@19 103 allocate.
Chris@19 104
Chris@19 105 <ul class="menu">
Chris@19 106 <li><a accesskey="1" href="Basic-and-advanced-distribution-interfaces.html#Basic-and-advanced-distribution-interfaces">Basic and advanced distribution interfaces</a>
Chris@19 107 <li><a accesskey="2" href="Load-balancing.html#Load-balancing">Load balancing</a>
Chris@19 108 <li><a accesskey="3" href="Transposed-distributions.html#Transposed-distributions">Transposed distributions</a>
Chris@19 109 <li><a accesskey="4" href="One_002ddimensional-distributions.html#One_002ddimensional-distributions">One-dimensional distributions</a>
Chris@19 110 </ul>
Chris@19 111
Chris@19 112 </body></html>
Chris@19 113