| cannam@127 | 1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> | 
| cannam@127 | 2 <html> | 
| cannam@127 | 3 <!-- This manual is for FFTW | 
| cannam@127 | 4 (version 3.3.5, 30 July 2016). | 
| cannam@127 | 5 | 
| cannam@127 | 6 Copyright (C) 2003 Matteo Frigo. | 
| cannam@127 | 7 | 
| cannam@127 | 8 Copyright (C) 2003 Massachusetts Institute of Technology. | 
| cannam@127 | 9 | 
| cannam@127 | 10 Permission is granted to make and distribute verbatim copies of this | 
| cannam@127 | 11 manual provided the copyright notice and this permission notice are | 
| cannam@127 | 12 preserved on all copies. | 
| cannam@127 | 13 | 
| cannam@127 | 14 Permission is granted to copy and distribute modified versions of this | 
| cannam@127 | 15 manual under the conditions for verbatim copying, provided that the | 
| cannam@127 | 16 entire resulting derived work is distributed under the terms of a | 
| cannam@127 | 17 permission notice identical to this one. | 
| cannam@127 | 18 | 
| cannam@127 | 19 Permission is granted to copy and distribute translations of this manual | 
| cannam@127 | 20 into another language, under the above conditions for modified versions, | 
| cannam@127 | 21 except that this permission notice may be stated in a translation | 
| cannam@127 | 22 approved by the Free Software Foundation. --> | 
| cannam@127 | 23 <!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ --> | 
| cannam@127 | 24 <head> | 
| cannam@127 | 25 <title>FFTW 3.3.5: 2d MPI example</title> | 
| cannam@127 | 26 | 
| cannam@127 | 27 <meta name="description" content="FFTW 3.3.5: 2d MPI example"> | 
| cannam@127 | 28 <meta name="keywords" content="FFTW 3.3.5: 2d MPI example"> | 
| cannam@127 | 29 <meta name="resource-type" content="document"> | 
| cannam@127 | 30 <meta name="distribution" content="global"> | 
| cannam@127 | 31 <meta name="Generator" content="makeinfo"> | 
| cannam@127 | 32 <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> | 
| cannam@127 | 33 <link href="index.html#Top" rel="start" title="Top"> | 
| cannam@127 | 34 <link href="Concept-Index.html#Concept-Index" rel="index" title="Concept Index"> | 
| cannam@127 | 35 <link href="index.html#SEC_Contents" rel="contents" title="Table of Contents"> | 
| cannam@127 | 36 <link href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" rel="up" title="Distributed-memory FFTW with MPI"> | 
| cannam@127 | 37 <link href="MPI-Data-Distribution.html#MPI-Data-Distribution" rel="next" title="MPI Data Distribution"> | 
| cannam@127 | 38 <link href="Linking-and-Initializing-MPI-FFTW.html#Linking-and-Initializing-MPI-FFTW" rel="prev" title="Linking and Initializing MPI FFTW"> | 
| cannam@127 | 39 <style type="text/css"> | 
| cannam@127 | 40 <!-- | 
| cannam@127 | 41 a.summary-letter {text-decoration: none} | 
| cannam@127 | 42 blockquote.smallquotation {font-size: smaller} | 
| cannam@127 | 43 div.display {margin-left: 3.2em} | 
| cannam@127 | 44 div.example {margin-left: 3.2em} | 
| cannam@127 | 45 div.indentedblock {margin-left: 3.2em} | 
| cannam@127 | 46 div.lisp {margin-left: 3.2em} | 
| cannam@127 | 47 div.smalldisplay {margin-left: 3.2em} | 
| cannam@127 | 48 div.smallexample {margin-left: 3.2em} | 
| cannam@127 | 49 div.smallindentedblock {margin-left: 3.2em; font-size: smaller} | 
| cannam@127 | 50 div.smalllisp {margin-left: 3.2em} | 
| cannam@127 | 51 kbd {font-style:oblique} | 
| cannam@127 | 52 pre.display {font-family: inherit} | 
| cannam@127 | 53 pre.format {font-family: inherit} | 
| cannam@127 | 54 pre.menu-comment {font-family: serif} | 
| cannam@127 | 55 pre.menu-preformatted {font-family: serif} | 
| cannam@127 | 56 pre.smalldisplay {font-family: inherit; font-size: smaller} | 
| cannam@127 | 57 pre.smallexample {font-size: smaller} | 
| cannam@127 | 58 pre.smallformat {font-family: inherit; font-size: smaller} | 
| cannam@127 | 59 pre.smalllisp {font-size: smaller} | 
| cannam@127 | 60 span.nocodebreak {white-space:nowrap} | 
| cannam@127 | 61 span.nolinebreak {white-space:nowrap} | 
| cannam@127 | 62 span.roman {font-family:serif; font-weight:normal} | 
| cannam@127 | 63 span.sansserif {font-family:sans-serif; font-weight:normal} | 
| cannam@127 | 64 ul.no-bullet {list-style: none} | 
| cannam@127 | 65 --> | 
| cannam@127 | 66 </style> | 
| cannam@127 | 67 | 
| cannam@127 | 68 | 
| cannam@127 | 69 </head> | 
| cannam@127 | 70 | 
| cannam@127 | 71 <body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000"> | 
| cannam@127 | 72 <a name="g_t2d-MPI-example"></a> | 
| cannam@127 | 73 <div class="header"> | 
| cannam@127 | 74 <p> | 
| cannam@127 | 75 Next: <a href="MPI-Data-Distribution.html#MPI-Data-Distribution" accesskey="n" rel="next">MPI Data Distribution</a>, Previous: <a href="Linking-and-Initializing-MPI-FFTW.html#Linking-and-Initializing-MPI-FFTW" accesskey="p" rel="prev">Linking and Initializing MPI FFTW</a>, Up: <a href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" accesskey="u" rel="up">Distributed-memory FFTW with MPI</a>   [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html#Concept-Index" title="Index" rel="index">Index</a>]</p> | 
| cannam@127 | 76 </div> | 
| cannam@127 | 77 <hr> | 
| cannam@127 | 78 <a name="g_t2d-MPI-example-1"></a> | 
| cannam@127 | 79 <h3 class="section">6.3 2d MPI example</h3> | 
| cannam@127 | 80 | 
| cannam@127 | 81 <p>Before we document the FFTW MPI interface in detail, we begin with a | 
| cannam@127 | 82 simple example outlining how one would perform a two-dimensional | 
| cannam@127 | 83 <code>N0</code> by <code>N1</code> complex DFT. | 
| cannam@127 | 84 </p> | 
| cannam@127 | 85 <div class="example"> | 
| cannam@127 | 86 <pre class="example">#include <fftw3-mpi.h> | 
| cannam@127 | 87 | 
| cannam@127 | 88 int main(int argc, char **argv) | 
| cannam@127 | 89 { | 
| cannam@127 | 90     const ptrdiff_t N0 = ..., N1 = ...; | 
| cannam@127 | 91     fftw_plan plan; | 
| cannam@127 | 92     fftw_complex *data; | 
| cannam@127 | 93     ptrdiff_t alloc_local, local_n0, local_0_start, i, j; | 
| cannam@127 | 94 | 
| cannam@127 | 95     MPI_Init(&argc, &argv); | 
| cannam@127 | 96     fftw_mpi_init(); | 
| cannam@127 | 97 | 
| cannam@127 | 98     /* <span class="roman">get local data size and allocate</span> */ | 
| cannam@127 | 99     alloc_local = fftw_mpi_local_size_2d(N0, N1, MPI_COMM_WORLD, | 
| cannam@127 | 100                                          &local_n0, &local_0_start); | 
| cannam@127 | 101     data = fftw_alloc_complex(alloc_local); | 
| cannam@127 | 102 | 
| cannam@127 | 103     /* <span class="roman">create plan for in-place forward DFT</span> */ | 
| cannam@127 | 104     plan = fftw_mpi_plan_dft_2d(N0, N1, data, data, MPI_COMM_WORLD, | 
| cannam@127 | 105                                 FFTW_FORWARD, FFTW_ESTIMATE); | 
| cannam@127 | 106 | 
| cannam@127 | 107     /* <span class="roman">initialize data to some function</span> my_function(x,y) */ | 
| cannam@127 | 108     for (i = 0; i < local_n0; ++i) for (j = 0; j < N1; ++j) | 
| cannam@127 | 109        data[i*N1 + j] = my_function(local_0_start + i, j); | 
| cannam@127 | 110 | 
| cannam@127 | 111     /* <span class="roman">compute transforms, in-place, as many times as desired</span> */ | 
| cannam@127 | 112     fftw_execute(plan); | 
| cannam@127 | 113 | 
| cannam@127 | 114     fftw_destroy_plan(plan); | 
| cannam@127 | 115 | 
| cannam@127 | 116     MPI_Finalize(); | 
| cannam@127 | 117 } | 
| cannam@127 | 118 </pre></div> | 
| cannam@127 | 119 | 
| cannam@127 | 120 <p>As can be seen above, the MPI interface follows the same basic style | 
| cannam@127 | 121 of allocate/plan/execute/destroy as the serial FFTW routines.  All of | 
| cannam@127 | 122 the MPI-specific routines are prefixed with ‘<samp>fftw_mpi_</samp>’ instead | 
| cannam@127 | 123 of ‘<samp>fftw_</samp>’.  There are a few important differences, however: | 
| cannam@127 | 124 </p> | 
| cannam@127 | 125 <p>First, we must call <code>fftw_mpi_init()</code> after calling | 
| cannam@127 | 126 <code>MPI_Init</code> (required in all MPI programs) and before calling any | 
| cannam@127 | 127 other ‘<samp>fftw_mpi_</samp>’ routine. | 
| cannam@127 | 128 <a name="index-MPI_005fInit"></a> | 
| cannam@127 | 129 <a name="index-fftw_005fmpi_005finit-1"></a> | 
| cannam@127 | 130 </p> | 
| cannam@127 | 131 | 
| cannam@127 | 132 <p>Second, when we create the plan with <code>fftw_mpi_plan_dft_2d</code>, | 
| cannam@127 | 133 analogous to <code>fftw_plan_dft_2d</code>, we pass an additional argument: | 
| cannam@127 | 134 the communicator, indicating which processes will participate in the | 
| cannam@127 | 135 transform (here <code>MPI_COMM_WORLD</code>, indicating all processes). | 
| cannam@127 | 136 Whenever you create, execute, or destroy a plan for an MPI transform, | 
| cannam@127 | 137 you must call the corresponding FFTW routine on <em>all</em> processes | 
| cannam@127 | 138 in the communicator for that transform.  (That is, these are | 
| cannam@127 | 139 <em>collective</em> calls.)  Note that the plan for the MPI transform | 
| cannam@127 | 140 uses the standard <code>fftw_execute</code> and <code>fftw_destroy</code> routines | 
| cannam@127 | 141 (on the other hand, there are MPI-specific new-array execute functions | 
| cannam@127 | 142 documented below). | 
| cannam@127 | 143 <a name="index-collective-function"></a> | 
| cannam@127 | 144 <a name="index-fftw_005fmpi_005fplan_005fdft_005f2d"></a> | 
| cannam@127 | 145 <a name="index-MPI_005fCOMM_005fWORLD-1"></a> | 
| cannam@127 | 146 </p> | 
| cannam@127 | 147 | 
| cannam@127 | 148 <p>Third, all of the FFTW MPI routines take <code>ptrdiff_t</code> arguments | 
| cannam@127 | 149 instead of <code>int</code> as for the serial FFTW.  <code>ptrdiff_t</code> is a | 
| cannam@127 | 150 standard C integer type which is (at least) 32 bits wide on a 32-bit | 
| cannam@127 | 151 machine and 64 bits wide on a 64-bit machine.  This is to make it easy | 
| cannam@127 | 152 to specify very large parallel transforms on a 64-bit machine.  (You | 
| cannam@127 | 153 can specify 64-bit transform sizes in the serial FFTW, too, but only | 
| cannam@127 | 154 by using the ‘<samp>guru64</samp>’ planner interface.  See <a href="64_002dbit-Guru-Interface.html#g_t64_002dbit-Guru-Interface">64-bit Guru Interface</a>.) | 
| cannam@127 | 155 <a name="index-ptrdiff_005ft-1"></a> | 
| cannam@127 | 156 <a name="index-64_002dbit-architecture-1"></a> | 
| cannam@127 | 157 </p> | 
| cannam@127 | 158 | 
| cannam@127 | 159 <p>Fourth, and most importantly, you don’t allocate the entire | 
| cannam@127 | 160 two-dimensional array on each process.  Instead, you call | 
| cannam@127 | 161 <code>fftw_mpi_local_size_2d</code> to find out what <em>portion</em> of the | 
| cannam@127 | 162 array resides on each processor, and how much space to allocate. | 
| cannam@127 | 163 Here, the portion of the array on each process is a <code>local_n0</code> by | 
| cannam@127 | 164 <code>N1</code> slice of the total array, starting at index | 
| cannam@127 | 165 <code>local_0_start</code>.  The total number of <code>fftw_complex</code> numbers | 
| cannam@127 | 166 to allocate is given by the <code>alloc_local</code> return value, which | 
| cannam@127 | 167 <em>may</em> be greater than <code>local_n0 * N1</code> (in case some | 
| cannam@127 | 168 intermediate calculations require additional storage).  The data | 
| cannam@127 | 169 distribution in FFTW’s MPI interface is described in more detail by | 
| cannam@127 | 170 the next section. | 
| cannam@127 | 171 <a name="index-fftw_005fmpi_005flocal_005fsize_005f2d"></a> | 
| cannam@127 | 172 <a name="index-data-distribution-1"></a> | 
| cannam@127 | 173 </p> | 
| cannam@127 | 174 | 
| cannam@127 | 175 <p>Given the portion of the array that resides on the local process, it | 
| cannam@127 | 176 is straightforward to initialize the data (here to a function | 
| cannam@127 | 177 <code>myfunction</code>) and otherwise manipulate it.  Of course, at the end | 
| cannam@127 | 178 of the program you may want to output the data somehow, but | 
| cannam@127 | 179 synchronizing this output is up to you and is beyond the scope of this | 
| cannam@127 | 180 manual.  (One good way to output a large multi-dimensional distributed | 
| cannam@127 | 181 array in MPI to a portable binary file is to use the free HDF5 | 
| cannam@127 | 182 library; see the <a href="http://www.hdfgroup.org/">HDF home page</a>.) | 
| cannam@127 | 183 <a name="index-HDF5"></a> | 
| cannam@127 | 184 <a name="index-MPI-I_002fO"></a> | 
| cannam@127 | 185 </p> | 
| cannam@127 | 186 <hr> | 
| cannam@127 | 187 <div class="header"> | 
| cannam@127 | 188 <p> | 
| cannam@127 | 189 Next: <a href="MPI-Data-Distribution.html#MPI-Data-Distribution" accesskey="n" rel="next">MPI Data Distribution</a>, Previous: <a href="Linking-and-Initializing-MPI-FFTW.html#Linking-and-Initializing-MPI-FFTW" accesskey="p" rel="prev">Linking and Initializing MPI FFTW</a>, Up: <a href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" accesskey="u" rel="up">Distributed-memory FFTW with MPI</a>   [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html#Concept-Index" title="Index" rel="index">Index</a>]</p> | 
| cannam@127 | 190 </div> | 
| cannam@127 | 191 | 
| cannam@127 | 192 | 
| cannam@127 | 193 | 
| cannam@127 | 194 </body> | 
| cannam@127 | 195 </html> |