annotate src/fftw-3.3.3/doc/html/Combining-MPI-and-Threads.html @ 83:ae30d91d2ffe

Replace these with versions built using an older toolset (so as to avoid ABI compatibilities when linking on Ubuntu 14.04 for packaging purposes)
author Chris Cannam
date Fri, 07 Feb 2020 11:51:13 +0000
parents 37bf6b4a2645
children
rev   line source
Chris@10 1 <html lang="en">
Chris@10 2 <head>
Chris@10 3 <title>Combining MPI and Threads - FFTW 3.3.3</title>
Chris@10 4 <meta http-equiv="Content-Type" content="text/html">
Chris@10 5 <meta name="description" content="FFTW 3.3.3">
Chris@10 6 <meta name="generator" content="makeinfo 4.13">
Chris@10 7 <link title="Top" rel="start" href="index.html#Top">
Chris@10 8 <link rel="up" href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" title="Distributed-memory FFTW with MPI">
Chris@10 9 <link rel="prev" href="FFTW-MPI-Performance-Tips.html#FFTW-MPI-Performance-Tips" title="FFTW MPI Performance Tips">
Chris@10 10 <link rel="next" href="FFTW-MPI-Reference.html#FFTW-MPI-Reference" title="FFTW MPI Reference">
Chris@10 11 <link href="http://www.gnu.org/software/texinfo/" rel="generator-home" title="Texinfo Homepage">
Chris@10 12 <!--
Chris@10 13 This manual is for FFTW
Chris@10 14 (version 3.3.3, 25 November 2012).
Chris@10 15
Chris@10 16 Copyright (C) 2003 Matteo Frigo.
Chris@10 17
Chris@10 18 Copyright (C) 2003 Massachusetts Institute of Technology.
Chris@10 19
Chris@10 20 Permission is granted to make and distribute verbatim copies of
Chris@10 21 this manual provided the copyright notice and this permission
Chris@10 22 notice are preserved on all copies.
Chris@10 23
Chris@10 24 Permission is granted to copy and distribute modified versions of
Chris@10 25 this manual under the conditions for verbatim copying, provided
Chris@10 26 that the entire resulting derived work is distributed under the
Chris@10 27 terms of a permission notice identical to this one.
Chris@10 28
Chris@10 29 Permission is granted to copy and distribute translations of this
Chris@10 30 manual into another language, under the above conditions for
Chris@10 31 modified versions, except that this permission notice may be
Chris@10 32 stated in a translation approved by the Free Software Foundation.
Chris@10 33 -->
Chris@10 34 <meta http-equiv="Content-Style-Type" content="text/css">
Chris@10 35 <style type="text/css"><!--
Chris@10 36 pre.display { font-family:inherit }
Chris@10 37 pre.format { font-family:inherit }
Chris@10 38 pre.smalldisplay { font-family:inherit; font-size:smaller }
Chris@10 39 pre.smallformat { font-family:inherit; font-size:smaller }
Chris@10 40 pre.smallexample { font-size:smaller }
Chris@10 41 pre.smalllisp { font-size:smaller }
Chris@10 42 span.sc { font-variant:small-caps }
Chris@10 43 span.roman { font-family:serif; font-weight:normal; }
Chris@10 44 span.sansserif { font-family:sans-serif; font-weight:normal; }
Chris@10 45 --></style>
Chris@10 46 </head>
Chris@10 47 <body>
Chris@10 48 <div class="node">
Chris@10 49 <a name="Combining-MPI-and-Threads"></a>
Chris@10 50 <p>
Chris@10 51 Next:&nbsp;<a rel="next" accesskey="n" href="FFTW-MPI-Reference.html#FFTW-MPI-Reference">FFTW MPI Reference</a>,
Chris@10 52 Previous:&nbsp;<a rel="previous" accesskey="p" href="FFTW-MPI-Performance-Tips.html#FFTW-MPI-Performance-Tips">FFTW MPI Performance Tips</a>,
Chris@10 53 Up:&nbsp;<a rel="up" accesskey="u" href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI">Distributed-memory FFTW with MPI</a>
Chris@10 54 <hr>
Chris@10 55 </div>
Chris@10 56
Chris@10 57 <h3 class="section">6.11 Combining MPI and Threads</h3>
Chris@10 58
Chris@10 59 <p><a name="index-threads-427"></a>
Chris@10 60 In certain cases, it may be advantageous to combine MPI
Chris@10 61 (distributed-memory) and threads (shared-memory) parallelization.
Chris@10 62 FFTW supports this, with certain caveats. For example, if you have a
Chris@10 63 cluster of 4-processor shared-memory nodes, you may want to use
Chris@10 64 threads within the nodes and MPI between the nodes, instead of MPI for
Chris@10 65 all parallelization.
Chris@10 66
Chris@10 67 <p>In particular, it is possible to seamlessly combine the MPI FFTW
Chris@10 68 routines with the multi-threaded FFTW routines (see <a href="Multi_002dthreaded-FFTW.html#Multi_002dthreaded-FFTW">Multi-threaded FFTW</a>). However, some care must be taken in the initialization code,
Chris@10 69 which should look something like this:
Chris@10 70
Chris@10 71 <pre class="example"> int threads_ok;
Chris@10 72
Chris@10 73 int main(int argc, char **argv)
Chris@10 74 {
Chris@10 75 int provided;
Chris@10 76 MPI_Init_thread(&amp;argc, &amp;argv, MPI_THREAD_FUNNELED, &amp;provided);
Chris@10 77 threads_ok = provided &gt;= MPI_THREAD_FUNNELED;
Chris@10 78
Chris@10 79 if (threads_ok) threads_ok = fftw_init_threads();
Chris@10 80 fftw_mpi_init();
Chris@10 81
Chris@10 82 ...
Chris@10 83 if (threads_ok) fftw_plan_with_nthreads(...);
Chris@10 84 ...
Chris@10 85
Chris@10 86 MPI_Finalize();
Chris@10 87 }
Chris@10 88 </pre>
Chris@10 89 <p><a name="index-fftw_005fmpi_005finit-428"></a><a name="index-fftw_005finit_005fthreads-429"></a><a name="index-fftw_005fplan_005fwith_005fnthreads-430"></a>
Chris@10 90 First, note that instead of calling <code>MPI_Init</code>, you should call
Chris@10 91 <code>MPI_Init_threads</code>, which is the initialization routine defined
Chris@10 92 by the MPI-2 standard to indicate to MPI that your program will be
Chris@10 93 multithreaded. We pass <code>MPI_THREAD_FUNNELED</code>, which indicates
Chris@10 94 that we will only call MPI routines from the main thread. (FFTW will
Chris@10 95 launch additional threads internally, but the extra threads will not
Chris@10 96 call MPI code.) (You may also pass <code>MPI_THREAD_SERIALIZED</code> or
Chris@10 97 <code>MPI_THREAD_MULTIPLE</code>, which requests additional multithreading
Chris@10 98 support from the MPI implementation, but this is not required by
Chris@10 99 FFTW.) The <code>provided</code> parameter returns what level of threads
Chris@10 100 support is actually supported by your MPI implementation; this
Chris@10 101 <em>must</em> be at least <code>MPI_THREAD_FUNNELED</code> if you want to call
Chris@10 102 the FFTW threads routines, so we define a global variable
Chris@10 103 <code>threads_ok</code> to record this. You should only call
Chris@10 104 <code>fftw_init_threads</code> or <code>fftw_plan_with_nthreads</code> if
Chris@10 105 <code>threads_ok</code> is true. For more information on thread safety in
Chris@10 106 MPI, see the
Chris@10 107 <a href="http://www.mpi-forum.org/docs/mpi-20-html/node162.htm">MPI and Threads</a> section of the MPI-2 standard.
Chris@10 108 <a name="index-thread-safety-431"></a>
Chris@10 109
Chris@10 110 <p>Second, we must call <code>fftw_init_threads</code> <em>before</em>
Chris@10 111 <code>fftw_mpi_init</code>. This is critical for technical reasons having
Chris@10 112 to do with how FFTW initializes its list of algorithms.
Chris@10 113
Chris@10 114 <p>Then, if you call <code>fftw_plan_with_nthreads(N)</code>, <em>every</em> MPI
Chris@10 115 process will launch (up to) <code>N</code> threads to parallelize its transforms.
Chris@10 116
Chris@10 117 <p>For example, in the hypothetical cluster of 4-processor nodes, you
Chris@10 118 might wish to launch only a single MPI process per node, and then call
Chris@10 119 <code>fftw_plan_with_nthreads(4)</code> on each process to use all
Chris@10 120 processors in the nodes.
Chris@10 121
Chris@10 122 <p>This may or may not be faster than simply using as many MPI processes
Chris@10 123 as you have processors, however. On the one hand, using threads
Chris@10 124 within a node eliminates the need for explicit message passing within
Chris@10 125 the node. On the other hand, FFTW's transpose routines are not
Chris@10 126 multi-threaded, and this means that the communications that do take
Chris@10 127 place will not benefit from parallelization within the node.
Chris@10 128 Moreover, many MPI implementations already have optimizations to
Chris@10 129 exploit shared memory when it is available, so adding the
Chris@10 130 multithreaded FFTW on top of this may be superfluous.
Chris@10 131 <a name="index-transpose-432"></a>
Chris@10 132 <!-- -->
Chris@10 133
Chris@10 134 </body></html>
Chris@10 135