Mercurial > hg > sv-dependency-builds

annotate src/fftw-3.3.5/doc/html/Combining-MPI-and-Threads.html @ 42:2cd0e3b3e1fd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Current fftw source
author Chris Cannam
date Tue, 18 Oct 2016 13:40:26 +0100
parents
children
rev   line source
Chris@42 1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
Chris@42 2 <html>
Chris@42 3 <!-- This manual is for FFTW
Chris@42 4 (version 3.3.5, 30 July 2016).
Chris@42 5
Chris@42 6 Copyright (C) 2003 Matteo Frigo.
Chris@42 7
Chris@42 8 Copyright (C) 2003 Massachusetts Institute of Technology.
Chris@42 9
Chris@42 10 Permission is granted to make and distribute verbatim copies of this
Chris@42 11 manual provided the copyright notice and this permission notice are
Chris@42 12 preserved on all copies.
Chris@42 13
Chris@42 14 Permission is granted to copy and distribute modified versions of this
Chris@42 15 manual under the conditions for verbatim copying, provided that the
Chris@42 16 entire resulting derived work is distributed under the terms of a
Chris@42 17 permission notice identical to this one.
Chris@42 18
Chris@42 19 Permission is granted to copy and distribute translations of this manual
Chris@42 20 into another language, under the above conditions for modified versions,
Chris@42 21 except that this permission notice may be stated in a translation
Chris@42 22 approved by the Free Software Foundation. -->
Chris@42 23 <!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
Chris@42 24 <head>
Chris@42 25 <title>FFTW 3.3.5: Combining MPI and Threads</title>
Chris@42 26
Chris@42 27 <meta name="description" content="FFTW 3.3.5: Combining MPI and Threads">
Chris@42 28 <meta name="keywords" content="FFTW 3.3.5: Combining MPI and Threads">
Chris@42 29 <meta name="resource-type" content="document">
Chris@42 30 <meta name="distribution" content="global">
Chris@42 31 <meta name="Generator" content="makeinfo">
Chris@42 32 <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
Chris@42 33 <link href="index.html#Top" rel="start" title="Top">
Chris@42 34 <link href="Concept-Index.html#Concept-Index" rel="index" title="Concept Index">
Chris@42 35 <link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
Chris@42 36 <link href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" rel="up" title="Distributed-memory FFTW with MPI">
Chris@42 37 <link href="FFTW-MPI-Reference.html#FFTW-MPI-Reference" rel="next" title="FFTW MPI Reference">
Chris@42 38 <link href="FFTW-MPI-Performance-Tips.html#FFTW-MPI-Performance-Tips" rel="prev" title="FFTW MPI Performance Tips">
Chris@42 39 <style type="text/css">
Chris@42 40 <!--
Chris@42 41 a.summary-letter {text-decoration: none}
Chris@42 42 blockquote.smallquotation {font-size: smaller}
Chris@42 43 div.display {margin-left: 3.2em}
Chris@42 44 div.example {margin-left: 3.2em}
Chris@42 45 div.indentedblock {margin-left: 3.2em}
Chris@42 46 div.lisp {margin-left: 3.2em}
Chris@42 47 div.smalldisplay {margin-left: 3.2em}
Chris@42 48 div.smallexample {margin-left: 3.2em}
Chris@42 49 div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
Chris@42 50 div.smalllisp {margin-left: 3.2em}
Chris@42 51 kbd {font-style:oblique}
Chris@42 52 pre.display {font-family: inherit}
Chris@42 53 pre.format {font-family: inherit}
Chris@42 54 pre.menu-comment {font-family: serif}
Chris@42 55 pre.menu-preformatted {font-family: serif}
Chris@42 56 pre.smalldisplay {font-family: inherit; font-size: smaller}
Chris@42 57 pre.smallexample {font-size: smaller}
Chris@42 58 pre.smallformat {font-family: inherit; font-size: smaller}
Chris@42 59 pre.smalllisp {font-size: smaller}
Chris@42 60 span.nocodebreak {white-space:nowrap}
Chris@42 61 span.nolinebreak {white-space:nowrap}
Chris@42 62 span.roman {font-family:serif; font-weight:normal}
Chris@42 63 span.sansserif {font-family:sans-serif; font-weight:normal}
Chris@42 64 ul.no-bullet {list-style: none}
Chris@42 65 -->
Chris@42 66 </style>
Chris@42 67
Chris@42 68
Chris@42 69 </head>
Chris@42 70
Chris@42 71 <body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
Chris@42 72 <a name="Combining-MPI-and-Threads"></a>
Chris@42 73 <div class="header">
Chris@42 74 <p>
Chris@42 75 Next: <a href="FFTW-MPI-Reference.html#FFTW-MPI-Reference" accesskey="n" rel="next">FFTW MPI Reference</a>, Previous: <a href="FFTW-MPI-Performance-Tips.html#FFTW-MPI-Performance-Tips" accesskey="p" rel="prev">FFTW MPI Performance Tips</a>, Up: <a href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" accesskey="u" rel="up">Distributed-memory FFTW with MPI</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html#Concept-Index" title="Index" rel="index">Index</a>]</p>
Chris@42 76 </div>
Chris@42 77 <hr>
Chris@42 78 <a name="Combining-MPI-and-Threads-1"></a>
Chris@42 79 <h3 class="section">6.11 Combining MPI and Threads</h3>
Chris@42 80 <a name="index-threads-2"></a>
Chris@42 81
Chris@42 82 <p>In certain cases, it may be advantageous to combine MPI
Chris@42 83 (distributed-memory) and threads (shared-memory) parallelization.
Chris@42 84 FFTW supports this, with certain caveats. For example, if you have a
Chris@42 85 cluster of 4-processor shared-memory nodes, you may want to use
Chris@42 86 threads within the nodes and MPI between the nodes, instead of MPI for
Chris@42 87 all parallelization.
Chris@42 88 </p>
Chris@42 89 <p>In particular, it is possible to seamlessly combine the MPI FFTW
Chris@42 90 routines with the multi-threaded FFTW routines (see <a href="Multi_002dthreaded-FFTW.html#Multi_002dthreaded-FFTW">Multi-threaded FFTW</a>). However, some care must be taken in the initialization code,
Chris@42 91 which should look something like this:
Chris@42 92 </p>
Chris@42 93 <div class="example">
Chris@42 94 <pre class="example">int threads_ok;
Chris@42 95
Chris@42 96 int main(int argc, char **argv)
Chris@42 97 {
Chris@42 98 int provided;
Chris@42 99 MPI_Init_thread(&amp;argc, &amp;argv, MPI_THREAD_FUNNELED, &amp;provided);
Chris@42 100 threads_ok = provided &gt;= MPI_THREAD_FUNNELED;
Chris@42 101
Chris@42 102 if (threads_ok) threads_ok = fftw_init_threads();
Chris@42 103 fftw_mpi_init();
Chris@42 104
Chris@42 105 ...
Chris@42 106 if (threads_ok) fftw_plan_with_nthreads(...);
Chris@42 107 ...
Chris@42 108
Chris@42 109 MPI_Finalize();
Chris@42 110 }
Chris@42 111 </pre></div>
Chris@42 112 <a name="index-fftw_005fmpi_005finit-3"></a>
Chris@42 113 <a name="index-fftw_005finit_005fthreads-2"></a>
Chris@42 114 <a name="index-fftw_005fplan_005fwith_005fnthreads-1"></a>
Chris@42 115
Chris@42 116 <p>First, note that instead of calling <code>MPI_Init</code>, you should call
Chris@42 117 <code>MPI_Init_threads</code>, which is the initialization routine defined
Chris@42 118 by the MPI-2 standard to indicate to MPI that your program will be
Chris@42 119 multithreaded. We pass <code>MPI_THREAD_FUNNELED</code>, which indicates
Chris@42 120 that we will only call MPI routines from the main thread. (FFTW will
Chris@42 121 launch additional threads internally, but the extra threads will not
Chris@42 122 call MPI code.) (You may also pass <code>MPI_THREAD_SERIALIZED</code> or
Chris@42 123 <code>MPI_THREAD_MULTIPLE</code>, which requests additional multithreading
Chris@42 124 support from the MPI implementation, but this is not required by
Chris@42 125 FFTW.) The <code>provided</code> parameter returns what level of threads
Chris@42 126 support is actually supported by your MPI implementation; this
Chris@42 127 <em>must</em> be at least <code>MPI_THREAD_FUNNELED</code> if you want to call
Chris@42 128 the FFTW threads routines, so we define a global variable
Chris@42 129 <code>threads_ok</code> to record this. You should only call
Chris@42 130 <code>fftw_init_threads</code> or <code>fftw_plan_with_nthreads</code> if
Chris@42 131 <code>threads_ok</code> is true. For more information on thread safety in
Chris@42 132 MPI, see the
Chris@42 133 <a href="http://www.mpi-forum.org/docs/mpi-20-html/node162.htm">MPI and
Chris@42 134 Threads</a> section of the MPI-2 standard.
Chris@42 135 <a name="index-thread-safety-2"></a>
Chris@42 136 </p>
Chris@42 137
Chris@42 138 <p>Second, we must call <code>fftw_init_threads</code> <em>before</em>
Chris@42 139 <code>fftw_mpi_init</code>. This is critical for technical reasons having
Chris@42 140 to do with how FFTW initializes its list of algorithms.
Chris@42 141 </p>
Chris@42 142 <p>Then, if you call <code>fftw_plan_with_nthreads(N)</code>, <em>every</em> MPI
Chris@42 143 process will launch (up to) <code>N</code> threads to parallelize its transforms.
Chris@42 144 </p>
Chris@42 145 <p>For example, in the hypothetical cluster of 4-processor nodes, you
Chris@42 146 might wish to launch only a single MPI process per node, and then call
Chris@42 147 <code>fftw_plan_with_nthreads(4)</code> on each process to use all
Chris@42 148 processors in the nodes.
Chris@42 149 </p>
Chris@42 150 <p>This may or may not be faster than simply using as many MPI processes
Chris@42 151 as you have processors, however. On the one hand, using threads
Chris@42 152 within a node eliminates the need for explicit message passing within
Chris@42 153 the node. On the other hand, FFTW&rsquo;s transpose routines are not
Chris@42 154 multi-threaded, and this means that the communications that do take
Chris@42 155 place will not benefit from parallelization within the node.
Chris@42 156 Moreover, many MPI implementations already have optimizations to
Chris@42 157 exploit shared memory when it is available, so adding the
Chris@42 158 multithreaded FFTW on top of this may be superfluous.
Chris@42 159 <a name="index-transpose-4"></a>
Chris@42 160 </p>
Chris@42 161 <hr>
Chris@42 162 <div class="header">
Chris@42 163 <p>
Chris@42 164 Next: <a href="FFTW-MPI-Reference.html#FFTW-MPI-Reference" accesskey="n" rel="next">FFTW MPI Reference</a>, Previous: <a href="FFTW-MPI-Performance-Tips.html#FFTW-MPI-Performance-Tips" accesskey="p" rel="prev">FFTW MPI Performance Tips</a>, Up: <a href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" accesskey="u" rel="up">Distributed-memory FFTW with MPI</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html#Concept-Index" title="Index" rel="index">Index</a>]</p>
Chris@42 165 </div>
Chris@42 166
Chris@42 167
Chris@42 168
Chris@42 169 </body>
Chris@42 170 </html>