Mercurial > hg > sv-dependency-builds

annotate src/bzip2-1.0.6/manual.html @ 169:223a55898ab9 tip default

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add null config files
author Chris Cannam <cannam@all-day-breakfast.com>
date Mon, 02 Mar 2020 14:03:47 +0000
parents 8a15ff55d9af
children
rev   line source
cannam@89 1 <html>
cannam@89 2 <head>
cannam@89 3 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
cannam@89 4 <title>bzip2 and libbzip2, version 1.0.6</title>
cannam@89 5 <meta name="generator" content="DocBook XSL Stylesheets V1.75.2">
cannam@89 6 <style type="text/css" media="screen">/* Colours:
cannam@89 7 #74240f dark brown h1, h2, h3, h4
cannam@89 8 #336699 medium blue links
cannam@89 9 #339999 turquoise link hover colour
cannam@89 10 #202020 almost black general text
cannam@89 11 #761596 purple md5sum text
cannam@89 12 #626262 dark gray pre border
cannam@89 13 #eeeeee very light gray pre background
cannam@89 14 #f2f2f9 very light blue nav table background
cannam@89 15 #3366cc medium blue nav table border
cannam@89 16 */
cannam@89 17
cannam@89 18 a, a:link, a:visited, a:active { color: #336699; }
cannam@89 19 a:hover { color: #339999; }
cannam@89 20
cannam@89 21 body { font: 80%/126% sans-serif; }
cannam@89 22 h1, h2, h3, h4 { color: #74240f; }
cannam@89 23
cannam@89 24 dt { color: #336699; font-weight: bold }
cannam@89 25 dd {
cannam@89 26 margin-left: 1.5em;
cannam@89 27 padding-bottom: 0.8em;
cannam@89 28 }
cannam@89 29
cannam@89 30 /* -- ruler -- */
cannam@89 31 div.hr_blue {
cannam@89 32 height: 3px;
cannam@89 33 background:#ffffff url("/images/hr_blue.png") repeat-x; }
cannam@89 34 div.hr_blue hr { display:none; }
cannam@89 35
cannam@89 36 /* release styles */
cannam@89 37 #release p { margin-top: 0.4em; }
cannam@89 38 #release .md5sum { color: #761596; }
cannam@89 39
cannam@89 40
cannam@89 41 /* ------ styles for docs|manuals|howto ------ */
cannam@89 42 /* -- lists -- */
cannam@89 43 ul {
cannam@89 44 margin: 0px 4px 16px 16px;
cannam@89 45 padding: 0px;
cannam@89 46 list-style: url("/images/li-blue.png");
cannam@89 47 }
cannam@89 48 ul li {
cannam@89 49 margin-bottom: 10px;
cannam@89 50 }
cannam@89 51 ul ul {
cannam@89 52 list-style-type: none;
cannam@89 53 list-style-image: none;
cannam@89 54 margin-left: 0px;
cannam@89 55 }
cannam@89 56
cannam@89 57 /* header / footer nav tables */
cannam@89 58 table.nav {
cannam@89 59 border: solid 1px #3366cc;
cannam@89 60 background: #f2f2f9;
cannam@89 61 background-color: #f2f2f9;
cannam@89 62 margin-bottom: 0.5em;
cannam@89 63 }
cannam@89 64 /* don't have underlined links in chunked nav menus */
cannam@89 65 table.nav a { text-decoration: none; }
cannam@89 66 table.nav a:hover { text-decoration: underline; }
cannam@89 67 table.nav td { font-size: 85%; }
cannam@89 68
cannam@89 69 code, tt, pre { font-size: 120%; }
cannam@89 70 code, tt { color: #761596; }
cannam@89 71
cannam@89 72 div.literallayout, pre.programlisting, pre.screen {
cannam@89 73 color: #000000;
cannam@89 74 padding: 0.5em;
cannam@89 75 background: #eeeeee;
cannam@89 76 border: 1px solid #626262;
cannam@89 77 background-color: #eeeeee;
cannam@89 78 margin: 4px 0px 4px 0px;
cannam@89 79 }
cannam@89 80 </style>
cannam@89 81 </head>
cannam@89 82 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div lang="en" class="book" title="bzip2 and libbzip2, version 1.0.6">
cannam@89 83 <div class="titlepage">
cannam@89 84 <div>
cannam@89 85 <div><h1 class="title">
cannam@89 86 <a name="userman"></a>bzip2 and libbzip2, version 1.0.6</h1></div>
cannam@89 87 <div><h2 class="subtitle">A program and library for data compression</h2></div>
cannam@89 88 <div><div class="authorgroup"><div class="author">
cannam@89 89 <h3 class="author">
cannam@89 90 <span class="firstname">Julian</span> <span class="surname">Seward</span>
cannam@89 91 </h3>
cannam@89 92 <div class="affiliation"><span class="orgname">http://www.bzip.org<br></span></div>
cannam@89 93 </div></div></div>
cannam@89 94 <div><p class="releaseinfo">Version 1.0.6 of 6 September 2010</p></div>
cannam@89 95 <div><p class="copyright">Copyright © 1996-2010 Julian Seward</p></div>
cannam@89 96 <div><div class="legalnotice" title="Legal Notice">
cannam@89 97 <a name="id537185"></a><p>This program, <code class="computeroutput">bzip2</code>, the
cannam@89 98 associated library <code class="computeroutput">libbzip2</code>, and
cannam@89 99 all documentation, are copyright © 1996-2010 Julian Seward.
cannam@89 100 All rights reserved.</p>
cannam@89 101 <p>Redistribution and use in source and binary forms, with
cannam@89 102 or without modification, are permitted provided that the
cannam@89 103 following conditions are met:</p>
cannam@89 104 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 105 <li class="listitem" style="list-style-type: disc"><p>Redistributions of source code must retain the
cannam@89 106 above copyright notice, this list of conditions and the
cannam@89 107 following disclaimer.</p></li>
cannam@89 108 <li class="listitem" style="list-style-type: disc"><p>The origin of this software must not be
cannam@89 109 misrepresented; you must not claim that you wrote the original
cannam@89 110 software. If you use this software in a product, an
cannam@89 111 acknowledgment in the product documentation would be
cannam@89 112 appreciated but is not required.</p></li>
cannam@89 113 <li class="listitem" style="list-style-type: disc"><p>Altered source versions must be plainly marked
cannam@89 114 as such, and must not be misrepresented as being the original
cannam@89 115 software.</p></li>
cannam@89 116 <li class="listitem" style="list-style-type: disc"><p>The name of the author may not be used to
cannam@89 117 endorse or promote products derived from this software without
cannam@89 118 specific prior written permission.</p></li>
cannam@89 119 </ul></div>
cannam@89 120 <p>THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY
cannam@89 121 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
cannam@89 122 THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
cannam@89 123 PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
cannam@89 124 AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
cannam@89 125 EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
cannam@89 126 TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
cannam@89 127 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
cannam@89 128 ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
cannam@89 129 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
cannam@89 130 IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
cannam@89 131 THE POSSIBILITY OF SUCH DAMAGE.</p>
cannam@89 132 <p>PATENTS: To the best of my knowledge,
cannam@89 133 <code class="computeroutput">bzip2</code> and
cannam@89 134 <code class="computeroutput">libbzip2</code> do not use any patented
cannam@89 135 algorithms. However, I do not have the resources to carry
cannam@89 136 out a patent search. Therefore I cannot give any guarantee of
cannam@89 137 the above statement.
cannam@89 138 </p>
cannam@89 139 </div></div>
cannam@89 140 </div>
cannam@89 141 <hr>
cannam@89 142 </div>
cannam@89 143 <div class="toc">
cannam@89 144 <p><b>Table of Contents</b></p>
cannam@89 145 <dl>
cannam@89 146 <dt><span class="chapter"><a href="#intro">1. Introduction</a></span></dt>
cannam@89 147 <dt><span class="chapter"><a href="#using">2. How to use bzip2</a></span></dt>
cannam@89 148 <dd><dl>
cannam@89 149 <dt><span class="sect1"><a href="#name">2.1. NAME</a></span></dt>
cannam@89 150 <dt><span class="sect1"><a href="#synopsis">2.2. SYNOPSIS</a></span></dt>
cannam@89 151 <dt><span class="sect1"><a href="#description">2.3. DESCRIPTION</a></span></dt>
cannam@89 152 <dt><span class="sect1"><a href="#options">2.4. OPTIONS</a></span></dt>
cannam@89 153 <dt><span class="sect1"><a href="#memory-management">2.5. MEMORY MANAGEMENT</a></span></dt>
cannam@89 154 <dt><span class="sect1"><a href="#recovering">2.6. RECOVERING DATA FROM DAMAGED FILES</a></span></dt>
cannam@89 155 <dt><span class="sect1"><a href="#performance">2.7. PERFORMANCE NOTES</a></span></dt>
cannam@89 156 <dt><span class="sect1"><a href="#caveats">2.8. CAVEATS</a></span></dt>
cannam@89 157 <dt><span class="sect1"><a href="#author">2.9. AUTHOR</a></span></dt>
cannam@89 158 </dl></dd>
cannam@89 159 <dt><span class="chapter"><a href="#libprog">3.
cannam@89 160 Programming with <code class="computeroutput">libbzip2</code>
cannam@89 161 </a></span></dt>
cannam@89 162 <dd><dl>
cannam@89 163 <dt><span class="sect1"><a href="#top-level">3.1. Top-level structure</a></span></dt>
cannam@89 164 <dd><dl>
cannam@89 165 <dt><span class="sect2"><a href="#ll-summary">3.1.1. Low-level summary</a></span></dt>
cannam@89 166 <dt><span class="sect2"><a href="#hl-summary">3.1.2. High-level summary</a></span></dt>
cannam@89 167 <dt><span class="sect2"><a href="#util-fns-summary">3.1.3. Utility functions summary</a></span></dt>
cannam@89 168 </dl></dd>
cannam@89 169 <dt><span class="sect1"><a href="#err-handling">3.2. Error handling</a></span></dt>
cannam@89 170 <dt><span class="sect1"><a href="#low-level">3.3. Low-level interface</a></span></dt>
cannam@89 171 <dd><dl>
cannam@89 172 <dt><span class="sect2"><a href="#bzcompress-init">3.3.1. BZ2_bzCompressInit</a></span></dt>
cannam@89 173 <dt><span class="sect2"><a href="#bzCompress">3.3.2. BZ2_bzCompress</a></span></dt>
cannam@89 174 <dt><span class="sect2"><a href="#bzCompress-end">3.3.3. BZ2_bzCompressEnd</a></span></dt>
cannam@89 175 <dt><span class="sect2"><a href="#bzDecompress-init">3.3.4. BZ2_bzDecompressInit</a></span></dt>
cannam@89 176 <dt><span class="sect2"><a href="#bzDecompress">3.3.5. BZ2_bzDecompress</a></span></dt>
cannam@89 177 <dt><span class="sect2"><a href="#bzDecompress-end">3.3.6. BZ2_bzDecompressEnd</a></span></dt>
cannam@89 178 </dl></dd>
cannam@89 179 <dt><span class="sect1"><a href="#hl-interface">3.4. High-level interface</a></span></dt>
cannam@89 180 <dd><dl>
cannam@89 181 <dt><span class="sect2"><a href="#bzreadopen">3.4.1. BZ2_bzReadOpen</a></span></dt>
cannam@89 182 <dt><span class="sect2"><a href="#bzread">3.4.2. BZ2_bzRead</a></span></dt>
cannam@89 183 <dt><span class="sect2"><a href="#bzreadgetunused">3.4.3. BZ2_bzReadGetUnused</a></span></dt>
cannam@89 184 <dt><span class="sect2"><a href="#bzreadclose">3.4.4. BZ2_bzReadClose</a></span></dt>
cannam@89 185 <dt><span class="sect2"><a href="#bzwriteopen">3.4.5. BZ2_bzWriteOpen</a></span></dt>
cannam@89 186 <dt><span class="sect2"><a href="#bzwrite">3.4.6. BZ2_bzWrite</a></span></dt>
cannam@89 187 <dt><span class="sect2"><a href="#bzwriteclose">3.4.7. BZ2_bzWriteClose</a></span></dt>
cannam@89 188 <dt><span class="sect2"><a href="#embed">3.4.8. Handling embedded compressed data streams</a></span></dt>
cannam@89 189 <dt><span class="sect2"><a href="#std-rdwr">3.4.9. Standard file-reading/writing code</a></span></dt>
cannam@89 190 </dl></dd>
cannam@89 191 <dt><span class="sect1"><a href="#util-fns">3.5. Utility functions</a></span></dt>
cannam@89 192 <dd><dl>
cannam@89 193 <dt><span class="sect2"><a href="#bzbufftobuffcompress">3.5.1. BZ2_bzBuffToBuffCompress</a></span></dt>
cannam@89 194 <dt><span class="sect2"><a href="#bzbufftobuffdecompress">3.5.2. BZ2_bzBuffToBuffDecompress</a></span></dt>
cannam@89 195 </dl></dd>
cannam@89 196 <dt><span class="sect1"><a href="#zlib-compat">3.6. zlib compatibility functions</a></span></dt>
cannam@89 197 <dt><span class="sect1"><a href="#stdio-free">3.7. Using the library in a stdio-free environment</a></span></dt>
cannam@89 198 <dd><dl>
cannam@89 199 <dt><span class="sect2"><a href="#stdio-bye">3.7.1. Getting rid of stdio</a></span></dt>
cannam@89 200 <dt><span class="sect2"><a href="#critical-error">3.7.2. Critical error handling</a></span></dt>
cannam@89 201 </dl></dd>
cannam@89 202 <dt><span class="sect1"><a href="#win-dll">3.8. Making a Windows DLL</a></span></dt>
cannam@89 203 </dl></dd>
cannam@89 204 <dt><span class="chapter"><a href="#misc">4. Miscellanea</a></span></dt>
cannam@89 205 <dd><dl>
cannam@89 206 <dt><span class="sect1"><a href="#limits">4.1. Limitations of the compressed file format</a></span></dt>
cannam@89 207 <dt><span class="sect1"><a href="#port-issues">4.2. Portability issues</a></span></dt>
cannam@89 208 <dt><span class="sect1"><a href="#bugs">4.3. Reporting bugs</a></span></dt>
cannam@89 209 <dt><span class="sect1"><a href="#package">4.4. Did you get the right package?</a></span></dt>
cannam@89 210 <dt><span class="sect1"><a href="#reading">4.5. Further Reading</a></span></dt>
cannam@89 211 </dl></dd>
cannam@89 212 </dl>
cannam@89 213 </div>
cannam@89 214 <div class="chapter" title="1. Introduction">
cannam@89 215 <div class="titlepage"><div><div><h2 class="title">
cannam@89 216 <a name="intro"></a>1. Introduction</h2></div></div></div>
cannam@89 217 <p><code class="computeroutput">bzip2</code> compresses files
cannam@89 218 using the Burrows-Wheeler block-sorting text compression
cannam@89 219 algorithm, and Huffman coding. Compression is generally
cannam@89 220 considerably better than that achieved by more conventional
cannam@89 221 LZ77/LZ78-based compressors, and approaches the performance of
cannam@89 222 the PPM family of statistical compressors.</p>
cannam@89 223 <p><code class="computeroutput">bzip2</code> is built on top of
cannam@89 224 <code class="computeroutput">libbzip2</code>, a flexible library for
cannam@89 225 handling compressed data in the
cannam@89 226 <code class="computeroutput">bzip2</code> format. This manual
cannam@89 227 describes both how to use the program and how to work with the
cannam@89 228 library interface. Most of the manual is devoted to this
cannam@89 229 library, not the program, which is good news if your interest is
cannam@89 230 only in the program.</p>
cannam@89 231 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 232 <li class="listitem" style="list-style-type: disc"><p><a class="xref" href="#using" title="2. How to use bzip2">How to use bzip2</a> describes how to use
cannam@89 233 <code class="computeroutput">bzip2</code>; this is the only part
cannam@89 234 you need to read if you just want to know how to operate the
cannam@89 235 program.</p></li>
cannam@89 236 <li class="listitem" style="list-style-type: disc"><p><a class="xref" href="#libprog" title="3.  Programming with libbzip2">Programming with libbzip2</a> describes the
cannam@89 237 programming interfaces in detail, and</p></li>
cannam@89 238 <li class="listitem" style="list-style-type: disc"><p><a class="xref" href="#misc" title="4. Miscellanea">Miscellanea</a> records some
cannam@89 239 miscellaneous notes which I thought ought to be recorded
cannam@89 240 somewhere.</p></li>
cannam@89 241 </ul></div>
cannam@89 242 </div>
cannam@89 243 <div class="chapter" title="2. How to use bzip2">
cannam@89 244 <div class="titlepage"><div><div><h2 class="title">
cannam@89 245 <a name="using"></a>2. How to use bzip2</h2></div></div></div>
cannam@89 246 <div class="toc">
cannam@89 247 <p><b>Table of Contents</b></p>
cannam@89 248 <dl>
cannam@89 249 <dt><span class="sect1"><a href="#name">2.1. NAME</a></span></dt>
cannam@89 250 <dt><span class="sect1"><a href="#synopsis">2.2. SYNOPSIS</a></span></dt>
cannam@89 251 <dt><span class="sect1"><a href="#description">2.3. DESCRIPTION</a></span></dt>
cannam@89 252 <dt><span class="sect1"><a href="#options">2.4. OPTIONS</a></span></dt>
cannam@89 253 <dt><span class="sect1"><a href="#memory-management">2.5. MEMORY MANAGEMENT</a></span></dt>
cannam@89 254 <dt><span class="sect1"><a href="#recovering">2.6. RECOVERING DATA FROM DAMAGED FILES</a></span></dt>
cannam@89 255 <dt><span class="sect1"><a href="#performance">2.7. PERFORMANCE NOTES</a></span></dt>
cannam@89 256 <dt><span class="sect1"><a href="#caveats">2.8. CAVEATS</a></span></dt>
cannam@89 257 <dt><span class="sect1"><a href="#author">2.9. AUTHOR</a></span></dt>
cannam@89 258 </dl>
cannam@89 259 </div>
cannam@89 260 <p>This chapter contains a copy of the
cannam@89 261 <code class="computeroutput">bzip2</code> man page, and nothing
cannam@89 262 else.</p>
cannam@89 263 <div class="sect1" title="2.1. NAME">
cannam@89 264 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 265 <a name="name"></a>2.1. NAME</h2></div></div></div>
cannam@89 266 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 267 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2</code>,
cannam@89 268 <code class="computeroutput">bunzip2</code> - a block-sorting file
cannam@89 269 compressor, v1.0.6</p></li>
cannam@89 270 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzcat</code> -
cannam@89 271 decompresses files to stdout</p></li>
cannam@89 272 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2recover</code> -
cannam@89 273 recovers data from damaged bzip2 files</p></li>
cannam@89 274 </ul></div>
cannam@89 275 </div>
cannam@89 276 <div class="sect1" title="2.2. SYNOPSIS">
cannam@89 277 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 278 <a name="synopsis"></a>2.2. SYNOPSIS</h2></div></div></div>
cannam@89 279 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 280 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2</code> [
cannam@89 281 -cdfkqstvzVL123456789 ] [ filenames ... ]</p></li>
cannam@89 282 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bunzip2</code> [
cannam@89 283 -fkvsVL ] [ filenames ... ]</p></li>
cannam@89 284 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzcat</code> [ -s ] [
cannam@89 285 filenames ... ]</p></li>
cannam@89 286 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2recover</code>
cannam@89 287 filename</p></li>
cannam@89 288 </ul></div>
cannam@89 289 </div>
cannam@89 290 <div class="sect1" title="2.3. DESCRIPTION">
cannam@89 291 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 292 <a name="description"></a>2.3. DESCRIPTION</h2></div></div></div>
cannam@89 293 <p><code class="computeroutput">bzip2</code> compresses files
cannam@89 294 using the Burrows-Wheeler block sorting text compression
cannam@89 295 algorithm, and Huffman coding. Compression is generally
cannam@89 296 considerably better than that achieved by more conventional
cannam@89 297 LZ77/LZ78-based compressors, and approaches the performance of
cannam@89 298 the PPM family of statistical compressors.</p>
cannam@89 299 <p>The command-line options are deliberately very similar to
cannam@89 300 those of GNU <code class="computeroutput">gzip</code>, but they are
cannam@89 301 not identical.</p>
cannam@89 302 <p><code class="computeroutput">bzip2</code> expects a list of
cannam@89 303 file names to accompany the command-line flags. Each file is
cannam@89 304 replaced by a compressed version of itself, with the name
cannam@89 305 <code class="computeroutput">original_name.bz2</code>. Each
cannam@89 306 compressed file has the same modification date, permissions, and,
cannam@89 307 when possible, ownership as the corresponding original, so that
cannam@89 308 these properties can be correctly restored at decompression time.
cannam@89 309 File name handling is naive in the sense that there is no
cannam@89 310 mechanism for preserving original file names, permissions,
cannam@89 311 ownerships or dates in filesystems which lack these concepts, or
cannam@89 312 have serious file name length restrictions, such as
cannam@89 313 MS-DOS.</p>
cannam@89 314 <p><code class="computeroutput">bzip2</code> and
cannam@89 315 <code class="computeroutput">bunzip2</code> will by default not
cannam@89 316 overwrite existing files. If you want this to happen, specify
cannam@89 317 the <code class="computeroutput">-f</code> flag.</p>
cannam@89 318 <p>If no file names are specified,
cannam@89 319 <code class="computeroutput">bzip2</code> compresses from standard
cannam@89 320 input to standard output. In this case,
cannam@89 321 <code class="computeroutput">bzip2</code> will decline to write
cannam@89 322 compressed output to a terminal, as this would be entirely
cannam@89 323 incomprehensible and therefore pointless.</p>
cannam@89 324 <p><code class="computeroutput">bunzip2</code> (or
cannam@89 325 <code class="computeroutput">bzip2 -d</code>) decompresses all
cannam@89 326 specified files. Files which were not created by
cannam@89 327 <code class="computeroutput">bzip2</code> will be detected and
cannam@89 328 ignored, and a warning issued.
cannam@89 329 <code class="computeroutput">bzip2</code> attempts to guess the
cannam@89 330 filename for the decompressed file from that of the compressed
cannam@89 331 file as follows:</p>
cannam@89 332 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 333 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.bz2 </code>
cannam@89 334 becomes
cannam@89 335 <code class="computeroutput">filename</code></p></li>
cannam@89 336 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.bz </code>
cannam@89 337 becomes
cannam@89 338 <code class="computeroutput">filename</code></p></li>
cannam@89 339 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.tbz2</code>
cannam@89 340 becomes
cannam@89 341 <code class="computeroutput">filename.tar</code></p></li>
cannam@89 342 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.tbz </code>
cannam@89 343 becomes
cannam@89 344 <code class="computeroutput">filename.tar</code></p></li>
cannam@89 345 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">anyothername </code>
cannam@89 346 becomes
cannam@89 347 <code class="computeroutput">anyothername.out</code></p></li>
cannam@89 348 </ul></div>
cannam@89 349 <p>If the file does not end in one of the recognised endings,
cannam@89 350 <code class="computeroutput">.bz2</code>,
cannam@89 351 <code class="computeroutput">.bz</code>,
cannam@89 352 <code class="computeroutput">.tbz2</code> or
cannam@89 353 <code class="computeroutput">.tbz</code>,
cannam@89 354 <code class="computeroutput">bzip2</code> complains that it cannot
cannam@89 355 guess the name of the original file, and uses the original name
cannam@89 356 with <code class="computeroutput">.out</code> appended.</p>
cannam@89 357 <p>As with compression, supplying no filenames causes
cannam@89 358 decompression from standard input to standard output.</p>
cannam@89 359 <p><code class="computeroutput">bunzip2</code> will correctly
cannam@89 360 decompress a file which is the concatenation of two or more
cannam@89 361 compressed files. The result is the concatenation of the
cannam@89 362 corresponding uncompressed files. Integrity testing
cannam@89 363 (<code class="computeroutput">-t</code>) of concatenated compressed
cannam@89 364 files is also supported.</p>
cannam@89 365 <p>You can also compress or decompress files to the standard
cannam@89 366 output by giving the <code class="computeroutput">-c</code> flag.
cannam@89 367 Multiple files may be compressed and decompressed like this. The
cannam@89 368 resulting outputs are fed sequentially to stdout. Compression of
cannam@89 369 multiple files in this manner generates a stream containing
cannam@89 370 multiple compressed file representations. Such a stream can be
cannam@89 371 decompressed correctly only by
cannam@89 372 <code class="computeroutput">bzip2</code> version 0.9.0 or later.
cannam@89 373 Earlier versions of <code class="computeroutput">bzip2</code> will
cannam@89 374 stop after decompressing the first file in the stream.</p>
cannam@89 375 <p><code class="computeroutput">bzcat</code> (or
cannam@89 376 <code class="computeroutput">bzip2 -dc</code>) decompresses all
cannam@89 377 specified files to the standard output.</p>
cannam@89 378 <p><code class="computeroutput">bzip2</code> will read arguments
cannam@89 379 from the environment variables
cannam@89 380 <code class="computeroutput">BZIP2</code> and
cannam@89 381 <code class="computeroutput">BZIP</code>, in that order, and will
cannam@89 382 process them before any arguments read from the command line.
cannam@89 383 This gives a convenient way to supply default arguments.</p>
cannam@89 384 <p>Compression is always performed, even if the compressed
cannam@89 385 file is slightly larger than the original. Files of less than
cannam@89 386 about one hundred bytes tend to get larger, since the compression
cannam@89 387 mechanism has a constant overhead in the region of 50 bytes.
cannam@89 388 Random data (including the output of most file compressors) is
cannam@89 389 coded at about 8.05 bits per byte, giving an expansion of around
cannam@89 390 0.5%.</p>
cannam@89 391 <p>As a self-check for your protection,
cannam@89 392 <code class="computeroutput">bzip2</code> uses 32-bit CRCs to make
cannam@89 393 sure that the decompressed version of a file is identical to the
cannam@89 394 original. This guards against corruption of the compressed data,
cannam@89 395 and against undetected bugs in
cannam@89 396 <code class="computeroutput">bzip2</code> (hopefully very unlikely).
cannam@89 397 The chances of data corruption going undetected is microscopic,
cannam@89 398 about one chance in four billion for each file processed. Be
cannam@89 399 aware, though, that the check occurs upon decompression, so it
cannam@89 400 can only tell you that something is wrong. It can't help you
cannam@89 401 recover the original uncompressed data. You can use
cannam@89 402 <code class="computeroutput">bzip2recover</code> to try to recover
cannam@89 403 data from damaged files.</p>
cannam@89 404 <p>Return values: 0 for a normal exit, 1 for environmental
cannam@89 405 problems (file not found, invalid flags, I/O errors, etc.), 2
cannam@89 406 to indicate a corrupt compressed file, 3 for an internal
cannam@89 407 consistency error (eg, bug) which caused
cannam@89 408 <code class="computeroutput">bzip2</code> to panic.</p>
cannam@89 409 </div>
cannam@89 410 <div class="sect1" title="2.4. OPTIONS">
cannam@89 411 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 412 <a name="options"></a>2.4. OPTIONS</h2></div></div></div>
cannam@89 413 <div class="variablelist"><dl>
cannam@89 414 <dt><span class="term"><code class="computeroutput">-c --stdout</code></span></dt>
cannam@89 415 <dd><p>Compress or decompress to standard
cannam@89 416 output.</p></dd>
cannam@89 417 <dt><span class="term"><code class="computeroutput">-d --decompress</code></span></dt>
cannam@89 418 <dd><p>Force decompression.
cannam@89 419 <code class="computeroutput">bzip2</code>,
cannam@89 420 <code class="computeroutput">bunzip2</code> and
cannam@89 421 <code class="computeroutput">bzcat</code> are really the same
cannam@89 422 program, and the decision about what actions to take is done on
cannam@89 423 the basis of which name is used. This flag overrides that
cannam@89 424 mechanism, and forces bzip2 to decompress.</p></dd>
cannam@89 425 <dt><span class="term"><code class="computeroutput">-z --compress</code></span></dt>
cannam@89 426 <dd><p>The complement to
cannam@89 427 <code class="computeroutput">-d</code>: forces compression,
cannam@89 428 regardless of the invokation name.</p></dd>
cannam@89 429 <dt><span class="term"><code class="computeroutput">-t --test</code></span></dt>
cannam@89 430 <dd><p>Check integrity of the specified file(s), but
cannam@89 431 don't decompress them. This really performs a trial
cannam@89 432 decompression and throws away the result.</p></dd>
cannam@89 433 <dt><span class="term"><code class="computeroutput">-f --force</code></span></dt>
cannam@89 434 <dd>
cannam@89 435 <p>Force overwrite of output files. Normally,
cannam@89 436 <code class="computeroutput">bzip2</code> will not overwrite
cannam@89 437 existing output files. Also forces
cannam@89 438 <code class="computeroutput">bzip2</code> to break hard links to
cannam@89 439 files, which it otherwise wouldn't do.</p>
cannam@89 440 <p><code class="computeroutput">bzip2</code> normally declines
cannam@89 441 to decompress files which don't have the correct magic header
cannam@89 442 bytes. If forced (<code class="computeroutput">-f</code>),
cannam@89 443 however, it will pass such files through unmodified. This is
cannam@89 444 how GNU <code class="computeroutput">gzip</code> behaves.</p>
cannam@89 445 </dd>
cannam@89 446 <dt><span class="term"><code class="computeroutput">-k --keep</code></span></dt>
cannam@89 447 <dd><p>Keep (don't delete) input files during
cannam@89 448 compression or decompression.</p></dd>
cannam@89 449 <dt><span class="term"><code class="computeroutput">-s --small</code></span></dt>
cannam@89 450 <dd>
cannam@89 451 <p>Reduce memory usage, for compression,
cannam@89 452 decompression and testing. Files are decompressed and tested
cannam@89 453 using a modified algorithm which only requires 2.5 bytes per
cannam@89 454 block byte. This means any file can be decompressed in 2300k
cannam@89 455 of memory, albeit at about half the normal speed.</p>
cannam@89 456 <p>During compression, <code class="computeroutput">-s</code>
cannam@89 457 selects a block size of 200k, which limits memory use to around
cannam@89 458 the same figure, at the expense of your compression ratio. In
cannam@89 459 short, if your machine is low on memory (8 megabytes or less),
cannam@89 460 use <code class="computeroutput">-s</code> for everything. See
cannam@89 461 <a class="xref" href="#memory-management" title="2.5. MEMORY MANAGEMENT">MEMORY MANAGEMENT</a> below.</p>
cannam@89 462 </dd>
cannam@89 463 <dt><span class="term"><code class="computeroutput">-q --quiet</code></span></dt>
cannam@89 464 <dd><p>Suppress non-essential warning messages.
cannam@89 465 Messages pertaining to I/O errors and other critical events
cannam@89 466 will not be suppressed.</p></dd>
cannam@89 467 <dt><span class="term"><code class="computeroutput">-v --verbose</code></span></dt>
cannam@89 468 <dd><p>Verbose mode -- show the compression ratio for
cannam@89 469 each file processed. Further
cannam@89 470 <code class="computeroutput">-v</code>'s increase the verbosity
cannam@89 471 level, spewing out lots of information which is primarily of
cannam@89 472 interest for diagnostic purposes.</p></dd>
cannam@89 473 <dt><span class="term"><code class="computeroutput">-L --license -V --version</code></span></dt>
cannam@89 474 <dd><p>Display the software version, license terms and
cannam@89 475 conditions.</p></dd>
cannam@89 476 <dt><span class="term"><code class="computeroutput">-1</code> (or
cannam@89 477 <code class="computeroutput">--fast</code>) to
cannam@89 478 <code class="computeroutput">-9</code> (or
cannam@89 479 <code class="computeroutput">-best</code>)</span></dt>
cannam@89 480 <dd><p>Set the block size to 100 k, 200 k ... 900 k
cannam@89 481 when compressing. Has no effect when decompressing. See <a class="xref" href="#memory-management" title="2.5. MEMORY MANAGEMENT">MEMORY MANAGEMENT</a> below. The
cannam@89 482 <code class="computeroutput">--fast</code> and
cannam@89 483 <code class="computeroutput">--best</code> aliases are primarily
cannam@89 484 for GNU <code class="computeroutput">gzip</code> compatibility.
cannam@89 485 In particular, <code class="computeroutput">--fast</code> doesn't
cannam@89 486 make things significantly faster. And
cannam@89 487 <code class="computeroutput">--best</code> merely selects the
cannam@89 488 default behaviour.</p></dd>
cannam@89 489 <dt><span class="term"><code class="computeroutput">--</code></span></dt>
cannam@89 490 <dd><p>Treats all subsequent arguments as file names,
cannam@89 491 even if they start with a dash. This is so you can handle
cannam@89 492 files with names beginning with a dash, for example:
cannam@89 493 <code class="computeroutput">bzip2 --
cannam@89 494 -myfilename</code>.</p></dd>
cannam@89 495 <dt>
cannam@89 496 <span class="term"><code class="computeroutput">--repetitive-fast</code>, </span><span class="term"><code class="computeroutput">--repetitive-best</code></span>
cannam@89 497 </dt>
cannam@89 498 <dd><p>These flags are redundant in versions 0.9.5 and
cannam@89 499 above. They provided some coarse control over the behaviour of
cannam@89 500 the sorting algorithm in earlier versions, which was sometimes
cannam@89 501 useful. 0.9.5 and above have an improved algorithm which
cannam@89 502 renders these flags irrelevant.</p></dd>
cannam@89 503 </dl></div>
cannam@89 504 </div>
cannam@89 505 <div class="sect1" title="2.5. MEMORY MANAGEMENT">
cannam@89 506 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 507 <a name="memory-management"></a>2.5. MEMORY MANAGEMENT</h2></div></div></div>
cannam@89 508 <p><code class="computeroutput">bzip2</code> compresses large
cannam@89 509 files in blocks. The block size affects both the compression
cannam@89 510 ratio achieved, and the amount of memory needed for compression
cannam@89 511 and decompression. The flags <code class="computeroutput">-1</code>
cannam@89 512 through <code class="computeroutput">-9</code> specify the block
cannam@89 513 size to be 100,000 bytes through 900,000 bytes (the default)
cannam@89 514 respectively. At decompression time, the block size used for
cannam@89 515 compression is read from the header of the compressed file, and
cannam@89 516 <code class="computeroutput">bunzip2</code> then allocates itself
cannam@89 517 just enough memory to decompress the file. Since block sizes are
cannam@89 518 stored in compressed files, it follows that the flags
cannam@89 519 <code class="computeroutput">-1</code> to
cannam@89 520 <code class="computeroutput">-9</code> are irrelevant to and so
cannam@89 521 ignored during decompression.</p>
cannam@89 522 <p>Compression and decompression requirements, in bytes, can be
cannam@89 523 estimated as:</p>
cannam@89 524 <pre class="programlisting">Compression: 400k + ( 8 x block size )
cannam@89 525
cannam@89 526 Decompression: 100k + ( 4 x block size ), or
cannam@89 527 100k + ( 2.5 x block size )</pre>
cannam@89 528 <p>Larger block sizes give rapidly diminishing marginal
cannam@89 529 returns. Most of the compression comes from the first two or
cannam@89 530 three hundred k of block size, a fact worth bearing in mind when
cannam@89 531 using <code class="computeroutput">bzip2</code> on small machines.
cannam@89 532 It is also important to appreciate that the decompression memory
cannam@89 533 requirement is set at compression time by the choice of block
cannam@89 534 size.</p>
cannam@89 535 <p>For files compressed with the default 900k block size,
cannam@89 536 <code class="computeroutput">bunzip2</code> will require about 3700
cannam@89 537 kbytes to decompress. To support decompression of any file on a
cannam@89 538 4 megabyte machine, <code class="computeroutput">bunzip2</code> has
cannam@89 539 an option to decompress using approximately half this amount of
cannam@89 540 memory, about 2300 kbytes. Decompression speed is also halved,
cannam@89 541 so you should use this option only where necessary. The relevant
cannam@89 542 flag is <code class="computeroutput">-s</code>.</p>
cannam@89 543 <p>In general, try and use the largest block size memory
cannam@89 544 constraints allow, since that maximises the compression achieved.
cannam@89 545 Compression and decompression speed are virtually unaffected by
cannam@89 546 block size.</p>
cannam@89 547 <p>Another significant point applies to files which fit in a
cannam@89 548 single block -- that means most files you'd encounter using a
cannam@89 549 large block size. The amount of real memory touched is
cannam@89 550 proportional to the size of the file, since the file is smaller
cannam@89 551 than a block. For example, compressing a file 20,000 bytes long
cannam@89 552 with the flag <code class="computeroutput">-9</code> will cause the
cannam@89 553 compressor to allocate around 7600k of memory, but only touch
cannam@89 554 400k + 20000 * 8 = 560 kbytes of it. Similarly, the decompressor
cannam@89 555 will allocate 3700k but only touch 100k + 20000 * 4 = 180
cannam@89 556 kbytes.</p>
cannam@89 557 <p>Here is a table which summarises the maximum memory usage
cannam@89 558 for different block sizes. Also recorded is the total compressed
cannam@89 559 size for 14 files of the Calgary Text Compression Corpus
cannam@89 560 totalling 3,141,622 bytes. This column gives some feel for how
cannam@89 561 compression varies with block size. These figures tend to
cannam@89 562 understate the advantage of larger block sizes for larger files,
cannam@89 563 since the Corpus is dominated by smaller files.</p>
cannam@89 564 <pre class="programlisting"> Compress Decompress Decompress Corpus
cannam@89 565 Flag usage usage -s usage Size
cannam@89 566
cannam@89 567 -1 1200k 500k 350k 914704
cannam@89 568 -2 2000k 900k 600k 877703
cannam@89 569 -3 2800k 1300k 850k 860338
cannam@89 570 -4 3600k 1700k 1100k 846899
cannam@89 571 -5 4400k 2100k 1350k 845160
cannam@89 572 -6 5200k 2500k 1600k 838626
cannam@89 573 -7 6100k 2900k 1850k 834096
cannam@89 574 -8 6800k 3300k 2100k 828642
cannam@89 575 -9 7600k 3700k 2350k 828642</pre>
cannam@89 576 </div>
cannam@89 577 <div class="sect1" title="2.6. RECOVERING DATA FROM DAMAGED FILES">
cannam@89 578 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 579 <a name="recovering"></a>2.6. RECOVERING DATA FROM DAMAGED FILES</h2></div></div></div>
cannam@89 580 <p><code class="computeroutput">bzip2</code> compresses files in
cannam@89 581 blocks, usually 900kbytes long. Each block is handled
cannam@89 582 independently. If a media or transmission error causes a
cannam@89 583 multi-block <code class="computeroutput">.bz2</code> file to become
cannam@89 584 damaged, it may be possible to recover data from the undamaged
cannam@89 585 blocks in the file.</p>
cannam@89 586 <p>The compressed representation of each block is delimited by
cannam@89 587 a 48-bit pattern, which makes it possible to find the block
cannam@89 588 boundaries with reasonable certainty. Each block also carries
cannam@89 589 its own 32-bit CRC, so damaged blocks can be distinguished from
cannam@89 590 undamaged ones.</p>
cannam@89 591 <p><code class="computeroutput">bzip2recover</code> is a simple
cannam@89 592 program whose purpose is to search for blocks in
cannam@89 593 <code class="computeroutput">.bz2</code> files, and write each block
cannam@89 594 out into its own <code class="computeroutput">.bz2</code> file. You
cannam@89 595 can then use <code class="computeroutput">bzip2 -t</code> to test
cannam@89 596 the integrity of the resulting files, and decompress those which
cannam@89 597 are undamaged.</p>
cannam@89 598 <p><code class="computeroutput">bzip2recover</code> takes a
cannam@89 599 single argument, the name of the damaged file, and writes a
cannam@89 600 number of files <code class="computeroutput">rec0001file.bz2</code>,
cannam@89 601 <code class="computeroutput">rec0002file.bz2</code>, etc, containing
cannam@89 602 the extracted blocks. The output filenames are designed so that
cannam@89 603 the use of wildcards in subsequent processing -- for example,
cannam@89 604 <code class="computeroutput">bzip2 -dc rec*file.bz2 &gt;
cannam@89 605 recovered_data</code> -- lists the files in the correct
cannam@89 606 order.</p>
cannam@89 607 <p><code class="computeroutput">bzip2recover</code> should be of
cannam@89 608 most use dealing with large <code class="computeroutput">.bz2</code>
cannam@89 609 files, as these will contain many blocks. It is clearly futile
cannam@89 610 to use it on damaged single-block files, since a damaged block
cannam@89 611 cannot be recovered. If you wish to minimise any potential data
cannam@89 612 loss through media or transmission errors, you might consider
cannam@89 613 compressing with a smaller block size.</p>
cannam@89 614 </div>
cannam@89 615 <div class="sect1" title="2.7. PERFORMANCE NOTES">
cannam@89 616 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 617 <a name="performance"></a>2.7. PERFORMANCE NOTES</h2></div></div></div>
cannam@89 618 <p>The sorting phase of compression gathers together similar
cannam@89 619 strings in the file. Because of this, files containing very long
cannam@89 620 runs of repeated symbols, like "aabaabaabaab ..." (repeated
cannam@89 621 several hundred times) may compress more slowly than normal.
cannam@89 622 Versions 0.9.5 and above fare much better than previous versions
cannam@89 623 in this respect. The ratio between worst-case and average-case
cannam@89 624 compression time is in the region of 10:1. For previous
cannam@89 625 versions, this figure was more like 100:1. You can use the
cannam@89 626 <code class="computeroutput">-vvvv</code> option to monitor progress
cannam@89 627 in great detail, if you want.</p>
cannam@89 628 <p>Decompression speed is unaffected by these
cannam@89 629 phenomena.</p>
cannam@89 630 <p><code class="computeroutput">bzip2</code> usually allocates
cannam@89 631 several megabytes of memory to operate in, and then charges all
cannam@89 632 over it in a fairly random fashion. This means that performance,
cannam@89 633 both for compressing and decompressing, is largely determined by
cannam@89 634 the speed at which your machine can service cache misses.
cannam@89 635 Because of this, small changes to the code to reduce the miss
cannam@89 636 rate have been observed to give disproportionately large
cannam@89 637 performance improvements. I imagine
cannam@89 638 <code class="computeroutput">bzip2</code> will perform best on
cannam@89 639 machines with very large caches.</p>
cannam@89 640 </div>
cannam@89 641 <div class="sect1" title="2.8. CAVEATS">
cannam@89 642 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 643 <a name="caveats"></a>2.8. CAVEATS</h2></div></div></div>
cannam@89 644 <p>I/O error messages are not as helpful as they could be.
cannam@89 645 <code class="computeroutput">bzip2</code> tries hard to detect I/O
cannam@89 646 errors and exit cleanly, but the details of what the problem is
cannam@89 647 sometimes seem rather misleading.</p>
cannam@89 648 <p>This manual page pertains to version 1.0.6 of
cannam@89 649 <code class="computeroutput">bzip2</code>. Compressed data created by
cannam@89 650 this version is entirely forwards and backwards compatible with the
cannam@89 651 previous public releases, versions 0.1pl2, 0.9.0 and 0.9.5, 1.0.0,
cannam@89 652 1.0.1, 1.0.2 and 1.0.3, but with the following exception: 0.9.0 and
cannam@89 653 above can correctly decompress multiple concatenated compressed files.
cannam@89 654 0.1pl2 cannot do this; it will stop after decompressing just the first
cannam@89 655 file in the stream.</p>
cannam@89 656 <p><code class="computeroutput">bzip2recover</code> versions
cannam@89 657 prior to 1.0.2 used 32-bit integers to represent bit positions in
cannam@89 658 compressed files, so it could not handle compressed files more
cannam@89 659 than 512 megabytes long. Versions 1.0.2 and above use 64-bit ints
cannam@89 660 on some platforms which support them (GNU supported targets, and
cannam@89 661 Windows). To establish whether or not
cannam@89 662 <code class="computeroutput">bzip2recover</code> was built with such
cannam@89 663 a limitation, run it without arguments. In any event you can
cannam@89 664 build yourself an unlimited version if you can recompile it with
cannam@89 665 <code class="computeroutput">MaybeUInt64</code> set to be an
cannam@89 666 unsigned 64-bit integer.</p>
cannam@89 667 </div>
cannam@89 668 <div class="sect1" title="2.9. AUTHOR">
cannam@89 669 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 670 <a name="author"></a>2.9. AUTHOR</h2></div></div></div>
cannam@89 671 <p>Julian Seward,
cannam@89 672 <code class="computeroutput">jseward@bzip.org</code></p>
cannam@89 673 <p>The ideas embodied in
cannam@89 674 <code class="computeroutput">bzip2</code> are due to (at least) the
cannam@89 675 following people: Michael Burrows and David Wheeler (for the
cannam@89 676 block sorting transformation), David Wheeler (again, for the
cannam@89 677 Huffman coder), Peter Fenwick (for the structured coding model in
cannam@89 678 the original <code class="computeroutput">bzip</code>, and many
cannam@89 679 refinements), and Alistair Moffat, Radford Neal and Ian Witten
cannam@89 680 (for the arithmetic coder in the original
cannam@89 681 <code class="computeroutput">bzip</code>). I am much indebted for
cannam@89 682 their help, support and advice. See the manual in the source
cannam@89 683 distribution for pointers to sources of documentation. Christian
cannam@89 684 von Roques encouraged me to look for faster sorting algorithms,
cannam@89 685 so as to speed up compression. Bela Lubkin encouraged me to
cannam@89 686 improve the worst-case compression performance.
cannam@89 687 Donna Robinson XMLised the documentation.
cannam@89 688 Many people sent
cannam@89 689 patches, helped with portability problems, lent machines, gave
cannam@89 690 advice and were generally helpful.</p>
cannam@89 691 </div>
cannam@89 692 </div>
cannam@89 693 <div class="chapter" title="3.  Programming with libbzip2">
cannam@89 694 <div class="titlepage"><div><div><h2 class="title">
cannam@89 695 <a name="libprog"></a>3. 
cannam@89 696 Programming with <code class="computeroutput">libbzip2</code>
cannam@89 697 </h2></div></div></div>
cannam@89 698 <div class="toc">
cannam@89 699 <p><b>Table of Contents</b></p>
cannam@89 700 <dl>
cannam@89 701 <dt><span class="sect1"><a href="#top-level">3.1. Top-level structure</a></span></dt>
cannam@89 702 <dd><dl>
cannam@89 703 <dt><span class="sect2"><a href="#ll-summary">3.1.1. Low-level summary</a></span></dt>
cannam@89 704 <dt><span class="sect2"><a href="#hl-summary">3.1.2. High-level summary</a></span></dt>
cannam@89 705 <dt><span class="sect2"><a href="#util-fns-summary">3.1.3. Utility functions summary</a></span></dt>
cannam@89 706 </dl></dd>
cannam@89 707 <dt><span class="sect1"><a href="#err-handling">3.2. Error handling</a></span></dt>
cannam@89 708 <dt><span class="sect1"><a href="#low-level">3.3. Low-level interface</a></span></dt>
cannam@89 709 <dd><dl>
cannam@89 710 <dt><span class="sect2"><a href="#bzcompress-init">3.3.1. BZ2_bzCompressInit</a></span></dt>
cannam@89 711 <dt><span class="sect2"><a href="#bzCompress">3.3.2. BZ2_bzCompress</a></span></dt>
cannam@89 712 <dt><span class="sect2"><a href="#bzCompress-end">3.3.3. BZ2_bzCompressEnd</a></span></dt>
cannam@89 713 <dt><span class="sect2"><a href="#bzDecompress-init">3.3.4. BZ2_bzDecompressInit</a></span></dt>
cannam@89 714 <dt><span class="sect2"><a href="#bzDecompress">3.3.5. BZ2_bzDecompress</a></span></dt>
cannam@89 715 <dt><span class="sect2"><a href="#bzDecompress-end">3.3.6. BZ2_bzDecompressEnd</a></span></dt>
cannam@89 716 </dl></dd>
cannam@89 717 <dt><span class="sect1"><a href="#hl-interface">3.4. High-level interface</a></span></dt>
cannam@89 718 <dd><dl>
cannam@89 719 <dt><span class="sect2"><a href="#bzreadopen">3.4.1. BZ2_bzReadOpen</a></span></dt>
cannam@89 720 <dt><span class="sect2"><a href="#bzread">3.4.2. BZ2_bzRead</a></span></dt>
cannam@89 721 <dt><span class="sect2"><a href="#bzreadgetunused">3.4.3. BZ2_bzReadGetUnused</a></span></dt>
cannam@89 722 <dt><span class="sect2"><a href="#bzreadclose">3.4.4. BZ2_bzReadClose</a></span></dt>
cannam@89 723 <dt><span class="sect2"><a href="#bzwriteopen">3.4.5. BZ2_bzWriteOpen</a></span></dt>
cannam@89 724 <dt><span class="sect2"><a href="#bzwrite">3.4.6. BZ2_bzWrite</a></span></dt>
cannam@89 725 <dt><span class="sect2"><a href="#bzwriteclose">3.4.7. BZ2_bzWriteClose</a></span></dt>
cannam@89 726 <dt><span class="sect2"><a href="#embed">3.4.8. Handling embedded compressed data streams</a></span></dt>
cannam@89 727 <dt><span class="sect2"><a href="#std-rdwr">3.4.9. Standard file-reading/writing code</a></span></dt>
cannam@89 728 </dl></dd>
cannam@89 729 <dt><span class="sect1"><a href="#util-fns">3.5. Utility functions</a></span></dt>
cannam@89 730 <dd><dl>
cannam@89 731 <dt><span class="sect2"><a href="#bzbufftobuffcompress">3.5.1. BZ2_bzBuffToBuffCompress</a></span></dt>
cannam@89 732 <dt><span class="sect2"><a href="#bzbufftobuffdecompress">3.5.2. BZ2_bzBuffToBuffDecompress</a></span></dt>
cannam@89 733 </dl></dd>
cannam@89 734 <dt><span class="sect1"><a href="#zlib-compat">3.6. zlib compatibility functions</a></span></dt>
cannam@89 735 <dt><span class="sect1"><a href="#stdio-free">3.7. Using the library in a stdio-free environment</a></span></dt>
cannam@89 736 <dd><dl>
cannam@89 737 <dt><span class="sect2"><a href="#stdio-bye">3.7.1. Getting rid of stdio</a></span></dt>
cannam@89 738 <dt><span class="sect2"><a href="#critical-error">3.7.2. Critical error handling</a></span></dt>
cannam@89 739 </dl></dd>
cannam@89 740 <dt><span class="sect1"><a href="#win-dll">3.8. Making a Windows DLL</a></span></dt>
cannam@89 741 </dl>
cannam@89 742 </div>
cannam@89 743 <p>This chapter describes the programming interface to
cannam@89 744 <code class="computeroutput">libbzip2</code>.</p>
cannam@89 745 <p>For general background information, particularly about
cannam@89 746 memory use and performance aspects, you'd be well advised to read
cannam@89 747 <a class="xref" href="#using" title="2. How to use bzip2">How to use bzip2</a> as well.</p>
cannam@89 748 <div class="sect1" title="3.1. Top-level structure">
cannam@89 749 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 750 <a name="top-level"></a>3.1. Top-level structure</h2></div></div></div>
cannam@89 751 <p><code class="computeroutput">libbzip2</code> is a flexible
cannam@89 752 library for compressing and decompressing data in the
cannam@89 753 <code class="computeroutput">bzip2</code> data format. Although
cannam@89 754 packaged as a single entity, it helps to regard the library as
cannam@89 755 three separate parts: the low level interface, and the high level
cannam@89 756 interface, and some utility functions.</p>
cannam@89 757 <p>The structure of
cannam@89 758 <code class="computeroutput">libbzip2</code>'s interfaces is similar
cannam@89 759 to that of Jean-loup Gailly's and Mark Adler's excellent
cannam@89 760 <code class="computeroutput">zlib</code> library.</p>
cannam@89 761 <p>All externally visible symbols have names beginning
cannam@89 762 <code class="computeroutput">BZ2_</code>. This is new in version
cannam@89 763 1.0. The intention is to minimise pollution of the namespaces of
cannam@89 764 library clients.</p>
cannam@89 765 <p>To use any part of the library, you need to
cannam@89 766 <code class="computeroutput">#include &lt;bzlib.h&gt;</code>
cannam@89 767 into your sources.</p>
cannam@89 768 <div class="sect2" title="3.1.1. Low-level summary">
cannam@89 769 <div class="titlepage"><div><div><h3 class="title">
cannam@89 770 <a name="ll-summary"></a>3.1.1. Low-level summary</h3></div></div></div>
cannam@89 771 <p>This interface provides services for compressing and
cannam@89 772 decompressing data in memory. There's no provision for dealing
cannam@89 773 with files, streams or any other I/O mechanisms, just straight
cannam@89 774 memory-to-memory work. In fact, this part of the library can be
cannam@89 775 compiled without inclusion of
cannam@89 776 <code class="computeroutput">stdio.h</code>, which may be helpful
cannam@89 777 for embedded applications.</p>
cannam@89 778 <p>The low-level part of the library has no global variables
cannam@89 779 and is therefore thread-safe.</p>
cannam@89 780 <p>Six routines make up the low level interface:
cannam@89 781 <code class="computeroutput">BZ2_bzCompressInit</code>,
cannam@89 782 <code class="computeroutput">BZ2_bzCompress</code>, and
cannam@89 783 <code class="computeroutput">BZ2_bzCompressEnd</code> for
cannam@89 784 compression, and a corresponding trio
cannam@89 785 <code class="computeroutput">BZ2_bzDecompressInit</code>,
cannam@89 786 <code class="computeroutput">BZ2_bzDecompress</code> and
cannam@89 787 <code class="computeroutput">BZ2_bzDecompressEnd</code> for
cannam@89 788 decompression. The <code class="computeroutput">*Init</code>
cannam@89 789 functions allocate memory for compression/decompression and do
cannam@89 790 other initialisations, whilst the
cannam@89 791 <code class="computeroutput">*End</code> functions close down
cannam@89 792 operations and release memory.</p>
cannam@89 793 <p>The real work is done by
cannam@89 794 <code class="computeroutput">BZ2_bzCompress</code> and
cannam@89 795 <code class="computeroutput">BZ2_bzDecompress</code>. These
cannam@89 796 compress and decompress data from a user-supplied input buffer to
cannam@89 797 a user-supplied output buffer. These buffers can be any size;
cannam@89 798 arbitrary quantities of data are handled by making repeated calls
cannam@89 799 to these functions. This is a flexible mechanism allowing a
cannam@89 800 consumer-pull style of activity, or producer-push, or a mixture
cannam@89 801 of both.</p>
cannam@89 802 </div>
cannam@89 803 <div class="sect2" title="3.1.2. High-level summary">
cannam@89 804 <div class="titlepage"><div><div><h3 class="title">
cannam@89 805 <a name="hl-summary"></a>3.1.2. High-level summary</h3></div></div></div>
cannam@89 806 <p>This interface provides some handy wrappers around the
cannam@89 807 low-level interface to facilitate reading and writing
cannam@89 808 <code class="computeroutput">bzip2</code> format files
cannam@89 809 (<code class="computeroutput">.bz2</code> files). The routines
cannam@89 810 provide hooks to facilitate reading files in which the
cannam@89 811 <code class="computeroutput">bzip2</code> data stream is embedded
cannam@89 812 within some larger-scale file structure, or where there are
cannam@89 813 multiple <code class="computeroutput">bzip2</code> data streams
cannam@89 814 concatenated end-to-end.</p>
cannam@89 815 <p>For reading files,
cannam@89 816 <code class="computeroutput">BZ2_bzReadOpen</code>,
cannam@89 817 <code class="computeroutput">BZ2_bzRead</code>,
cannam@89 818 <code class="computeroutput">BZ2_bzReadClose</code> and
cannam@89 819 <code class="computeroutput">BZ2_bzReadGetUnused</code> are
cannam@89 820 supplied. For writing files,
cannam@89 821 <code class="computeroutput">BZ2_bzWriteOpen</code>,
cannam@89 822 <code class="computeroutput">BZ2_bzWrite</code> and
cannam@89 823 <code class="computeroutput">BZ2_bzWriteFinish</code> are
cannam@89 824 available.</p>
cannam@89 825 <p>As with the low-level library, no global variables are used
cannam@89 826 so the library is per se thread-safe. However, if I/O errors
cannam@89 827 occur whilst reading or writing the underlying compressed files,
cannam@89 828 you may have to consult <code class="computeroutput">errno</code> to
cannam@89 829 determine the cause of the error. In that case, you'd need a C
cannam@89 830 library which correctly supports
cannam@89 831 <code class="computeroutput">errno</code> in a multithreaded
cannam@89 832 environment.</p>
cannam@89 833 <p>To make the library a little simpler and more portable,
cannam@89 834 <code class="computeroutput">BZ2_bzReadOpen</code> and
cannam@89 835 <code class="computeroutput">BZ2_bzWriteOpen</code> require you to
cannam@89 836 pass them file handles (<code class="computeroutput">FILE*</code>s)
cannam@89 837 which have previously been opened for reading or writing
cannam@89 838 respectively. That avoids portability problems associated with
cannam@89 839 file operations and file attributes, whilst not being much of an
cannam@89 840 imposition on the programmer.</p>
cannam@89 841 </div>
cannam@89 842 <div class="sect2" title="3.1.3. Utility functions summary">
cannam@89 843 <div class="titlepage"><div><div><h3 class="title">
cannam@89 844 <a name="util-fns-summary"></a>3.1.3. Utility functions summary</h3></div></div></div>
cannam@89 845 <p>For very simple needs,
cannam@89 846 <code class="computeroutput">BZ2_bzBuffToBuffCompress</code> and
cannam@89 847 <code class="computeroutput">BZ2_bzBuffToBuffDecompress</code> are
cannam@89 848 provided. These compress data in memory from one buffer to
cannam@89 849 another buffer in a single function call. You should assess
cannam@89 850 whether these functions fulfill your memory-to-memory
cannam@89 851 compression/decompression requirements before investing effort in
cannam@89 852 understanding the more general but more complex low-level
cannam@89 853 interface.</p>
cannam@89 854 <p>Yoshioka Tsuneo
cannam@89 855 (<code class="computeroutput">tsuneo@rr.iij4u.or.jp</code>) has
cannam@89 856 contributed some functions to give better
cannam@89 857 <code class="computeroutput">zlib</code> compatibility. These
cannam@89 858 functions are <code class="computeroutput">BZ2_bzopen</code>,
cannam@89 859 <code class="computeroutput">BZ2_bzread</code>,
cannam@89 860 <code class="computeroutput">BZ2_bzwrite</code>,
cannam@89 861 <code class="computeroutput">BZ2_bzflush</code>,
cannam@89 862 <code class="computeroutput">BZ2_bzclose</code>,
cannam@89 863 <code class="computeroutput">BZ2_bzerror</code> and
cannam@89 864 <code class="computeroutput">BZ2_bzlibVersion</code>. You may find
cannam@89 865 these functions more convenient for simple file reading and
cannam@89 866 writing, than those in the high-level interface. These functions
cannam@89 867 are not (yet) officially part of the library, and are minimally
cannam@89 868 documented here. If they break, you get to keep all the pieces.
cannam@89 869 I hope to document them properly when time permits.</p>
cannam@89 870 <p>Yoshioka also contributed modifications to allow the
cannam@89 871 library to be built as a Windows DLL.</p>
cannam@89 872 </div>
cannam@89 873 </div>
cannam@89 874 <div class="sect1" title="3.2. Error handling">
cannam@89 875 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 876 <a name="err-handling"></a>3.2. Error handling</h2></div></div></div>
cannam@89 877 <p>The library is designed to recover cleanly in all
cannam@89 878 situations, including the worst-case situation of decompressing
cannam@89 879 random data. I'm not 100% sure that it can always do this, so
cannam@89 880 you might want to add a signal handler to catch segmentation
cannam@89 881 violations during decompression if you are feeling especially
cannam@89 882 paranoid. I would be interested in hearing more about the
cannam@89 883 robustness of the library to corrupted compressed data.</p>
cannam@89 884 <p>Version 1.0.3 more robust in this respect than any
cannam@89 885 previous version. Investigations with Valgrind (a tool for detecting
cannam@89 886 problems with memory management) indicate
cannam@89 887 that, at least for the few files I tested, all single-bit errors
cannam@89 888 in the decompressed data are caught properly, with no
cannam@89 889 segmentation faults, no uses of uninitialised data, no out of
cannam@89 890 range reads or writes, and no infinite looping in the decompressor.
cannam@89 891 So it's certainly pretty robust, although
cannam@89 892 I wouldn't claim it to be totally bombproof.</p>
cannam@89 893 <p>The file <code class="computeroutput">bzlib.h</code> contains
cannam@89 894 all definitions needed to use the library. In particular, you
cannam@89 895 should definitely not include
cannam@89 896 <code class="computeroutput">bzlib_private.h</code>.</p>
cannam@89 897 <p>In <code class="computeroutput">bzlib.h</code>, the various
cannam@89 898 return values are defined. The following list is not intended as
cannam@89 899 an exhaustive description of the circumstances in which a given
cannam@89 900 value may be returned -- those descriptions are given later.
cannam@89 901 Rather, it is intended to convey the rough meaning of each return
cannam@89 902 value. The first five actions are normal and not intended to
cannam@89 903 denote an error situation.</p>
cannam@89 904 <div class="variablelist"><dl>
cannam@89 905 <dt><span class="term"><code class="computeroutput">BZ_OK</code></span></dt>
cannam@89 906 <dd><p>The requested action was completed
cannam@89 907 successfully.</p></dd>
cannam@89 908 <dt><span class="term"><code class="computeroutput">BZ_RUN_OK, BZ_FLUSH_OK,
cannam@89 909 BZ_FINISH_OK</code></span></dt>
cannam@89 910 <dd><p>In
cannam@89 911 <code class="computeroutput">BZ2_bzCompress</code>, the requested
cannam@89 912 flush/finish/nothing-special action was completed
cannam@89 913 successfully.</p></dd>
cannam@89 914 <dt><span class="term"><code class="computeroutput">BZ_STREAM_END</code></span></dt>
cannam@89 915 <dd><p>Compression of data was completed, or the
cannam@89 916 logical stream end was detected during
cannam@89 917 decompression.</p></dd>
cannam@89 918 </dl></div>
cannam@89 919 <p>The following return values indicate an error of some
cannam@89 920 kind.</p>
cannam@89 921 <div class="variablelist"><dl>
cannam@89 922 <dt><span class="term"><code class="computeroutput">BZ_CONFIG_ERROR</code></span></dt>
cannam@89 923 <dd><p>Indicates that the library has been improperly
cannam@89 924 compiled on your platform -- a major configuration error.
cannam@89 925 Specifically, it means that
cannam@89 926 <code class="computeroutput">sizeof(char)</code>,
cannam@89 927 <code class="computeroutput">sizeof(short)</code> and
cannam@89 928 <code class="computeroutput">sizeof(int)</code> are not 1, 2 and
cannam@89 929 4 respectively, as they should be. Note that the library
cannam@89 930 should still work properly on 64-bit platforms which follow
cannam@89 931 the LP64 programming model -- that is, where
cannam@89 932 <code class="computeroutput">sizeof(long)</code> and
cannam@89 933 <code class="computeroutput">sizeof(void*)</code> are 8. Under
cannam@89 934 LP64, <code class="computeroutput">sizeof(int)</code> is still 4,
cannam@89 935 so <code class="computeroutput">libbzip2</code>, which doesn't
cannam@89 936 use the <code class="computeroutput">long</code> type, is
cannam@89 937 OK.</p></dd>
cannam@89 938 <dt><span class="term"><code class="computeroutput">BZ_SEQUENCE_ERROR</code></span></dt>
cannam@89 939 <dd><p>When using the library, it is important to call
cannam@89 940 the functions in the correct sequence and with data structures
cannam@89 941 (buffers etc) in the correct states.
cannam@89 942 <code class="computeroutput">libbzip2</code> checks as much as it
cannam@89 943 can to ensure this is happening, and returns
cannam@89 944 <code class="computeroutput">BZ_SEQUENCE_ERROR</code> if not.
cannam@89 945 Code which complies precisely with the function semantics, as
cannam@89 946 detailed below, should never receive this value; such an event
cannam@89 947 denotes buggy code which you should
cannam@89 948 investigate.</p></dd>
cannam@89 949 <dt><span class="term"><code class="computeroutput">BZ_PARAM_ERROR</code></span></dt>
cannam@89 950 <dd><p>Returned when a parameter to a function call is
cannam@89 951 out of range or otherwise manifestly incorrect. As with
cannam@89 952 <code class="computeroutput">BZ_SEQUENCE_ERROR</code>, this
cannam@89 953 denotes a bug in the client code. The distinction between
cannam@89 954 <code class="computeroutput">BZ_PARAM_ERROR</code> and
cannam@89 955 <code class="computeroutput">BZ_SEQUENCE_ERROR</code> is a bit
cannam@89 956 hazy, but still worth making.</p></dd>
cannam@89 957 <dt><span class="term"><code class="computeroutput">BZ_MEM_ERROR</code></span></dt>
cannam@89 958 <dd><p>Returned when a request to allocate memory
cannam@89 959 failed. Note that the quantity of memory needed to decompress
cannam@89 960 a stream cannot be determined until the stream's header has
cannam@89 961 been read. So
cannam@89 962 <code class="computeroutput">BZ2_bzDecompress</code> and
cannam@89 963 <code class="computeroutput">BZ2_bzRead</code> may return
cannam@89 964 <code class="computeroutput">BZ_MEM_ERROR</code> even though some
cannam@89 965 of the compressed data has been read. The same is not true
cannam@89 966 for compression; once
cannam@89 967 <code class="computeroutput">BZ2_bzCompressInit</code> or
cannam@89 968 <code class="computeroutput">BZ2_bzWriteOpen</code> have
cannam@89 969 successfully completed,
cannam@89 970 <code class="computeroutput">BZ_MEM_ERROR</code> cannot
cannam@89 971 occur.</p></dd>
cannam@89 972 <dt><span class="term"><code class="computeroutput">BZ_DATA_ERROR</code></span></dt>
cannam@89 973 <dd><p>Returned when a data integrity error is
cannam@89 974 detected during decompression. Most importantly, this means
cannam@89 975 when stored and computed CRCs for the data do not match. This
cannam@89 976 value is also returned upon detection of any other anomaly in
cannam@89 977 the compressed data.</p></dd>
cannam@89 978 <dt><span class="term"><code class="computeroutput">BZ_DATA_ERROR_MAGIC</code></span></dt>
cannam@89 979 <dd><p>As a special case of
cannam@89 980 <code class="computeroutput">BZ_DATA_ERROR</code>, it is
cannam@89 981 sometimes useful to know when the compressed stream does not
cannam@89 982 start with the correct magic bytes (<code class="computeroutput">'B' 'Z'
cannam@89 983 'h'</code>).</p></dd>
cannam@89 984 <dt><span class="term"><code class="computeroutput">BZ_IO_ERROR</code></span></dt>
cannam@89 985 <dd><p>Returned by
cannam@89 986 <code class="computeroutput">BZ2_bzRead</code> and
cannam@89 987 <code class="computeroutput">BZ2_bzWrite</code> when there is an
cannam@89 988 error reading or writing in the compressed file, and by
cannam@89 989 <code class="computeroutput">BZ2_bzReadOpen</code> and
cannam@89 990 <code class="computeroutput">BZ2_bzWriteOpen</code> for attempts
cannam@89 991 to use a file for which the error indicator (viz,
cannam@89 992 <code class="computeroutput">ferror(f)</code>) is set. On
cannam@89 993 receipt of <code class="computeroutput">BZ_IO_ERROR</code>, the
cannam@89 994 caller should consult <code class="computeroutput">errno</code>
cannam@89 995 and/or <code class="computeroutput">perror</code> to acquire
cannam@89 996 operating-system specific information about the
cannam@89 997 problem.</p></dd>
cannam@89 998 <dt><span class="term"><code class="computeroutput">BZ_UNEXPECTED_EOF</code></span></dt>
cannam@89 999 <dd><p>Returned by
cannam@89 1000 <code class="computeroutput">BZ2_bzRead</code> when the
cannam@89 1001 compressed file finishes before the logical end of stream is
cannam@89 1002 detected.</p></dd>
cannam@89 1003 <dt><span class="term"><code class="computeroutput">BZ_OUTBUFF_FULL</code></span></dt>
cannam@89 1004 <dd><p>Returned by
cannam@89 1005 <code class="computeroutput">BZ2_bzBuffToBuffCompress</code> and
cannam@89 1006 <code class="computeroutput">BZ2_bzBuffToBuffDecompress</code> to
cannam@89 1007 indicate that the output data will not fit into the output
cannam@89 1008 buffer provided.</p></dd>
cannam@89 1009 </dl></div>
cannam@89 1010 </div>
cannam@89 1011 <div class="sect1" title="3.3. Low-level interface">
cannam@89 1012 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 1013 <a name="low-level"></a>3.3. Low-level interface</h2></div></div></div>
cannam@89 1014 <div class="sect2" title="3.3.1. BZ2_bzCompressInit">
cannam@89 1015 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1016 <a name="bzcompress-init"></a>3.3.1. BZ2_bzCompressInit</h3></div></div></div>
cannam@89 1017 <pre class="programlisting">typedef struct {
cannam@89 1018 char *next_in;
cannam@89 1019 unsigned int avail_in;
cannam@89 1020 unsigned int total_in_lo32;
cannam@89 1021 unsigned int total_in_hi32;
cannam@89 1022
cannam@89 1023 char *next_out;
cannam@89 1024 unsigned int avail_out;
cannam@89 1025 unsigned int total_out_lo32;
cannam@89 1026 unsigned int total_out_hi32;
cannam@89 1027
cannam@89 1028 void *state;
cannam@89 1029
cannam@89 1030 void *(*bzalloc)(void *,int,int);
cannam@89 1031 void (*bzfree)(void *,void *);
cannam@89 1032 void *opaque;
cannam@89 1033 } bz_stream;
cannam@89 1034
cannam@89 1035 int BZ2_bzCompressInit ( bz_stream *strm,
cannam@89 1036 int blockSize100k,
cannam@89 1037 int verbosity,
cannam@89 1038 int workFactor );</pre>
cannam@89 1039 <p>Prepares for compression. The
cannam@89 1040 <code class="computeroutput">bz_stream</code> structure holds all
cannam@89 1041 data pertaining to the compression activity. A
cannam@89 1042 <code class="computeroutput">bz_stream</code> structure should be
cannam@89 1043 allocated and initialised prior to the call. The fields of
cannam@89 1044 <code class="computeroutput">bz_stream</code> comprise the entirety
cannam@89 1045 of the user-visible data. <code class="computeroutput">state</code>
cannam@89 1046 is a pointer to the private data structures required for
cannam@89 1047 compression.</p>
cannam@89 1048 <p>Custom memory allocators are supported, via fields
cannam@89 1049 <code class="computeroutput">bzalloc</code>,
cannam@89 1050 <code class="computeroutput">bzfree</code>, and
cannam@89 1051 <code class="computeroutput">opaque</code>. The value
cannam@89 1052 <code class="computeroutput">opaque</code> is passed to as the first
cannam@89 1053 argument to all calls to <code class="computeroutput">bzalloc</code>
cannam@89 1054 and <code class="computeroutput">bzfree</code>, but is otherwise
cannam@89 1055 ignored by the library. The call <code class="computeroutput">bzalloc (
cannam@89 1056 opaque, n, m )</code> is expected to return a pointer
cannam@89 1057 <code class="computeroutput">p</code> to <code class="computeroutput">n *
cannam@89 1058 m</code> bytes of memory, and <code class="computeroutput">bzfree (
cannam@89 1059 opaque, p )</code> should free that memory.</p>
cannam@89 1060 <p>If you don't want to use a custom memory allocator, set
cannam@89 1061 <code class="computeroutput">bzalloc</code>,
cannam@89 1062 <code class="computeroutput">bzfree</code> and
cannam@89 1063 <code class="computeroutput">opaque</code> to
cannam@89 1064 <code class="computeroutput">NULL</code>, and the library will then
cannam@89 1065 use the standard <code class="computeroutput">malloc</code> /
cannam@89 1066 <code class="computeroutput">free</code> routines.</p>
cannam@89 1067 <p>Before calling
cannam@89 1068 <code class="computeroutput">BZ2_bzCompressInit</code>, fields
cannam@89 1069 <code class="computeroutput">bzalloc</code>,
cannam@89 1070 <code class="computeroutput">bzfree</code> and
cannam@89 1071 <code class="computeroutput">opaque</code> should be filled
cannam@89 1072 appropriately, as just described. Upon return, the internal
cannam@89 1073 state will have been allocated and initialised, and
cannam@89 1074 <code class="computeroutput">total_in_lo32</code>,
cannam@89 1075 <code class="computeroutput">total_in_hi32</code>,
cannam@89 1076 <code class="computeroutput">total_out_lo32</code> and
cannam@89 1077 <code class="computeroutput">total_out_hi32</code> will have been
cannam@89 1078 set to zero. These four fields are used by the library to inform
cannam@89 1079 the caller of the total amount of data passed into and out of the
cannam@89 1080 library, respectively. You should not try to change them. As of
cannam@89 1081 version 1.0, 64-bit counts are maintained, even on 32-bit
cannam@89 1082 platforms, using the <code class="computeroutput">_hi32</code>
cannam@89 1083 fields to store the upper 32 bits of the count. So, for example,
cannam@89 1084 the total amount of data in is <code class="computeroutput">(total_in_hi32
cannam@89 1085 &lt;&lt; 32) + total_in_lo32</code>.</p>
cannam@89 1086 <p>Parameter <code class="computeroutput">blockSize100k</code>
cannam@89 1087 specifies the block size to be used for compression. It should
cannam@89 1088 be a value between 1 and 9 inclusive, and the actual block size
cannam@89 1089 used is 100000 x this figure. 9 gives the best compression but
cannam@89 1090 takes most memory.</p>
cannam@89 1091 <p>Parameter <code class="computeroutput">verbosity</code> should
cannam@89 1092 be set to a number between 0 and 4 inclusive. 0 is silent, and
cannam@89 1093 greater numbers give increasingly verbose monitoring/debugging
cannam@89 1094 output. If the library has been compiled with
cannam@89 1095 <code class="computeroutput">-DBZ_NO_STDIO</code>, no such output
cannam@89 1096 will appear for any verbosity setting.</p>
cannam@89 1097 <p>Parameter <code class="computeroutput">workFactor</code>
cannam@89 1098 controls how the compression phase behaves when presented with
cannam@89 1099 worst case, highly repetitive, input data. If compression runs
cannam@89 1100 into difficulties caused by repetitive data, the library switches
cannam@89 1101 from the standard sorting algorithm to a fallback algorithm. The
cannam@89 1102 fallback is slower than the standard algorithm by perhaps a
cannam@89 1103 factor of three, but always behaves reasonably, no matter how bad
cannam@89 1104 the input.</p>
cannam@89 1105 <p>Lower values of <code class="computeroutput">workFactor</code>
cannam@89 1106 reduce the amount of effort the standard algorithm will expend
cannam@89 1107 before resorting to the fallback. You should set this parameter
cannam@89 1108 carefully; too low, and many inputs will be handled by the
cannam@89 1109 fallback algorithm and so compress rather slowly, too high, and
cannam@89 1110 your average-to-worst case compression times can become very
cannam@89 1111 large. The default value of 30 gives reasonable behaviour over a
cannam@89 1112 wide range of circumstances.</p>
cannam@89 1113 <p>Allowable values range from 0 to 250 inclusive. 0 is a
cannam@89 1114 special case, equivalent to using the default value of 30.</p>
cannam@89 1115 <p>Note that the compressed output generated is the same
cannam@89 1116 regardless of whether or not the fallback algorithm is
cannam@89 1117 used.</p>
cannam@89 1118 <p>Be aware also that this parameter may disappear entirely in
cannam@89 1119 future versions of the library. In principle it should be
cannam@89 1120 possible to devise a good way to automatically choose which
cannam@89 1121 algorithm to use. Such a mechanism would render the parameter
cannam@89 1122 obsolete.</p>
cannam@89 1123 <p>Possible return values:</p>
cannam@89 1124 <pre class="programlisting">BZ_CONFIG_ERROR
cannam@89 1125 if the library has been mis-compiled
cannam@89 1126 BZ_PARAM_ERROR
cannam@89 1127 if strm is NULL
cannam@89 1128 or blockSize &lt; 1 or blockSize &gt; 9
cannam@89 1129 or verbosity &lt; 0 or verbosity &gt; 4
cannam@89 1130 or workFactor &lt; 0 or workFactor &gt; 250
cannam@89 1131 BZ_MEM_ERROR
cannam@89 1132 if not enough memory is available
cannam@89 1133 BZ_OK
cannam@89 1134 otherwise</pre>
cannam@89 1135 <p>Allowable next actions:</p>
cannam@89 1136 <pre class="programlisting">BZ2_bzCompress
cannam@89 1137 if BZ_OK is returned
cannam@89 1138 no specific action needed in case of error</pre>
cannam@89 1139 </div>
cannam@89 1140 <div class="sect2" title="3.3.2. BZ2_bzCompress">
cannam@89 1141 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1142 <a name="bzCompress"></a>3.3.2. BZ2_bzCompress</h3></div></div></div>
cannam@89 1143 <pre class="programlisting">int BZ2_bzCompress ( bz_stream *strm, int action );</pre>
cannam@89 1144 <p>Provides more input and/or output buffer space for the
cannam@89 1145 library. The caller maintains input and output buffers, and
cannam@89 1146 calls <code class="computeroutput">BZ2_bzCompress</code> to transfer
cannam@89 1147 data between them.</p>
cannam@89 1148 <p>Before each call to
cannam@89 1149 <code class="computeroutput">BZ2_bzCompress</code>,
cannam@89 1150 <code class="computeroutput">next_in</code> should point at the data
cannam@89 1151 to be compressed, and <code class="computeroutput">avail_in</code>
cannam@89 1152 should indicate how many bytes the library may read.
cannam@89 1153 <code class="computeroutput">BZ2_bzCompress</code> updates
cannam@89 1154 <code class="computeroutput">next_in</code>,
cannam@89 1155 <code class="computeroutput">avail_in</code> and
cannam@89 1156 <code class="computeroutput">total_in</code> to reflect the number
cannam@89 1157 of bytes it has read.</p>
cannam@89 1158 <p>Similarly, <code class="computeroutput">next_out</code> should
cannam@89 1159 point to a buffer in which the compressed data is to be placed,
cannam@89 1160 with <code class="computeroutput">avail_out</code> indicating how
cannam@89 1161 much output space is available.
cannam@89 1162 <code class="computeroutput">BZ2_bzCompress</code> updates
cannam@89 1163 <code class="computeroutput">next_out</code>,
cannam@89 1164 <code class="computeroutput">avail_out</code> and
cannam@89 1165 <code class="computeroutput">total_out</code> to reflect the number
cannam@89 1166 of bytes output.</p>
cannam@89 1167 <p>You may provide and remove as little or as much data as you
cannam@89 1168 like on each call of
cannam@89 1169 <code class="computeroutput">BZ2_bzCompress</code>. In the limit,
cannam@89 1170 it is acceptable to supply and remove data one byte at a time,
cannam@89 1171 although this would be terribly inefficient. You should always
cannam@89 1172 ensure that at least one byte of output space is available at
cannam@89 1173 each call.</p>
cannam@89 1174 <p>A second purpose of
cannam@89 1175 <code class="computeroutput">BZ2_bzCompress</code> is to request a
cannam@89 1176 change of mode of the compressed stream.</p>
cannam@89 1177 <p>Conceptually, a compressed stream can be in one of four
cannam@89 1178 states: IDLE, RUNNING, FLUSHING and FINISHING. Before
cannam@89 1179 initialisation
cannam@89 1180 (<code class="computeroutput">BZ2_bzCompressInit</code>) and after
cannam@89 1181 termination (<code class="computeroutput">BZ2_bzCompressEnd</code>),
cannam@89 1182 a stream is regarded as IDLE.</p>
cannam@89 1183 <p>Upon initialisation
cannam@89 1184 (<code class="computeroutput">BZ2_bzCompressInit</code>), the stream
cannam@89 1185 is placed in the RUNNING state. Subsequent calls to
cannam@89 1186 <code class="computeroutput">BZ2_bzCompress</code> should pass
cannam@89 1187 <code class="computeroutput">BZ_RUN</code> as the requested action;
cannam@89 1188 other actions are illegal and will result in
cannam@89 1189 <code class="computeroutput">BZ_SEQUENCE_ERROR</code>.</p>
cannam@89 1190 <p>At some point, the calling program will have provided all
cannam@89 1191 the input data it wants to. It will then want to finish up -- in
cannam@89 1192 effect, asking the library to process any data it might have
cannam@89 1193 buffered internally. In this state,
cannam@89 1194 <code class="computeroutput">BZ2_bzCompress</code> will no longer
cannam@89 1195 attempt to read data from
cannam@89 1196 <code class="computeroutput">next_in</code>, but it will want to
cannam@89 1197 write data to <code class="computeroutput">next_out</code>. Because
cannam@89 1198 the output buffer supplied by the user can be arbitrarily small,
cannam@89 1199 the finishing-up operation cannot necessarily be done with a
cannam@89 1200 single call of
cannam@89 1201 <code class="computeroutput">BZ2_bzCompress</code>.</p>
cannam@89 1202 <p>Instead, the calling program passes
cannam@89 1203 <code class="computeroutput">BZ_FINISH</code> as an action to
cannam@89 1204 <code class="computeroutput">BZ2_bzCompress</code>. This changes
cannam@89 1205 the stream's state to FINISHING. Any remaining input (ie,
cannam@89 1206 <code class="computeroutput">next_in[0 .. avail_in-1]</code>) is
cannam@89 1207 compressed and transferred to the output buffer. To do this,
cannam@89 1208 <code class="computeroutput">BZ2_bzCompress</code> must be called
cannam@89 1209 repeatedly until all the output has been consumed. At that
cannam@89 1210 point, <code class="computeroutput">BZ2_bzCompress</code> returns
cannam@89 1211 <code class="computeroutput">BZ_STREAM_END</code>, and the stream's
cannam@89 1212 state is set back to IDLE.
cannam@89 1213 <code class="computeroutput">BZ2_bzCompressEnd</code> should then be
cannam@89 1214 called.</p>
cannam@89 1215 <p>Just to make sure the calling program does not cheat, the
cannam@89 1216 library makes a note of <code class="computeroutput">avail_in</code>
cannam@89 1217 at the time of the first call to
cannam@89 1218 <code class="computeroutput">BZ2_bzCompress</code> which has
cannam@89 1219 <code class="computeroutput">BZ_FINISH</code> as an action (ie, at
cannam@89 1220 the time the program has announced its intention to not supply
cannam@89 1221 any more input). By comparing this value with that of
cannam@89 1222 <code class="computeroutput">avail_in</code> over subsequent calls
cannam@89 1223 to <code class="computeroutput">BZ2_bzCompress</code>, the library
cannam@89 1224 can detect any attempts to slip in more data to compress. Any
cannam@89 1225 calls for which this is detected will return
cannam@89 1226 <code class="computeroutput">BZ_SEQUENCE_ERROR</code>. This
cannam@89 1227 indicates a programming mistake which should be corrected.</p>
cannam@89 1228 <p>Instead of asking to finish, the calling program may ask
cannam@89 1229 <code class="computeroutput">BZ2_bzCompress</code> to take all the
cannam@89 1230 remaining input, compress it and terminate the current
cannam@89 1231 (Burrows-Wheeler) compression block. This could be useful for
cannam@89 1232 error control purposes. The mechanism is analogous to that for
cannam@89 1233 finishing: call <code class="computeroutput">BZ2_bzCompress</code>
cannam@89 1234 with an action of <code class="computeroutput">BZ_FLUSH</code>,
cannam@89 1235 remove output data, and persist with the
cannam@89 1236 <code class="computeroutput">BZ_FLUSH</code> action until the value
cannam@89 1237 <code class="computeroutput">BZ_RUN</code> is returned. As with
cannam@89 1238 finishing, <code class="computeroutput">BZ2_bzCompress</code>
cannam@89 1239 detects any attempt to provide more input data once the flush has
cannam@89 1240 begun.</p>
cannam@89 1241 <p>Once the flush is complete, the stream returns to the
cannam@89 1242 normal RUNNING state.</p>
cannam@89 1243 <p>This all sounds pretty complex, but isn't really. Here's a
cannam@89 1244 table which shows which actions are allowable in each state, what
cannam@89 1245 action will be taken, what the next state is, and what the
cannam@89 1246 non-error return values are. Note that you can't explicitly ask
cannam@89 1247 what state the stream is in, but nor do you need to -- it can be
cannam@89 1248 inferred from the values returned by
cannam@89 1249 <code class="computeroutput">BZ2_bzCompress</code>.</p>
cannam@89 1250 <pre class="programlisting">IDLE/any
cannam@89 1251 Illegal. IDLE state only exists after BZ2_bzCompressEnd or
cannam@89 1252 before BZ2_bzCompressInit.
cannam@89 1253 Return value = BZ_SEQUENCE_ERROR
cannam@89 1254
cannam@89 1255 RUNNING/BZ_RUN
cannam@89 1256 Compress from next_in to next_out as much as possible.
cannam@89 1257 Next state = RUNNING
cannam@89 1258 Return value = BZ_RUN_OK
cannam@89 1259
cannam@89 1260 RUNNING/BZ_FLUSH
cannam@89 1261 Remember current value of next_in. Compress from next_in
cannam@89 1262 to next_out as much as possible, but do not accept any more input.
cannam@89 1263 Next state = FLUSHING
cannam@89 1264 Return value = BZ_FLUSH_OK
cannam@89 1265
cannam@89 1266 RUNNING/BZ_FINISH
cannam@89 1267 Remember current value of next_in. Compress from next_in
cannam@89 1268 to next_out as much as possible, but do not accept any more input.
cannam@89 1269 Next state = FINISHING
cannam@89 1270 Return value = BZ_FINISH_OK
cannam@89 1271
cannam@89 1272 FLUSHING/BZ_FLUSH
cannam@89 1273 Compress from next_in to next_out as much as possible,
cannam@89 1274 but do not accept any more input.
cannam@89 1275 If all the existing input has been used up and all compressed
cannam@89 1276 output has been removed
cannam@89 1277 Next state = RUNNING; Return value = BZ_RUN_OK
cannam@89 1278 else
cannam@89 1279 Next state = FLUSHING; Return value = BZ_FLUSH_OK
cannam@89 1280
cannam@89 1281 FLUSHING/other
cannam@89 1282 Illegal.
cannam@89 1283 Return value = BZ_SEQUENCE_ERROR
cannam@89 1284
cannam@89 1285 FINISHING/BZ_FINISH
cannam@89 1286 Compress from next_in to next_out as much as possible,
cannam@89 1287 but to not accept any more input.
cannam@89 1288 If all the existing input has been used up and all compressed
cannam@89 1289 output has been removed
cannam@89 1290 Next state = IDLE; Return value = BZ_STREAM_END
cannam@89 1291 else
cannam@89 1292 Next state = FINISHING; Return value = BZ_FINISH_OK
cannam@89 1293
cannam@89 1294 FINISHING/other
cannam@89 1295 Illegal.
cannam@89 1296 Return value = BZ_SEQUENCE_ERROR</pre>
cannam@89 1297 <p>That still looks complicated? Well, fair enough. The
cannam@89 1298 usual sequence of calls for compressing a load of data is:</p>
cannam@89 1299 <div class="orderedlist"><ol class="orderedlist" type="1">
cannam@89 1300 <li class="listitem"><p>Get started with
cannam@89 1301 <code class="computeroutput">BZ2_bzCompressInit</code>.</p></li>
cannam@89 1302 <li class="listitem"><p>Shovel data in and shlurp out its compressed form
cannam@89 1303 using zero or more calls of
cannam@89 1304 <code class="computeroutput">BZ2_bzCompress</code> with action =
cannam@89 1305 <code class="computeroutput">BZ_RUN</code>.</p></li>
cannam@89 1306 <li class="listitem"><p>Finish up. Repeatedly call
cannam@89 1307 <code class="computeroutput">BZ2_bzCompress</code> with action =
cannam@89 1308 <code class="computeroutput">BZ_FINISH</code>, copying out the
cannam@89 1309 compressed output, until
cannam@89 1310 <code class="computeroutput">BZ_STREAM_END</code> is
cannam@89 1311 returned.</p></li>
cannam@89 1312 <li class="listitem"><p>Close up and go home. Call
cannam@89 1313 <code class="computeroutput">BZ2_bzCompressEnd</code>.</p></li>
cannam@89 1314 </ol></div>
cannam@89 1315 <p>If the data you want to compress fits into your input
cannam@89 1316 buffer all at once, you can skip the calls of
cannam@89 1317 <code class="computeroutput">BZ2_bzCompress ( ..., BZ_RUN )</code>
cannam@89 1318 and just do the <code class="computeroutput">BZ2_bzCompress ( ..., BZ_FINISH
cannam@89 1319 )</code> calls.</p>
cannam@89 1320 <p>All required memory is allocated by
cannam@89 1321 <code class="computeroutput">BZ2_bzCompressInit</code>. The
cannam@89 1322 compression library can accept any data at all (obviously). So
cannam@89 1323 you shouldn't get any error return values from the
cannam@89 1324 <code class="computeroutput">BZ2_bzCompress</code> calls. If you
cannam@89 1325 do, they will be
cannam@89 1326 <code class="computeroutput">BZ_SEQUENCE_ERROR</code>, and indicate
cannam@89 1327 a bug in your programming.</p>
cannam@89 1328 <p>Trivial other possible return values:</p>
cannam@89 1329 <pre class="programlisting">BZ_PARAM_ERROR
cannam@89 1330 if strm is NULL, or strm-&gt;s is NULL</pre>
cannam@89 1331 </div>
cannam@89 1332 <div class="sect2" title="3.3.3. BZ2_bzCompressEnd">
cannam@89 1333 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1334 <a name="bzCompress-end"></a>3.3.3. BZ2_bzCompressEnd</h3></div></div></div>
cannam@89 1335 <pre class="programlisting">int BZ2_bzCompressEnd ( bz_stream *strm );</pre>
cannam@89 1336 <p>Releases all memory associated with a compression
cannam@89 1337 stream.</p>
cannam@89 1338 <p>Possible return values:</p>
cannam@89 1339 <pre class="programlisting">BZ_PARAM_ERROR if strm is NULL or strm-&gt;s is NULL
cannam@89 1340 BZ_OK otherwise</pre>
cannam@89 1341 </div>
cannam@89 1342 <div class="sect2" title="3.3.4. BZ2_bzDecompressInit">
cannam@89 1343 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1344 <a name="bzDecompress-init"></a>3.3.4. BZ2_bzDecompressInit</h3></div></div></div>
cannam@89 1345 <pre class="programlisting">int BZ2_bzDecompressInit ( bz_stream *strm, int verbosity, int small );</pre>
cannam@89 1346 <p>Prepares for decompression. As with
cannam@89 1347 <code class="computeroutput">BZ2_bzCompressInit</code>, a
cannam@89 1348 <code class="computeroutput">bz_stream</code> record should be
cannam@89 1349 allocated and initialised before the call. Fields
cannam@89 1350 <code class="computeroutput">bzalloc</code>,
cannam@89 1351 <code class="computeroutput">bzfree</code> and
cannam@89 1352 <code class="computeroutput">opaque</code> should be set if a custom
cannam@89 1353 memory allocator is required, or made
cannam@89 1354 <code class="computeroutput">NULL</code> for the normal
cannam@89 1355 <code class="computeroutput">malloc</code> /
cannam@89 1356 <code class="computeroutput">free</code> routines. Upon return, the
cannam@89 1357 internal state will have been initialised, and
cannam@89 1358 <code class="computeroutput">total_in</code> and
cannam@89 1359 <code class="computeroutput">total_out</code> will be zero.</p>
cannam@89 1360 <p>For the meaning of parameter
cannam@89 1361 <code class="computeroutput">verbosity</code>, see
cannam@89 1362 <code class="computeroutput">BZ2_bzCompressInit</code>.</p>
cannam@89 1363 <p>If <code class="computeroutput">small</code> is nonzero, the
cannam@89 1364 library will use an alternative decompression algorithm which
cannam@89 1365 uses less memory but at the cost of decompressing more slowly
cannam@89 1366 (roughly speaking, half the speed, but the maximum memory
cannam@89 1367 requirement drops to around 2300k). See <a class="xref" href="#using" title="2. How to use bzip2">How to use bzip2</a>
cannam@89 1368 for more information on memory management.</p>
cannam@89 1369 <p>Note that the amount of memory needed to decompress a
cannam@89 1370 stream cannot be determined until the stream's header has been
cannam@89 1371 read, so even if
cannam@89 1372 <code class="computeroutput">BZ2_bzDecompressInit</code> succeeds, a
cannam@89 1373 subsequent <code class="computeroutput">BZ2_bzDecompress</code>
cannam@89 1374 could fail with
cannam@89 1375 <code class="computeroutput">BZ_MEM_ERROR</code>.</p>
cannam@89 1376 <p>Possible return values:</p>
cannam@89 1377 <pre class="programlisting">BZ_CONFIG_ERROR
cannam@89 1378 if the library has been mis-compiled
cannam@89 1379 BZ_PARAM_ERROR
cannam@89 1380 if ( small != 0 &amp;&amp; small != 1 )
cannam@89 1381 or (verbosity &lt;; 0 || verbosity &gt; 4)
cannam@89 1382 BZ_MEM_ERROR
cannam@89 1383 if insufficient memory is available</pre>
cannam@89 1384 <p>Allowable next actions:</p>
cannam@89 1385 <pre class="programlisting">BZ2_bzDecompress
cannam@89 1386 if BZ_OK was returned
cannam@89 1387 no specific action required in case of error</pre>
cannam@89 1388 </div>
cannam@89 1389 <div class="sect2" title="3.3.5. BZ2_bzDecompress">
cannam@89 1390 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1391 <a name="bzDecompress"></a>3.3.5. BZ2_bzDecompress</h3></div></div></div>
cannam@89 1392 <pre class="programlisting">int BZ2_bzDecompress ( bz_stream *strm );</pre>
cannam@89 1393 <p>Provides more input and/out output buffer space for the
cannam@89 1394 library. The caller maintains input and output buffers, and uses
cannam@89 1395 <code class="computeroutput">BZ2_bzDecompress</code> to transfer
cannam@89 1396 data between them.</p>
cannam@89 1397 <p>Before each call to
cannam@89 1398 <code class="computeroutput">BZ2_bzDecompress</code>,
cannam@89 1399 <code class="computeroutput">next_in</code> should point at the
cannam@89 1400 compressed data, and <code class="computeroutput">avail_in</code>
cannam@89 1401 should indicate how many bytes the library may read.
cannam@89 1402 <code class="computeroutput">BZ2_bzDecompress</code> updates
cannam@89 1403 <code class="computeroutput">next_in</code>,
cannam@89 1404 <code class="computeroutput">avail_in</code> and
cannam@89 1405 <code class="computeroutput">total_in</code> to reflect the number
cannam@89 1406 of bytes it has read.</p>
cannam@89 1407 <p>Similarly, <code class="computeroutput">next_out</code> should
cannam@89 1408 point to a buffer in which the uncompressed output is to be
cannam@89 1409 placed, with <code class="computeroutput">avail_out</code>
cannam@89 1410 indicating how much output space is available.
cannam@89 1411 <code class="computeroutput">BZ2_bzCompress</code> updates
cannam@89 1412 <code class="computeroutput">next_out</code>,
cannam@89 1413 <code class="computeroutput">avail_out</code> and
cannam@89 1414 <code class="computeroutput">total_out</code> to reflect the number
cannam@89 1415 of bytes output.</p>
cannam@89 1416 <p>You may provide and remove as little or as much data as you
cannam@89 1417 like on each call of
cannam@89 1418 <code class="computeroutput">BZ2_bzDecompress</code>. In the limit,
cannam@89 1419 it is acceptable to supply and remove data one byte at a time,
cannam@89 1420 although this would be terribly inefficient. You should always
cannam@89 1421 ensure that at least one byte of output space is available at
cannam@89 1422 each call.</p>
cannam@89 1423 <p>Use of <code class="computeroutput">BZ2_bzDecompress</code> is
cannam@89 1424 simpler than
cannam@89 1425 <code class="computeroutput">BZ2_bzCompress</code>.</p>
cannam@89 1426 <p>You should provide input and remove output as described
cannam@89 1427 above, and repeatedly call
cannam@89 1428 <code class="computeroutput">BZ2_bzDecompress</code> until
cannam@89 1429 <code class="computeroutput">BZ_STREAM_END</code> is returned.
cannam@89 1430 Appearance of <code class="computeroutput">BZ_STREAM_END</code>
cannam@89 1431 denotes that <code class="computeroutput">BZ2_bzDecompress</code>
cannam@89 1432 has detected the logical end of the compressed stream.
cannam@89 1433 <code class="computeroutput">BZ2_bzDecompress</code> will not
cannam@89 1434 produce <code class="computeroutput">BZ_STREAM_END</code> until all
cannam@89 1435 output data has been placed into the output buffer, so once
cannam@89 1436 <code class="computeroutput">BZ_STREAM_END</code> appears, you are
cannam@89 1437 guaranteed to have available all the decompressed output, and
cannam@89 1438 <code class="computeroutput">BZ2_bzDecompressEnd</code> can safely
cannam@89 1439 be called.</p>
cannam@89 1440 <p>If case of an error return value, you should call
cannam@89 1441 <code class="computeroutput">BZ2_bzDecompressEnd</code> to clean up
cannam@89 1442 and release memory.</p>
cannam@89 1443 <p>Possible return values:</p>
cannam@89 1444 <pre class="programlisting">BZ_PARAM_ERROR
cannam@89 1445 if strm is NULL or strm-&gt;s is NULL
cannam@89 1446 or strm-&gt;avail_out &lt; 1
cannam@89 1447 BZ_DATA_ERROR
cannam@89 1448 if a data integrity error is detected in the compressed stream
cannam@89 1449 BZ_DATA_ERROR_MAGIC
cannam@89 1450 if the compressed stream doesn't begin with the right magic bytes
cannam@89 1451 BZ_MEM_ERROR
cannam@89 1452 if there wasn't enough memory available
cannam@89 1453 BZ_STREAM_END
cannam@89 1454 if the logical end of the data stream was detected and all
cannam@89 1455 output in has been consumed, eg s--&gt;avail_out &gt; 0
cannam@89 1456 BZ_OK
cannam@89 1457 otherwise</pre>
cannam@89 1458 <p>Allowable next actions:</p>
cannam@89 1459 <pre class="programlisting">BZ2_bzDecompress
cannam@89 1460 if BZ_OK was returned
cannam@89 1461 BZ2_bzDecompressEnd
cannam@89 1462 otherwise</pre>
cannam@89 1463 </div>
cannam@89 1464 <div class="sect2" title="3.3.6. BZ2_bzDecompressEnd">
cannam@89 1465 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1466 <a name="bzDecompress-end"></a>3.3.6. BZ2_bzDecompressEnd</h3></div></div></div>
cannam@89 1467 <pre class="programlisting">int BZ2_bzDecompressEnd ( bz_stream *strm );</pre>
cannam@89 1468 <p>Releases all memory associated with a decompression
cannam@89 1469 stream.</p>
cannam@89 1470 <p>Possible return values:</p>
cannam@89 1471 <pre class="programlisting">BZ_PARAM_ERROR
cannam@89 1472 if strm is NULL or strm-&gt;s is NULL
cannam@89 1473 BZ_OK
cannam@89 1474 otherwise</pre>
cannam@89 1475 <p>Allowable next actions:</p>
cannam@89 1476 <pre class="programlisting"> None.</pre>
cannam@89 1477 </div>
cannam@89 1478 </div>
cannam@89 1479 <div class="sect1" title="3.4. High-level interface">
cannam@89 1480 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 1481 <a name="hl-interface"></a>3.4. High-level interface</h2></div></div></div>
cannam@89 1482 <p>This interface provides functions for reading and writing
cannam@89 1483 <code class="computeroutput">bzip2</code> format files. First, some
cannam@89 1484 general points.</p>
cannam@89 1485 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 1486 <li class="listitem" style="list-style-type: disc"><p>All of the functions take an
cannam@89 1487 <code class="computeroutput">int*</code> first argument,
cannam@89 1488 <code class="computeroutput">bzerror</code>. After each call,
cannam@89 1489 <code class="computeroutput">bzerror</code> should be consulted
cannam@89 1490 first to determine the outcome of the call. If
cannam@89 1491 <code class="computeroutput">bzerror</code> is
cannam@89 1492 <code class="computeroutput">BZ_OK</code>, the call completed
cannam@89 1493 successfully, and only then should the return value of the
cannam@89 1494 function (if any) be consulted. If
cannam@89 1495 <code class="computeroutput">bzerror</code> is
cannam@89 1496 <code class="computeroutput">BZ_IO_ERROR</code>, there was an
cannam@89 1497 error reading/writing the underlying compressed file, and you
cannam@89 1498 should then consult <code class="computeroutput">errno</code> /
cannam@89 1499 <code class="computeroutput">perror</code> to determine the cause
cannam@89 1500 of the difficulty. <code class="computeroutput">bzerror</code>
cannam@89 1501 may also be set to various other values; precise details are
cannam@89 1502 given on a per-function basis below.</p></li>
cannam@89 1503 <li class="listitem" style="list-style-type: disc"><p>If <code class="computeroutput">bzerror</code> indicates
cannam@89 1504 an error (ie, anything except
cannam@89 1505 <code class="computeroutput">BZ_OK</code> and
cannam@89 1506 <code class="computeroutput">BZ_STREAM_END</code>), you should
cannam@89 1507 immediately call
cannam@89 1508 <code class="computeroutput">BZ2_bzReadClose</code> (or
cannam@89 1509 <code class="computeroutput">BZ2_bzWriteClose</code>, depending on
cannam@89 1510 whether you are attempting to read or to write) to free up all
cannam@89 1511 resources associated with the stream. Once an error has been
cannam@89 1512 indicated, behaviour of all calls except
cannam@89 1513 <code class="computeroutput">BZ2_bzReadClose</code>
cannam@89 1514 (<code class="computeroutput">BZ2_bzWriteClose</code>) is
cannam@89 1515 undefined. The implication is that (1)
cannam@89 1516 <code class="computeroutput">bzerror</code> should be checked
cannam@89 1517 after each call, and (2) if
cannam@89 1518 <code class="computeroutput">bzerror</code> indicates an error,
cannam@89 1519 <code class="computeroutput">BZ2_bzReadClose</code>
cannam@89 1520 (<code class="computeroutput">BZ2_bzWriteClose</code>) should then
cannam@89 1521 be called to clean up.</p></li>
cannam@89 1522 <li class="listitem" style="list-style-type: disc"><p>The <code class="computeroutput">FILE*</code> arguments
cannam@89 1523 passed to <code class="computeroutput">BZ2_bzReadOpen</code> /
cannam@89 1524 <code class="computeroutput">BZ2_bzWriteOpen</code> should be set
cannam@89 1525 to binary mode. Most Unix systems will do this by default, but
cannam@89 1526 other platforms, including Windows and Mac, will not. If you
cannam@89 1527 omit this, you may encounter problems when moving code to new
cannam@89 1528 platforms.</p></li>
cannam@89 1529 <li class="listitem" style="list-style-type: disc"><p>Memory allocation requests are handled by
cannam@89 1530 <code class="computeroutput">malloc</code> /
cannam@89 1531 <code class="computeroutput">free</code>. At present there is no
cannam@89 1532 facility for user-defined memory allocators in the file I/O
cannam@89 1533 functions (could easily be added, though).</p></li>
cannam@89 1534 </ul></div>
cannam@89 1535 <div class="sect2" title="3.4.1. BZ2_bzReadOpen">
cannam@89 1536 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1537 <a name="bzreadopen"></a>3.4.1. BZ2_bzReadOpen</h3></div></div></div>
cannam@89 1538 <pre class="programlisting">typedef void BZFILE;
cannam@89 1539
cannam@89 1540 BZFILE *BZ2_bzReadOpen( int *bzerror, FILE *f,
cannam@89 1541 int verbosity, int small,
cannam@89 1542 void *unused, int nUnused );</pre>
cannam@89 1543 <p>Prepare to read compressed data from file handle
cannam@89 1544 <code class="computeroutput">f</code>.
cannam@89 1545 <code class="computeroutput">f</code> should refer to a file which
cannam@89 1546 has been opened for reading, and for which the error indicator
cannam@89 1547 (<code class="computeroutput">ferror(f)</code>)is not set. If
cannam@89 1548 <code class="computeroutput">small</code> is 1, the library will try
cannam@89 1549 to decompress using less memory, at the expense of speed.</p>
cannam@89 1550 <p>For reasons explained below,
cannam@89 1551 <code class="computeroutput">BZ2_bzRead</code> will decompress the
cannam@89 1552 <code class="computeroutput">nUnused</code> bytes starting at
cannam@89 1553 <code class="computeroutput">unused</code>, before starting to read
cannam@89 1554 from the file <code class="computeroutput">f</code>. At most
cannam@89 1555 <code class="computeroutput">BZ_MAX_UNUSED</code> bytes may be
cannam@89 1556 supplied like this. If this facility is not required, you should
cannam@89 1557 pass <code class="computeroutput">NULL</code> and
cannam@89 1558 <code class="computeroutput">0</code> for
cannam@89 1559 <code class="computeroutput">unused</code> and
cannam@89 1560 n<code class="computeroutput">Unused</code> respectively.</p>
cannam@89 1561 <p>For the meaning of parameters
cannam@89 1562 <code class="computeroutput">small</code> and
cannam@89 1563 <code class="computeroutput">verbosity</code>, see
cannam@89 1564 <code class="computeroutput">BZ2_bzDecompressInit</code>.</p>
cannam@89 1565 <p>The amount of memory needed to decompress a file cannot be
cannam@89 1566 determined until the file's header has been read. So it is
cannam@89 1567 possible that <code class="computeroutput">BZ2_bzReadOpen</code>
cannam@89 1568 returns <code class="computeroutput">BZ_OK</code> but a subsequent
cannam@89 1569 call of <code class="computeroutput">BZ2_bzRead</code> will return
cannam@89 1570 <code class="computeroutput">BZ_MEM_ERROR</code>.</p>
cannam@89 1571 <p>Possible assignments to
cannam@89 1572 <code class="computeroutput">bzerror</code>:</p>
cannam@89 1573 <pre class="programlisting">BZ_CONFIG_ERROR
cannam@89 1574 if the library has been mis-compiled
cannam@89 1575 BZ_PARAM_ERROR
cannam@89 1576 if f is NULL
cannam@89 1577 or small is neither 0 nor 1
cannam@89 1578 or ( unused == NULL &amp;&amp; nUnused != 0 )
cannam@89 1579 or ( unused != NULL &amp;&amp; !(0 &lt;= nUnused &lt;= BZ_MAX_UNUSED) )
cannam@89 1580 BZ_IO_ERROR
cannam@89 1581 if ferror(f) is nonzero
cannam@89 1582 BZ_MEM_ERROR
cannam@89 1583 if insufficient memory is available
cannam@89 1584 BZ_OK
cannam@89 1585 otherwise.</pre>
cannam@89 1586 <p>Possible return values:</p>
cannam@89 1587 <pre class="programlisting">Pointer to an abstract BZFILE
cannam@89 1588 if bzerror is BZ_OK
cannam@89 1589 NULL
cannam@89 1590 otherwise</pre>
cannam@89 1591 <p>Allowable next actions:</p>
cannam@89 1592 <pre class="programlisting">BZ2_bzRead
cannam@89 1593 if bzerror is BZ_OK
cannam@89 1594 BZ2_bzClose
cannam@89 1595 otherwise</pre>
cannam@89 1596 </div>
cannam@89 1597 <div class="sect2" title="3.4.2. BZ2_bzRead">
cannam@89 1598 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1599 <a name="bzread"></a>3.4.2. BZ2_bzRead</h3></div></div></div>
cannam@89 1600 <pre class="programlisting">int BZ2_bzRead ( int *bzerror, BZFILE *b, void *buf, int len );</pre>
cannam@89 1601 <p>Reads up to <code class="computeroutput">len</code>
cannam@89 1602 (uncompressed) bytes from the compressed file
cannam@89 1603 <code class="computeroutput">b</code> into the buffer
cannam@89 1604 <code class="computeroutput">buf</code>. If the read was
cannam@89 1605 successful, <code class="computeroutput">bzerror</code> is set to
cannam@89 1606 <code class="computeroutput">BZ_OK</code> and the number of bytes
cannam@89 1607 read is returned. If the logical end-of-stream was detected,
cannam@89 1608 <code class="computeroutput">bzerror</code> will be set to
cannam@89 1609 <code class="computeroutput">BZ_STREAM_END</code>, and the number of
cannam@89 1610 bytes read is returned. All other
cannam@89 1611 <code class="computeroutput">bzerror</code> values denote an
cannam@89 1612 error.</p>
cannam@89 1613 <p><code class="computeroutput">BZ2_bzRead</code> will supply
cannam@89 1614 <code class="computeroutput">len</code> bytes, unless the logical
cannam@89 1615 stream end is detected or an error occurs. Because of this, it
cannam@89 1616 is possible to detect the stream end by observing when the number
cannam@89 1617 of bytes returned is less than the number requested.
cannam@89 1618 Nevertheless, this is regarded as inadvisable; you should instead
cannam@89 1619 check <code class="computeroutput">bzerror</code> after every call
cannam@89 1620 and watch out for
cannam@89 1621 <code class="computeroutput">BZ_STREAM_END</code>.</p>
cannam@89 1622 <p>Internally, <code class="computeroutput">BZ2_bzRead</code>
cannam@89 1623 copies data from the compressed file in chunks of size
cannam@89 1624 <code class="computeroutput">BZ_MAX_UNUSED</code> bytes before
cannam@89 1625 decompressing it. If the file contains more bytes than strictly
cannam@89 1626 needed to reach the logical end-of-stream,
cannam@89 1627 <code class="computeroutput">BZ2_bzRead</code> will almost certainly
cannam@89 1628 read some of the trailing data before signalling
cannam@89 1629 <code class="computeroutput">BZ_SEQUENCE_END</code>. To collect the
cannam@89 1630 read but unused data once
cannam@89 1631 <code class="computeroutput">BZ_SEQUENCE_END</code> has appeared,
cannam@89 1632 call <code class="computeroutput">BZ2_bzReadGetUnused</code>
cannam@89 1633 immediately before
cannam@89 1634 <code class="computeroutput">BZ2_bzReadClose</code>.</p>
cannam@89 1635 <p>Possible assignments to
cannam@89 1636 <code class="computeroutput">bzerror</code>:</p>
cannam@89 1637 <pre class="programlisting">BZ_PARAM_ERROR
cannam@89 1638 if b is NULL or buf is NULL or len &lt; 0
cannam@89 1639 BZ_SEQUENCE_ERROR
cannam@89 1640 if b was opened with BZ2_bzWriteOpen
cannam@89 1641 BZ_IO_ERROR
cannam@89 1642 if there is an error reading from the compressed file
cannam@89 1643 BZ_UNEXPECTED_EOF
cannam@89 1644 if the compressed file ended before
cannam@89 1645 the logical end-of-stream was detected
cannam@89 1646 BZ_DATA_ERROR
cannam@89 1647 if a data integrity error was detected in the compressed stream
cannam@89 1648 BZ_DATA_ERROR_MAGIC
cannam@89 1649 if the stream does not begin with the requisite header bytes
cannam@89 1650 (ie, is not a bzip2 data file). This is really
cannam@89 1651 a special case of BZ_DATA_ERROR.
cannam@89 1652 BZ_MEM_ERROR
cannam@89 1653 if insufficient memory was available
cannam@89 1654 BZ_STREAM_END
cannam@89 1655 if the logical end of stream was detected.
cannam@89 1656 BZ_OK
cannam@89 1657 otherwise.</pre>
cannam@89 1658 <p>Possible return values:</p>
cannam@89 1659 <pre class="programlisting">number of bytes read
cannam@89 1660 if bzerror is BZ_OK or BZ_STREAM_END
cannam@89 1661 undefined
cannam@89 1662 otherwise</pre>
cannam@89 1663 <p>Allowable next actions:</p>
cannam@89 1664 <pre class="programlisting">collect data from buf, then BZ2_bzRead or BZ2_bzReadClose
cannam@89 1665 if bzerror is BZ_OK
cannam@89 1666 collect data from buf, then BZ2_bzReadClose or BZ2_bzReadGetUnused
cannam@89 1667 if bzerror is BZ_SEQUENCE_END
cannam@89 1668 BZ2_bzReadClose
cannam@89 1669 otherwise</pre>
cannam@89 1670 </div>
cannam@89 1671 <div class="sect2" title="3.4.3. BZ2_bzReadGetUnused">
cannam@89 1672 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1673 <a name="bzreadgetunused"></a>3.4.3. BZ2_bzReadGetUnused</h3></div></div></div>
cannam@89 1674 <pre class="programlisting">void BZ2_bzReadGetUnused( int* bzerror, BZFILE *b,
cannam@89 1675 void** unused, int* nUnused );</pre>
cannam@89 1676 <p>Returns data which was read from the compressed file but
cannam@89 1677 was not needed to get to the logical end-of-stream.
cannam@89 1678 <code class="computeroutput">*unused</code> is set to the address of
cannam@89 1679 the data, and <code class="computeroutput">*nUnused</code> to the
cannam@89 1680 number of bytes. <code class="computeroutput">*nUnused</code> will
cannam@89 1681 be set to a value between <code class="computeroutput">0</code> and
cannam@89 1682 <code class="computeroutput">BZ_MAX_UNUSED</code> inclusive.</p>
cannam@89 1683 <p>This function may only be called once
cannam@89 1684 <code class="computeroutput">BZ2_bzRead</code> has signalled
cannam@89 1685 <code class="computeroutput">BZ_STREAM_END</code> but before
cannam@89 1686 <code class="computeroutput">BZ2_bzReadClose</code>.</p>
cannam@89 1687 <p>Possible assignments to
cannam@89 1688 <code class="computeroutput">bzerror</code>:</p>
cannam@89 1689 <pre class="programlisting">BZ_PARAM_ERROR
cannam@89 1690 if b is NULL
cannam@89 1691 or unused is NULL or nUnused is NULL
cannam@89 1692 BZ_SEQUENCE_ERROR
cannam@89 1693 if BZ_STREAM_END has not been signalled
cannam@89 1694 or if b was opened with BZ2_bzWriteOpen
cannam@89 1695 BZ_OK
cannam@89 1696 otherwise</pre>
cannam@89 1697 <p>Allowable next actions:</p>
cannam@89 1698 <pre class="programlisting">BZ2_bzReadClose</pre>
cannam@89 1699 </div>
cannam@89 1700 <div class="sect2" title="3.4.4. BZ2_bzReadClose">
cannam@89 1701 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1702 <a name="bzreadclose"></a>3.4.4. BZ2_bzReadClose</h3></div></div></div>
cannam@89 1703 <pre class="programlisting">void BZ2_bzReadClose ( int *bzerror, BZFILE *b );</pre>
cannam@89 1704 <p>Releases all memory pertaining to the compressed file
cannam@89 1705 <code class="computeroutput">b</code>.
cannam@89 1706 <code class="computeroutput">BZ2_bzReadClose</code> does not call
cannam@89 1707 <code class="computeroutput">fclose</code> on the underlying file
cannam@89 1708 handle, so you should do that yourself if appropriate.
cannam@89 1709 <code class="computeroutput">BZ2_bzReadClose</code> should be called
cannam@89 1710 to clean up after all error situations.</p>
cannam@89 1711 <p>Possible assignments to
cannam@89 1712 <code class="computeroutput">bzerror</code>:</p>
cannam@89 1713 <pre class="programlisting">BZ_SEQUENCE_ERROR
cannam@89 1714 if b was opened with BZ2_bzOpenWrite
cannam@89 1715 BZ_OK
cannam@89 1716 otherwise</pre>
cannam@89 1717 <p>Allowable next actions:</p>
cannam@89 1718 <pre class="programlisting">none</pre>
cannam@89 1719 </div>
cannam@89 1720 <div class="sect2" title="3.4.5. BZ2_bzWriteOpen">
cannam@89 1721 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1722 <a name="bzwriteopen"></a>3.4.5. BZ2_bzWriteOpen</h3></div></div></div>
cannam@89 1723 <pre class="programlisting">BZFILE *BZ2_bzWriteOpen( int *bzerror, FILE *f,
cannam@89 1724 int blockSize100k, int verbosity,
cannam@89 1725 int workFactor );</pre>
cannam@89 1726 <p>Prepare to write compressed data to file handle
cannam@89 1727 <code class="computeroutput">f</code>.
cannam@89 1728 <code class="computeroutput">f</code> should refer to a file which
cannam@89 1729 has been opened for writing, and for which the error indicator
cannam@89 1730 (<code class="computeroutput">ferror(f)</code>)is not set.</p>
cannam@89 1731 <p>For the meaning of parameters
cannam@89 1732 <code class="computeroutput">blockSize100k</code>,
cannam@89 1733 <code class="computeroutput">verbosity</code> and
cannam@89 1734 <code class="computeroutput">workFactor</code>, see
cannam@89 1735 <code class="computeroutput">BZ2_bzCompressInit</code>.</p>
cannam@89 1736 <p>All required memory is allocated at this stage, so if the
cannam@89 1737 call completes successfully,
cannam@89 1738 <code class="computeroutput">BZ_MEM_ERROR</code> cannot be signalled
cannam@89 1739 by a subsequent call to
cannam@89 1740 <code class="computeroutput">BZ2_bzWrite</code>.</p>
cannam@89 1741 <p>Possible assignments to
cannam@89 1742 <code class="computeroutput">bzerror</code>:</p>
cannam@89 1743 <pre class="programlisting">BZ_CONFIG_ERROR
cannam@89 1744 if the library has been mis-compiled
cannam@89 1745 BZ_PARAM_ERROR
cannam@89 1746 if f is NULL
cannam@89 1747 or blockSize100k &lt; 1 or blockSize100k &gt; 9
cannam@89 1748 BZ_IO_ERROR
cannam@89 1749 if ferror(f) is nonzero
cannam@89 1750 BZ_MEM_ERROR
cannam@89 1751 if insufficient memory is available
cannam@89 1752 BZ_OK
cannam@89 1753 otherwise</pre>
cannam@89 1754 <p>Possible return values:</p>
cannam@89 1755 <pre class="programlisting">Pointer to an abstract BZFILE
cannam@89 1756 if bzerror is BZ_OK
cannam@89 1757 NULL
cannam@89 1758 otherwise</pre>
cannam@89 1759 <p>Allowable next actions:</p>
cannam@89 1760 <pre class="programlisting">BZ2_bzWrite
cannam@89 1761 if bzerror is BZ_OK
cannam@89 1762 (you could go directly to BZ2_bzWriteClose, but this would be pretty pointless)
cannam@89 1763 BZ2_bzWriteClose
cannam@89 1764 otherwise</pre>
cannam@89 1765 </div>
cannam@89 1766 <div class="sect2" title="3.4.6. BZ2_bzWrite">
cannam@89 1767 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1768 <a name="bzwrite"></a>3.4.6. BZ2_bzWrite</h3></div></div></div>
cannam@89 1769 <pre class="programlisting">void BZ2_bzWrite ( int *bzerror, BZFILE *b, void *buf, int len );</pre>
cannam@89 1770 <p>Absorbs <code class="computeroutput">len</code> bytes from the
cannam@89 1771 buffer <code class="computeroutput">buf</code>, eventually to be
cannam@89 1772 compressed and written to the file.</p>
cannam@89 1773 <p>Possible assignments to
cannam@89 1774 <code class="computeroutput">bzerror</code>:</p>
cannam@89 1775 <pre class="programlisting">BZ_PARAM_ERROR
cannam@89 1776 if b is NULL or buf is NULL or len &lt; 0
cannam@89 1777 BZ_SEQUENCE_ERROR
cannam@89 1778 if b was opened with BZ2_bzReadOpen
cannam@89 1779 BZ_IO_ERROR
cannam@89 1780 if there is an error writing the compressed file.
cannam@89 1781 BZ_OK
cannam@89 1782 otherwise</pre>
cannam@89 1783 </div>
cannam@89 1784 <div class="sect2" title="3.4.7. BZ2_bzWriteClose">
cannam@89 1785 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1786 <a name="bzwriteclose"></a>3.4.7. BZ2_bzWriteClose</h3></div></div></div>
cannam@89 1787 <pre class="programlisting">void BZ2_bzWriteClose( int *bzerror, BZFILE* f,
cannam@89 1788 int abandon,
cannam@89 1789 unsigned int* nbytes_in,
cannam@89 1790 unsigned int* nbytes_out );
cannam@89 1791
cannam@89 1792 void BZ2_bzWriteClose64( int *bzerror, BZFILE* f,
cannam@89 1793 int abandon,
cannam@89 1794 unsigned int* nbytes_in_lo32,
cannam@89 1795 unsigned int* nbytes_in_hi32,
cannam@89 1796 unsigned int* nbytes_out_lo32,
cannam@89 1797 unsigned int* nbytes_out_hi32 );</pre>
cannam@89 1798 <p>Compresses and flushes to the compressed file all data so
cannam@89 1799 far supplied by <code class="computeroutput">BZ2_bzWrite</code>.
cannam@89 1800 The logical end-of-stream markers are also written, so subsequent
cannam@89 1801 calls to <code class="computeroutput">BZ2_bzWrite</code> are
cannam@89 1802 illegal. All memory associated with the compressed file
cannam@89 1803 <code class="computeroutput">b</code> is released.
cannam@89 1804 <code class="computeroutput">fflush</code> is called on the
cannam@89 1805 compressed file, but it is not
cannam@89 1806 <code class="computeroutput">fclose</code>'d.</p>
cannam@89 1807 <p>If <code class="computeroutput">BZ2_bzWriteClose</code> is
cannam@89 1808 called to clean up after an error, the only action is to release
cannam@89 1809 the memory. The library records the error codes issued by
cannam@89 1810 previous calls, so this situation will be detected automatically.
cannam@89 1811 There is no attempt to complete the compression operation, nor to
cannam@89 1812 <code class="computeroutput">fflush</code> the compressed file. You
cannam@89 1813 can force this behaviour to happen even in the case of no error,
cannam@89 1814 by passing a nonzero value to
cannam@89 1815 <code class="computeroutput">abandon</code>.</p>
cannam@89 1816 <p>If <code class="computeroutput">nbytes_in</code> is non-null,
cannam@89 1817 <code class="computeroutput">*nbytes_in</code> will be set to be the
cannam@89 1818 total volume of uncompressed data handled. Similarly,
cannam@89 1819 <code class="computeroutput">nbytes_out</code> will be set to the
cannam@89 1820 total volume of compressed data written. For compatibility with
cannam@89 1821 older versions of the library,
cannam@89 1822 <code class="computeroutput">BZ2_bzWriteClose</code> only yields the
cannam@89 1823 lower 32 bits of these counts. Use
cannam@89 1824 <code class="computeroutput">BZ2_bzWriteClose64</code> if you want
cannam@89 1825 the full 64 bit counts. These two functions are otherwise
cannam@89 1826 absolutely identical.</p>
cannam@89 1827 <p>Possible assignments to
cannam@89 1828 <code class="computeroutput">bzerror</code>:</p>
cannam@89 1829 <pre class="programlisting">BZ_SEQUENCE_ERROR
cannam@89 1830 if b was opened with BZ2_bzReadOpen
cannam@89 1831 BZ_IO_ERROR
cannam@89 1832 if there is an error writing the compressed file
cannam@89 1833 BZ_OK
cannam@89 1834 otherwise</pre>
cannam@89 1835 </div>
cannam@89 1836 <div class="sect2" title="3.4.8. Handling embedded compressed data streams">
cannam@89 1837 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1838 <a name="embed"></a>3.4.8. Handling embedded compressed data streams</h3></div></div></div>
cannam@89 1839 <p>The high-level library facilitates use of
cannam@89 1840 <code class="computeroutput">bzip2</code> data streams which form
cannam@89 1841 some part of a surrounding, larger data stream.</p>
cannam@89 1842 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 1843 <li class="listitem" style="list-style-type: disc"><p>For writing, the library takes an open file handle,
cannam@89 1844 writes compressed data to it,
cannam@89 1845 <code class="computeroutput">fflush</code>es it but does not
cannam@89 1846 <code class="computeroutput">fclose</code> it. The calling
cannam@89 1847 application can write its own data before and after the
cannam@89 1848 compressed data stream, using that same file handle.</p></li>
cannam@89 1849 <li class="listitem" style="list-style-type: disc"><p>Reading is more complex, and the facilities are not as
cannam@89 1850 general as they could be since generality is hard to reconcile
cannam@89 1851 with efficiency. <code class="computeroutput">BZ2_bzRead</code>
cannam@89 1852 reads from the compressed file in blocks of size
cannam@89 1853 <code class="computeroutput">BZ_MAX_UNUSED</code> bytes, and in
cannam@89 1854 doing so probably will overshoot the logical end of compressed
cannam@89 1855 stream. To recover this data once decompression has ended,
cannam@89 1856 call <code class="computeroutput">BZ2_bzReadGetUnused</code> after
cannam@89 1857 the last call of <code class="computeroutput">BZ2_bzRead</code>
cannam@89 1858 (the one returning
cannam@89 1859 <code class="computeroutput">BZ_STREAM_END</code>) but before
cannam@89 1860 calling
cannam@89 1861 <code class="computeroutput">BZ2_bzReadClose</code>.</p></li>
cannam@89 1862 </ul></div>
cannam@89 1863 <p>This mechanism makes it easy to decompress multiple
cannam@89 1864 <code class="computeroutput">bzip2</code> streams placed end-to-end.
cannam@89 1865 As the end of one stream, when
cannam@89 1866 <code class="computeroutput">BZ2_bzRead</code> returns
cannam@89 1867 <code class="computeroutput">BZ_STREAM_END</code>, call
cannam@89 1868 <code class="computeroutput">BZ2_bzReadGetUnused</code> to collect
cannam@89 1869 the unused data (copy it into your own buffer somewhere). That
cannam@89 1870 data forms the start of the next compressed stream. To start
cannam@89 1871 uncompressing that next stream, call
cannam@89 1872 <code class="computeroutput">BZ2_bzReadOpen</code> again, feeding in
cannam@89 1873 the unused data via the <code class="computeroutput">unused</code> /
cannam@89 1874 <code class="computeroutput">nUnused</code> parameters. Keep doing
cannam@89 1875 this until <code class="computeroutput">BZ_STREAM_END</code> return
cannam@89 1876 coincides with the physical end of file
cannam@89 1877 (<code class="computeroutput">feof(f)</code>). In this situation
cannam@89 1878 <code class="computeroutput">BZ2_bzReadGetUnused</code> will of
cannam@89 1879 course return no data.</p>
cannam@89 1880 <p>This should give some feel for how the high-level interface
cannam@89 1881 can be used. If you require extra flexibility, you'll have to
cannam@89 1882 bite the bullet and get to grips with the low-level
cannam@89 1883 interface.</p>
cannam@89 1884 </div>
cannam@89 1885 <div class="sect2" title="3.4.9. Standard file-reading/writing code">
cannam@89 1886 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1887 <a name="std-rdwr"></a>3.4.9. Standard file-reading/writing code</h3></div></div></div>
cannam@89 1888 <p>Here's how you'd write data to a compressed file:</p>
cannam@89 1889 <pre class="programlisting">FILE* f;
cannam@89 1890 BZFILE* b;
cannam@89 1891 int nBuf;
cannam@89 1892 char buf[ /* whatever size you like */ ];
cannam@89 1893 int bzerror;
cannam@89 1894 int nWritten;
cannam@89 1895
cannam@89 1896 f = fopen ( "myfile.bz2", "w" );
cannam@89 1897 if ( !f ) {
cannam@89 1898 /* handle error */
cannam@89 1899 }
cannam@89 1900 b = BZ2_bzWriteOpen( &amp;bzerror, f, 9 );
cannam@89 1901 if (bzerror != BZ_OK) {
cannam@89 1902 BZ2_bzWriteClose ( b );
cannam@89 1903 /* handle error */
cannam@89 1904 }
cannam@89 1905
cannam@89 1906 while ( /* condition */ ) {
cannam@89 1907 /* get data to write into buf, and set nBuf appropriately */
cannam@89 1908 nWritten = BZ2_bzWrite ( &amp;bzerror, b, buf, nBuf );
cannam@89 1909 if (bzerror == BZ_IO_ERROR) {
cannam@89 1910 BZ2_bzWriteClose ( &amp;bzerror, b );
cannam@89 1911 /* handle error */
cannam@89 1912 }
cannam@89 1913 }
cannam@89 1914
cannam@89 1915 BZ2_bzWriteClose( &amp;bzerror, b );
cannam@89 1916 if (bzerror == BZ_IO_ERROR) {
cannam@89 1917 /* handle error */
cannam@89 1918 }</pre>
cannam@89 1919 <p>And to read from a compressed file:</p>
cannam@89 1920 <pre class="programlisting">FILE* f;
cannam@89 1921 BZFILE* b;
cannam@89 1922 int nBuf;
cannam@89 1923 char buf[ /* whatever size you like */ ];
cannam@89 1924 int bzerror;
cannam@89 1925 int nWritten;
cannam@89 1926
cannam@89 1927 f = fopen ( "myfile.bz2", "r" );
cannam@89 1928 if ( !f ) {
cannam@89 1929 /* handle error */
cannam@89 1930 }
cannam@89 1931 b = BZ2_bzReadOpen ( &amp;bzerror, f, 0, NULL, 0 );
cannam@89 1932 if ( bzerror != BZ_OK ) {
cannam@89 1933 BZ2_bzReadClose ( &amp;bzerror, b );
cannam@89 1934 /* handle error */
cannam@89 1935 }
cannam@89 1936
cannam@89 1937 bzerror = BZ_OK;
cannam@89 1938 while ( bzerror == BZ_OK &amp;&amp; /* arbitrary other conditions */) {
cannam@89 1939 nBuf = BZ2_bzRead ( &amp;bzerror, b, buf, /* size of buf */ );
cannam@89 1940 if ( bzerror == BZ_OK ) {
cannam@89 1941 /* do something with buf[0 .. nBuf-1] */
cannam@89 1942 }
cannam@89 1943 }
cannam@89 1944 if ( bzerror != BZ_STREAM_END ) {
cannam@89 1945 BZ2_bzReadClose ( &amp;bzerror, b );
cannam@89 1946 /* handle error */
cannam@89 1947 } else {
cannam@89 1948 BZ2_bzReadClose ( &amp;bzerror, b );
cannam@89 1949 }</pre>
cannam@89 1950 </div>
cannam@89 1951 </div>
cannam@89 1952 <div class="sect1" title="3.5. Utility functions">
cannam@89 1953 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 1954 <a name="util-fns"></a>3.5. Utility functions</h2></div></div></div>
cannam@89 1955 <div class="sect2" title="3.5.1. BZ2_bzBuffToBuffCompress">
cannam@89 1956 <div class="titlepage"><div><div><h3 class="title">
cannam@89 1957 <a name="bzbufftobuffcompress"></a>3.5.1. BZ2_bzBuffToBuffCompress</h3></div></div></div>
cannam@89 1958 <pre class="programlisting">int BZ2_bzBuffToBuffCompress( char* dest,
cannam@89 1959 unsigned int* destLen,
cannam@89 1960 char* source,
cannam@89 1961 unsigned int sourceLen,
cannam@89 1962 int blockSize100k,
cannam@89 1963 int verbosity,
cannam@89 1964 int workFactor );</pre>
cannam@89 1965 <p>Attempts to compress the data in <code class="computeroutput">source[0
cannam@89 1966 .. sourceLen-1]</code> into the destination buffer,
cannam@89 1967 <code class="computeroutput">dest[0 .. *destLen-1]</code>. If the
cannam@89 1968 destination buffer is big enough,
cannam@89 1969 <code class="computeroutput">*destLen</code> is set to the size of
cannam@89 1970 the compressed data, and <code class="computeroutput">BZ_OK</code>
cannam@89 1971 is returned. If the compressed data won't fit,
cannam@89 1972 <code class="computeroutput">*destLen</code> is unchanged, and
cannam@89 1973 <code class="computeroutput">BZ_OUTBUFF_FULL</code> is
cannam@89 1974 returned.</p>
cannam@89 1975 <p>Compression in this manner is a one-shot event, done with a
cannam@89 1976 single call to this function. The resulting compressed data is a
cannam@89 1977 complete <code class="computeroutput">bzip2</code> format data
cannam@89 1978 stream. There is no mechanism for making additional calls to
cannam@89 1979 provide extra input data. If you want that kind of mechanism,
cannam@89 1980 use the low-level interface.</p>
cannam@89 1981 <p>For the meaning of parameters
cannam@89 1982 <code class="computeroutput">blockSize100k</code>,
cannam@89 1983 <code class="computeroutput">verbosity</code> and
cannam@89 1984 <code class="computeroutput">workFactor</code>, see
cannam@89 1985 <code class="computeroutput">BZ2_bzCompressInit</code>.</p>
cannam@89 1986 <p>To guarantee that the compressed data will fit in its
cannam@89 1987 buffer, allocate an output buffer of size 1% larger than the
cannam@89 1988 uncompressed data, plus six hundred extra bytes.</p>
cannam@89 1989 <p><code class="computeroutput">BZ2_bzBuffToBuffDecompress</code>
cannam@89 1990 will not write data at or beyond
cannam@89 1991 <code class="computeroutput">dest[*destLen]</code>, even in case of
cannam@89 1992 buffer overflow.</p>
cannam@89 1993 <p>Possible return values:</p>
cannam@89 1994 <pre class="programlisting">BZ_CONFIG_ERROR
cannam@89 1995 if the library has been mis-compiled
cannam@89 1996 BZ_PARAM_ERROR
cannam@89 1997 if dest is NULL or destLen is NULL
cannam@89 1998 or blockSize100k &lt; 1 or blockSize100k &gt; 9
cannam@89 1999 or verbosity &lt; 0 or verbosity &gt; 4
cannam@89 2000 or workFactor &lt; 0 or workFactor &gt; 250
cannam@89 2001 BZ_MEM_ERROR
cannam@89 2002 if insufficient memory is available
cannam@89 2003 BZ_OUTBUFF_FULL
cannam@89 2004 if the size of the compressed data exceeds *destLen
cannam@89 2005 BZ_OK
cannam@89 2006 otherwise</pre>
cannam@89 2007 </div>
cannam@89 2008 <div class="sect2" title="3.5.2. BZ2_bzBuffToBuffDecompress">
cannam@89 2009 <div class="titlepage"><div><div><h3 class="title">
cannam@89 2010 <a name="bzbufftobuffdecompress"></a>3.5.2. BZ2_bzBuffToBuffDecompress</h3></div></div></div>
cannam@89 2011 <pre class="programlisting">int BZ2_bzBuffToBuffDecompress( char* dest,
cannam@89 2012 unsigned int* destLen,
cannam@89 2013 char* source,
cannam@89 2014 unsigned int sourceLen,
cannam@89 2015 int small,
cannam@89 2016 int verbosity );</pre>
cannam@89 2017 <p>Attempts to decompress the data in <code class="computeroutput">source[0
cannam@89 2018 .. sourceLen-1]</code> into the destination buffer,
cannam@89 2019 <code class="computeroutput">dest[0 .. *destLen-1]</code>. If the
cannam@89 2020 destination buffer is big enough,
cannam@89 2021 <code class="computeroutput">*destLen</code> is set to the size of
cannam@89 2022 the uncompressed data, and <code class="computeroutput">BZ_OK</code>
cannam@89 2023 is returned. If the compressed data won't fit,
cannam@89 2024 <code class="computeroutput">*destLen</code> is unchanged, and
cannam@89 2025 <code class="computeroutput">BZ_OUTBUFF_FULL</code> is
cannam@89 2026 returned.</p>
cannam@89 2027 <p><code class="computeroutput">source</code> is assumed to hold
cannam@89 2028 a complete <code class="computeroutput">bzip2</code> format data
cannam@89 2029 stream.
cannam@89 2030 <code class="computeroutput">BZ2_bzBuffToBuffDecompress</code> tries
cannam@89 2031 to decompress the entirety of the stream into the output
cannam@89 2032 buffer.</p>
cannam@89 2033 <p>For the meaning of parameters
cannam@89 2034 <code class="computeroutput">small</code> and
cannam@89 2035 <code class="computeroutput">verbosity</code>, see
cannam@89 2036 <code class="computeroutput">BZ2_bzDecompressInit</code>.</p>
cannam@89 2037 <p>Because the compression ratio of the compressed data cannot
cannam@89 2038 be known in advance, there is no easy way to guarantee that the
cannam@89 2039 output buffer will be big enough. You may of course make
cannam@89 2040 arrangements in your code to record the size of the uncompressed
cannam@89 2041 data, but such a mechanism is beyond the scope of this
cannam@89 2042 library.</p>
cannam@89 2043 <p><code class="computeroutput">BZ2_bzBuffToBuffDecompress</code>
cannam@89 2044 will not write data at or beyond
cannam@89 2045 <code class="computeroutput">dest[*destLen]</code>, even in case of
cannam@89 2046 buffer overflow.</p>
cannam@89 2047 <p>Possible return values:</p>
cannam@89 2048 <pre class="programlisting">BZ_CONFIG_ERROR
cannam@89 2049 if the library has been mis-compiled
cannam@89 2050 BZ_PARAM_ERROR
cannam@89 2051 if dest is NULL or destLen is NULL
cannam@89 2052 or small != 0 &amp;&amp; small != 1
cannam@89 2053 or verbosity &lt; 0 or verbosity &gt; 4
cannam@89 2054 BZ_MEM_ERROR
cannam@89 2055 if insufficient memory is available
cannam@89 2056 BZ_OUTBUFF_FULL
cannam@89 2057 if the size of the compressed data exceeds *destLen
cannam@89 2058 BZ_DATA_ERROR
cannam@89 2059 if a data integrity error was detected in the compressed data
cannam@89 2060 BZ_DATA_ERROR_MAGIC
cannam@89 2061 if the compressed data doesn't begin with the right magic bytes
cannam@89 2062 BZ_UNEXPECTED_EOF
cannam@89 2063 if the compressed data ends unexpectedly
cannam@89 2064 BZ_OK
cannam@89 2065 otherwise</pre>
cannam@89 2066 </div>
cannam@89 2067 </div>
cannam@89 2068 <div class="sect1" title="3.6. zlib compatibility functions">
cannam@89 2069 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 2070 <a name="zlib-compat"></a>3.6. zlib compatibility functions</h2></div></div></div>
cannam@89 2071 <p>Yoshioka Tsuneo has contributed some functions to give
cannam@89 2072 better <code class="computeroutput">zlib</code> compatibility.
cannam@89 2073 These functions are <code class="computeroutput">BZ2_bzopen</code>,
cannam@89 2074 <code class="computeroutput">BZ2_bzread</code>,
cannam@89 2075 <code class="computeroutput">BZ2_bzwrite</code>,
cannam@89 2076 <code class="computeroutput">BZ2_bzflush</code>,
cannam@89 2077 <code class="computeroutput">BZ2_bzclose</code>,
cannam@89 2078 <code class="computeroutput">BZ2_bzerror</code> and
cannam@89 2079 <code class="computeroutput">BZ2_bzlibVersion</code>. These
cannam@89 2080 functions are not (yet) officially part of the library. If they
cannam@89 2081 break, you get to keep all the pieces. Nevertheless, I think
cannam@89 2082 they work ok.</p>
cannam@89 2083 <pre class="programlisting">typedef void BZFILE;
cannam@89 2084
cannam@89 2085 const char * BZ2_bzlibVersion ( void );</pre>
cannam@89 2086 <p>Returns a string indicating the library version.</p>
cannam@89 2087 <pre class="programlisting">BZFILE * BZ2_bzopen ( const char *path, const char *mode );
cannam@89 2088 BZFILE * BZ2_bzdopen ( int fd, const char *mode );</pre>
cannam@89 2089 <p>Opens a <code class="computeroutput">.bz2</code> file for
cannam@89 2090 reading or writing, using either its name or a pre-existing file
cannam@89 2091 descriptor. Analogous to <code class="computeroutput">fopen</code>
cannam@89 2092 and <code class="computeroutput">fdopen</code>.</p>
cannam@89 2093 <pre class="programlisting">int BZ2_bzread ( BZFILE* b, void* buf, int len );
cannam@89 2094 int BZ2_bzwrite ( BZFILE* b, void* buf, int len );</pre>
cannam@89 2095 <p>Reads/writes data from/to a previously opened
cannam@89 2096 <code class="computeroutput">BZFILE</code>. Analogous to
cannam@89 2097 <code class="computeroutput">fread</code> and
cannam@89 2098 <code class="computeroutput">fwrite</code>.</p>
cannam@89 2099 <pre class="programlisting">int BZ2_bzflush ( BZFILE* b );
cannam@89 2100 void BZ2_bzclose ( BZFILE* b );</pre>
cannam@89 2101 <p>Flushes/closes a <code class="computeroutput">BZFILE</code>.
cannam@89 2102 <code class="computeroutput">BZ2_bzflush</code> doesn't actually do
cannam@89 2103 anything. Analogous to <code class="computeroutput">fflush</code>
cannam@89 2104 and <code class="computeroutput">fclose</code>.</p>
cannam@89 2105 <pre class="programlisting">const char * BZ2_bzerror ( BZFILE *b, int *errnum )</pre>
cannam@89 2106 <p>Returns a string describing the more recent error status of
cannam@89 2107 <code class="computeroutput">b</code>, and also sets
cannam@89 2108 <code class="computeroutput">*errnum</code> to its numerical
cannam@89 2109 value.</p>
cannam@89 2110 </div>
cannam@89 2111 <div class="sect1" title="3.7. Using the library in a stdio-free environment">
cannam@89 2112 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 2113 <a name="stdio-free"></a>3.7. Using the library in a stdio-free environment</h2></div></div></div>
cannam@89 2114 <div class="sect2" title="3.7.1. Getting rid of stdio">
cannam@89 2115 <div class="titlepage"><div><div><h3 class="title">
cannam@89 2116 <a name="stdio-bye"></a>3.7.1. Getting rid of stdio</h3></div></div></div>
cannam@89 2117 <p>In a deeply embedded application, you might want to use
cannam@89 2118 just the memory-to-memory functions. You can do this
cannam@89 2119 conveniently by compiling the library with preprocessor symbol
cannam@89 2120 <code class="computeroutput">BZ_NO_STDIO</code> defined. Doing this
cannam@89 2121 gives you a library containing only the following eight
cannam@89 2122 functions:</p>
cannam@89 2123 <p><code class="computeroutput">BZ2_bzCompressInit</code>,
cannam@89 2124 <code class="computeroutput">BZ2_bzCompress</code>,
cannam@89 2125 <code class="computeroutput">BZ2_bzCompressEnd</code>
cannam@89 2126 <code class="computeroutput">BZ2_bzDecompressInit</code>,
cannam@89 2127 <code class="computeroutput">BZ2_bzDecompress</code>,
cannam@89 2128 <code class="computeroutput">BZ2_bzDecompressEnd</code>
cannam@89 2129 <code class="computeroutput">BZ2_bzBuffToBuffCompress</code>,
cannam@89 2130 <code class="computeroutput">BZ2_bzBuffToBuffDecompress</code></p>
cannam@89 2131 <p>When compiled like this, all functions will ignore
cannam@89 2132 <code class="computeroutput">verbosity</code> settings.</p>
cannam@89 2133 </div>
cannam@89 2134 <div class="sect2" title="3.7.2. Critical error handling">
cannam@89 2135 <div class="titlepage"><div><div><h3 class="title">
cannam@89 2136 <a name="critical-error"></a>3.7.2. Critical error handling</h3></div></div></div>
cannam@89 2137 <p><code class="computeroutput">libbzip2</code> contains a number
cannam@89 2138 of internal assertion checks which should, needless to say, never
cannam@89 2139 be activated. Nevertheless, if an assertion should fail,
cannam@89 2140 behaviour depends on whether or not the library was compiled with
cannam@89 2141 <code class="computeroutput">BZ_NO_STDIO</code> set.</p>
cannam@89 2142 <p>For a normal compile, an assertion failure yields the
cannam@89 2143 message:</p>
cannam@89 2144 <div class="blockquote"><blockquote class="blockquote">
cannam@89 2145 <p>bzip2/libbzip2: internal error number N.</p>
cannam@89 2146 <p>This is a bug in bzip2/libbzip2, 1.0.6 of 6 September 2010.
cannam@89 2147 Please report it to me at: jseward@bzip.org. If this happened
cannam@89 2148 when you were using some program which uses libbzip2 as a
cannam@89 2149 component, you should also report this bug to the author(s)
cannam@89 2150 of that program. Please make an effort to report this bug;
cannam@89 2151 timely and accurate bug reports eventually lead to higher
cannam@89 2152 quality software. Thanks. Julian Seward, 6 September 2010.
cannam@89 2153 </p>
cannam@89 2154 </blockquote></div>
cannam@89 2155 <p>where <code class="computeroutput">N</code> is some error code
cannam@89 2156 number. If <code class="computeroutput">N == 1007</code>, it also
cannam@89 2157 prints some extra text advising the reader that unreliable memory
cannam@89 2158 is often associated with internal error 1007. (This is a
cannam@89 2159 frequently-observed-phenomenon with versions 1.0.0/1.0.1).</p>
cannam@89 2160 <p><code class="computeroutput">exit(3)</code> is then
cannam@89 2161 called.</p>
cannam@89 2162 <p>For a <code class="computeroutput">stdio</code>-free library,
cannam@89 2163 assertion failures result in a call to a function declared
cannam@89 2164 as:</p>
cannam@89 2165 <pre class="programlisting">extern void bz_internal_error ( int errcode );</pre>
cannam@89 2166 <p>The relevant code is passed as a parameter. You should
cannam@89 2167 supply such a function.</p>
cannam@89 2168 <p>In either case, once an assertion failure has occurred, any
cannam@89 2169 <code class="computeroutput">bz_stream</code> records involved can
cannam@89 2170 be regarded as invalid. You should not attempt to resume normal
cannam@89 2171 operation with them.</p>
cannam@89 2172 <p>You may, of course, change critical error handling to suit
cannam@89 2173 your needs. As I said above, critical errors indicate bugs in
cannam@89 2174 the library and should not occur. All "normal" error situations
cannam@89 2175 are indicated via error return codes from functions, and can be
cannam@89 2176 recovered from.</p>
cannam@89 2177 </div>
cannam@89 2178 </div>
cannam@89 2179 <div class="sect1" title="3.8. Making a Windows DLL">
cannam@89 2180 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 2181 <a name="win-dll"></a>3.8. Making a Windows DLL</h2></div></div></div>
cannam@89 2182 <p>Everything related to Windows has been contributed by
cannam@89 2183 Yoshioka Tsuneo
cannam@89 2184 (<code class="computeroutput">tsuneo@rr.iij4u.or.jp</code>), so
cannam@89 2185 you should send your queries to him (but perhaps Cc: me,
cannam@89 2186 <code class="computeroutput">jseward@bzip.org</code>).</p>
cannam@89 2187 <p>My vague understanding of what to do is: using Visual C++
cannam@89 2188 5.0, open the project file
cannam@89 2189 <code class="computeroutput">libbz2.dsp</code>, and build. That's
cannam@89 2190 all.</p>
cannam@89 2191 <p>If you can't open the project file for some reason, make a
cannam@89 2192 new one, naming these files:
cannam@89 2193 <code class="computeroutput">blocksort.c</code>,
cannam@89 2194 <code class="computeroutput">bzlib.c</code>,
cannam@89 2195 <code class="computeroutput">compress.c</code>,
cannam@89 2196 <code class="computeroutput">crctable.c</code>,
cannam@89 2197 <code class="computeroutput">decompress.c</code>,
cannam@89 2198 <code class="computeroutput">huffman.c</code>,
cannam@89 2199 <code class="computeroutput">randtable.c</code> and
cannam@89 2200 <code class="computeroutput">libbz2.def</code>. You will also need
cannam@89 2201 to name the header files <code class="computeroutput">bzlib.h</code>
cannam@89 2202 and <code class="computeroutput">bzlib_private.h</code>.</p>
cannam@89 2203 <p>If you don't use VC++, you may need to define the
cannam@89 2204 proprocessor symbol
cannam@89 2205 <code class="computeroutput">_WIN32</code>.</p>
cannam@89 2206 <p>Finally, <code class="computeroutput">dlltest.c</code> is a
cannam@89 2207 sample program using the DLL. It has a project file,
cannam@89 2208 <code class="computeroutput">dlltest.dsp</code>.</p>
cannam@89 2209 <p>If you just want a makefile for Visual C, have a look at
cannam@89 2210 <code class="computeroutput">makefile.msc</code>.</p>
cannam@89 2211 <p>Be aware that if you compile
cannam@89 2212 <code class="computeroutput">bzip2</code> itself on Win32, you must
cannam@89 2213 set <code class="computeroutput">BZ_UNIX</code> to 0 and
cannam@89 2214 <code class="computeroutput">BZ_LCCWIN32</code> to 1, in the file
cannam@89 2215 <code class="computeroutput">bzip2.c</code>, before compiling.
cannam@89 2216 Otherwise the resulting binary won't work correctly.</p>
cannam@89 2217 <p>I haven't tried any of this stuff myself, but it all looks
cannam@89 2218 plausible.</p>
cannam@89 2219 </div>
cannam@89 2220 </div>
cannam@89 2221 <div class="chapter" title="4. Miscellanea">
cannam@89 2222 <div class="titlepage"><div><div><h2 class="title">
cannam@89 2223 <a name="misc"></a>4. Miscellanea</h2></div></div></div>
cannam@89 2224 <div class="toc">
cannam@89 2225 <p><b>Table of Contents</b></p>
cannam@89 2226 <dl>
cannam@89 2227 <dt><span class="sect1"><a href="#limits">4.1. Limitations of the compressed file format</a></span></dt>
cannam@89 2228 <dt><span class="sect1"><a href="#port-issues">4.2. Portability issues</a></span></dt>
cannam@89 2229 <dt><span class="sect1"><a href="#bugs">4.3. Reporting bugs</a></span></dt>
cannam@89 2230 <dt><span class="sect1"><a href="#package">4.4. Did you get the right package?</a></span></dt>
cannam@89 2231 <dt><span class="sect1"><a href="#reading">4.5. Further Reading</a></span></dt>
cannam@89 2232 </dl>
cannam@89 2233 </div>
cannam@89 2234 <p>These are just some random thoughts of mine. Your mileage
cannam@89 2235 may vary.</p>
cannam@89 2236 <div class="sect1" title="4.1. Limitations of the compressed file format">
cannam@89 2237 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 2238 <a name="limits"></a>4.1. Limitations of the compressed file format</h2></div></div></div>
cannam@89 2239 <p><code class="computeroutput">bzip2-1.0.X</code>,
cannam@89 2240 <code class="computeroutput">0.9.5</code> and
cannam@89 2241 <code class="computeroutput">0.9.0</code> use exactly the same file
cannam@89 2242 format as the original version,
cannam@89 2243 <code class="computeroutput">bzip2-0.1</code>. This decision was
cannam@89 2244 made in the interests of stability. Creating yet another
cannam@89 2245 incompatible compressed file format would create further
cannam@89 2246 confusion and disruption for users.</p>
cannam@89 2247 <p>Nevertheless, this is not a painless decision. Development
cannam@89 2248 work since the release of
cannam@89 2249 <code class="computeroutput">bzip2-0.1</code> in August 1997 has
cannam@89 2250 shown complexities in the file format which slow down
cannam@89 2251 decompression and, in retrospect, are unnecessary. These
cannam@89 2252 are:</p>
cannam@89 2253 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 2254 <li class="listitem" style="list-style-type: disc"><p>The run-length encoder, which is the first of the
cannam@89 2255 compression transformations, is entirely irrelevant. The
cannam@89 2256 original purpose was to protect the sorting algorithm from the
cannam@89 2257 very worst case input: a string of repeated symbols. But
cannam@89 2258 algorithm steps Q6a and Q6b in the original Burrows-Wheeler
cannam@89 2259 technical report (SRC-124) show how repeats can be handled
cannam@89 2260 without difficulty in block sorting.</p></li>
cannam@89 2261 <li class="listitem" style="list-style-type: disc">
cannam@89 2262 <p>The randomisation mechanism doesn't really need to be
cannam@89 2263 there. Udi Manber and Gene Myers published a suffix array
cannam@89 2264 construction algorithm a few years back, which can be employed
cannam@89 2265 to sort any block, no matter how repetitive, in O(N log N)
cannam@89 2266 time. Subsequent work by Kunihiko Sadakane has produced a
cannam@89 2267 derivative O(N (log N)^2) algorithm which usually outperforms
cannam@89 2268 the Manber-Myers algorithm.</p>
cannam@89 2269 <p>I could have changed to Sadakane's algorithm, but I find
cannam@89 2270 it to be slower than <code class="computeroutput">bzip2</code>'s
cannam@89 2271 existing algorithm for most inputs, and the randomisation
cannam@89 2272 mechanism protects adequately against bad cases. I didn't
cannam@89 2273 think it was a good tradeoff to make. Partly this is due to
cannam@89 2274 the fact that I was not flooded with email complaints about
cannam@89 2275 <code class="computeroutput">bzip2-0.1</code>'s performance on
cannam@89 2276 repetitive data, so perhaps it isn't a problem for real
cannam@89 2277 inputs.</p>
cannam@89 2278 <p>Probably the best long-term solution, and the one I have
cannam@89 2279 incorporated into 0.9.5 and above, is to use the existing
cannam@89 2280 sorting algorithm initially, and fall back to a O(N (log N)^2)
cannam@89 2281 algorithm if the standard algorithm gets into
cannam@89 2282 difficulties.</p>
cannam@89 2283 </li>
cannam@89 2284 <li class="listitem" style="list-style-type: disc"><p>The compressed file format was never designed to be
cannam@89 2285 handled by a library, and I have had to jump though some hoops
cannam@89 2286 to produce an efficient implementation of decompression. It's
cannam@89 2287 a bit hairy. Try passing
cannam@89 2288 <code class="computeroutput">decompress.c</code> through the C
cannam@89 2289 preprocessor and you'll see what I mean. Much of this
cannam@89 2290 complexity could have been avoided if the compressed size of
cannam@89 2291 each block of data was recorded in the data stream.</p></li>
cannam@89 2292 <li class="listitem" style="list-style-type: disc"><p>An Adler-32 checksum, rather than a CRC32 checksum,
cannam@89 2293 would be faster to compute.</p></li>
cannam@89 2294 </ul></div>
cannam@89 2295 <p>It would be fair to say that the
cannam@89 2296 <code class="computeroutput">bzip2</code> format was frozen before I
cannam@89 2297 properly and fully understood the performance consequences of
cannam@89 2298 doing so.</p>
cannam@89 2299 <p>Improvements which I was able to incorporate into 0.9.0,
cannam@89 2300 despite using the same file format, are:</p>
cannam@89 2301 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 2302 <li class="listitem" style="list-style-type: disc"><p>Single array implementation of the inverse BWT. This
cannam@89 2303 significantly speeds up decompression, presumably because it
cannam@89 2304 reduces the number of cache misses.</p></li>
cannam@89 2305 <li class="listitem" style="list-style-type: disc"><p>Faster inverse MTF transform for large MTF values.
cannam@89 2306 The new implementation is based on the notion of sliding blocks
cannam@89 2307 of values.</p></li>
cannam@89 2308 <li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2-0.9.0</code> now reads
cannam@89 2309 and writes files with <code class="computeroutput">fread</code>
cannam@89 2310 and <code class="computeroutput">fwrite</code>; version 0.1 used
cannam@89 2311 <code class="computeroutput">putc</code> and
cannam@89 2312 <code class="computeroutput">getc</code>. Duh! Well, you live
cannam@89 2313 and learn.</p></li>
cannam@89 2314 </ul></div>
cannam@89 2315 <p>Further ahead, it would be nice to be able to do random
cannam@89 2316 access into files. This will require some careful design of
cannam@89 2317 compressed file formats.</p>
cannam@89 2318 </div>
cannam@89 2319 <div class="sect1" title="4.2. Portability issues">
cannam@89 2320 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 2321 <a name="port-issues"></a>4.2. Portability issues</h2></div></div></div>
cannam@89 2322 <p>After some consideration, I have decided not to use GNU
cannam@89 2323 <code class="computeroutput">autoconf</code> to configure 0.9.5 or
cannam@89 2324 1.0.</p>
cannam@89 2325 <p><code class="computeroutput">autoconf</code>, admirable and
cannam@89 2326 wonderful though it is, mainly assists with portability problems
cannam@89 2327 between Unix-like platforms. But
cannam@89 2328 <code class="computeroutput">bzip2</code> doesn't have much in the
cannam@89 2329 way of portability problems on Unix; most of the difficulties
cannam@89 2330 appear when porting to the Mac, or to Microsoft's operating
cannam@89 2331 systems. <code class="computeroutput">autoconf</code> doesn't help
cannam@89 2332 in those cases, and brings in a whole load of new
cannam@89 2333 complexity.</p>
cannam@89 2334 <p>Most people should be able to compile the library and
cannam@89 2335 program under Unix straight out-of-the-box, so to speak,
cannam@89 2336 especially if you have a version of GNU C available.</p>
cannam@89 2337 <p>There are a couple of
cannam@89 2338 <code class="computeroutput">__inline__</code> directives in the
cannam@89 2339 code. GNU C (<code class="computeroutput">gcc</code>) should be
cannam@89 2340 able to handle them. If you're not using GNU C, your C compiler
cannam@89 2341 shouldn't see them at all. If your compiler does, for some
cannam@89 2342 reason, see them and doesn't like them, just
cannam@89 2343 <code class="computeroutput">#define</code>
cannam@89 2344 <code class="computeroutput">__inline__</code> to be
cannam@89 2345 <code class="computeroutput">/* */</code>. One easy way to do this
cannam@89 2346 is to compile with the flag
cannam@89 2347 <code class="computeroutput">-D__inline__=</code>, which should be
cannam@89 2348 understood by most Unix compilers.</p>
cannam@89 2349 <p>If you still have difficulties, try compiling with the
cannam@89 2350 macro <code class="computeroutput">BZ_STRICT_ANSI</code> defined.
cannam@89 2351 This should enable you to build the library in a strictly ANSI
cannam@89 2352 compliant environment. Building the program itself like this is
cannam@89 2353 dangerous and not supported, since you remove
cannam@89 2354 <code class="computeroutput">bzip2</code>'s checks against
cannam@89 2355 compressing directories, symbolic links, devices, and other
cannam@89 2356 not-really-a-file entities. This could cause filesystem
cannam@89 2357 corruption!</p>
cannam@89 2358 <p>One other thing: if you create a
cannam@89 2359 <code class="computeroutput">bzip2</code> binary for public distribution,
cannam@89 2360 please consider linking it statically (<code class="computeroutput">gcc
cannam@89 2361 -static</code>). This avoids all sorts of library-version
cannam@89 2362 issues that others may encounter later on.</p>
cannam@89 2363 <p>If you build <code class="computeroutput">bzip2</code> on
cannam@89 2364 Win32, you must set <code class="computeroutput">BZ_UNIX</code> to 0
cannam@89 2365 and <code class="computeroutput">BZ_LCCWIN32</code> to 1, in the
cannam@89 2366 file <code class="computeroutput">bzip2.c</code>, before compiling.
cannam@89 2367 Otherwise the resulting binary won't work correctly.</p>
cannam@89 2368 </div>
cannam@89 2369 <div class="sect1" title="4.3. Reporting bugs">
cannam@89 2370 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 2371 <a name="bugs"></a>4.3. Reporting bugs</h2></div></div></div>
cannam@89 2372 <p>I tried pretty hard to make sure
cannam@89 2373 <code class="computeroutput">bzip2</code> is bug free, both by
cannam@89 2374 design and by testing. Hopefully you'll never need to read this
cannam@89 2375 section for real.</p>
cannam@89 2376 <p>Nevertheless, if <code class="computeroutput">bzip2</code> dies
cannam@89 2377 with a segmentation fault, a bus error or an internal assertion
cannam@89 2378 failure, it will ask you to email me a bug report. Experience from
cannam@89 2379 years of feedback of bzip2 users indicates that almost all these
cannam@89 2380 problems can be traced to either compiler bugs or hardware
cannam@89 2381 problems.</p>
cannam@89 2382 <div class="itemizedlist"><ul class="itemizedlist" type="bullet">
cannam@89 2383 <li class="listitem" style="list-style-type: disc">
cannam@89 2384 <p>Recompile the program with no optimisation, and
cannam@89 2385 see if it works. And/or try a different compiler. I heard all
cannam@89 2386 sorts of stories about various flavours of GNU C (and other
cannam@89 2387 compilers) generating bad code for
cannam@89 2388 <code class="computeroutput">bzip2</code>, and I've run across two
cannam@89 2389 such examples myself.</p>
cannam@89 2390 <p>2.7.X versions of GNU C are known to generate bad code
cannam@89 2391 from time to time, at high optimisation levels. If you get
cannam@89 2392 problems, try using the flags
cannam@89 2393 <code class="computeroutput">-O2</code>
cannam@89 2394 <code class="computeroutput">-fomit-frame-pointer</code>
cannam@89 2395 <code class="computeroutput">-fno-strength-reduce</code>. You
cannam@89 2396 should specifically <span class="emphasis"><em>not</em></span> use
cannam@89 2397 <code class="computeroutput">-funroll-loops</code>.</p>
cannam@89 2398 <p>You may notice that the Makefile runs six tests as part
cannam@89 2399 of the build process. If the program passes all of these, it's
cannam@89 2400 a pretty good (but not 100%) indication that the compiler has
cannam@89 2401 done its job correctly.</p>
cannam@89 2402 </li>
cannam@89 2403 <li class="listitem" style="list-style-type: disc">
cannam@89 2404 <p>If <code class="computeroutput">bzip2</code>
cannam@89 2405 crashes randomly, and the crashes are not repeatable, you may
cannam@89 2406 have a flaky memory subsystem.
cannam@89 2407 <code class="computeroutput">bzip2</code> really hammers your
cannam@89 2408 memory hierarchy, and if it's a bit marginal, you may get these
cannam@89 2409 problems. Ditto if your disk or I/O subsystem is slowly
cannam@89 2410 failing. Yup, this really does happen.</p>
cannam@89 2411 <p>Try using a different machine of the same type, and see
cannam@89 2412 if you can repeat the problem.</p>
cannam@89 2413 </li>
cannam@89 2414 <li class="listitem" style="list-style-type: disc"><p>This isn't really a bug, but ... If
cannam@89 2415 <code class="computeroutput">bzip2</code> tells you your file is
cannam@89 2416 corrupted on decompression, and you obtained the file via FTP,
cannam@89 2417 there is a possibility that you forgot to tell FTP to do a
cannam@89 2418 binary mode transfer. That absolutely will cause the file to
cannam@89 2419 be non-decompressible. You'll have to transfer it
cannam@89 2420 again.</p></li>
cannam@89 2421 </ul></div>
cannam@89 2422 <p>If you've incorporated
cannam@89 2423 <code class="computeroutput">libbzip2</code> into your own program
cannam@89 2424 and are getting problems, please, please, please, check that the
cannam@89 2425 parameters you are passing in calls to the library, are correct,
cannam@89 2426 and in accordance with what the documentation says is allowable.
cannam@89 2427 I have tried to make the library robust against such problems,
cannam@89 2428 but I'm sure I haven't succeeded.</p>
cannam@89 2429 <p>Finally, if the above comments don't help, you'll have to
cannam@89 2430 send me a bug report. Now, it's just amazing how many people
cannam@89 2431 will send me a bug report saying something like:</p>
cannam@89 2432 <pre class="programlisting">bzip2 crashed with segmentation fault on my machine</pre>
cannam@89 2433 <p>and absolutely nothing else. Needless to say, a such a
cannam@89 2434 report is <span class="emphasis"><em>totally, utterly, completely and
cannam@89 2435 comprehensively 100% useless; a waste of your time, my time, and
cannam@89 2436 net bandwidth</em></span>. With no details at all, there's no way
cannam@89 2437 I can possibly begin to figure out what the problem is.</p>
cannam@89 2438 <p>The rules of the game are: facts, facts, facts. Don't omit
cannam@89 2439 them because "oh, they won't be relevant". At the bare
cannam@89 2440 minimum:</p>
cannam@89 2441 <pre class="programlisting">Machine type. Operating system version.
cannam@89 2442 Exact version of bzip2 (do bzip2 -V).
cannam@89 2443 Exact version of the compiler used.
cannam@89 2444 Flags passed to the compiler.</pre>
cannam@89 2445 <p>However, the most important single thing that will help me
cannam@89 2446 is the file that you were trying to compress or decompress at the
cannam@89 2447 time the problem happened. Without that, my ability to do
cannam@89 2448 anything more than speculate about the cause, is limited.</p>
cannam@89 2449 </div>
cannam@89 2450 <div class="sect1" title="4.4. Did you get the right package?">
cannam@89 2451 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 2452 <a name="package"></a>4.4. Did you get the right package?</h2></div></div></div>
cannam@89 2453 <p><code class="computeroutput">bzip2</code> is a resource hog.
cannam@89 2454 It soaks up large amounts of CPU cycles and memory. Also, it
cannam@89 2455 gives very large latencies. In the worst case, you can feed many
cannam@89 2456 megabytes of uncompressed data into the library before getting
cannam@89 2457 any compressed output, so this probably rules out applications
cannam@89 2458 requiring interactive behaviour.</p>
cannam@89 2459 <p>These aren't faults of my implementation, I hope, but more
cannam@89 2460 an intrinsic property of the Burrows-Wheeler transform
cannam@89 2461 (unfortunately). Maybe this isn't what you want.</p>
cannam@89 2462 <p>If you want a compressor and/or library which is faster,
cannam@89 2463 uses less memory but gets pretty good compression, and has
cannam@89 2464 minimal latency, consider Jean-loup Gailly's and Mark Adler's
cannam@89 2465 work, <code class="computeroutput">zlib-1.2.1</code> and
cannam@89 2466 <code class="computeroutput">gzip-1.2.4</code>. Look for them at
cannam@89 2467 <a class="ulink" href="http://www.zlib.org" target="_top">http://www.zlib.org</a> and
cannam@89 2468 <a class="ulink" href="http://www.gzip.org" target="_top">http://www.gzip.org</a>
cannam@89 2469 respectively.</p>
cannam@89 2470 <p>For something faster and lighter still, you might try Markus F
cannam@89 2471 X J Oberhumer's <code class="computeroutput">LZO</code> real-time
cannam@89 2472 compression/decompression library, at
cannam@89 2473 <a class="ulink" href="http://www.oberhumer.com/opensource" target="_top">http://www.oberhumer.com/opensource</a>.</p>
cannam@89 2474 </div>
cannam@89 2475 <div class="sect1" title="4.5. Further Reading">
cannam@89 2476 <div class="titlepage"><div><div><h2 class="title" style="clear: both">
cannam@89 2477 <a name="reading"></a>4.5. Further Reading</h2></div></div></div>
cannam@89 2478 <p><code class="computeroutput">bzip2</code> is not research
cannam@89 2479 work, in the sense that it doesn't present any new ideas.
cannam@89 2480 Rather, it's an engineering exercise based on existing
cannam@89 2481 ideas.</p>
cannam@89 2482 <p>Four documents describe essentially all the ideas behind
cannam@89 2483 <code class="computeroutput">bzip2</code>:</p>
cannam@89 2484 <div class="literallayout"><p>Michael Burrows and D. J. Wheeler:<br>
cannam@89 2485   "A block-sorting lossless data compression algorithm"<br>
cannam@89 2486    10th May 1994. <br>
cannam@89 2487    Digital SRC Research Report 124.<br>
cannam@89 2488    ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.gz<br>
cannam@89 2489    If you have trouble finding it, try searching at the<br>
cannam@89 2490    New Zealand Digital Library, http://www.nzdl.org.<br>
cannam@89 2491 <br>
cannam@89 2492 Daniel S. Hirschberg and Debra A. LeLewer<br>
cannam@89 2493   "Efficient Decoding of Prefix Codes"<br>
cannam@89 2494    Communications of the ACM, April 1990, Vol 33, Number 4.<br>
cannam@89 2495    You might be able to get an electronic copy of this<br>
cannam@89 2496    from the ACM Digital Library.<br>
cannam@89 2497 <br>
cannam@89 2498 David J. Wheeler<br>
cannam@89 2499    Program bred3.c and accompanying document bred3.ps.<br>
cannam@89 2500    This contains the idea behind the multi-table Huffman coding scheme.<br>
cannam@89 2501    ftp://ftp.cl.cam.ac.uk/users/djw3/<br>
cannam@89 2502 <br>
cannam@89 2503 Jon L. Bentley and Robert Sedgewick<br>
cannam@89 2504   "Fast Algorithms for Sorting and Searching Strings"<br>
cannam@89 2505    Available from Sedgewick's web page,<br>
cannam@89 2506    www.cs.princeton.edu/~rs<br>
cannam@89 2507 </p></div>
cannam@89 2508 <p>The following paper gives valuable additional insights into
cannam@89 2509 the algorithm, but is not immediately the basis of any code used
cannam@89 2510 in bzip2.</p>
cannam@89 2511 <div class="literallayout"><p>Peter Fenwick:<br>
cannam@89 2512    Block Sorting Text Compression<br>
cannam@89 2513    Proceedings of the 19th Australasian Computer Science Conference,<br>
cannam@89 2514      Melbourne, Australia.  Jan 31 - Feb 2, 1996.<br>
cannam@89 2515    ftp://ftp.cs.auckland.ac.nz/pub/peter-f/ACSC96paper.ps</p></div>
cannam@89 2516 <p>Kunihiko Sadakane's sorting algorithm, mentioned above, is
cannam@89 2517 available from:</p>
cannam@89 2518 <div class="literallayout"><p>http://naomi.is.s.u-tokyo.ac.jp/~sada/papers/Sada98b.ps.gz<br>
cannam@89 2519 </p></div>
cannam@89 2520 <p>The Manber-Myers suffix array construction algorithm is
cannam@89 2521 described in a paper available from:</p>
cannam@89 2522 <div class="literallayout"><p>http://www.cs.arizona.edu/people/gene/PAPERS/suffix.ps<br>
cannam@89 2523 </p></div>
cannam@89 2524 <p>Finally, the following papers document some
cannam@89 2525 investigations I made into the performance of sorting
cannam@89 2526 and decompression algorithms:</p>
cannam@89 2527 <div class="literallayout"><p>Julian Seward<br>
cannam@89 2528    On the Performance of BWT Sorting Algorithms<br>
cannam@89 2529    Proceedings of the IEEE Data Compression Conference 2000<br>
cannam@89 2530      Snowbird, Utah.  28-30 March 2000.<br>
cannam@89 2531 <br>
cannam@89 2532 Julian Seward<br>
cannam@89 2533    Space-time Tradeoffs in the Inverse B-W Transform<br>
cannam@89 2534    Proceedings of the IEEE Data Compression Conference 2001<br>
cannam@89 2535      Snowbird, Utah.  27-29 March 2001.<br>
cannam@89 2536 </p></div>
cannam@89 2537 </div>
cannam@89 2538 </div>
cannam@89 2539 </div></body>
cannam@89 2540 </html>