Mercurial > hg > sv-dependency-builds
comparison src/bzip2-1.0.6/bzip2.txt @ 89:8a15ff55d9af
Add bzip2, zlib, liblo, portaudio sources
author | Chris Cannam <cannam@all-day-breakfast.com> |
---|---|
date | Wed, 20 Mar 2013 13:59:52 +0000 |
parents | |
children |
comparison
equal
deleted
inserted
replaced
88:fe7c3a0b0259 | 89:8a15ff55d9af |
---|---|
1 | |
2 NAME | |
3 bzip2, bunzip2 - a block-sorting file compressor, v1.0.6 | |
4 bzcat - decompresses files to stdout | |
5 bzip2recover - recovers data from damaged bzip2 files | |
6 | |
7 | |
8 SYNOPSIS | |
9 bzip2 [ -cdfkqstvzVL123456789 ] [ filenames ... ] | |
10 bunzip2 [ -fkvsVL ] [ filenames ... ] | |
11 bzcat [ -s ] [ filenames ... ] | |
12 bzip2recover filename | |
13 | |
14 | |
15 DESCRIPTION | |
16 bzip2 compresses files using the Burrows-Wheeler block | |
17 sorting text compression algorithm, and Huffman coding. | |
18 Compression is generally considerably better than that | |
19 achieved by more conventional LZ77/LZ78-based compressors, | |
20 and approaches the performance of the PPM family of sta- | |
21 tistical compressors. | |
22 | |
23 The command-line options are deliberately very similar to | |
24 those of GNU gzip, but they are not identical. | |
25 | |
26 bzip2 expects a list of file names to accompany the com- | |
27 mand-line flags. Each file is replaced by a compressed | |
28 version of itself, with the name "original_name.bz2". | |
29 Each compressed file has the same modification date, per- | |
30 missions, and, when possible, ownership as the correspond- | |
31 ing original, so that these properties can be correctly | |
32 restored at decompression time. File name handling is | |
33 naive in the sense that there is no mechanism for preserv- | |
34 ing original file names, permissions, ownerships or dates | |
35 in filesystems which lack these concepts, or have serious | |
36 file name length restrictions, such as MS-DOS. | |
37 | |
38 bzip2 and bunzip2 will by default not overwrite existing | |
39 files. If you want this to happen, specify the -f flag. | |
40 | |
41 If no file names are specified, bzip2 compresses from | |
42 standard input to standard output. In this case, bzip2 | |
43 will decline to write compressed output to a terminal, as | |
44 this would be entirely incomprehensible and therefore | |
45 pointless. | |
46 | |
47 bunzip2 (or bzip2 -d) decompresses all specified files. | |
48 Files which were not created by bzip2 will be detected and | |
49 ignored, and a warning issued. bzip2 attempts to guess | |
50 the filename for the decompressed file from that of the | |
51 compressed file as follows: | |
52 | |
53 filename.bz2 becomes filename | |
54 filename.bz becomes filename | |
55 filename.tbz2 becomes filename.tar | |
56 filename.tbz becomes filename.tar | |
57 anyothername becomes anyothername.out | |
58 | |
59 If the file does not end in one of the recognised endings, | |
60 .bz2, .bz, .tbz2 or .tbz, bzip2 complains that it cannot | |
61 guess the name of the original file, and uses the original | |
62 name with .out appended. | |
63 | |
64 As with compression, supplying no filenames causes decom- | |
65 pression from standard input to standard output. | |
66 | |
67 bunzip2 will correctly decompress a file which is the con- | |
68 catenation of two or more compressed files. The result is | |
69 the concatenation of the corresponding uncompressed files. | |
70 Integrity testing (-t) of concatenated compressed files is | |
71 also supported. | |
72 | |
73 You can also compress or decompress files to the standard | |
74 output by giving the -c flag. Multiple files may be com- | |
75 pressed and decompressed like this. The resulting outputs | |
76 are fed sequentially to stdout. Compression of multiple | |
77 files in this manner generates a stream containing multi- | |
78 ple compressed file representations. Such a stream can be | |
79 decompressed correctly only by bzip2 version 0.9.0 or | |
80 later. Earlier versions of bzip2 will stop after decom- | |
81 pressing the first file in the stream. | |
82 | |
83 bzcat (or bzip2 -dc) decompresses all specified files to | |
84 the standard output. | |
85 | |
86 bzip2 will read arguments from the environment variables | |
87 BZIP2 and BZIP, in that order, and will process them | |
88 before any arguments read from the command line. This | |
89 gives a convenient way to supply default arguments. | |
90 | |
91 Compression is always performed, even if the compressed | |
92 file is slightly larger than the original. Files of less | |
93 than about one hundred bytes tend to get larger, since the | |
94 compression mechanism has a constant overhead in the | |
95 region of 50 bytes. Random data (including the output of | |
96 most file compressors) is coded at about 8.05 bits per | |
97 byte, giving an expansion of around 0.5%. | |
98 | |
99 As a self-check for your protection, bzip2 uses 32-bit | |
100 CRCs to make sure that the decompressed version of a file | |
101 is identical to the original. This guards against corrup- | |
102 tion of the compressed data, and against undetected bugs | |
103 in bzip2 (hopefully very unlikely). The chances of data | |
104 corruption going undetected is microscopic, about one | |
105 chance in four billion for each file processed. Be aware, | |
106 though, that the check occurs upon decompression, so it | |
107 can only tell you that something is wrong. It can't help | |
108 you recover the original uncompressed data. You can use | |
109 bzip2recover to try to recover data from damaged files. | |
110 | |
111 Return values: 0 for a normal exit, 1 for environmental | |
112 problems (file not found, invalid flags, I/O errors, &c), | |
113 2 to indicate a corrupt compressed file, 3 for an internal | |
114 consistency error (eg, bug) which caused bzip2 to panic. | |
115 | |
116 | |
117 OPTIONS | |
118 -c --stdout | |
119 Compress or decompress to standard output. | |
120 | |
121 -d --decompress | |
122 Force decompression. bzip2, bunzip2 and bzcat are | |
123 really the same program, and the decision about | |
124 what actions to take is done on the basis of which | |
125 name is used. This flag overrides that mechanism, | |
126 and forces bzip2 to decompress. | |
127 | |
128 -z --compress | |
129 The complement to -d: forces compression, | |
130 regardless of the invocation name. | |
131 | |
132 -t --test | |
133 Check integrity of the specified file(s), but don't | |
134 decompress them. This really performs a trial | |
135 decompression and throws away the result. | |
136 | |
137 -f --force | |
138 Force overwrite of output files. Normally, bzip2 | |
139 will not overwrite existing output files. Also | |
140 forces bzip2 to break hard links to files, which it | |
141 otherwise wouldn't do. | |
142 | |
143 bzip2 normally declines to decompress files which | |
144 don't have the correct magic header bytes. If | |
145 forced (-f), however, it will pass such files | |
146 through unmodified. This is how GNU gzip behaves. | |
147 | |
148 -k --keep | |
149 Keep (don't delete) input files during compression | |
150 or decompression. | |
151 | |
152 -s --small | |
153 Reduce memory usage, for compression, decompression | |
154 and testing. Files are decompressed and tested | |
155 using a modified algorithm which only requires 2.5 | |
156 bytes per block byte. This means any file can be | |
157 decompressed in 2300k of memory, albeit at about | |
158 half the normal speed. | |
159 | |
160 During compression, -s selects a block size of | |
161 200k, which limits memory use to around the same | |
162 figure, at the expense of your compression ratio. | |
163 In short, if your machine is low on memory (8 | |
164 megabytes or less), use -s for everything. See | |
165 MEMORY MANAGEMENT below. | |
166 | |
167 -q --quiet | |
168 Suppress non-essential warning messages. Messages | |
169 pertaining to I/O errors and other critical events | |
170 will not be suppressed. | |
171 | |
172 -v --verbose | |
173 Verbose mode -- show the compression ratio for each | |
174 file processed. Further -v's increase the ver- | |
175 bosity level, spewing out lots of information which | |
176 is primarily of interest for diagnostic purposes. | |
177 | |
178 -L --license -V --version | |
179 Display the software version, license terms and | |
180 conditions. | |
181 | |
182 -1 (or --fast) to -9 (or --best) | |
183 Set the block size to 100 k, 200 k .. 900 k when | |
184 compressing. Has no effect when decompressing. | |
185 See MEMORY MANAGEMENT below. The --fast and --best | |
186 aliases are primarily for GNU gzip compatibility. | |
187 In particular, --fast doesn't make things signifi- | |
188 cantly faster. And --best merely selects the | |
189 default behaviour. | |
190 | |
191 -- Treats all subsequent arguments as file names, even | |
192 if they start with a dash. This is so you can han- | |
193 dle files with names beginning with a dash, for | |
194 example: bzip2 -- -myfilename. | |
195 | |
196 --repetitive-fast --repetitive-best | |
197 These flags are redundant in versions 0.9.5 and | |
198 above. They provided some coarse control over the | |
199 behaviour of the sorting algorithm in earlier ver- | |
200 sions, which was sometimes useful. 0.9.5 and above | |
201 have an improved algorithm which renders these | |
202 flags irrelevant. | |
203 | |
204 | |
205 MEMORY MANAGEMENT | |
206 bzip2 compresses large files in blocks. The block size | |
207 affects both the compression ratio achieved, and the | |
208 amount of memory needed for compression and decompression. | |
209 The flags -1 through -9 specify the block size to be | |
210 100,000 bytes through 900,000 bytes (the default) respec- | |
211 tively. At decompression time, the block size used for | |
212 compression is read from the header of the compressed | |
213 file, and bunzip2 then allocates itself just enough memory | |
214 to decompress the file. Since block sizes are stored in | |
215 compressed files, it follows that the flags -1 to -9 are | |
216 irrelevant to and so ignored during decompression. | |
217 | |
218 Compression and decompression requirements, in bytes, can | |
219 be estimated as: | |
220 | |
221 Compression: 400k + ( 8 x block size ) | |
222 | |
223 Decompression: 100k + ( 4 x block size ), or | |
224 100k + ( 2.5 x block size ) | |
225 | |
226 Larger block sizes give rapidly diminishing marginal | |
227 returns. Most of the compression comes from the first two | |
228 or three hundred k of block size, a fact worth bearing in | |
229 mind when using bzip2 on small machines. It is also | |
230 important to appreciate that the decompression memory | |
231 requirement is set at compression time by the choice of | |
232 block size. | |
233 | |
234 For files compressed with the default 900k block size, | |
235 bunzip2 will require about 3700 kbytes to decompress. To | |
236 support decompression of any file on a 4 megabyte machine, | |
237 bunzip2 has an option to decompress using approximately | |
238 half this amount of memory, about 2300 kbytes. Decompres- | |
239 sion speed is also halved, so you should use this option | |
240 only where necessary. The relevant flag is -s. | |
241 | |
242 In general, try and use the largest block size memory con- | |
243 straints allow, since that maximises the compression | |
244 achieved. Compression and decompression speed are virtu- | |
245 ally unaffected by block size. | |
246 | |
247 Another significant point applies to files which fit in a | |
248 single block -- that means most files you'd encounter | |
249 using a large block size. The amount of real memory | |
250 touched is proportional to the size of the file, since the | |
251 file is smaller than a block. For example, compressing a | |
252 file 20,000 bytes long with the flag -9 will cause the | |
253 compressor to allocate around 7600k of memory, but only | |
254 touch 400k + 20000 * 8 = 560 kbytes of it. Similarly, the | |
255 decompressor will allocate 3700k but only touch 100k + | |
256 20000 * 4 = 180 kbytes. | |
257 | |
258 Here is a table which summarises the maximum memory usage | |
259 for different block sizes. Also recorded is the total | |
260 compressed size for 14 files of the Calgary Text Compres- | |
261 sion Corpus totalling 3,141,622 bytes. This column gives | |
262 some feel for how compression varies with block size. | |
263 These figures tend to understate the advantage of larger | |
264 block sizes for larger files, since the Corpus is domi- | |
265 nated by smaller files. | |
266 | |
267 Compress Decompress Decompress Corpus | |
268 Flag usage usage -s usage Size | |
269 | |
270 -1 1200k 500k 350k 914704 | |
271 -2 2000k 900k 600k 877703 | |
272 -3 2800k 1300k 850k 860338 | |
273 -4 3600k 1700k 1100k 846899 | |
274 -5 4400k 2100k 1350k 845160 | |
275 -6 5200k 2500k 1600k 838626 | |
276 -7 6100k 2900k 1850k 834096 | |
277 -8 6800k 3300k 2100k 828642 | |
278 -9 7600k 3700k 2350k 828642 | |
279 | |
280 | |
281 RECOVERING DATA FROM DAMAGED FILES | |
282 bzip2 compresses files in blocks, usually 900kbytes long. | |
283 Each block is handled independently. If a media or trans- | |
284 mission error causes a multi-block .bz2 file to become | |
285 damaged, it may be possible to recover data from the | |
286 undamaged blocks in the file. | |
287 | |
288 The compressed representation of each block is delimited | |
289 by a 48-bit pattern, which makes it possible to find the | |
290 block boundaries with reasonable certainty. Each block | |
291 also carries its own 32-bit CRC, so damaged blocks can be | |
292 distinguished from undamaged ones. | |
293 | |
294 bzip2recover is a simple program whose purpose is to | |
295 search for blocks in .bz2 files, and write each block out | |
296 into its own .bz2 file. You can then use bzip2 -t to test | |
297 the integrity of the resulting files, and decompress those | |
298 which are undamaged. | |
299 | |
300 bzip2recover takes a single argument, the name of the dam- | |
301 aged file, and writes a number of files | |
302 "rec00001file.bz2", "rec00002file.bz2", etc, containing | |
303 the extracted blocks. The output filenames are | |
304 designed so that the use of wildcards in subsequent pro- | |
305 cessing -- for example, "bzip2 -dc rec*file.bz2 > recov- | |
306 ered_data" -- processes the files in the correct order. | |
307 | |
308 bzip2recover should be of most use dealing with large .bz2 | |
309 files, as these will contain many blocks. It is clearly | |
310 futile to use it on damaged single-block files, since a | |
311 damaged block cannot be recovered. If you wish to min- | |
312 imise any potential data loss through media or transmis- | |
313 sion errors, you might consider compressing with a smaller | |
314 block size. | |
315 | |
316 | |
317 PERFORMANCE NOTES | |
318 The sorting phase of compression gathers together similar | |
319 strings in the file. Because of this, files containing | |
320 very long runs of repeated symbols, like "aabaabaabaab | |
321 ..." (repeated several hundred times) may compress more | |
322 slowly than normal. Versions 0.9.5 and above fare much | |
323 better than previous versions in this respect. The ratio | |
324 between worst-case and average-case compression time is in | |
325 the region of 10:1. For previous versions, this figure | |
326 was more like 100:1. You can use the -vvvv option to mon- | |
327 itor progress in great detail, if you want. | |
328 | |
329 Decompression speed is unaffected by these phenomena. | |
330 | |
331 bzip2 usually allocates several megabytes of memory to | |
332 operate in, and then charges all over it in a fairly ran- | |
333 dom fashion. This means that performance, both for com- | |
334 pressing and decompressing, is largely determined by the | |
335 speed at which your machine can service cache misses. | |
336 Because of this, small changes to the code to reduce the | |
337 miss rate have been observed to give disproportionately | |
338 large performance improvements. I imagine bzip2 will per- | |
339 form best on machines with very large caches. | |
340 | |
341 | |
342 CAVEATS | |
343 I/O error messages are not as helpful as they could be. | |
344 bzip2 tries hard to detect I/O errors and exit cleanly, | |
345 but the details of what the problem is sometimes seem | |
346 rather misleading. | |
347 | |
348 This manual page pertains to version 1.0.6 of bzip2. Com- | |
349 pressed data created by this version is entirely forwards | |
350 and backwards compatible with the previous public | |
351 releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1, | |
352 1.0.2 and above, but with the following exception: 0.9.0 | |
353 and above can correctly decompress multiple concatenated | |
354 compressed files. 0.1pl2 cannot do this; it will stop | |
355 after decompressing just the first file in the stream. | |
356 | |
357 bzip2recover versions prior to 1.0.2 used 32-bit integers | |
358 to represent bit positions in compressed files, so they | |
359 could not handle compressed files more than 512 megabytes | |
360 long. Versions 1.0.2 and above use 64-bit ints on some | |
361 platforms which support them (GNU supported targets, and | |
362 Windows). To establish whether or not bzip2recover was | |
363 built with such a limitation, run it without arguments. | |
364 In any event you can build yourself an unlimited version | |
365 if you can recompile it with MaybeUInt64 set to be an | |
366 unsigned 64-bit integer. | |
367 | |
368 | |
369 AUTHOR | |
370 Julian Seward, jsewardbzip.org. | |
371 | |
372 http://www.bzip.org | |
373 | |
374 The ideas embodied in bzip2 are due to (at least) the fol- | |
375 lowing people: Michael Burrows and David Wheeler (for the | |
376 block sorting transformation), David Wheeler (again, for | |
377 the Huffman coder), Peter Fenwick (for the structured cod- | |
378 ing model in the original bzip, and many refinements), and | |
379 Alistair Moffat, Radford Neal and Ian Witten (for the | |
380 arithmetic coder in the original bzip). I am much | |
381 indebted for their help, support and advice. See the man- | |
382 ual in the source distribution for pointers to sources of | |
383 documentation. Christian von Roques encouraged me to look | |
384 for faster sorting algorithms, so as to speed up compres- | |
385 sion. Bela Lubkin encouraged me to improve the worst-case | |
386 compression performance. Donna Robinson XMLised the docu- | |
387 mentation. The bz* scripts are derived from those of GNU | |
388 gzip. Many people sent patches, helped with portability | |
389 problems, lent machines, gave advice and were generally | |
390 helpful. | |
391 |