annotate src/libogg-1.3.0/doc/ogg-multiplex.html @ 86:98c1576536ae

Bring in flac, ogg, vorbis
author Chris Cannam <cannam@all-day-breakfast.com>
date Tue, 19 Mar 2013 17:37:49 +0000
parents
children
rev   line source
cannam@86 1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
cannam@86 2 <html>
cannam@86 3 <head>
cannam@86 4
cannam@86 5 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-15"/>
cannam@86 6 <title>Ogg Documentation</title>
cannam@86 7
cannam@86 8 <style type="text/css">
cannam@86 9 body {
cannam@86 10 margin: 0 18px 0 18px;
cannam@86 11 padding-bottom: 30px;
cannam@86 12 font-family: Verdana, Arial, Helvetica, sans-serif;
cannam@86 13 color: #333333;
cannam@86 14 font-size: .8em;
cannam@86 15 }
cannam@86 16
cannam@86 17 a {
cannam@86 18 color: #3366cc;
cannam@86 19 }
cannam@86 20
cannam@86 21 img {
cannam@86 22 border: 0;
cannam@86 23 }
cannam@86 24
cannam@86 25 #xiphlogo {
cannam@86 26 margin: 30px 0 16px 0;
cannam@86 27 }
cannam@86 28
cannam@86 29 #content p {
cannam@86 30 line-height: 1.4;
cannam@86 31 }
cannam@86 32
cannam@86 33 h1, h1 a, h2, h2 a, h3, h3 a, h4, h4 a {
cannam@86 34 font-weight: bold;
cannam@86 35 color: #ff9900;
cannam@86 36 margin: 1.3em 0 8px 0;
cannam@86 37 }
cannam@86 38
cannam@86 39 h1 {
cannam@86 40 font-size: 1.3em;
cannam@86 41 }
cannam@86 42
cannam@86 43 h2 {
cannam@86 44 font-size: 1.2em;
cannam@86 45 }
cannam@86 46
cannam@86 47 h3 {
cannam@86 48 font-size: 1.1em;
cannam@86 49 }
cannam@86 50
cannam@86 51 li {
cannam@86 52 line-height: 1.4;
cannam@86 53 }
cannam@86 54
cannam@86 55 #copyright {
cannam@86 56 margin-top: 30px;
cannam@86 57 line-height: 1.5em;
cannam@86 58 text-align: center;
cannam@86 59 font-size: .8em;
cannam@86 60 color: #888888;
cannam@86 61 clear: both;
cannam@86 62 }
cannam@86 63 </style>
cannam@86 64
cannam@86 65 </head>
cannam@86 66
cannam@86 67 <body>
cannam@86 68
cannam@86 69 <div id="xiphlogo">
cannam@86 70 <a href="http://www.xiph.org/"><img src="fish_xiph_org.png" alt="Fish Logo and Xiph.org"/></a>
cannam@86 71 </div>
cannam@86 72
cannam@86 73 <h1>Page Multiplexing and Ordering in a Physical Ogg Stream</h1>
cannam@86 74
cannam@86 75 <p>The low-level mechanisms of an Ogg stream (as described in the Ogg
cannam@86 76 Bitstream Overview) provide means for mixing multiple logical streams
cannam@86 77 and media types into a single linear-chronological stream. This
cannam@86 78 document specifies the high-level arrangement and use of page
cannam@86 79 structure to multiplex multiple streams of mixed media type within a
cannam@86 80 physical Ogg stream.</p>
cannam@86 81
cannam@86 82 <h2>Design Elements</h2>
cannam@86 83
cannam@86 84 <p>The design and arrangement of the Ogg container format is governed by
cannam@86 85 several high-level design decisions that form the reasoning behind
cannam@86 86 specific low-level design decisions.</p>
cannam@86 87
cannam@86 88 <h3>Linear media</h3>
cannam@86 89
cannam@86 90 <p>The Ogg bitstream is intended to encapsulate chronological,
cannam@86 91 time-linear mixed media into a single delivery stream or file. The
cannam@86 92 design is such that an application can always encode and/or decode a
cannam@86 93 full-featured bitstream in one pass with no seeking and minimal
cannam@86 94 buffering. Seeking to provide optimized encoding (such as two-pass
cannam@86 95 encoding) or interactive decoding (such as scrubbing or instant
cannam@86 96 replay) is not disallowed or discouraged, however no bitstream feature
cannam@86 97 must require nonlinear operation on the bitstream.</p>
cannam@86 98
cannam@86 99 <h3>Multiplexing</h3>
cannam@86 100
cannam@86 101 <p>Ogg bitstreams multiplex multiple logical streams into a single
cannam@86 102 physical stream at the page level. Each page contains an abstract
cannam@86 103 time stamp (the Granule Position) that represents an absolute time
cannam@86 104 landmark within the stream. After the pages representing stream
cannam@86 105 headers (all logical stream headers occur at the beginning of a
cannam@86 106 physical bitstream section before any logical stream data), logical
cannam@86 107 stream data pages are arranged in a physical bitstream in strict
cannam@86 108 non-decreasing order by chronological absolute time as
cannam@86 109 specified by the granule position.</p>
cannam@86 110
cannam@86 111 <p>The only exception to arranging pages in strictly ascending time order
cannam@86 112 by granule position is those pages that do not set the granule
cannam@86 113 position value. This is a special case when exceptionally large
cannam@86 114 packets span multiple pages; the specifics of handling this special
cannam@86 115 case are described later under 'Continuous and Discontinuous
cannam@86 116 Streams'.</p>
cannam@86 117
cannam@86 118 <h3>Seeking</h3>
cannam@86 119
cannam@86 120 <p>Ogg is designed to use an interpolated bisection search to
cannam@86 121 implement exact positional seeking. Interpolated bisection search is
cannam@86 122 a spec-mandated mechanism.</p>
cannam@86 123
cannam@86 124 <p><i>An index may improve objective performance, but it seldom
cannam@86 125 improves subjective performance outside of a few high-latency use
cannam@86 126 cases and adds no additional functionality as bisection search
cannam@86 127 delivers the same functionality for both one- and two-pass stream
cannam@86 128 types. For these reasons, use of indexes is discouraged, except in
cannam@86 129 cases where an index provides demonstrable and noticable performance
cannam@86 130 improvement.</i></p>
cannam@86 131
cannam@86 132 <p>Seek operations are by absolute time; a direct bisection search must
cannam@86 133 find the exact time position requested. Information in the Ogg
cannam@86 134 bitstream is arranged such that all information to be presented for
cannam@86 135 playback from the desired seek point will occur at or after the
cannam@86 136 desired seek point. Seek operations are neither 'fuzzy' nor
cannam@86 137 heuristic.</p>
cannam@86 138
cannam@86 139 <p><i>Although key frame handling in video appears to be an exception to
cannam@86 140 "all needed playback information lies ahead of a given seek",
cannam@86 141 key frames can still be handled directly within this indexless
cannam@86 142 framework. Seeking to a key frame in video (as well as seeking in other
cannam@86 143 media types with analogous restraints) is handled as two seeks; first
cannam@86 144 a seek to the desired time which extracts state information that
cannam@86 145 decodes to the time of the last key frame, followed by a second seek
cannam@86 146 directly to the key frame. The location of the previous key frame is
cannam@86 147 embedded as state information in the granulepos; this mechanism is
cannam@86 148 described in more detail later.</i></p>
cannam@86 149
cannam@86 150 <h3>Continuous and Discontinuous Streams</h3>
cannam@86 151
cannam@86 152 <p>Logical streams within a physical Ogg stream belong to one of two
cannam@86 153 categories, "Continuous" streams and "Discontinuous" streams.
cannam@86 154 Although these are discussed in more detail later, the distinction is
cannam@86 155 important to a high-level understanding of how to buffer an Ogg
cannam@86 156 stream.</p>
cannam@86 157
cannam@86 158 <p>A stream that provides a gapless, time-continuous media type with a
cannam@86 159 fine-grained timebase is considered to be 'Continuous'. A continuous
cannam@86 160 stream should never be starved of data. Clear examples of continuous
cannam@86 161 data types include broadcast audio and video.</p>
cannam@86 162
cannam@86 163 <p>A stream that delivers data in a potentially irregular pattern or with
cannam@86 164 widely spaced timing gaps is considered to be 'Discontinuous'. A
cannam@86 165 discontinuous stream may be best thought of as data representing
cannam@86 166 scattered events; although they happen in order, they are typically
cannam@86 167 unconnected data often located far apart. One possible example of a
cannam@86 168 discontinuous stream types would be captioning. Although it's
cannam@86 169 possible to design captions as a continuous stream type, it's most
cannam@86 170 natural to think of captions as widely spaced pieces of text with
cannam@86 171 little happening between.</p>
cannam@86 172
cannam@86 173 <p>The fundamental design distinction between continuous and
cannam@86 174 discontinuous streams concerns buffering.</p>
cannam@86 175
cannam@86 176 <h3>Buffering</h3>
cannam@86 177
cannam@86 178 <p>Because a continuous stream is, by definition, gapless, Ogg buffering
cannam@86 179 is based on the simple premise of never allowing any active continuous
cannam@86 180 stream to starve for data during decode; buffering proceeds ahead
cannam@86 181 until all continuous streams in a physical stream have data ready to
cannam@86 182 decode on demand.</p>
cannam@86 183
cannam@86 184 <p>Discontinuous stream data may occur on a fairly regular basis, but the
cannam@86 185 timing of, for example, a specific caption is impossible to predict
cannam@86 186 with certainty in most captioning systems. Thus the buffering system
cannam@86 187 should take discontinuous data 'as it comes' rather than working ahead
cannam@86 188 (for a potentially unbounded period) to look for future discontinuous
cannam@86 189 data. As such, discontinuous streams are ignored when managing
cannam@86 190 buffering; their pages simply 'fall out' of the stream when continuous
cannam@86 191 streams are handled properly.</p>
cannam@86 192
cannam@86 193 <p>Buffering requirements need not be explicitly declared or managed for
cannam@86 194 the encoded stream; the decoder simply reads as much data as is
cannam@86 195 necessary to keep all continuous stream types gapless (also ensuring
cannam@86 196 discontinuous data arrives in time) and no more, resulting in optimum
cannam@86 197 implicit buffer usage for a given stream. Because all pages of all
cannam@86 198 data types are stamped with absolute timing information within the
cannam@86 199 stream, inter-stream synchronization timing is always explicitly
cannam@86 200 maintained without the need for explicitly declared buffer-ahead
cannam@86 201 hinting.</p>
cannam@86 202
cannam@86 203 <p>Further details, mechanisms and reasons for the differing arrangement
cannam@86 204 and behavior of continuous and discontinuous streams is discussed
cannam@86 205 later.</p>
cannam@86 206
cannam@86 207 <h3>Whole-stream navigation</h3>
cannam@86 208
cannam@86 209 <p>Ogg is designed so that the simplest navigation operations treat the
cannam@86 210 physical Ogg stream as a whole summary of its streams, rather than
cannam@86 211 navigating each interleaved stream as a separate entity.</p>
cannam@86 212
cannam@86 213 <p>First Example: seeking to a desired time position in a multiplexed (or
cannam@86 214 unmultiplexed) Ogg stream can be accomplished through a bisection
cannam@86 215 search on time position of all pages in the stream (as encoded in the
cannam@86 216 granule position). More powerful searches (such as a key frame-aware
cannam@86 217 seek within video) are also possible with additional search
cannam@86 218 complexity, but similar computational complexity.</p>
cannam@86 219
cannam@86 220 <p>Second Example: A bitstream section may consist of three multiplexed
cannam@86 221 streams of differing lengths. The result of multiplexing these
cannam@86 222 streams should be thought of as a single mixed stream with a length
cannam@86 223 equal to the longest of the three component streams. Although it is
cannam@86 224 also possible to think of the multiplexed results as three concurrent
cannam@86 225 streams of different lengths and it is possible to recover the three
cannam@86 226 original streams, it will also become obvious that once multiplexed,
cannam@86 227 it isn't possible to find the internal lengths of the component
cannam@86 228 streams without a linear search of the whole bitstream section.
cannam@86 229 However, it is possible to find the length of the whole bitstream
cannam@86 230 section easily (in near-constant time per section) just as it is for a
cannam@86 231 single-media unmultiplexed stream.</p>
cannam@86 232
cannam@86 233 <h2>Granule Position</h2>
cannam@86 234
cannam@86 235 <h3>Description</h3>
cannam@86 236
cannam@86 237 <p>The Granule Position is a signed 64 bit field appearing in the header
cannam@86 238 of every Ogg page. Although the granule position represents absolute
cannam@86 239 time within a logical stream, its value does not necessarily directly
cannam@86 240 encode a simple timestamp. It may represent frames elapsed (as in
cannam@86 241 Vorbis), a simple timestamp, or a more complex bit-division encoding
cannam@86 242 (such as in Theora). The exact encoding of the granule position is up
cannam@86 243 to a specific codec.</p>
cannam@86 244
cannam@86 245 <p>The granule position is governed by the following rules:</p>
cannam@86 246
cannam@86 247 <ul>
cannam@86 248
cannam@86 249 <li>Granule Position must always increase forward or remain equal from
cannam@86 250 page to page, be unset, or be zero for a header page. The absolute
cannam@86 251 time to which any correct sequence of granule position maps must
cannam@86 252 similarly always increase forward or remain equal. <i>(A codec may
cannam@86 253 make use of data, such as a control sequence, that only affects codec
cannam@86 254 working state without producing data and thus advancing granule
cannam@86 255 position and time. Although the packet sequence number increases in
cannam@86 256 this case, the granule position, and thus the time position, do
cannam@86 257 not.)</i></li>
cannam@86 258
cannam@86 259 <li>Granule position may only be unset if there no packet defining a
cannam@86 260 time boundary on the page (that is, if no packet in a continuous
cannam@86 261 stream ends on the page, or no packet in a discontinuous stream begins
cannam@86 262 on the page. This will be discussed in more detail under Continuous
cannam@86 263 and Discontinuous streams).</li>
cannam@86 264
cannam@86 265 <li>A codec must be able to translate a given granule position value
cannam@86 266 to a unique, deterministic absolute time value through direct
cannam@86 267 calculation. A codec is not required to be able to translate an
cannam@86 268 absolute time value into a unique granule position value.</li>
cannam@86 269
cannam@86 270 <li>Codecs shall choose a granule position definition that allows that
cannam@86 271 codec means to seek as directly as possible to an immediately
cannam@86 272 decodable point, such as the bit-divided granule position encoding of
cannam@86 273 Theora allows the codec to seek efficiently to key frame without using
cannam@86 274 an index. That is, additional information other than absolute time
cannam@86 275 may be encoded into a granule position value so long as the granule
cannam@86 276 position obeys the above points.</li>
cannam@86 277
cannam@86 278 </ul>
cannam@86 279
cannam@86 280 <h4>Example: timestamp</h4>
cannam@86 281
cannam@86 282 <p>In general, a codec/stream type should choose the simplest granule
cannam@86 283 position encoding that addresses its requirements. The examples here
cannam@86 284 are by no means exhaustive of the possibilities within Ogg.</p>
cannam@86 285
cannam@86 286 <p>A simple granule position could encode a timestamp directly. For
cannam@86 287 example, a granule position that encoded milliseconds from beginning
cannam@86 288 of stream would allow a logical stream length of over 100,000,000,000
cannam@86 289 days before beginning a new logical stream (to avoid the granule
cannam@86 290 position wrapping).</p>
cannam@86 291
cannam@86 292 <h4>Example: framestamp</h4>
cannam@86 293
cannam@86 294 <p>A simple millisecond timestamp granule encoding might suit many stream
cannam@86 295 types, but a millisecond resolution is inappropriate to, eg, most
cannam@86 296 audio encodings where exact single-sample resolution is generally a
cannam@86 297 requirement. A millisecond is both too large a granule and often does
cannam@86 298 not represent an integer number of samples.</p>
cannam@86 299
cannam@86 300 <p>In the event that audio frames are always encoded as the same number of
cannam@86 301 samples, the granule position could simply be a linear count of frames
cannam@86 302 since beginning of stream. This has the advantages of being exact and
cannam@86 303 efficient. Position in time would simply be <tt>[granule_position] *
cannam@86 304 [samples_per_frame] / [samples_per_second]</tt>.</p>
cannam@86 305
cannam@86 306 <h4>Example: samplestamp (Vorbis)</h4>
cannam@86 307
cannam@86 308 <p>Frame counting is insufficient in codecs such as Vorbis where an audio
cannam@86 309 frame [packet] encodes a variable number of samples. In Vorbis's
cannam@86 310 case, the granule position is a count of the number of raw samples
cannam@86 311 from the beginning of stream; the absolute time of
cannam@86 312 a granule position is <tt>[granule_position] /
cannam@86 313 [samples_per_second]</tt>.</p>
cannam@86 314
cannam@86 315 <h4>Example: bit-divided framestamp (Theora)</h4>
cannam@86 316
cannam@86 317 <p>Some video codecs may be able to use the simple framestamp scheme for
cannam@86 318 granule position. However, most modern video codecs introduce at
cannam@86 319 least the following complications:</p>
cannam@86 320
cannam@86 321 <ul>
cannam@86 322
cannam@86 323 <li>video frames are relatively far apart compared to audio samples;
cannam@86 324 for this reason, the point at which a video frame changes to the next
cannam@86 325 frame is usually a strictly defined offset within the frame 'period'.
cannam@86 326 That is, video at 50fps could just as easily define frame transitions
cannam@86 327 &lt;.015, .035, .055...&gt; as at &lt;.00, .02, .04...&gt;.</li>
cannam@86 328
cannam@86 329 <li>frame rates often include drop-frames, leap-frames or other
cannam@86 330 rational-but-non-integer timings.</li>
cannam@86 331
cannam@86 332 <li>Decode must begin at a 'key frame' or 'I frame'. Keyframes usually
cannam@86 333 occur relatively seldom.</li>
cannam@86 334
cannam@86 335 </ul>
cannam@86 336
cannam@86 337 <p>The first two points can be handled straightforwardly via the fact
cannam@86 338 that the codec has complete control mapping granule position to
cannam@86 339 absolute time; non-integer frame rates and offsets can be set in the
cannam@86 340 codec's initial header, and the rest is just arithmetic.</p>
cannam@86 341
cannam@86 342 <p>The third point appears trickier at first glance, but it too can be
cannam@86 343 handled through the granule position mapping mechanism. Here we
cannam@86 344 arrange the granule position in such a way that granule positions of
cannam@86 345 key frames are easy to find. Divide the granule position into two
cannam@86 346 fields; the most-significant bits are an absolute frame counter, but
cannam@86 347 it's only updated at each key frame. The least significant bits encode
cannam@86 348 the number of frames since the last key frame. In this way, each
cannam@86 349 granule position both encodes the absolute time of the current frame
cannam@86 350 as well as the absolute time of the last key frame.</p>
cannam@86 351
cannam@86 352 <p>Seeking to a most recent preceding key frame is then accomplished by
cannam@86 353 first seeking to the original desired point, inspecting the granulepos
cannam@86 354 of the resulting video page, extracting from that granulepos the
cannam@86 355 absolute time of the desired key frame, and then seeking directly to
cannam@86 356 that key frame's page. Of course, it's still possible for an
cannam@86 357 application to ignore key frames and use a simpler seeking algorithm
cannam@86 358 (decode would be unable to present decoded video until the next
cannam@86 359 key frame). Surprisingly many player applications do choose the
cannam@86 360 simpler approach.</p>
cannam@86 361
cannam@86 362 <h3>granule position, packets and pages</h3>
cannam@86 363
cannam@86 364 <p>Although each packet of data in a logical stream theoretically has a
cannam@86 365 specific granule position, only one granule position is encoded
cannam@86 366 per page. It is possible to encode a logical stream such that each
cannam@86 367 page contains only a single packet (so that granule positions are
cannam@86 368 preserved for each packet), however a one-to-one packet/page mapping
cannam@86 369 is not intended to be the general case.</p>
cannam@86 370
cannam@86 371 <p>Because Ogg functions at the page, not packet, level, this
cannam@86 372 once-per-page time information provides Ogg with the finest-grained
cannam@86 373 time information is can use. Ogg passes this granule positioning data
cannam@86 374 to the codec (along with the packets extracted from a page); it is the
cannam@86 375 responsibility of codecs to track timing information at granularities
cannam@86 376 finer than a single page.</p>
cannam@86 377
cannam@86 378 <h3>start-time and end-time positioning</h3>
cannam@86 379
cannam@86 380 <p>A granule position represents the <em>instantaneous time location
cannam@86 381 between two pages</em>. However, continuous streams and discontinuous
cannam@86 382 streams differ on whether the granulepos represents the end-time of
cannam@86 383 the data on a page or the start-time. Continuous streams are
cannam@86 384 'end-time' encoded; the granulepos represents the point in time
cannam@86 385 immediately after the last data decoded from a page. Discontinuous
cannam@86 386 streams are 'start-time' encoded; the granulepos represents the point
cannam@86 387 in time of the first data decoded from the page.</p>
cannam@86 388
cannam@86 389 <p>An Ogg stream type is declared continuous or discontinuous by its
cannam@86 390 codec. A given codec may support both continuous and discontinuous
cannam@86 391 operation so long as any given logical stream is continuous or
cannam@86 392 discontinuous for its entirety and the codec is able to ascertain (and
cannam@86 393 inform the Ogg layer) as to which after decoding the initial stream
cannam@86 394 header. The majority of codecs will always be continuous (such as
cannam@86 395 Vorbis) or discontinuous (such as Writ).</p>
cannam@86 396
cannam@86 397 <p>Start- and end-time encoding do not affect multiplexing sort-order;
cannam@86 398 pages are still sorted by the absolute time a given granulepos maps to
cannam@86 399 regardless of whether that granulepos represents start- or
cannam@86 400 end-time.</p>
cannam@86 401
cannam@86 402 <h2>Multiplex/Demultiplex Division of Labor</h2>
cannam@86 403
cannam@86 404 <p>The Ogg multiplex/demultiplex layer provides mechanisms for encoding
cannam@86 405 raw packets into Ogg pages, decoding Ogg pages back into the original
cannam@86 406 codec packets, determining the logical structure of an Ogg stream, and
cannam@86 407 navigating through and synchronizing with an Ogg stream at a desired
cannam@86 408 stream location. Strict multiplex/demultiplex operations are entirely
cannam@86 409 in the Ogg domain and require no intervention from codecs.</p>
cannam@86 410
cannam@86 411 <p>Implementation of more complex operations does require codec
cannam@86 412 knowledge, however. Unlike other framing systems, Ogg maintains
cannam@86 413 strict separation between framing and the framed bitstream data; Ogg
cannam@86 414 does not replicate codec-specific information in the page/framing
cannam@86 415 data, nor does Ogg blur the line between framing and stream
cannam@86 416 data/metadata. Because Ogg is fully data-agnostic toward the data it
cannam@86 417 frames, operations which require specifics of bitstream data (such as
cannam@86 418 'seek to key frame') also require interaction with the codec layer
cannam@86 419 (because, in this example, the Ogg layer is not aware of the concept
cannam@86 420 of key frames). This is different from systems that blur the
cannam@86 421 separation between framing and stream data in order to simplify the
cannam@86 422 separation of code. The Ogg system purposely keeps the distinction in
cannam@86 423 data simple so that later codec innovations are not constrained by
cannam@86 424 framing design.</p>
cannam@86 425
cannam@86 426 <p>For this reason, however, complex seeking operations require
cannam@86 427 interaction with the codecs in order to decode the granule position of
cannam@86 428 a given stream type back to absolute time or in order to find
cannam@86 429 'decodable points' such as key frames in video.</p>
cannam@86 430
cannam@86 431 <h2>Unsorted Discussion Points</h2>
cannam@86 432
cannam@86 433 <p>flushes around key frames? RFC suggestion: repaginating or building a
cannam@86 434 stream this way is nice but not required</p>
cannam@86 435
cannam@86 436 <h2>Appendix A: multiplexing examples</h2>
cannam@86 437
cannam@86 438 <div id="copyright">
cannam@86 439 The Xiph Fish Logo is a
cannam@86 440 trademark (&trade;) of Xiph.Org.<br/>
cannam@86 441
cannam@86 442 These pages &copy; 1994 - 2005 Xiph.Org. All rights reserved.
cannam@86 443 </div>
cannam@86 444
cannam@86 445 </body>
cannam@86 446 </html>