Chris@1: Chris@1: Chris@1:
Chris@1:
This document provides a high level description of the Vorbis codec’s construction. A bit-by-bit Chris@1: specification appears beginning in Section 4, “Codec Setup and Packet Decode”. The later Chris@1: sections assume a high-level understanding of the Vorbis decode process, which is provided Chris@1: here. Chris@1:
Chris@1:
Vorbis is a general purpose perceptual audio CODEC intended to allow maximum encoder Chris@1: flexibility, thus allowing it to scale competitively over an exceptionally wide range of bitrates. At Chris@1: the high quality/bitrate end of the scale (CD or DAT rate stereo, 16/24 bits) it is in the same Chris@1: league as MPEG-2 and MPC. Similarly, the 1.0 encoder can encode high-quality CD and DAT Chris@1: rate stereo at below 48kbps without resampling to a lower rate. Vorbis is also intended for lower Chris@1: and higher sample rates (from 8kHz telephony to 192kHz digital masters) and a range of channel Chris@1: representations (monaural, polyphonic, stereo, quadraphonic, 5.1, ambisonic, or up to 255 Chris@1: discrete channels). Chris@1:
Chris@1:
Vorbis I is a forward-adaptive monolithic transform CODEC based on the Modified Discrete Chris@1: Cosine Transform. The codec is structured to allow addition of a hybrid wavelet filterbank in Chris@1: Vorbis II to offer better transient response and reproduction using a transform better suited to Chris@1: localized time events. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
The Vorbis CODEC design assumes a complex, psychoacoustically-aware encoder and simple, Chris@1: low-complexity decoder. Vorbis decode is computationally simpler than mp3, although it does Chris@1: require more working memory as Vorbis has no static probability model; the vector codebooks Chris@1: used in the first stage of decoding from the bitstream are packed in their entirety into the Vorbis Chris@1: bitstream headers. In packed form, these codebooks occupy only a few kilobytes; the extent to Chris@1: which they are pre-decoded into a cache is the dominant factor in decoder memory Chris@1: usage. Chris@1:
Vorbis provides none of its own framing, synchronization or protection against errors; it Chris@1: is solely a method of accepting input audio, dividing it into individual frames and Chris@1: compressing these frames into raw, unformatted ’packets’. The decoder then accepts Chris@1: these raw packets in sequence, decodes them, synthesizes audio frames from them, and Chris@1: reassembles the frames into a facsimile of the original audio stream. Vorbis is a free-form Chris@1: variable bit rate (VBR) codec and packets have no minimum size, maximum size, or Chris@1: fixed/expected size. Packets are designed that they may be truncated (or padded) Chris@1: and remain decodable; this is not to be considered an error condition and is used Chris@1: extensively in bitrate management in peeling. Both the transport mechanism and Chris@1: decoder must allow that a packet may be any size, or end before or after packet decode Chris@1: expects. Chris@1:
Vorbis packets are thus intended to be used with a transport mechanism that provides free-form Chris@1: framing, sync, positioning and error correction in accordance with these design assumptions, such Chris@1: as Ogg (for file transport) or RTP (for network multicast). For purposes of a few examples in this Chris@1: document, we will assume that Vorbis is to be embedded in an Ogg stream specifically, Chris@1: although this is by no means a requirement or fundamental assumption in the Vorbis Chris@1: design. Chris@1:
The specification for embedding Vorbis into an Ogg transport stream is in Section A, Chris@1: “Embedding Vorbis into an Ogg stream”. Chris@1:
Chris@1:
Vorbis’ heritage is as a research CODEC and its current design reflects a desire to allow multiple Chris@1: decades of continuous encoder improvement before running out of room within the codec Chris@1: specification. For these reasons, configurable aspects of codec setup intentionally lean toward the Chris@1: extreme of forward adaptive. Chris@1: Chris@1: Chris@1: Chris@1:
The single most controversial design decision in Vorbis (and the most unusual for a Vorbis Chris@1: developer to keep in mind) is that the entire probability model of the codec, the Huffman and Chris@1: VQ codebooks, is packed into the bitstream header along with extensive CODEC setup Chris@1: parameters (often several hundred fields). This makes it impossible, as it would be with Chris@1: MPEG audio layers, to embed a simple frame type flag in each audio packet, or begin Chris@1: decode at any frame in the stream without having previously fetched the codec setup Chris@1: header. Chris@1:
Note: Vorbis can initiate decode at any arbitrary packet within a bitstream so long as the codec Chris@1: has been initialized/setup with the setup headers. Chris@1:
Thus, Vorbis headers are both required for decode to begin and relatively large as bitstream Chris@1: headers go. The header size is unbounded, although for streaming a rule-of-thumb of 4kB or less Chris@1: is recommended (and Xiph.Org’s Vorbis encoder follows this suggestion). Chris@1:
Our own design work indicates the primary liability of the required header is in mindshare; it is Chris@1: an unusual design and thus causes some amount of complaint among engineers as this runs Chris@1: against current design trends (and also points out limitations in some existing software/interface Chris@1: designs, such as Windows’ ACM codec framework). However, we find that it does not Chris@1: fundamentally limit Vorbis’ suitable application space. Chris@1:
Chris@1:
The Vorbis format is well-defined by its decode specification; any encoder that produces packets Chris@1: that are correctly decoded by the reference Vorbis decoder described below may be considered Chris@1: a proper Vorbis encoder. A decoder must faithfully and completely implement the Chris@1: specification defined below (except where noted) to be considered a proper Vorbis Chris@1: decoder. Chris@1:
Chris@1:
Although Vorbis decode is computationally simple, it may still run into specific limitations of an Chris@1: embedded design. For this reason, embedded designs are allowed to deviate in limited ways from Chris@1: the ‘full’ decode specification yet still be certified compliant. These optional omissions are Chris@1: labelled in the spec where relevant. Chris@1:
Chris@1:
Decoder setup consists of configuration of multiple, self-contained component abstractions that Chris@1: perform specific functions in the decode pipeline. Each different component instance of a specific Chris@1: type is semantically interchangeable; decoder configuration consists both of internal component Chris@1: configuration, as well as arrangement of specific instances into a decode pipeline. Componentry Chris@1: arrangement is roughly as follows: Chris@1:
Chris@1: Chris@1:
Chris@1:
Chris@1:
Global codec configuration consists of a few audio related fields (sample rate, channels), Vorbis Chris@1: version (always ’0’ in Vorbis I), bitrate hints, and the lists of component instances. All other Chris@1: configuration is in the context of specific components. Chris@1:
Chris@1:
Each Vorbis frame is coded according to a master ’mode’. A bitstream may use one or many Chris@1: modes. Chris@1:
The mode mechanism is used to encode a frame according to one of multiple possible Chris@1: methods with the intention of choosing a method best suited to that frame. Different Chris@1: modes are, e.g. how frame size is changed from frame to frame. The mode number of a Chris@1: frame serves as a top level configuration switch for all other specific aspects of frame Chris@1: decode. Chris@1:
A ’mode’ configuration consists of a frame size setting, window type (always 0, the Vorbis Chris@1: window, in Vorbis I), transform type (always type 0, the MDCT, in Vorbis I) and a mapping Chris@1: number. The mapping number specifies which mapping configuration instance to use for low-level Chris@1: packet decode and synthesis. Chris@1:
Chris@1:
A mapping contains a channel coupling description and a list of ’submaps’ that bundle sets Chris@1: of channel vectors together for grouped encoding and decoding. These submaps are Chris@1: not references to external components; the submap list is internal and specific to a Chris@1: mapping. Chris@1:
A ’submap’ is a configuration/grouping that applies to a subset of floor and residue vectors Chris@1: within a mapping. The submap functions as a last layer of indirection such that specific special Chris@1: floor or residue settings can be applied not only to all the vectors in a given mode, but also Chris@1: specific vectors in a specific mode. Each submap specifies the proper floor and residue Chris@1: instance number to use for decoding that submap’s spectral floor and spectral residue Chris@1: vectors. Chris@1:
As an example: Chris@1:
Assume a Vorbis stream that contains six channels in the standard 5.1 format. The sixth Chris@1: channel, as is normal in 5.1, is bass only. Therefore it would be wasteful to encode a Chris@1: full-spectrum version of it as with the other channels. The submapping mechanism can be used Chris@1: to apply a full range floor and residue encoding to channels 0 through 4, and a bass-only Chris@1: representation to the bass channel, thus saving space. In this example, channels 0-4 belong to Chris@1: submap 0 (which indicates use of a full-range floor) and channel 5 belongs to submap 1, which Chris@1: uses a bass-only representation. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
Vorbis encodes a spectral ’floor’ vector for each PCM channel. This vector is a low-resolution Chris@1: representation of the audio spectrum for the given channel in the current frame, generally used Chris@1: akin to a whitening filter. It is named a ’floor’ because the Xiph.Org reference encoder has Chris@1: historically used it as a unit-baseline for spectral resolution. Chris@1:
A floor encoding may be of two types. Floor 0 uses a packed LSP representation on a dB Chris@1: amplitude scale and Bark frequency scale. Floor 1 represents the curve as a piecewise linear Chris@1: interpolated representation on a dB amplitude scale and linear frequency scale. The two floors Chris@1: are semantically interchangeable in encoding/decoding. However, floor type 1 provides more Chris@1: stable inter-frame behavior, and so is the preferred choice in all coupled-stereo and Chris@1: high bitrate modes. Floor 1 is also considerably less expensive to decode than floor Chris@1: 0. Chris@1:
Floor 0 is not to be considered deprecated, but it is of limited modern use. No known Vorbis Chris@1: encoder past Xiph.Org’s own beta 4 makes use of floor 0. Chris@1:
The values coded/decoded by a floor are both compactly formatted and make use of entropy Chris@1: coding to save space. For this reason, a floor configuration generally refers to multiple Chris@1: codebooks in the codebook component list. Entropy coding is thus provided as an Chris@1: abstraction, and each floor instance may choose from any and all available codebooks when Chris@1: coding/decoding. Chris@1:
Chris@1:
The spectral residue is the fine structure of the audio spectrum once the floor curve has been Chris@1: subtracted out. In simplest terms, it is coded in the bitstream using cascaded (multi-pass) vector Chris@1: quantization according to one of three specific packing/coding algorithms numbered Chris@1: 0 through 2. The packing algorithm details are configured by residue instance. As Chris@1: with the floor components, the final VQ/entropy encoding is provided by external Chris@1: codebook instances and each residue instance may choose from any and all available Chris@1: codebooks. Chris@1:
Chris@1: Chris@1: Chris@1: Chris@1:
Codebooks are a self-contained abstraction that perform entropy decoding and, optionally, use Chris@1: the entropy-decoded integer value as an offset into an index of output value vectors, returning Chris@1: the indicated vector of values. Chris@1:
The entropy coding in a Vorbis I codebook is provided by a standard Huffman binary tree Chris@1: representation. This tree is tightly packed using one of several methods, depending on whether Chris@1: codeword lengths are ordered or unordered, or the tree is sparse. Chris@1:
The codebook vector index is similarly packed according to index characteristic. Most commonly, Chris@1: the vector index is encoded as a single list of values of possible values that are then permuted Chris@1: into a list of n-dimensional rows (lattice VQ). Chris@1:
Chris@1:
Chris@1:
Before decoding can begin, a decoder must initialize using the bitstream headers matching the Chris@1: stream to be decoded. Vorbis uses three header packets; all are required, in-order, by Chris@1: this specification. Once set up, decode may begin at any audio packet belonging to Chris@1: the Vorbis stream. In Vorbis I, all packets after the three initial headers are audio Chris@1: packets. Chris@1:
The header packets are, in order, the identification header, the comments header, and the setup Chris@1: header. Chris@1:
Identification Header Chris@1: The identification header identifies the bitstream as Vorbis, Vorbis version, and the simple audio Chris@1: characteristics of the stream such as sample rate and number of channels. Chris@1: Chris@1: Chris@1: Chris@1:
Comment Header Chris@1: The comment header includes user text comments (“tags”) and a vendor string for the Chris@1: application/library that produced the bitstream. The encoding and proper use of the comment Chris@1: header is described in Section 5, “comment field and header specification”. Chris@1:
Setup Header Chris@1: The setup header includes extensive CODEC setup information as well as the complete VQ and Chris@1: Huffman codebooks needed for decode. Chris@1:
Chris@1:
The decoding and synthesis procedure for all audio packets is fundamentally the same. Chris@1:
Note that clever rearrangement of the synthesis arithmetic is possible; as an example, one can Chris@1: take advantage of symmetries in the MDCT to store the right-hand transform data of a partial Chris@1: MDCT for a 50% inter-frame buffer space savings, and then complete the transform later before Chris@1: overlap/add with the next frame. This optimization produces entirely equivalent output and is Chris@1: naturally perfectly legal. The decoder must be entirely mathematically equivalent to the Chris@1: specification, it need not be a literal semantic implementation. Chris@1:
Packet type decode Chris@1: Vorbis I uses four packet types. The first three packet types mark each of the three Vorbis Chris@1: headers described above. The fourth packet type marks an audio packet. All other packet types Chris@1: are reserved; packets marked with a reserved type should be ignored. Chris@1:
Following the three header packets, all packets in a Vorbis I stream are audio. The first step of Chris@1: audio packet decode is to read and verify the packet type; a non-audio packet when audio is Chris@1: expected indicates stream corruption or a non-compliant stream. The decoder must ignore the Chris@1: packet and not attempt decoding it to audio. Chris@1:
Mode decode Chris@1: Vorbis allows an encoder to set up multiple, numbered packet ’modes’, as described earlier, all of Chris@1: which may be used in a given Vorbis stream. The mode is encoded as an integer used as a direct Chris@1: offset into the mode instance index. Chris@1:
Window shape decode (long windows only) Chris@1: Vorbis frames may be one of two PCM sample sizes specified during codec setup. In Vorbis I, Chris@1: legal frame sizes are powers of two from 64 to 8192 samples. Aside from coupling, Vorbis Chris@1: handles channels as independent vectors and these frame sizes are in samples per Chris@1: channel. Chris@1: Chris@1: Chris@1: Chris@1:
Vorbis uses an overlapping transform, namely the MDCT, to blend one frame into the next, Chris@1: avoiding most inter-frame block boundary artifacts. The MDCT output of one frame is windowed Chris@1: according to MDCT requirements, overlapped 50% with the output of the previous frame and Chris@1: added. The window shape assures seamless reconstruction. Chris@1:
This is easy to visualize in the case of equal sized-windows: Chris@1:
Chris@1: Chris@1:
Chris@1:
And slightly more complex in the case of overlapping unequal sized windows: Chris@1:
Chris@1: Chris@1:
Chris@1:
In the unequal-sized window case, the window shape of the long window must be modified for Chris@1: seamless lapping as above. It is possible to correctly infer window shape to be applied to the Chris@1: current window from knowing the sizes of the current, previous and next window. It is legal for a Chris@1: decoder to use this method. However, in the case of a long window (short windows require no Chris@1: modification), Vorbis also codes two flag bits to specify pre- and post- window shape. Although Chris@1: not strictly necessary for function, this minor redundancy allows a packet to be fully decoded to Chris@1: the point of lapping entirely independently of any other packet, allowing easier abstraction of Chris@1: decode layers as well as allowing a greater level of easy parallelism in encode and Chris@1: decode. Chris@1:
A description of valid window functions for use with an inverse MDCT can be found in [1]. Chris@1: Vorbis windows all use the slope function Chris@1:
Chris@1:
floor decode Chris@1: Each floor is encoded/decoded in channel order, however each floor belongs to a ’submap’ that Chris@1: specifies which floor configuration to use. All floors are decoded before residue decode Chris@1: begins. Chris@1:
residue decode Chris@1: Although the number of residue vectors equals the number of channels, channel coupling may Chris@1: mean that the raw residue vectors extracted during decode do not map directly to specific Chris@1: channels. When channel coupling is in use, some vectors will correspond to coupled magnitude or Chris@1: angle. The coupling relationships are described in the codec setup and may differ from frame to Chris@1: frame, due to different mode numbers. Chris@1:
Vorbis codes residue vectors in groups by submap; the coding is done in submap order from Chris@1: submap 0 through n-1. This differs from floors which are coded using a configuration provided by Chris@1: submap number, but are coded individually in channel order. Chris@1:
inverse channel coupling Chris@1: A detailed discussion of stereo in the Vorbis codec can be found in the document Chris@1: Stereo Channel Coupling in the Vorbis CODEC. Vorbis is not limited to only stereo Chris@1: coupling, but the stereo document also gives a good overview of the generic coupling Chris@1: mechanism. Chris@1:
Vorbis coupling applies to pairs of residue vectors at a time; decoupling is done in-place a Chris@1: pair at a time in the order and using the vectors specified in the current mapping Chris@1: configuration. The decoupling operation is the same for all pairs, converting square polar Chris@1: representation (where one vector is magnitude and the second angle) back to Cartesian Chris@1: representation. Chris@1:
After decoupling, in order, each pair of vectors on the coupling list, the resulting residue vectors Chris@1: represent the fine spectral detail of each output channel. Chris@1: Chris@1: Chris@1: Chris@1:
generate floor curve Chris@1: The decoder may choose to generate the floor curve at any appropriate time. It is reasonable to Chris@1: generate the output curve when the floor data is decoded from the raw packet, or it Chris@1: can be generated after inverse coupling and applied to the spectral residue directly, Chris@1: combining generation and the dot product into one step and eliminating some working Chris@1: space. Chris@1:
Both floor 0 and floor 1 generate a linear-range, linear-domain output vector to be multiplied Chris@1: (dot product) by the linear-range, linear-domain spectral residue. Chris@1:
compute floor/residue dot product Chris@1: This step is straightforward; for each output channel, the decoder multiplies the floor curve and Chris@1: residue vectors element by element, producing the finished audio spectrum of each Chris@1: channel. Chris@1:
One point is worth mentioning about this dot product; a common mistake in a fixed point Chris@1: implementation might be to assume that a 32 bit fixed-point representation for floor and Chris@1: residue and direct multiplication of the vectors is sufficient for acceptable spectral depth Chris@1: in all cases because it happens to mostly work with the current Xiph.Org reference Chris@1: encoder. Chris@1:
However, floor vector values can span ∼140dB (∼24 bits unsigned), and the audio spectrum Chris@1: vector should represent a minimum of 120dB (∼21 bits with sign), even when output is to a 16 Chris@1: bit PCM device. For the residue vector to represent full scale if the floor is nailed Chris@1: to −140dB, it must be able to span 0 to +140dB. For the residue vector to reach Chris@1: full scale if the floor is nailed at 0dB, it must be able to represent −140dB to +0dB. Chris@1: Thus, in order to handle full range dynamics, a residue vector may span −140dB to Chris@1: +140dB entirely within spec. A 280dB range is approximately 48 bits with sign; thus the Chris@1: residue vector must be able to represent a 48 bit range and the dot product must Chris@1: be able to handle an effective 48 bit times 24 bit multiplication. This range may be Chris@1: achieved using large (64 bit or larger) integers, or implementing a movable binary point Chris@1: representation. Chris@1:
inverse monolithic transform (MDCT) Chris@1: The audio spectrum is converted back into time domain PCM audio via an inverse Modified Chris@1: Discrete Cosine Transform (MDCT). A detailed description of the MDCT is available in Chris@1: [1]. Chris@1:
Note that the PCM produced directly from the MDCT is not yet finished audio; it must be Chris@1: Chris@1: Chris@1: Chris@1: lapped with surrounding frames using an appropriate window (such as the Vorbis window) before Chris@1: the MDCT can be considered orthogonal. Chris@1:
overlap/add data Chris@1: Windowed MDCT output is overlapped and added with the right hand data of the previous Chris@1: window such that the 3/4 point of the previous window is aligned with the 1/4 point of the Chris@1: current window (as illustrated in the window overlap diagram). At this point, the audio data Chris@1: between the center of the previous frame and the center of the current frame is now finished and Chris@1: ready to be returned. Chris@1:
cache right hand data Chris@1: The decoder must cache the right hand portion of the current frame to be lapped with the left Chris@1: hand portion of the next frame. Chris@1:
return finished audio data Chris@1: The overlapped portion produced from overlapping the previous and current frame data Chris@1: is finished data to be returned by the decoder. This data spans from the center of Chris@1: the previous window to the center of the current window. In the case of same-sized Chris@1: windows, the amount of data to return is one-half block consisting of and only of the Chris@1: overlapped portions. When overlapping a short and long window, much of the returned Chris@1: range is not actually overlap. This does not damage transform orthogonality. Pay Chris@1: attention however to returning the correct data range; the amount of data to be returned Chris@1: is: Chris@1:
Chris@1:
Chris@1:from the center of the previous window to the center of the current window. Chris@1:
Data is not returned from the first frame; it must be used to ’prime’ the decode engine. The Chris@1: encoder accounts for this priming when calculating PCM offsets; after the first frame, the proper Chris@1: PCM output offset is ’0’ (as no data has been returned yet). Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
The Vorbis codec uses relatively unstructured raw packets containing arbitrary-width binary Chris@1: integer fields. Logically, these packets are a bitstream in which bits are coded one-by-one by the Chris@1: encoder and then read one-by-one in the same monotonically increasing order by the decoder. Chris@1: Most current binary storage arrangements group bits into a native word size of eight bits Chris@1: (octets), sixteen bits, thirty-two bits or, less commonly other fixed word sizes. The Vorbis Chris@1: bitpacking convention specifies the correct mapping of the logical packet bitstream into an actual Chris@1: representation in fixed-width words. Chris@1:
Chris@1:
In most contemporary architectures, a ’byte’ is synonymous with an ’octet’, that is, eight bits. Chris@1: This has not always been the case; seven, ten, eleven and sixteen bit ’bytes’ have been used. Chris@1: For purposes of the bitpacking convention, a byte implies the native, smallest integer Chris@1: storage representation offered by a platform. On modern platforms, this is generally Chris@1: assumed to be eight bits (not necessarily because of the processor but because of the Chris@1: filesystem/memory architecture. Modern filesystems invariably offer bytes as the fundamental Chris@1: atom of storage). A ’word’ is an integer size that is a grouped multiple of this smallest Chris@1: size. Chris@1:
The most ubiquitous architectures today consider a ’byte’ to be an octet (eight bits) and a word Chris@1: to be a group of two, four or eight bytes (16, 32 or 64 bits). Note however that the Vorbis Chris@1: bitpacking convention is still well defined for any native byte size; Vorbis uses the native Chris@1: bit-width of a given storage system. This document assumes that a byte is one octet for purposes Chris@1: of example. Chris@1:
Chris@1: Chris@1: Chris@1: Chris@1:
A byte has a well-defined ’least significant’ bit (LSb), which is the only bit set when the byte is Chris@1: storing the two’s complement integer value +1. A byte’s ’most significant’ bit (MSb) is at the Chris@1: opposite end of the byte. Bits in a byte are numbered from zero at the LSb to n (n = 7 in an Chris@1: octet) for the MSb. Chris@1:
Chris@1:
Words are native groupings of multiple bytes. Several byte orderings are possible in a word; the Chris@1: common ones are 3-2-1-0 (’big endian’ or ’most significant byte first’ in which the Chris@1: highest-valued byte comes first), 0-1-2-3 (’little endian’ or ’least significant byte first’ in Chris@1: which the lowest value byte comes first) and less commonly 3-1-2-0 and 0-2-1-3 (’mixed Chris@1: endian’). Chris@1:
The Vorbis bitpacking convention specifies storage and bitstream manipulation at the byte, not Chris@1: word, level, thus host word ordering is of a concern only during optimization when writing high Chris@1: performance code that operates on a word of storage at a time rather than by byte. Chris@1: Logically, bytes are always coded and decoded in order from byte zero through byte Chris@1: n. Chris@1:
Chris@1:
The Vorbis codec has need to code arbitrary bit-width integers, from zero to 32 bits Chris@1: wide, into packets. These integer fields are not aligned to the boundaries of the byte Chris@1: representation; the next field is written at the bit position at which the previous field Chris@1: ends. Chris@1:
The encoder logically packs integers by writing the LSb of a binary integer to the logical Chris@1: bitstream first, followed by next least significant bit, etc, until the requested number of bits Chris@1: have been coded. When packing the bits into bytes, the encoder begins by placing Chris@1: the LSb of the integer to be written into the least significant unused bit position of Chris@1: the destination byte, followed by the next-least significant bit of the source integer Chris@1: and so on up to the requested number of bits. When all bits of the destination byte Chris@1: have been filled, encoding continues by zeroing all bits of the next byte and writing Chris@1: the next bit into the bit position 0 of that byte. Decoding follows the same process Chris@1: Chris@1: Chris@1: Chris@1: as encoding, but by reading bits from the byte stream and reassembling them into Chris@1: integers. Chris@1:
Chris@1:
The signedness of a specific number resulting from decode is to be interpreted by the decoder Chris@1: given decode context. That is, the three bit binary pattern ’b111’ can be taken to represent Chris@1: either ’seven’ as an unsigned integer, or ’-1’ as a signed, two’s complement integer. The Chris@1: encoder and decoder are responsible for knowing if fields are to be treated as signed or Chris@1: unsigned. Chris@1:
Chris@1:
Code the 4 bit integer value ’12’ [b1100] into an empty bytestream. Bytestream result: Chris@1:
Chris@1:
Continue by coding the 3 bit integer value ’-1’ [b111]: Chris@1:
Chris@1:
Continue by coding the 7 bit integer value ’17’ [b0010001]: Chris@1:
Chris@1:
Continue by coding the 13 bit integer value ’6969’ [b110 11001110 01]: Chris@1:
Chris@1:
Chris@1:
Reading from the beginning of the bytestream encoded in the above example: Chris@1:
Chris@1:
We read two, two-bit integer fields, resulting in the returned numbers ’b00’ and ’b11’. Two things Chris@1: are worth noting here: Chris@1:
Chris@1:
The typical use of bitpacking is to produce many independent byte-aligned packets which are Chris@1: embedded into a larger byte-aligned container structure, such as an Ogg transport bitstream. Chris@1: Externally, each bytestream (encoded bitstream) must begin and end on a byte boundary. Often, Chris@1: the encoded bitstream is not an integer number of bytes, and so there is unused (uncoded) space Chris@1: in the last byte of a packet. Chris@1:
Unused space in the last byte of a bytestream is always zeroed during the coding process. Thus, Chris@1: should this unused space be read, it will return binary zeroes. Chris@1:
Attempting to read past the end of an encoded packet results in an ’end-of-packet’ condition. Chris@1: End-of-packet is not to be considered an error; it is merely a state indicating that there is Chris@1: insufficient remaining data to fulfill the desired read size. Vorbis uses truncated packets as a Chris@1: normal mode of operation, and as such, decoders must handle reading past the end of a packet as Chris@1: a typical mode of operation. Any further read operations after an ’end-of-packet’ condition shall Chris@1: also return ’end-of-packet’. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
Reading a zero-bit-wide integer returns the value ’0’ and does not increment the stream cursor. Chris@1: Reading to the end of the packet (but not past, such that an ’end-of-packet’ condition has not Chris@1: triggered) and then reading a zero bit integer shall succeed, returning 0, and not trigger an Chris@1: end-of-packet condition. Reading a zero-bit-wide integer after a previous read sets ’end-of-packet’ Chris@1: shall also fail with ’end-of-packet’. Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
Unlike practically every other mainstream audio codec, Vorbis has no statically configured Chris@1: probability model, instead packing all entropy decoding configuration, VQ and Huffman, into the Chris@1: bitstream itself in the third header, the codec setup header. This packed configuration consists of Chris@1: multiple ’codebooks’, each containing a specific Huffman-equivalent representation for decoding Chris@1: compressed codewords as well as an optional lookup table of output vector values to which a Chris@1: decoded Huffman value is applied as an offset, generating the final decoded output corresponding Chris@1: to a given compressed codeword. Chris@1:
Chris@1:
The codebook mechanism is built on top of the vorbis bitpacker. Both the codebooks themselves Chris@1: and the codewords they decode are unrolled from a packet as a series of arbitrary-width values Chris@1: read from the stream according to Section 2, “Bitpacking Convention”. Chris@1:
Chris@1:
For purposes of the examples below, we assume that the storage system’s native byte width is Chris@1: eight bits. This is not universally true; see Section 2, “Bitpacking Convention” for discussion Chris@1: relating to non-eight-bit bytes. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
A codebook begins with a 24 bit sync pattern, 0x564342: Chris@1:
Chris@1:
16 bit [codebook_dimensions] and 24 bit [codebook_entries] fields: Chris@1:
Chris@1:
Next is the [ordered] bit flag: Chris@1:
Chris@1:
Chris@1:Each entry, numbering a total of [codebook_entries], is assigned a codeword length. Chris@1: We now read the list of codeword lengths and store these lengths in the array Chris@1: [codebook_codeword_lengths]. Decode of lengths is according to whether the [ordered] flag Chris@1: is set or unset. Chris@1:
The decoder first reads one additional bit flag, the [sparse] flag. This flag determines Chris@1: whether or not the codebook contains unused entries that are not to be included in Chris@1: Chris@1: Chris@1: Chris@1: the codeword decode tree: Chris@1:
Chris@1:
Chris@1:The decoder now performs for each of the [codebook_entries] codebook entries: Chris@1:
Chris@1:
Chris@1:
After all codeword lengths have been decoded, the decoder reads the vector lookup table. Vorbis Chris@1: I supports three lookup types: Chris@1:
The lookup table type is read as a four bit unsigned integer: Chris@1:
Chris@1:Codebook decode precedes according to [codebook_lookup_type]: Chris@1:
Chris@1:
An ’end of packet’ during any read operation in the above steps is considered an error condition Chris@1: rendering the stream undecodable. Chris@1:
Huffman decision tree representation Chris@1: The [codebook_codeword_lengths] array and [codebook_entries] value uniquely define the Chris@1: Huffman decision tree used for entropy decoding. Chris@1:
Briefly, each used codebook entry (recall that length-unordered codebooks support unused Chris@1: codeword entries) is assigned, in order, the lowest valued unused binary Huffman codeword Chris@1: possible. Assume the following codeword length list: Chris@1:
Chris@1:
Assigning codewords in order (lowest possible value of the appropriate length to highest) results Chris@1: in the following codeword list: Chris@1:
Chris@1:
Note: Unlike most binary numerical values in this document, we intend the above codewords to Chris@1: be read and used bit by bit from left to right, thus the codeword ’001’ is the bit string ’zero, zero, Chris@1: one’. When determining ’lowest possible value’ in the assignment definition above, the leftmost Chris@1: bit is the MSb. Chris@1:
It is clear that the codeword length list represents a Huffman decision tree with the entry Chris@1: numbers equivalent to the leaves numbered left-to-right: Chris@1:
Chris@1: Chris@1:
Chris@1:
As we assign codewords in order, we see that each choice constructs a new leaf in the leftmost Chris@1: possible position. Chris@1:
Note that it’s possible to underspecify or overspecify a Huffman tree via the length list. Chris@1: In the above example, if codeword seven were eliminated, it’s clear that the tree is Chris@1: unfinished: Chris@1:
Chris@1: Chris@1:
Chris@1:
Similarly, in the original codebook, it’s clear that the tree is fully populated and a ninth Chris@1: codeword is impossible. Both underspecified and overspecified trees are an error condition Chris@1: rendering the stream undecodable. Take special care that a codebook with a single used Chris@1: entry is handled properly; it consists of a single codework of zero bits and ’reading’ Chris@1: a value out of such a codebook always returns the single used value and sinks zero Chris@1: bits. Chris@1:
Codebook entries marked ’unused’ are simply skipped in the assigning process. They have no Chris@1: codeword and do not appear in the decision tree, thus it’s impossible for any bit pattern read Chris@1: from the stream to decode to that entry number. Chris@1: Chris@1: Chris@1: Chris@1:
VQ lookup table vector representation Chris@1: Unpacking the VQ lookup table vectors relies on the following values: Chris@1:
Decoding (unpacking) a specific vector in the vector lookup table proceeds according to Chris@1: [codebook_lookup_type]. The unpacked vector values are what a codebook would return Chris@1: during audio packet decode in a VQ context. Chris@1:
Vector value decode: Lookup type 1 Chris@1: Lookup type one specifies a lattice VQ lookup table built algorithmically from a list of Chris@1: scalar values. Calculate (unpack) the final values of a codebook entry vector from Chris@1: the entries in [codebook_multiplicands] as follows ([value_vector] is the output Chris@1: vector representing the vector of values for entry number [lookup_offset] in this Chris@1: codebook): Chris@1:
Chris@1:
Vector value decode: Lookup type 2 Chris@1: Lookup type two specifies a VQ lookup table in which each scalar in each vector is explicitly set Chris@1: by the [codebook_multiplicands] array in a one-to-one mapping. Calculate [unpack] the final Chris@1: values of a codebook entry vector from the entries in [codebook_multiplicands] as follows Chris@1: ([value_vector] is the output vector representing the vector of values for entry number Chris@1: [lookup_offset] in this codebook): Chris@1:
Chris@1:
Chris@1:
The decoder uses the codebook abstraction much as it does the bit-unpacking convention; a Chris@1: specific codebook reads a codeword from the bitstream, decoding it into an entry number, and Chris@1: then returns that entry number to the decoder (when used in a scalar entropy coding context), or Chris@1: uses that entry number as an offset into the VQ lookup table, returning a vector of values (when Chris@1: used in a context desiring a VQ value). Scalar or VQ context is always explicit; any Chris@1: call to the codebook mechanism requests either a scalar entry number or a lookup Chris@1: vector. Chris@1:
Note that VQ lookup type zero indicates that there is no lookup table; requesting Chris@1: decode using a codebook of lookup type 0 in any context expecting a vector return Chris@1: value (even in a case where a vector of dimension one) is forbidden. If decoder setup Chris@1: or decode requests such an action, that is an error condition rendering the packet Chris@1: Chris@1: Chris@1: Chris@1: undecodable. Chris@1:
Using a codebook to read from the packet bitstream consists first of reading and decoding the Chris@1: next codeword in the bitstream. The decoder reads bits until the accumulated bits match a Chris@1: codeword in the codebook. This process can be though of as logically walking the Chris@1: Huffman decode tree by reading one bit at a time from the bitstream, and using the Chris@1: bit as a decision boolean to take the 0 branch (left in the above examples) or the 1 Chris@1: branch (right in the above examples). Walking the tree finishes when the decode process Chris@1: hits a leaf in the decision tree; the result is the entry number corresponding to that Chris@1: leaf. Reading past the end of a packet propagates the ’end-of-stream’ condition to the Chris@1: decoder. Chris@1:
When used in a scalar context, the resulting codeword entry is the desired return Chris@1: value. Chris@1:
When used in a VQ context, the codeword entry number is used as an offset into the VQ lookup Chris@1: table. The value returned to the decoder is the vector of scalars corresponding to this Chris@1: offset. Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
This document serves as the top-level reference document for the bit-by-bit decode specification Chris@1: of Vorbis I. This document assumes a high-level understanding of the Vorbis decode Chris@1: process, which is provided in Section 1, “Introduction and Description”. Section 2, Chris@1: “Bitpacking Convention” covers reading and writing bit fields from and to bitstream Chris@1: packets. Chris@1:
Chris@1:
A Vorbis bitstream begins with three header packets. The header packets are, in order, the Chris@1: identification header, the comments header, and the setup header. All are required for decode Chris@1: compliance. An end-of-packet condition during decoding the first or third header packet renders Chris@1: the stream undecodable. End-of-packet decoding the comment header is a non-fatal error Chris@1: condition. Chris@1:
Chris@1:
Each header packet begins with the same header fields. Chris@1:
Chris@1:
Decode continues according to packet type; the identification header is type 1, the comment Chris@1: header type 3 and the setup header type 5 (these types are all odd as a packet with a leading Chris@1: single bit of ’0’ is an audio packet). The packets must occur in the order of identification, Chris@1: comment, setup. Chris@1:
Chris@1:
The identification header is a short header of only a few fields used to declare the stream Chris@1: definitively as Vorbis, and provide a few externally relevant pieces of information about the audio Chris@1: stream. The identification header is coded as follows: Chris@1:
Chris@1:
[vorbis_version] is to read ’0’ in order to be compatible with this document. Both Chris@1: [audio_channels] and [audio_sample_rate] must read greater than zero. Allowed final Chris@1: blocksize values are 64, 128, 256, 512, 1024, 2048, 4096 and 8192 in Vorbis I. [blocksize_0] Chris@1: must be less than or equal to [blocksize_1]. The framing bit must be nonzero. Failure to meet Chris@1: any of these conditions renders a stream undecodable. Chris@1:
The bitrate fields above are used only as hints. The nominal bitrate field especially may be Chris@1: considerably off in purely VBR streams. The fields are meaningful only when greater than Chris@1: zero. Chris@1:
Chris@1:
Comment header decode and data specification is covered in Section 5, “comment field and Chris@1: header specification”. Chris@1:
Chris@1:
Vorbis codec setup is configurable to an extreme degree: Chris@1:
Chris@1: Chris@1:
Chris@1:
The setup header contains the bulk of the codec setup information needed for decode. The setup Chris@1: header contains, in order, the lists of codebook configurations, time-domain transform Chris@1: configurations (placeholders in Vorbis I), floor configurations, residue configurations, channel Chris@1: mapping configurations and mode configurations. It finishes with a framing bit of ’1’. Header Chris@1: decode proceeds in the following order: Chris@1:
Codebooks Chris@1: Chris@1: Chris@1: Chris@1:
Time domain transforms Chris@1: These hooks are placeholders in Vorbis I. Nevertheless, the configuration placeholder values must Chris@1: be read to maintain bitstream sync. Chris@1:
Chris@1:
Floors Chris@1: Vorbis uses two floor types; header decode is handed to the decode abstraction of the appropriate Chris@1: type. Chris@1:
Chris@1:
Residues Chris@1: Vorbis uses three residue types; header decode of each type is identical. Chris@1:
Chris@1:
Mappings Chris@1: Mappings are used to set up specific pipelines for encoding multichannel audio with varying Chris@1: channel mapping applications. Vorbis I uses a single mapping type (0), with implicit PCM Chris@1: channel mappings. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
After reading mode descriptions, setup header decode is complete. Chris@1:
Chris@1:
Following the three header packets, all packets in a Vorbis I stream are audio. The first step of Chris@1: audio packet decode is to read and verify the packet type. A non-audio packet when audio is Chris@1: expected indicates stream corruption or a non-compliant stream. The decoder must ignore the Chris@1: packet and not attempt decoding it to audio. Chris@1:
Chris@1:
Chris@1:
Vorbis windows all use the slope function y = sin( ∗ sin 2((x + 0.5)∕n ∗ π)), where n is window
Chris@1: size and x ranges 0…n− 1, but dissimilar lapping requirements can affect overall shape. Window
Chris@1: generation proceeds as follows:
Chris@1:
Chris@1:
else Chris@1:
else Chris@1:
An end-of-packet condition up to this point should be considered an error that discards this Chris@1: packet from the stream. An end of packet condition past this point is to be considered a possible Chris@1: nominal occurrence. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
From this point on, we assume out decode context is using mode number [mode_number] Chris@1: from configuration array [vorbis_mode_configurations] and the map number Chris@1: [vorbis_mode_mapping] (specified by the current mode) taken from the mapping configuration Chris@1: array [vorbis_mapping_configurations]. Chris@1:
Floor curves are decoded one-by-one in channel order. Chris@1:
For each floor [i] of [audio_channels] Chris@1:
An end-of-packet condition during floor decode shall result in packet decode zeroing all channel Chris@1: output vectors and skipping to the add/overlap output stage. Chris@1:
Chris@1:
A possible result of floor decode is that a specific vector is marked ’unused’ which indicates that Chris@1: that final output vector is all-zero values (and the floor is zero). The residue for that vector is not Chris@1: coded in the stream, save for one complication. If some vectors are used and some are not, Chris@1: Chris@1: Chris@1: Chris@1: channel coupling could result in mixing a zeroed and nonzeroed vector to produce two nonzeroed Chris@1: vectors. Chris@1:
for each [i] from 0 ... [vorbis_mapping_coupling_steps]-1 Chris@1:
Chris@1:
Chris@1:
Unlike floors, which are decoded in channel order, the residue vectors are decoded in submap Chris@1: order. Chris@1:
for each submap [i] in order from 0 ... [vorbis_mapping_submaps]-1 Chris@1:
Chris@1:
else Chris@1:
Chris@1:
for each [i] from [vorbis_mapping_coupling_steps]-1 descending to 0 Chris@1:
Chris@1:
else Chris@1:
else Chris@1:
else Chris@1:
Chris@1:
For each channel, synthesize the floor curve from the decoded floor information, according to Chris@1: packet type. Note that the vector synthesis length for floor computation is [n]/2. Chris@1:
For each channel, multiply each element of the floor curve by each element of that Chris@1: channel’s residue vector. The result is the dot product of the floor and residue vectors for Chris@1: each channel; the produced vectors are the length [n]/2 audio spectrum for each Chris@1: channel. Chris@1:
One point is worth mentioning about this dot product; a common mistake in a fixed point Chris@1: implementation might be to assume that a 32 bit fixed-point representation for floor and Chris@1: residue and direct multiplication of the vectors is sufficient for acceptable spectral depth Chris@1: in all cases because it happens to mostly work with the current Xiph.Org reference Chris@1: encoder. Chris@1:
However, floor vector values can span ∼140dB (∼24 bits unsigned), and the audio spectrum Chris@1: vector should represent a minimum of 120dB (∼21 bits with sign), even when output is to a 16 Chris@1: bit PCM device. For the residue vector to represent full scale if the floor is nailed Chris@1: to −140dB, it must be able to span 0 to +140dB. For the residue vector to reach Chris@1: full scale if the floor is nailed at 0dB, it must be able to represent −140dB to +0dB. Chris@1: Thus, in order to handle full range dynamics, a residue vector may span −140dB to Chris@1: +140dB entirely within spec. A 280dB range is approximately 48 bits with sign; thus the Chris@1: residue vector must be able to represent a 48 bit range and the dot product must Chris@1: be able to handle an effective 48 bit times 24 bit multiplication. This range may be Chris@1: achieved using large (64 bit or larger) integers, or implementing a movable binary point Chris@1: representation. Chris@1:
Chris@1:
Convert the audio spectrum vector of each channel back into time domain PCM audio via an Chris@1: Chris@1: Chris@1: Chris@1: inverse Modified Discrete Cosine Transform (MDCT). A detailed description of the MDCT is Chris@1: available in [1]. The window function used for the MDCT is the function described Chris@1: earlier. Chris@1:
Chris@1:
Windowed MDCT output is overlapped and added with the right hand data of the previous Chris@1: window such that the 3/4 point of the previous window is aligned with the 1/4 point of the Chris@1: current window (as illustrated in paragraph 1.3.2, “Window shape decode (long windows Chris@1: only)”). The overlapped portion produced from overlapping the previous and current frame data Chris@1: is finished data to be returned by the decoder. This data spans from the center of Chris@1: the previous window to the center of the current window. In the case of same-sized Chris@1: windows, the amount of data to return is one-half block consisting of and only of the Chris@1: overlapped portions. When overlapping a short and long window, much of the returned Chris@1: range does not actually overlap. This does not damage transform orthogonality. Pay Chris@1: attention however to returning the correct data range; the amount of data to be returned Chris@1: is: Chris@1:
Chris@1:
Chris@1:from the center (element windowsize/2) of the previous window to the center (element Chris@1: windowsize/2-1, inclusive) of the current window. Chris@1:
Data is not returned from the first frame; it must be used to ’prime’ the decode engine. The Chris@1: encoder accounts for this priming when calculating PCM offsets; after the first frame, the proper Chris@1: PCM output offset is ’0’ (as no data has been returned yet). Chris@1:
Chris@1:
Vorbis I specifies only a channel mapping type 0. In mapping type 0, channel mapping is Chris@1: implicitly defined as follows for standard audio applications. As of revision 16781 (20100113), the Chris@1: specification adds defined channel locations for 6.1 and 7.1 surround. Ordering/location for Chris@1: Chris@1: Chris@1: Chris@1: greater-than-eight channels remains ’left to the implementation’. Chris@1:
These channel orderings refer to order within the encoded stream. It is naturally possible for a Chris@1: decoder to produce output with channels in any order. Any such decoder should explicitly Chris@1: document channel reordering behavior. Chris@1:
Chris@1:
Applications using Vorbis for dedicated purposes may define channel mapping as seen fit. Future Chris@1: channel mappings (such as three and four channel Ambisonics) will make use of channel Chris@1: mappings other than mapping 0. Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
The Vorbis text comment header is the second (of three) header packets that begin a Vorbis Chris@1: bitstream. It is meant for short text comments, not arbitrary metadata; arbitrary metadata Chris@1: belongs in a separate logical bitstream (usually an XML stream type) that provides greater Chris@1: structure and machine parseability. Chris@1:
The comment field is meant to be used much like someone jotting a quick note on the bottom of Chris@1: a CDR. It should be a little information to remember the disc by and explain it to others; a Chris@1: short, to-the-point text note that need not only be a couple words, but isn’t going to be more Chris@1: than a short paragraph. The essentials, in other words, whatever they turn out to be, Chris@1: eg: Chris@1:
Chris@1:
Honest Bob and the Factory-to-Dealer-Incentives, “I’m Still Around”, opening Chris@1: for Moxy Früvous, 1997.
Chris@1:
Chris@1:
The comment header is logically a list of eight-bit-clean vectors; the number of vectors is Chris@1: bounded to 232 − 1 and the length of each vector is limited to 232 − 1 bytes. The vector length is Chris@1: Chris@1: Chris@1: Chris@1: encoded; the vector contents themselves are not null terminated. In addition to the vector list, Chris@1: there is a single vector for vendor name (also 8 bit clean, length encoded in 32 bits). For Chris@1: example, the 1.0 release of libvorbis set the vendor string to “Xiph.Org libVorbis I Chris@1: 20020717”. Chris@1:
The vector lengths and number of vectors are stored lsb first, according to the bit Chris@1: packing conventions of the vorbis codec. However, since data in the comment header Chris@1: is octet-aligned, they can simply be read as unaligned 32 bit little endian unsigned Chris@1: integers. Chris@1:
The comment header is decoded as follows: Chris@1:
Chris@1:
Chris@1:
The comment vectors are structured similarly to a UNIX environment variable. That is, Chris@1: comment fields consist of a field name and a corresponding value and look like: Chris@1:
Chris@1:
Chris@1:
Chris@1:The field name is case-insensitive and may consist of ASCII 0x20 through 0x7D, 0x3D (’=’) Chris@1: excluded. ASCII 0x41 through 0x5A inclusive (characters A-Z) is to be considered equivalent to Chris@1: ASCII 0x61 through 0x7A inclusive (characters a-z). Chris@1:
The field name is immediately followed by ASCII 0x3D (’=’); this equals sign is used to Chris@1: terminate the field name. Chris@1:
0x3D is followed by 8 bit clean UTF-8 encoded value of the field contents to the end of the Chris@1: field. Chris@1:
Field names Chris@1: Below is a proposed, minimal list of standard field names with a description of intended use. No Chris@1: single or group of field names is mandatory; a comment header may contain one, all or none of Chris@1: the names in this list. Chris@1:
Chris@1:
Implications Chris@1: Field names should not be ’internationalized’; this is a concession to simplicity not Chris@1: an attempt to exclude the majority of the world that doesn’t speak English. Field Chris@1: contents, however, use the UTF-8 character encoding to allow easy representation of any Chris@1: language. Chris@1:
We have the length of the entirety of the field and restrictions on the field name so that Chris@1: the field name is bounded in a known way. Thus we also have the length of the field Chris@1: contents. Chris@1:
Individual ’vendors’ may use non-standard field names within reason. The proper Chris@1: use of comment fields should be clear through context at this point. Abuse will be Chris@1: discouraged. Chris@1: Chris@1: Chris@1: Chris@1:
There is no vendor-specific prefix to ’nonstandard’ field names. Vendors should make some effort Chris@1: to avoid arbitrarily polluting the common namespace. We will generally collect the more useful Chris@1: tags here to help with standardization. Chris@1:
Field names are not required to be unique (occur once) within a comment header. As an Chris@1: example, assume a track was recorded by three well know artists; the following is permissible, Chris@1: and encouraged: Chris@1:
Chris@1:
Chris@1:
Chris@1:Chris@1:
The comment header comprises the entirety of the second bitstream header packet. Unlike the Chris@1: first bitstream header packet, it is not generally the only packet on the second page and may not Chris@1: be restricted to within the second bitstream page. The length of the comment header packet is Chris@1: (practically) unbounded. The comment header packet is not optional; it must be present in the Chris@1: bitstream even if it is effectively empty. Chris@1:
The comment header is encoded as follows (as per Ogg’s standard bitstream mapping which Chris@1: renders least-significant-bit of the word to be coded into the least significant available bit of the Chris@1: current bitstream octet first): Chris@1:
Chris@1:
This is actually somewhat easier to describe in code; implementation of the above can be found Chris@1: in vorbis/lib/info.c, _vorbis_pack_comment() and _vorbis_unpack_comment(). Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
Vorbis floor type zero uses Line Spectral Pair (LSP, also alternately known as Line Spectral Chris@1: Frequency or LSF) representation to encode a smooth spectral envelope curve as the frequency Chris@1: response of the LSP filter. This representation is equivalent to a traditional all-pole infinite Chris@1: impulse response filter as would be used in linear predictive coding; LSP representation may be Chris@1: converted to LPC representation and vice-versa. Chris@1:
Chris@1:
Floor zero configuration consists of six integer fields and a list of VQ codebooks for use in Chris@1: coding/decoding the LSP filter coefficient values used by each frame. Chris@1:
Chris@1:
Configuration information for instances of floor zero decodes from the codec setup header (third Chris@1: packet). configuration decode proceeds as follows: Chris@1:
Chris@1:
An end-of-packet condition during any of these bitstream reads renders this stream undecodable. Chris@1: In addition, any element of the array [floor0_book_list] that is greater than the maximum Chris@1: codebook number for this bitstream is an error condition that also renders the stream Chris@1: undecodable. Chris@1:
Chris@1:
Extracting a floor0 curve from an audio packet consists of first decoding the curve Chris@1: amplitude and [floor0_order] LSP coefficient values from the bitstream, and then Chris@1: computing the floor curve, which is defined as the frequency response of the decoded LSP Chris@1: filter. Chris@1:
Packet decode proceeds as follows: Chris@1:
Take note of the following properties of decode: Chris@1:
Chris@1:
Given an [amplitude] integer and [coefficients] vector from packet decode as well as Chris@1: the [floor0_order], [floor0_rate], [floor0_bark_map_size], [floor0_amplitude_bits] and Chris@1: [floor0_amplitude_offset] values from floor setup, and an output vector size [n] specified by the Chris@1: decode process, we compute a floor output vector. Chris@1:
If the value [amplitude] is zero, the return value is a length [n] vector with all-zero Chris@1: scalars. Otherwise, begin by assuming the following definitions for the given vector to be Chris@1: synthesized: Chris@1:
Chris@1:
where Chris@1:
Chris@1:
and Chris@1:
Chris@1:
The above is used to synthesize the LSP curve on a Bark-scale frequency axis, then map the Chris@1: result to a linear-scale frequency axis. Similarly, the below calculation synthesizes the output Chris@1: LSP curve [output] on a log (dB) amplitude scale, mapping it to linear amplitude in the last Chris@1: step: Chris@1:
Chris@1:
else [floor0_order] is even Chris@1:
Chris@1:
Chris@1:
Vorbis floor type one uses a piecewise straight-line representation to encode a spectral envelope Chris@1: curve. The representation plots this curve mechanically on a linear frequency axis and a Chris@1: logarithmic (dB) amplitude axis. The integer plotting algorithm used is similar to Bresenham’s Chris@1: algorithm. Chris@1:
Chris@1:
Chris@1:
Floor type one represents a spectral curve as a series of line segments. Synthesis constructs a Chris@1: floor curve using iterative prediction in a process roughly equivalent to the following simplified Chris@1: description: Chris@1:
Consider the following example, with values chosen for ease of understanding rather than Chris@1: representing typical configuration: Chris@1:
For the below example, we assume a floor setup with an [n] of 128. The list of selected X values Chris@1: in increasing order is 0,16,32,48,64,80,96,112 and 128. In list order, the values interleave as 0, Chris@1: 128, 64, 32, 96, 16, 48, 80 and 112. The corresponding list-order Y values as decoded from an Chris@1: example packet are 110, 20, -5, -45, 0, -25, -10, 30 and -10. We compute the floor in the following Chris@1: way, beginning with the first line: Chris@1:
Chris@1: Chris@1:
Chris@1:
We now draw new logical lines to reflect the correction to new˙Y, and iterate for X positions 32 Chris@1: and 96: Chris@1:
Chris@1: Chris@1:
Chris@1:
Although the new Y value at X position 96 is unchanged, it is still used later as an endpoint for Chris@1: further refinement. From here on, the pattern should be clear; we complete the floor computation Chris@1: as follows: Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1: Chris@1:
Chris@1:
Chris@1: Chris@1:
Chris@1:
A more efficient algorithm with carefully defined integer rounding behavior is used for actual Chris@1: decode, as described later. The actual algorithm splits Y value computation and line plotting Chris@1: into two steps with modifications to the above algorithm to eliminate noise accumulation Chris@1: through integer roundoff/truncation. Chris@1:
Chris@1:
A list of floor X values is stored in the packet header in interleaved format (used in list order Chris@1: during packet decode and synthesis). This list is split into partitions, and each partition is Chris@1: assigned to a partition class. X positions 0 and [n] are implicit and do not belong to an explicit Chris@1: partition or partition class. Chris@1:
A partition class consists of a representation vector width (the number of Y values which Chris@1: the partition class encodes at once), a ’subclass’ value representing the number of Chris@1: alternate entropy books the partition class may use in representing Y values, the list of Chris@1: [subclass] books and a master book used to encode which alternate books were chosen Chris@1: for representation in a given packet. The master/subclass mechanism is meant to be Chris@1: used as a flexible representation cascade while still using codebooks only in a scalar Chris@1: context. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
An end-of-packet condition while reading any aspect of a floor 1 configuration during Chris@1: setup renders a stream undecodable. In addition, a [floor1_class_masterbooks] or Chris@1: [floor1_subclass_books] scalar element greater than the highest numbered codebook Chris@1: configured in this stream is an error condition that renders the stream undecodable. Vector Chris@1: [floor1_x_list] is limited to a maximum length of 65 elements; a setup indicating more than 65 Chris@1: total elements (including elements 0 and 1 set prior to the read loop) renders the stream Chris@1: undecodable. All vector [floor1_x_list] element values must be unique within the vector; a Chris@1: non-unique value renders the stream undecodable. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
Packet decode begins by checking the [nonzero] flag: Chris@1:
Chris@1:
Chris@1:If [nonzero] is unset, that indicates this channel contained no audio energy in this frame. Chris@1: Decode immediately returns a status indicating this floor curve (and thus this channel) is unused Chris@1: this frame. (A return status of ’unused’ is different from decoding a floor that has all Chris@1: points set to minimum representation amplitude, which happens to be approximately Chris@1: -140dB). Chris@1:
Assuming [nonzero] is set, decode proceeds as follows: Chris@1:
Chris@1:
An end-of-packet condition during curve decode should be considered a nominal occurrence; if Chris@1: end-of-packet is reached during any read operation above, floor decode is to return ’unused’ Chris@1: status as if the [nonzero] flag had been unset at the beginning of decode. Chris@1:
Vector [floor1_Y] contains the values from packet decode needed for floor 1 synthesis. Chris@1:
Chris@1:
Curve computation is split into two logical steps; the first step derives final Y amplitude values Chris@1: from the encoded, wrapped difference values taken from the bitstream. The second step Chris@1: plots the curve lines. Also, although zero-difference values are used in the iterative Chris@1: prediction to find final Y values, these points are conditionally skipped during final Chris@1: line computation in step two. Skipping zero-difference values allows a smoother line Chris@1: fit. Chris@1:
Although some aspects of the below algorithm look like inconsequential optimizations, Chris@1: implementors are warned to follow the details closely. Deviation from implementing a strictly Chris@1: equivalent algorithm can result in serious decoding errors. Chris@1:
Additional note: Although [floor1_final_Y] values in the prediction loop and at the end of Chris@1: step 1 are inherently limited by the prediction algorithm to [0, [range]), it is possible to abuse Chris@1: the setup and codebook machinery to produce negative or over-range results. We suggest that Chris@1: decoder implementations guard the values in vector [floor1_final_Y] by clamping each Chris@1: element to [0, [range]) after step 1. Variants of this suggestion are acceptable as valid floor1 Chris@1: setups cannot produce out of range values. Chris@1:
Chris@1:
Unwrap the always-positive-or-zero values read from the packet into +/- difference Chris@1: values, then apply to line prediction. Chris@1:
Chris@1:
Curve synthesis generates a return vector [floor] of length [n] (where [n] is provided by Chris@1: the decode process calling to floor decode). Floor 1 curve synthesis makes use of the Chris@1: [floor1_X_list], [floor1_final_Y] and [floor1_step2_flag] vectors, as well as Chris@1: [floor1_multiplier] and [floor1_values] values. Chris@1:
Decode begins by sorting the scalars from vectors [floor1_X_list], [floor1_final_Y] and Chris@1: [floor1_step2_flag] together into new vectors [floor1_X_list]’, [floor1_final_Y]’ Chris@1: and [floor1_step2_flag]’ according to ascending sort order of the values in Chris@1: [floor1_X_list]. That is, sort the values of [floor1_X_list] and then apply the same Chris@1: permutation to elements of the other two vectors so that the X, Y and step2_flag values Chris@1: still match. Chris@1:
Then compute the final curve in one pass: Chris@1:
Chris@1:
Chris@1:
A residue vector represents the fine detail of the audio spectrum of one channel in an audio frame Chris@1: after the encoder subtracts the floor curve and performs any channel coupling. A residue vector Chris@1: may represent spectral lines, spectral magnitude, spectral phase or hybrids as mixed by channel Chris@1: coupling. The exact semantic content of the vector does not matter to the residue Chris@1: abstraction. Chris@1:
Whatever the exact qualities, the Vorbis residue abstraction codes the residue vectors into the Chris@1: bitstream packet, and then reconstructs the vectors during decode. Vorbis makes use of three Chris@1: different encoding variants (numbered 0, 1 and 2) of the same basic vector encoding Chris@1: abstraction. Chris@1:
Chris@1:
Residue format partitions each vector in the vector bundle into chunks, classifies each Chris@1: chunk, encodes the chunk classifications and finally encodes the chunks themselves Chris@1: using the the specific VQ arrangement defined for each selected classification. The Chris@1: exact interleaving and partitioning vary by residue encoding number, however the Chris@1: high-level process used to classify and encode the residue vector is the same in all three Chris@1: variants. Chris@1:
A set of coded residue vectors are all of the same length. High level coding structure, ignoring for Chris@1: the moment exactly how a partition is encoded and simply trusting that it is, is as Chris@1: follows: Chris@1:
Chris@1: Chris@1:
Chris@1:
Chris@1:
Residue 0 and 1 differ only in the way the values within a residue partition are interleaved during Chris@1: partition encoding (visually treated as a black box–or cyan box or brown box–in the above Chris@1: figure). Chris@1:
Residue encoding 0 interleaves VQ encoding according to the dimension of the codebook used to Chris@1: Chris@1: Chris@1: Chris@1: encode a partition in a specific pass. The dimension of the codebook need not be the same in Chris@1: multiple passes, however the partition size must be an even multiple of the codebook Chris@1: dimension. Chris@1:
As an example, assume a partition vector of size eight, to be encoded by residue 0 using Chris@1: codebook sizes of 8, 4, 2 and 1: Chris@1:
Chris@1:
It is worth mentioning at this point that no configurable value in the residue coding setup is Chris@1: restricted to a power of two. Chris@1:
Chris@1:
Residue 1 does not interleave VQ encoding. It represents partition vector scalars in order. As Chris@1: with residue 0, however, partition length must be an integer multiple of the codebook dimension, Chris@1: although dimension may vary from pass to pass. Chris@1:
As an example, assume a partition vector of size eight, to be encoded by residue 0 using Chris@1: codebook sizes of 8, 4, 2 and 1: Chris@1:
Chris@1:
Chris@1:
Residue type two can be thought of as a variant of residue type 1. Rather than encoding multiple Chris@1: passed-in vectors as in residue type 1, the ch passed in vectors of length n are first interleaved Chris@1: and flattened into a single vector of length ch*n. Encoding then proceeds as in type 1. Decoding Chris@1: is as in type 1 with decode interleave reversed. If operating on a single vector to begin with, Chris@1: residue type 1 and type 2 are equivalent. Chris@1:
Chris@1: Chris@1:
Chris@1:
Chris@1:
Chris@1:
Header decode for all three residue types is identical. Chris@1:
[residue_begin] and [residue_end] select the specific sub-portion of each vector that is Chris@1: actually coded; it implements akin to a bandpass where, for coding purposes, the vector Chris@1: effectively begins at element [residue_begin] and ends at [residue_end]. Preceding and Chris@1: following values in the unpacked vectors are zeroed. Note that for residue type 2, these Chris@1: values as well as [residue_partition_size]apply to the interleaved vector, not the Chris@1: individual vectors before interleave. [residue_partition_size] is as explained above, Chris@1: [residue_classifications] is the number of possible classification to which a partition can Chris@1: belong and [residue_classbook] is the codebook number used to code classification Chris@1: codewords. The number of dimensions in book [residue_classbook] determines how Chris@1: many classification values are grouped into a single classification codeword. Note that Chris@1: the number of entries and dimensions in book [residue_classbook], along with Chris@1: [residue_classifications], overdetermines to possible number of classification Chris@1: codewords. If [residue_classifications]ˆ[residue_classbook].dimensions exceeds Chris@1: [residue_classbook].entries, the bitstream should be regarded to be undecodable. Chris@1:
Next we read a bitmap pattern that specifies which partition classes code values in which Chris@1: passes. Chris@1:
Chris@1:
Finally, we read in a list of book numbers, each corresponding to specific bit set in the cascade Chris@1: bitmap. We loop over the possible codebook classifications and the maximum possible number of Chris@1: encoding stages (8 in Vorbis I, as constrained by the elements of the cascade bitmap being eight Chris@1: bits): Chris@1:
Chris@1:
An end-of-packet condition at any point in header decode renders the stream undecodable. Chris@1: In addition, any codebook number greater than the maximum numbered codebook Chris@1: set up in this stream also renders the stream undecodable. All codebooks in array Chris@1: [residue_books] are required to have a value mapping. The presence of codebook in array Chris@1: [residue_books] without a value mapping (maptype equals zero) renders the stream Chris@1: undecodable. Chris@1:
Chris@1:
Format 0 and 1 packet decode is identical except for specific partition interleave. Format 2 packet Chris@1: decode can be built out of the format 1 decode process. Thus we describe first the decode Chris@1: infrastructure identical to all three formats. Chris@1:
In addition to configuration information, the residue decode process is passed the number of Chris@1: vectors in the submap bundle and a vector of flags indicating if any of the vectors are not to be Chris@1: decoded. If the passed in number of vectors is 3 and vector number 1 is marked ’do not decode’, Chris@1: decode skips vector 1 during the decode loop. However, even ’do not decode’ vectors are Chris@1: allocated and zeroed. Chris@1:
Depending on the values of [residue_begin] and [residue_end], it is obvious that the Chris@1: encoded portion of a residue vector may be the entire possible residue vector or some other strict Chris@1: subset of the actual residue vector size with zero padding at either uncoded end. However, it is Chris@1: also possible to set [residue_begin] and [residue_end] to specify a range partially or wholly Chris@1: beyond the maximum vector size. Before beginning residue decode, limit [residue_begin] Chris@1: and [residue_end] to the maximum possible vector size as follows. We assume that Chris@1: the number of vectors being encoded, [ch] is provided by the higher level decoding Chris@1: process. Chris@1:
Chris@1:
The following convenience values are conceptually useful to clarifying the decode process: Chris@1:
Chris@1:
Packet decode proceeds as follows, matching the description offered earlier in the document. Chris@1:
An end-of-packet condition during packet decode is to be considered a nominal occurrence. Chris@1: Decode returns the result of vector decode up to that point. Chris@1:
Chris@1:
Format zero decodes partitions exactly as described earlier in the ’Residue Format: residue 0’ Chris@1: section. The following pseudocode presents the same algorithm. Assume: Chris@1:
Chris@1:
Chris@1:
Format 1 decodes partitions exactly as described earlier in the ’Residue Format: residue 1’ Chris@1: section. The following pseudocode presents the same algorithm. Assume: Chris@1:
Chris@1:
Chris@1:
Format 2 is reducible to format 1. It may be implemented as an additional step prior to and an Chris@1: additional post-decode step after a normal format 1 decode. Chris@1: Chris@1: Chris@1: Chris@1:
Format 2 handles ’do not decode’ vectors differently than residue 0 or 1; if all vectors are marked Chris@1: ’do not decode’, no decode occurrs. However, if at least one vector is to be decoded, all Chris@1: the vectors are decoded. We then request normal format 1 to decode a single vector Chris@1: representing all output channels, rather than a vector for each channel. After decode, Chris@1: deinterleave the vector into independent vectors, one for each output channel. That Chris@1: is: Chris@1:
Chris@1:
Chris@1:
The equations below are used in multiple places by the Vorbis codec specification. Rather than Chris@1: cluttering up the main specification documents, they are defined here and referenced where Chris@1: appropriate. Chris@1:
Chris@1:
Chris@1:
The ”ilog(x)” function returns the position number (1 through n) of the highest set bit in the Chris@1: two’s complement integer value [x]. Values of [x] less than zero are defined to return Chris@1: zero. Chris@1:
Chris@1:
Examples: Chris@1:
Chris@1:
”float32_unpack(x)” is intended to translate the packed binary representation of a Vorbis Chris@1: codebook float value into the representation used by the decoder for floating point numbers. For Chris@1: purposes of this example, we will unpack a Vorbis float32 into a host-native floating point Chris@1: number. Chris@1:
Chris@1:
Chris@1:
”lookup1_values(codebook_entries,codebook_dimensions)” is used to compute the Chris@1: correct length of the value index for a codebook VQ lookup table of lookup type 1. Chris@1: The values on this list are permuted to construct the VQ vector lookup table of size Chris@1: [codebook_entries]. Chris@1:
The return value for this function is defined to be ’the greatest integer value for which Chris@1: [return_value] to the power of [codebook_dimensions] is less than or equal to Chris@1: [codebook_entries]’. Chris@1:
Chris@1:
”low_neighbor(v,x)” finds the position n in vector [v] of the greatest value scalar element for Chris@1: which n is less than [x] and vector [v] element n is less than vector [v] element Chris@1: [x]. Chris@1:
Chris@1:
”high_neighbor(v,x)” finds the position n in vector [v] of the lowest value scalar element for Chris@1: which n is less than [x] and vector [v] element n is greater than vector [v] element Chris@1: [x]. Chris@1:
Chris@1:
”render_point(x0,y0,x1,y1,X)” is used to find the Y value at point X along the line specified by Chris@1: x0, x1, y0 and y1. This function uses an integer algorithm to solve for the point directly without Chris@1: calculating intervening values along the line. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
Chris@1:
Floor decode type one uses the integer line drawing algorithm of ”render_line(x0, y0, x1, y1, v)” Chris@1: to construct an integer floor curve for contiguous piecewise line segments. Note that it has not Chris@1: been relevant elsewhere, but here we must define integer division as rounding division of both Chris@1: positive and negative numbers toward zero. Chris@1:
Chris@1:
Chris@1:
The vector [floor1_inverse_dB_table] is a 256 element static lookup table consiting of the Chris@1: following values (read left to right then top to bottom): Chris@1:
Chris@1:
Chris@1:
This document describes using Ogg logical and physical transport streams to encapsulate Vorbis Chris@1: compressed audio packet data into file form. Chris@1:
The Section 1, “Introduction and Description” provides an overview of the construction of Chris@1: Vorbis audio packets. Chris@1:
The Ogg bitstream overview and Ogg logical bitstream and framing spec provide detailed Chris@1: descriptions of Ogg transport streams. This specification document assumes a working Chris@1: knowledge of the concepts covered in these named backround documents. Please read them Chris@1: first. Chris@1:
Chris@1:
The Ogg/Vorbis I specification currently dictates that Ogg/Vorbis streams use Ogg transport Chris@1: streams in degenerate, unmultiplexed form only. That is: Chris@1:
This is not to say that it is not currently possible to multiplex Vorbis with other media Chris@1: types into a multi-stream Ogg file. At the time this document was written, Ogg was Chris@1: becoming a popular container for low-bitrate movies consisting of DivX video and Vorbis Chris@1: audio. However, a ’Vorbis I audio file’ is taken to imply Vorbis audio existing alone Chris@1: within a degenerate Ogg stream. A compliant ’Vorbis audio player’ is not required to Chris@1: implement Ogg support beyond the specific support of Vorbis within a degenrate Ogg Chris@1: stream (naturally, application authors are encouraged to support full multiplexed Ogg Chris@1: handling). Chris@1:
Chris@1:
The MIME type of Ogg files depend on the context. Specifically, complex multimedia and Chris@1: applications should use application/ogg, while visual media should use video/ogg, and audio Chris@1: audio/ogg. Vorbis data encapsulated in Ogg may appear in any of those types. RTP Chris@1: encapsulated Vorbis should use audio/vorbis + audio/vorbis-config. Chris@1:
Chris@1:
Ogg encapsulation of a Vorbis packet stream is straightforward. Chris@1:
Note that the last decoded (fully lapped) PCM sample from a packet is not Chris@1: necessarily the middle sample from that block. If, eg, the current Vorbis packet Chris@1: encodes a ”long block” and the next Vorbis packet encodes a ”short block”, the last Chris@1: decodable sample from the current packet be at position (3*long_block_length/4) - Chris@1: (short_block_length/4). Chris@1:
In both of these cases in which the initial audio PCM starting offset is nonzero, the Chris@1: second finished audio packet must flush the page on which it appears and the Chris@1: third packet begin a fresh page. This allows the decoder to always be able to Chris@1: perform PCM position adjustments before needing to return any PCM data from Chris@1: synthesis, resulting in correct positioning information without any aditional seeking Chris@1: logic. Chris@1:
Note: Failure to do so should, at worst, cause a decoder implementation to return Chris@1: incorrect positioning information for seeking operations at the very beginning of the Chris@1: stream. Chris@1:
Please consult RFC 5215 “RTP Payload Format for Vorbis Encoded Audio” for description of Chris@1: how to embed Vorbis audio in an RTP stream. Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
Ogg is a Xiph.Org Foundation effort to protect essential tenets of Internet multimedia from Chris@1: corporate hostage-taking; Open Source is the net’s greatest tool to keep everyone honest. See Chris@1: About the Xiph.Org Foundation for details. Chris@1:
Ogg Vorbis is the first Ogg audio CODEC. Anyone may freely use and distribute the Ogg and Chris@1: Vorbis specification, whether in a private, public or corporate capacity. However, the Xiph.Org Chris@1: Foundation and the Ogg project (xiph.org) reserve the right to set the Ogg Vorbis specification Chris@1: and certify specification compliance. Chris@1:
Xiph.Org’s Vorbis software CODEC implementation is distributed under a BSD-like license. This Chris@1: does not restrict third parties from distributing independent implementations of Vorbis software Chris@1: under other licenses. Chris@1:
Ogg, Vorbis, Xiph.Org Foundation and their logos are trademarks (tm) of the Xiph.Org Chris@1: Foundation. These pages are copyright (C) 1994-2007 Xiph.Org Foundation. All rights Chris@1: reserved. Chris@1:
This document is set using LATEX. Chris@1: Chris@1: Chris@1: Chris@1:
Chris@1:
Chris@1: [1] T. Sporer, K. Brandenburg and Chris@1: B. Edler, The use of multirate filter banks for coding of high quality digital audio, Chris@1: http://www.iocon.com/resource/docs/ps/eusipco_corrected.ps. Chris@1:
Chris@1: