H.264, also known as Moving Pictures Experts Group-4 (MPEG-4) Part 10 or MPEG-4 Advanced Video Coding (AVC), is a standard for video compression. A video includes a series of pictures (or frames), with each frame consisting of a two-dimensional array of pixels. The pixels are divided into macroblocks (a 16×16 array of pixels). Each macroblock has a macroblock number; in general, the macroblocks are consecutively numbered starting at the top-left of the frame, increasing monotonically from left-to-right and top-to-bottom. The macroblocks can be grouped into slices, and the slices can be grouped into slice groups. Macroblocks within a slice are arranged in ascending order by macroblock number. A slice can include any number of macroblocks, which may or may not be contiguous; that is, macroblocks in one slice may be interspersed among macroblocks of one or more other slices of other slice groups, although macroblocks from slices in the same slice group are not interspersed with each other.
The feature of H.264 that allows macroblocks to be grouped into slices as just described is generally referred to as flexible macroblock ordering (FMO). FMO is one of the error resiliency tools that can be used by a decoder to conceal errors if slices are lost or corrupted during transmission. Macroblocks in a missing or corrupted slice can be reconstructed by interpolating or extrapolating macroblock information from another slice.
Another H.264 feature is generally referred to as arbitrary slice ordering (ASO). With ASO, slices can be transmitted in any order. For example, a slice may be sent as soon as it is ready—that is, a slice may be streamed to a decoder as soon as all of the macroblocks that make up that slice are encoded. As a result, a slice from one slice group may be sent, followed by a slice from another slice group, followed by another slice from the first slice group, and so on.
Thus, generally speaking, H.264 allows encoded macroblocks to be sent “out of order.” That is, the macroblocks may be encoded in a particular order (e.g., from top-to-bottom and from left-to-right, in ascending order according to macroblock number), but the order in which the macroblocks are sent to, and hence received by, a decoding device may be different from the encoding order.
In addition to basic decoding operations, decoding devices perform deblocking operations. In deblocking, a deblocking filter is applied to decoded macroblocks in order to smooth the boundaries between adjacent macroblocks.
With reference to FIG. 1, in order to deblock macroblock M1, information may be needed for macroblock M2 (the nearest neighbor on the left of macroblock M1) and perhaps for macroblock M3 (the nearest neighbor above macroblock M1). However, macroblock M1 may be in one slice (e.g., slice S2) and the neighboring macroblocks M2 and M3 may be in a different slice (e.g., slice S1) or slices. With ASO enabled, slice S2 may be sent to the decoder before slice S1. Thus, macroblock M1 may be ready for deblocking, but deblocking may have to be delayed if slice S1 has not yet arrived at the decoder.
As a result, H.264 decoding is typically implemented using a two-pass approach. In the first pass, basic decoding (decoding prior to deblocking) is performed and the decoded data is saved as it is decoded. In the second pass, the decoded data is reloaded and deblocking is performed. Unfortunately, this type of implementation can place additional burdens on decoding devices. More specifically, decoding devices may need higher data bus bandwidths to accommodate the transfers of data to and from memory, additional memory to store the decoded data prior to deblocking, and longer computing times because the first and second passes are performed sequentially.
USPTO Backgrounds
USPTO Backgrounds