Digital video capabilities can be provided in a wide range of devices, including digital televisions, digital direct broadcast systems, wireless communication devices, personal digital assistants (PDAs), laptop computers, desktop computers, digital cameras, digital recording devices, cellular or satellite radio telephones, and the like. Digital video devices can provide significant improvements over conventional analog video systems in creating, modifying, transmitting, storing, recording and playing full motion video sequences.
A number of different video coding standards have been established for coding digital video sequences. The Moving Picture Experts Group (MPEG), for example, has developed a number of standards including MPEG-1, MPEG-2 and MPEG-4. Other standards include the International Telecommunication Union Telecommunications (ITU-T) H.263 standard, QuickTime™ technology developed by Apple Computer of Cupertino Calif., Video for Windows™ developed by Microsoft Corporation of Redmond, Wash., Indeo™ developed by Intel Corporation, RealVideo™ from RealNetworks, Inc. of Seattle, Wash., and Cinepak™ developed by SuperMac, Inc. Furthermore, new standards continue to emerge and evolve, including the ITU-T H.264 standard and a number of proprietary standards.
Many video coding standards allow for improved transmission rates of video sequences by coding data in a compressed fashion. Compression reduces the overall amount of data that needs to be transmitted for effective transmission of video frames. Most video coding standards, for example, utilize graphics and video compression techniques designed to facilitate video and image transmission over a narrower bandwidth than could be achieved without the compression. The standards typically set forth coding techniques that are used to create a standard compliant compressed residual block, but also allow for other techniques to be used, as long as the output bit stream complies with the standard.
Many standards, such as the MPEG standards and the ITU-T H.263 and ITU-T H.264 standards support video coding techniques that utilize similarities between successive video frames, referred to as temporal or inter-frame correlation, to provide inter-frame compression. The inter-frame compression techniques exploit data redundancy across frames by converting pixel-based representations of video frames to motion representations. In addition, some video coding techniques may utilize similarities within frames, referred to as spatial or intra-frame correlation, to further compress the video frames.
In order to achieve video frame compression, a digital video device typically includes an encoder for compressing digital video sequences, and a decoder for decompressing the digital video sequences. In many cases, the encoder and decoder form an integrated “CODEC” that operates on blocks of pixels within frames that define the video sequence. As used in this disclosure, the term “CODEC” refers to an encoder, a decoder or an integrated encoder/decoder.
In the MPEG-4 standard, the CODEC divides a video frame to be transmitted into video blocks referred to as “macroblocks.” The ITU-T H.264 standard supports 16 by 16 video blocks, 16 by 8 video blocks, 8 by 16 video blocks, 8 by 8 video blocks, 8 by 4 video blocks, 4 by 8 video blocks and 4 by 4 video blocks. Other standards may support differently sized video blocks. For each video block in the video frame, the CODEC searches similarly sized video blocks of one or more previously encoded video frames to identify a similar video block, referred to as the “prediction block.” The process of comparing a current video block to video blocks of other frames is generally referred to as motion estimation. Once a “prediction block” is identified for a current video block during motion estimation, the CODEC can code the differences between the current video block and the prediction block.
This process of coding the differences between a current video block and the prediction block includes a process referred to as motion compensation. Motion compensation involves creating a “residual,” which is a block of data indicative of the differences between the current video block to be coded and the prediction block. In particular, motion compensation usually refers to the act of fetching the prediction block using a motion vector, and then subtracting the prediction block from an input block to generate the residual. The residual typically includes substantially less data than the original video block that is represented by the difference block. “Intra” encoding also creates a “residual” but does so based on the differences between a video block to be encoded and a different video block of the same frame. The process of creating the residual in intra encoding is often referred to as “intra-prediction.”
In either case, after the residual has been created, a series of additional steps can also be performed to further code the residual and further compress the data. These additional steps may depend on the coding standard being used, but are generally referred to as “residual coding.” In MPEG-4 compliant CODECs, for example, the residual coding may include an 8×8 discrete cosine transform, followed by scalar quantization, followed by a raster-to-zigzag reordering, followed by run-length encoding, followed by Huffman encoding. The coded residual can be transmitted with a motion vector that indicates which video block from the previous frame (or subsequent frame) was used for the coding. A CODEC of a receive device receives the motion vector and the coded residual, and decodes the received information to reconstruct the video sequences.
The residual coding steps often become a bottleneck in video coding. This can be a problem for real time video, such as that used in video telephony (VT) applications. Moreover, limited computational resources and limited battery power can compound these issues for VT applications. Unfortunately, for most standards the residual coding steps are sequential processes that require the previous process to be performed before the subsequent process can be performed.
USPTO Backgrounds
USPTO Backgrounds