(1) Field of the Invention
The present invention can be used as any method of coding and decoding multimedia data and, more particularly, as a method of coding and decoding H.264/MPEG-4 AVC Multiview Video.
(2) Description of the Related Art
3D video has been around for a long time. There are several methods of producing the visual effects of 3D to a viewer. One of the methods is to present two sets of moving pictures separately to the viewer's left and right eyes. This is called stereoscopic imaging which involves the capturing of the two sets of moving pictures using two cameras. One of the techniques used previously to present stereoscopic video is using the filtering of the color components where each eye can only view. Such techniques reduce the resolution of a picture that reaches each eye.
Recently, with the advancement in display technologies, viewers are now able to view full resolution videos with each eye. The video standard H.264/MPEG-4 AVC Multiview Video Coding (MVC) is designed for the compression of such 3D imaging where each view is presented in the full resolution.
The video standard H.264/MPEG-4 AVC Multiview Video Coding (MVC) provides a set of compression tools which allow the efficient compression of moving pictures targeted for more than one set of views. The video standard MVC allows the compression of pictures using predictive coding from reconstructed pictures of a different set of views. This “inter-view” prediction utilizes the correlation of pictures captured by different cameras at approximately the same time to efficiently compress the pictures.
In the video standard MVC, the “inter-view” prediction is performed only on pictures of different views having the same picture order count information. The picture order count information is used to indicate the order of reconstructed pictures of the same view. In the video standard MVC, pictures (or view components as defined in the MVC standard) of different views having the same picture order count information are grouped together as a container called an access unit where the size of these access units are constrained by limits defined by a buffer model specified in the video standard. Such constraints are necessary to ensure correct decoding of the compressed video. A view component as defined in the MVC standard as well as in DESCRIPTION is a coded representation of a view in a single access unit. One example of a view is a picture.
The H.264/MPEG-4 AVC High Profile is widely used in various applications, such as High Definition Storage Media and High Definition Digital Video Broadcasting. The Multiview High Profile defined in the video standard MVC is designed as an extension of the H.264/MPEG-4 AVC High Profile where existing implementation of H.264/MPEG-4 AVC High Profile decoders can be easily modified to support the decoding of compressed video streams using the Multiview High Profile. In some implementation of the MVC standard, the decoding of each view can be achieved independently by using the implemented H.264/MPEG-4 AVC decoders.
Video content can be captured by a progressive scan or interlaced scan camera. In the case when the video is captured using the interlaced scan camera, the H.264/MPEG-4 AVC High Profile provides a set of coding tools targeted specifically to handle interlaced scan captured moving pictures. Each picture can be coded as a frame or fields as shown in FIGS. 1A and 1B. FIG. 1A shows pixel sampling locations of a frame picture including two of the interlaced scanned fields. FIG. 1B shows pixel sampling locations of field pictures where each interlaced scanned field is coded as a field picture. In the case when the two complementary fields are coded as a frame as shown in FIG. 1A, the frame has two picture order counts to present the order of each field.
One problem with the prior art (Multiview Video Coding Standard) is that the definition of the container, or access unit, when each of the pictures (or view components as defined in the MVC standard) of different views is coded differently either using frame coding or field coding. FIG. 2 shows an example of access units with different picture coding structures for each view. As shown in FIG. 2, an access unit container A as shown in a module 200 contains one frame picture of a view 2 and one field picture of a view 1 whereas an access unit container B as shown in a module 202 contains only one field picture of the view 1. This reduces the compression efficiency because the frame picture of the view 2 in the access unit A cannot fully utilize the inter-view prediction from the view 1.
The problem is that each access unit container does not always contain all the pictures (or view components) of different views. Thus, it is not possible to constrain the size of each access unit by the limits defined by a buffer model specified in the video standard to ensure proper decoding of the compressed video.
The other problem with the prior art (Multiview Video Coding Standard) is the constraints set to restrict the size of the access unit container which contains pictures (or view components as defined in the MVC standard) of more than one views. Even though the maximum size of an access unit is restricted, the maximum size of each picture (or view components as defined in the MVC standard) of each view within the access unit container is not constrained. This will create problems for some implementation of the MVC decoder where each view is decoded by a separate H.264/MPEG-4 AVC decoder.
The constraints defined to restrict the maximum size of an access unit does not ensure the proper decoding of each view component within the access unit when each view is separately decoded by an independent decoder. Such constraints include the maximum number of slices and the size of each picture (or view component as defined in the MVC standard) within an access unit.
The present invention has been conceived in view of the circumstances, and has an object of providing (i) a coding method and (ii) a decoding method for improving the coding efficiency of a group of pictures having different views, and (iii) a coding apparatus, (iv) a decoding apparatus, (v) a program, and (vi) an integrated circuit for implementing the coding method and the decoding method.
USPTO Backgrounds
USPTO Backgrounds