Images or moving images are one type of information transmitted in a communication system or recorded on a storage device. In the related art, there is known an image coding technology for transmitting and storing images (hereinafter, referred to as including moving images as well).
As a moving image coding scheme, there are known H.264/MPEG-4 advanced video coding (AVC) and high-efficiency video coding (HEVC) that is a follow-up codec of AVC (NPL 1).
In these moving image coding schemes, generally, a predicted image is generated on the basis of a locally decoded image obtained by coding/decoding an input image, and a prediction residual (may be called “difference image” or “residual difference image”) obtained by subtracting the predicted image from the input image (source image) is coded. As a method for generating the predicted image, inter-frame prediction (inter-prediction) and intra-frame prediction (intra-prediction) are exemplified.
HEVC employs a technology of realizing temporal scalability assuming that a content is reproduced at a temporally reduced frame rate such as when a content of 60 fps is reproduced at 30 fps. Specifically, HEVC imposes constraints such that a temporal identifier (TemporalID; called a sub-layer identifier as well) is assigned to each picture and that a picture of a higher temporal identifier does not reference a picture of a lower temporal identifier. Accordingly, when only pictures of a specific temporal identifier are thinned out for reproduction, it is not necessary to decode pictures to which a higher temporal identifier is attached.
Recently, there has been suggested a scalable coding technology or a hierarchical coding technology coding images hierarchically according to a necessary data rate. As a representative scalable coding scheme (hierarchical coding scheme), there are known scalable HEVC (SHVC) and multiview HEVC (MV-HEVC).
SHVC supports spatial scalability, temporal scalability, and SNR scalability. For example, in the case of spatial scalability, an image downsampled from a source image to a desired resolution is coded as a lower layer. Next, inter-layer prediction is performed in a higher layer to remove redundancy between layers (NPL 2).
MV-HEVC supports view scalability. For example, in a case of coding three viewpoint images of a viewpoint image 0 (layer 0), a viewpoint image 1 (layer 1), and a viewpoint image 2 (layer 2), redundancy between the layers can be removed by performing inter-layer prediction to predict higher layers of the viewpoint image 1 and the viewpoint image 2 from a lower layer (layer 0) (NPL 3).
The inter-layer prediction utilized in scalable coding schemes such as SHVC and MV-HEVC includes inter-layer image prediction and inter-layer motion prediction. In the inter-layer image prediction, texture information (image) of a decoded picture of a lower layer (or a layer different from a target layer) is utilized to generate a predicted image of the target layer. In the inter-layer motion prediction, motion information of a decoded picture of a lower layer (or a layer different from a target layer) is utilized to derive a predicted value of the motion information of the target layer. That is, inter-layer prediction is performed by utilizing the decoded picture of the lower layer (or a layer different from the target layer) as a reference picture for the target layer.
In NPL 1, a profile and a level are defined so as to define processing performance necessary for an image decoding device (decoder) to decode coded data (or hierarchically coded data) of an image.
A profile defines processing performance that a decoder complying with a standard has to exhibit, assuming a specific application, and is defined by a combination or a set of coding tools (underlying technologies). Defining a profile has the advantage of enabling reduction of complexity of a decoder/encoder because individual applications may only implement an appropriate profile, not the entire standard.
A level defines the upper limit of the processing performance of a decoder as well as the range of the size of circuitry and defines constraints on parameters such as the maximum number of pixels processed per time, the maximum resolution of an image, the maximum bit rate, the maximum size of a reference image buffer, and the minimum compression ratio. That is, a level defines the processing performance of a decoder as well as the complexity of a bitstream. A level also defines the extent to which a tool defined by each profile is to be supported. Thus, a higher level is required to support a lower level.
In NPL 1, for example, various parameters restricted by a level are exemplified by, as illustrated in FIG. 52(a), the maximum luminance picture size (Max luma picture size), the maximum bit rate (Max bit rate), the maximum CPB size (Max CPB size), the maximum number of slice segments per picture (Max slice segments per picture), the maximum number of tile rows per picture (Max number of tile rows), and the maximum number of tile columns per picture (Max number of tile columns). In addition, various parameters restricted by a level and applied to a specific profile are exemplified by, as illustrated in FIG. 52(b), the maximum luminance sample rate (Max luma sample rate), the maximum bit rate (Max bit rate), and the minimum compression ratio (Min compression ratio). As a sub-concept of a level, “tier” is employed to represent whether the maximum bit rate of a bitstream (coded data) corresponding to each level and the maximum CPB size for storing the bitstream are defined in a Main tier (for consumer applications) or are defined in a High tier (for professional applications).
In NPL 1, for example, a main profile is defined as a profile. A main profile defines constraints on a coding tool such as those illustrated in FIG. 53(a). A main profile also defines additional level constraints illustrated in FIG. 53(b) in addition to the constraints defined by a level illustrated in FIGS. 52(a) and 52(b).
In NPL 1, a profile that a bitstream complies with is specified by a profile identifier general_profile_idc (in a syntax group SYNPTL01 of FIG. 54) in profile/level information (hereinafter, referred to as PTL information as well) profile_tier_level( ) illustrated in FIG. 54. For example, when a bitstream complies with a main profile, the value of general_profile_idc is set to one.
In addition, general_profile_compatibility_flag[i] (in the syntax group SYNPTL01 of FIG. 54) indicates whether a current bitstream can be decoded by a decoder complying with a profile other than the profile specified by the profile identifier general_profile_idc. For example, when the profile is compatible with the main profile, general_profile_compatibility_flag[1] is set to one.
In addition, a level identifier general_level_idc (in SYNPTL02 of FIG. 54) in the PTL information profile_tier_level( ) specifies one of the levels of FIGS. 52(a) and 52(b) that the complexity of a bitstream, or a level indicating the performance of a decoder required to decode a bitstream, complies with. For example, the value of the level identifier general_level_idc indicating “61” corresponds to the level 6.1 of FIGS. 52(a) and 52(b), and the value of the level identifier general_level_idc indicating “10” corresponds to the level 1 of FIGS. 52(a) and 52(b). That is, the tens' place (first digit) and the ones' place (second digit) of the value indicated by the level identifier general_level_idc respectively correspond to the integer value and the fraction value of a level in FIGS. 52(a) and 52(b).
In addition, a tier flag general_tier_flag (in the syntax group SYNPTL01 of FIG. 54) indicates whether a tier in the level specified by the level identifier general_level_idc is the Main tier or the High tier. The value of the tier flag general_tier flag being zero indicates the Main tier, and the value being one indicates the High tier.
In the PTL information profile_tier_level( ) illustrated in FIG. 54, profile information (hereinafter, referred to as sub-layer profile information; a syntax group SYNPTL 05 in FIG. 54) per layer related to temporal scalability (hereinafter, referred to as a sub-layer or a temporal sub-layer as well) and level information (hereinafter, referred to as sub-layer level information; a syntax SYNPTL06 in FIG. 54) can be explicitly specified if each of a sub-layer profile presence flag sub_layer_profile_present_flag[i] (in a syntax group SYNPTL03 of FIG. 54) and a sub-layer level presence flag sub_layer_level_present_flag[i] (in the syntax group SYNPTL03 of FIG. 54) is set to one.
In NPL 1, the PTL information profile_tier_level( ) is signaled in both parameter sets of a video parameter set (VPS) and a sequence parameter set (SPS) illustrated in FIG. 10(a).
In scalable coding such as SHVC (NPL 2) or MV-HEVC (NPL 3), the PTL information applied to each layer set included in hierarchically coded data (bitstream) is signaled in the bitstream. Specifically, the PTL information profile_tier_level( ) related to a layer set 0 (base layer) is signaled in SYNVPS02 of a video parameter set in FIG. 55, and a PTL information list configured of the value of the total number of pieces of PTL information included in the bitstream—1 “vps_num_profile_tier_level_minus1” (SYNVPS0D in FIG. 56) and the vps_num_profile_tier_level_minus1 numbers of pieces of PTL information (SYNVPS0G in FIG. 56) is signaled in video parameter set extension data (FIG. 56). Next, a PTL information specifying index “profile_tier_level_idx[i]” (in a syntax group SYNVPS0H of FIG. 56) specifying the PTL information applied to each layer set (layer set i) is signaled. By so doing, the coding amount of the PTL information can be reduced in comparison with a case where the PTL information is signaled per layer set. In addition, if the PTL information occurring before an i-th iteration and including the same profile information as the i-th PTL information included in the PTL information list exists in the PTL information list, the value of a VPS profile presence flag “vps_profile_present_flag[i]” (SYNVPS0E in FIG. 56) is signaled as zero. Furthermore, an index (reference PTL information specifying index) “profile_ref_minus1[i]” (SYNVPS0F in FIG. 56) indicating the position of the corresponding PTL information on the PTL information list is signaled, and the profile information of the i-th PTL information inherits the profile information of the “profile_ref_minus1[i]+1”-th PTL information. By so doing, the redundancy of duplicate signaling of the profile information can be reduced (FIGS. 57, 62(a), and 62(b)).
In other words, the method of the related art for signaling the syntax structure related to the profile/level information profile_tier_level( ) for a layer set is summarized as follows. That is,                the list of the syntax structure related to the profile/level information profile_tier_level( ) is signaled, and        the PTL information specified by the i-th PTL information specifying index profile_level_tier_idx[i] is selected as the profile/level information applied to the i-th layer set from the PTL information list.        
Details of signaling the profile/level information profile_tier_level( ) in the list are as follows.                The i-th VPS profile presence flag vps_profile_present_flag[i] is signaled in each profile/level information profile_tier_level( ). When the flag is set to “0”, the i-th reference PTL information specifying index profile_ref_minus1[i] is signaled as well, and general profile information of the i-th profile/level information profile_tier_level( ) is estimated from the general profile information of the (profile_ref_minus1[i]+1)-th profile/level information profile_tier_level( ).        In signaling of the profile and level of each profile/level information profile_tier_level( ), each of the profile and level of the i-th sub-layer may be omitted by the sub-layer profile presence flag sub_layer_profile_present_flag and the i-th sub-layer level presence flag sub_layer_level_present_flag[i].        
FIG. 59 is a diagram illustrating a schematic configuration of a PTL decoder of the related art, and FIG. 61 is a diagram illustrating a summary of a decoding operation of the PTL decoder of the related art illustrated in FIG. 59. FIG. 60 is a diagram illustrating a schematic configuration of a PTL coder of the related art.
USPTO Backgrounds
USPTO Backgrounds