Moving image data generally has a very large data amount. For this reason, in an apparatus handing moving image data, compression by encoding of moving image data is broadly performed when transmitting moving image data to another apparatus, or when storing moving imaged data in a storage apparatus. As typical moving image encoding standards, various techniques set by the International Standardization Organization/International Electrotechnical Commission (ISO/IEC) have been broadly used. These techniques include those called Moving Picture Experts Group phase 2 (MPEG-2), MPEG-4, or H.264 MPEG-4 Advanced Video Coding (MPEG-4 AVC/H.264).
In these encoding standards, an inter-encoding method to encode a picture being the encoding target using both the picture being the encoding target and its preceding/subsequent pictures, and an intra-encoding method to encode using only the information that the picture being the encoding target has.
In the inter-encoding method, a part with a high correlation between the encoding target picture and a reference picture is searched. Then, the inter-encoding method encode with a motion vector representing the difference in the position between the encoding target picture and the reference picture, and the difference (called a prediction error) between the pixel values between the pixels that the two pictures correspond. Generally, the correlation between pictures is high in moving images, and the value of the prediction error becomes smaller than the original picture value. As a result, the inter-encoding method has high compression efficiency.
However, in the inter-encoding method, an error generated in encoding a certain picture is carried to the encoding of a picture that is encoded later than the picture. For this reason, when encoding a plurality of successive pictures by the inter-encoding method, the picture quality deteriorates more in pictures that are encoded later. In addition, for a moving image decoding apparatus, for example, when a picture to be the reference is not obtained such as when encoded moving image data is obtained from halfway, an inter-encoded inter picture based on the picture to be the reference is not decoded in the normal way.
On the other hand, for an intra picture encoded in the intra-encoding method, the pixel value is decoded in the normal way without referring to the decoding result of another picture. Therefore, the intra-encoding method is used for encoding the first picture of moving image data, and for encoding a picture in the middle in a predetermined cycle. By making a picture in the halfway an intra picture, when the image decoding apparatus starts a decoding process from halfway of encoded moving image data, each picture after the intra picture is recovered in the normal way. Meanwhile, when the decoding operation starts from a certain picture, in a case in which the pixel value is decoded in the normal way for the picture or a picture after a predetermined time, the picture from which the decoding operation starts is called a refresh picture.
However, generally, the compression efficiency of the intra-encoding method is lower than that of the inter-encoding method. For this reason, in order to maintain the picture quality of the intra picture and the picture quality of the inter picture after decoding approximately equal, generally, it is desired to make the information amount of the intra picture larger than the information amount of the inter picture.
When an encoded moving image data is handled, an encoding delay is generated by encoding moving image data and decoding it. The encoding delay is the time from when the picture is input to the moving image encoding apparatus to when the picture is output from the moving image decoding apparatus. For example, in an application to transmit and recover moving image data in real time such as a remote television conference system, it is desirable to make the encoding delay small. In order for the user to use such an application without feeling the inconvenience, it is preferable that the encoding delay is about 100 milliseconds or less.
In order to make the encoding delay about 100 milliseconds or less, it is preferable to match the encoding order of pictures with the time order of the pictures, and to minimize buffering delay, one of the causes of the delay generated in transmitting and receiving encoded moving image data. The buffering delay is set to perform transmission of encoded moving image in which the information amount varies for each picture in a network having an approximately equal transmission speed as the average information amount per unit time of the encoded moving image. The buffering delay is the time use for transmitting a picture having the largest information amount in an encoded moving image stream, which is, for example, about several-frame time to 20-frame time.
Therefore, in order to minimize the buffering delay, it is preferable to make the information amount of each picture approximately constant. However, as mentioned earlier, when the picture quality of each picture after decoding is constant, the information amount of the intra picture becomes larger than the information amount of the inter picture. Therefore, in order to make the information amount of each picture constant, the picture quality of the intra picture significantly deteriorates when the information amount of the intra picture is simply made approximately equal to the information amount of the inter picture. Then, the picture quality of the inter picture whose reference picture is the intra picture also deteriorates significantly, and as a result, the picture quality of the whole moving image data deteriorates significantly.
As a method to make the information amount of each picture approximately constant without losing the picture quality significantly, the intra-slice method has been proposed. In the intra-slice method, pictures other than the first picture of moving image data being the encoding target do not become the intra picture, but instead, a macroblock group to be intra-encoded is inserted into each picture so as to circulate in the picture in a predetermined cycle. The macroblock group to be intra-encoded is called an intra slice. In addition, in this method, normal decoding is performed sequentially in the picture, and the method is also called a gradual refresh.
For example, when the circulation direction is the direction from the top to the bottom in a picture, the picture having the intra slice highest is the refresh picture, and normal pixel value recovery is performed sequentially from the topmost of the picture. Then, in the picture having the intra slice lowest, the pixel value of the whole picture is recovered in the normal way. The interval of the refresh pictures, that is, the refresh cycle is the same as the circulation cycle of the intra slice.
Meanwhile, each macroblock in the intra slice does not have to be intra-encoded. That is, when starting decoding from the refresh picture, it will do as long as each macroblock in the intra slice so that, it is assured that, when starting decoding from a refresh picture, the pixel value of pictures after the refresh picture is recovered in the normal way. For example, the macroblock in the intra slice may be inter-encoded using a motion vector that refers to an area in an area in which it is assured that the pixel value is recovered in the normal way (such an area is called a “clean area”) in the picture.
However, when moving image data is encoded using the intra slice method, there is an area in which it is not assured that the pixel value is recovered in the normal way, in pictures before the picture refreshed by the intra slice. The area is called a “non-clean area”. In order to assure that the pixel value in the clean area is recovered in the normal way when moving image data is encoded using the intra slice method, there is a restriction that the macroblock in the clean area does not refer to the information of the pixel in the non-clean area.
For example, when an encoding target macro block in the clean area is inter-encoded, it is desirable that the whole block in the reference picture referred to be the encoding target macroblock is included in the clean area.
In addition, in the MPEG-4 AVC/H.264 standard, in encoding the motion vector, the average (or median) value of the motion vector of the macroblock immediately above or left of the encoding target macroblock is calculated as a predicted value PMVMED of the motion vector. Then, the difference between the predicted value PMVMED and the motion vector of the encoding target macroblock is entropy-encoded.
Incidentally, a technique to divide the picture being the encoding target into rectangular small areas called a “tile” and to process each block has been proposed. In this technique called the picture division encoding method, in order to process the respective tiles in parallel, restriction to prohibit reference across the tile boundary is permitted. The entropy encoding of each block is processed sequentially in the scanning order in the tile, and reset at the top of the tile.
Meanwhile, techniques described in each of the following documents have been known.                Document 1: Japanese Laid-open Patent Publication No. 2003-179938        Document 2: Japanese Laid-open Patent Publication No. 06-113286        Document 3: Japanese Laid-open Patent Publication No. 2005-260936        Document 4: International Publication Pamphlet No. 2009/037726        Document 5: Japanese Examined Patent Application Publication No. 06-101841        Document 6: JCTVC-F803, “WD4: Working Draft 4 of High-Efficiency Video Coding”, Joint Collaborative Team on Video Coding of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, July, 2011        Document 7: JCTVC-F335, “Tiles”, Joint Collaborative Team on Video Coding of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, July, 2011        
Incidentally, the picture division encoding method has the following two problems.
The first problem is the decrease in the encoding efficiency in the parallel processing of the compression process of each small area (the encoding process except entropy encoding).
In the existing picture division encoding method, absence/presence of the restriction of the reference across the small areas is applied to both the compression process and the entropy encoding at the same time. Furthermore, in the entropy encoding, reset is performed at the top of the small area. These restrictions for the entropy encoding is applied when encoding each small area in a completely separate manner, but does not need to be applied when processing only the entropy encoding by a single circuit, and application of these restrictions causes a decrease in the encoding efficiency.
Another problem is the delay in entropy encoding. This problem is explained using FIG. 1 and FIG. 2.
FIG. 1 schematically illustrates the structure in a case in which a boundary of two small areas is inserted into a picture, and represent the state in which two small areas 11 are inserted into a picture 10. Meanwhile, in FIG. 1, the processing order of the block in the existing picture division encoding method is expressed by the direction of an arrow 12.
When dividing the picture 10 into two left and right small areas 11 and sequentially processing the block in each small area in the scanning order as in FIG. 1, in order to start the entropy encoding of the small area 11 on the right in FIG. 1, it is desired that the entropy encoding of the small area 11 on the left is completed. Meanwhile, in order to complete the entropy encoding of the small area on the left, it is desired that at least the taking of the last line of the processed picture is completed. For this reason, a delay corresponding to one picture is generated.
As a method to solve the delay problem, a method to make the process order of each block the sequential scanning order by inserting the small area boundary between the respective block lines of the picture has been known. FIG. 2 schematically illustrates the structure in a case in which the small area boundary is inserted into the respective block lines of the picture, and represents the way in which a plurality of small areas 21 area inserted into a picture 20. Meanwhile, the height of the small area 21 is all “1”. Meanwhile, in FIG. 2, the processing order of the block in the existing picture division encoding method is expressed by the direction of an arrow 22. However, in this method, since the number of the small areas 21 in the picture 20 increases, the encoding efficiency by the reset of the entropy encoding further decreases.
USPTO Backgrounds
USPTO Backgrounds