The present invention relates to video coding for communication environments in which channel bandwidth may vary dynamically.
In real-time videoconferencing applications, the network communication channel bandwidth may vary greatly. Therefore, using all (or most) of the available bandwidth during a videoconferencing session is a priority so that the best possible quality can be delivered. Since the conditions of the network communication channel change dynamically during the real-time video conference, it would be beneficial to provide a videoconferencing system for efficiently utilizing the available network communication channel bandwidth while maintaining the presented video quality during a videoconference.
Video coders typically code video data according to a variety of data compression techniques to meet a target bit rate. Typically, this involves selection of coding parameters (e.g. quantization parameter, block/macroblock type selection(s), etc.) and application of coding techniques that induce data compression. The data compression techniques typically are lossy techniques, which incur data losses that cannot be recovered fully at decoding and playback. Accordingly, video coders typically operate according to coding “policies” that represent a balance between bit rate and recovered image quality throughout a coded video sequence. Accordingly, an encoder may be responsible for achieving a maximum recovered image quality given the bitrate restriction. When the network conditions stabilize, e.g. packets can be properly delivered to the receiver end with little or no loss or error, the encoder may usually choose the coding tools that maximize the coding efficiency. However, when network condition deteriorates, the encoder may start applying error resilience techniques, which can facilitate at the receiver end error recovery. For example, the known H.264 coding standard allows a video codec to code a frame as an intra-coded frame (I coding) at least once per 30 frames. Coding a frame by I coding techniques typically incurs higher bit rates than predictive coding techniques would have incurred by coding the frame according to predictive coding techniques. Forced mode selection, however, can be beneficial because it terminates accumulation of errors that might arise from long prediction chains in a video coding sequence and it also can contribute to error recovery if coded video data were lost due to transmission errors. In the event of a transmission error, decoding can resume when a decoder receives an intra coded frame.
The forced mode selection is an example of error resiliency built into the video coding protocol. Although a lower bit rate coding technique may be available to the video codec, the forced mode selection may generate a coded macroblock that has a higher bit rate than a macroblock coded by another mode selection with similar image quality. The forced mode selection, however, provides higher resiliency to error as noted. However, using error resiliency techniques, such as forced mode selection, during the coding of the bitstream may reduce the coding efficiency, depending upon the amount of error resiliency techniques used. In more detail, error resiliency features, such as forward error correction as well as redundant data and forced mode selection, when applied to a coded bitstream may decrease the coding efficiency because error resilience techniques usually introduces more data dependency during coding. The application of the error resiliency techniques usually can increase the probability of recovery from transmission errors (e.g. packet loss or error) at a receiver end. However, this improved ability to recover because of the applied error resiliency techniques may come at the cost of coding efficiency.
Coded video data may be transmitted from a video encoder to a video decoder in data packets. The packets include a packet header and a payload portion that carries the coded video data. The coded video data may have been processed by techniques to mitigate against data losses induced by transmission such as forward error correction, interleaving, etc. Many operating environments, such as packet-based communication networks, provide unpredictable channel bandwidth for communications. Prior to video delivery, it is possible for a transmitter to test network bandwidth and estimate a bit rate that a channel can support.
During operation, as video delivery is underway, a transmitting video codec may receive indications that channel impairments have arisen. For example, many packet-based networks engage in negative acknowledgement (“NACK”) protocols. FIG. 1 illustrates the problem graphically. Channel bandwidth is shown as operating at three different levels over time: At CH BWHI prior to time t1, at CH BWLOW from time t1 to time t2 and at CH WMED after time t2. When the channel bandwidth drops from CH BWHI to CH BWLOW at time t1, the transmitter does not receive an express indication of the change but may receive NACK indications. In response to the NACKs, the transmitting video codec may revise its target bit rate to TR BRLOW, a level at which it no longer receives NACKs.
A brute force approach to test for increased channel bandwidth might include an arbitrary increase of the target bit rate after a predetermined period of time elapses following stable operation. However, a transmitting video codec often has no way without potentially affecting the presentation quality of the decoded video to test the channel bandwidth to determine if the increased bandwidth is available. For example, it might happen that no increase in bandwidth has occurred (for example, the channel bandwidth may remain at CH BWLOW), and an arbitrary increase in the transmitted bit rate may result in a loss of data and, by extension, an abrupt drop in image quality.
Therefore, there is a need for increasing the transmitted bit rate while providing an acceptable probability of recovering the transmitted data in view of an increased risk of transmission errors.
USPTO Backgrounds
USPTO Backgrounds