Not Applicable.
1. Field of Invention
The present invention relates generally to packet-switched networks and, more particularly, to filters and methods for adaptively modifying the bit rate of synchronized video and audio streams distributed over such packet-switched networks.
2. Description of the Background
There are many formats used to encode movies (i.e., video and audio data) for transmission over networks of limited and varying bandwidth. For example, the MPEG (Motion Picture Experts Group) format is a widely accepted international standard for synchronously encoding video and audio data streams. MPEG-1, for example, was primarily designed for storing video data (moving pictures) and its associated audio data on digital storage media. As such, the MPEG standard deals with two aspects of encoding: compression and synchronization. MPEG specifies an algorithm for compressing video pictures (ISO-11172-2) and audio (ISO-11172-3) and then provides the facility to synchronize multiple audio and multiple video bitsreams (ISO-11172-1) in an MPEG system stream. MPEG-1 is intended for intermediate data rates on the order of 1.5 Mbit/sec.
An MPEG video stream distinguishes between different types of pictures, or frames, called I-frames, P-frames, and B-frames. These pictures differ in their coding schemes. The three types of coding provide three levels of compression by exploiting similarities within the picture or similarities to neighboring pictures. An MPEG-1 audio stream consists of audio coded using one of three algorithms, which offer different levels of complexity and subjective quality. These algorithms are referred as xe2x80x98layersxe2x80x99 in the coding standard. The coding algorithms use psycho-acoustic properties of the human hearing to compress the data (lossy compression). The MPEG system stream is responsible for combining one or more compressed audio and video bitstreams into a single bitstream. This is done by interleaving data from the video and audio streams, combined with meta-data that provides the time control and synchronization.
Distribution of encoded movies over a packet-switched network, such as the Internet, is a topic of great interest, but also poses a number of challenges. Because the movie is played as it is received, the transfer over the network must proceed at a specific rate to prevent buffer overflow or underflow at the player. If there is competing traffic in the network, however, there is always the risk of congestion, and consequently packets may be dropped or delayed. It is important to note that due to the high compression of video streams, dropped packets cause losses that are much higher than their proportionate size of the data streams, as discussed hereinbelow.
The infrastructure provided by today""s IP-based networks provides access to a large number of nodes. Unfortunately, IPv4 (the current standard) provides no framework for resource reservation. Users are competing for bandwidth, and if a link becomes congested (demand for bandwidth is higher than the link capacity), packets are dropped. Because traffic conditions change continuously, congestion can start and disappear at any time. Note that in the current Internet, there is an assumption that it is the source""s responsibility to reduce the data send rate when packet losses are observed to reduce congestion. For most applications, this reduction is done by TCP, the dominant Internet transport protocol.
Random packet losses can hurt MPEG system streams in two ways, besides the obvious fact that the information in the packets is lost. When the consequences of random packet losses are analyzed, it must be recognized that network packets may not correspond to MPEG packets, and the latter are a layer completely separate from the video frames. The amount of impact that the loss of a particular packet will have depends on its location in the stream and on the robustness of the player in recovering from errors. In the worst case, a network packet that contains meta-data of the whole MPEG stream (the MPEG system header) may be lost. As a result, players that rely solely on synchronization information found in the stream will be significantly impacted when such information is lost. In a typical scenario, it is most likely that a lost packet will contain some part of a video frame with meta-data (video being the predominant stream).
In the context of the MPEG layers, a network loss translates into a disruption in the system packet layer, and may result in concatenating parts from two different MPEG packets. This loss can induce corruption in the lower layers, e.g., corruption of the video or audio data. If video data has been affected, the frame is decoded incorrectly. An incorrect I-frame or P-frame propagates problems to all dependent frames and corrupts these as well. In the worst case, a whole group of pictures (GOP) may be lost, typically equivalent to half a second of video. For various MPEG streams, experiments have shown that a random loss of 1% of network packets can translate into as high as 10% damaged video frames. Similarly, it has been noticed that packet loss rates as low as 3% translated into frame error rates as high as 30%.
Accordingly, there exists a need for a manner to adapt the actual bandwidth requirements of an encoded movie data stream to the current conditions of the packet-switched network over which the encoded movie is being distributed. There further exists a need for such an adaptive manner to execute in real-time and without affecting the synchronization of the video and audio portions of the encoded movie.
The present invention is directed to a filter for adaptively modifying the bit rate of a synchronized video and audio stream transmitted over a packet-switched network. According to one embodiment, the filter includes a layer detection module, an adaptive frame removal module in communication with the layer detection module, and a network packetizing and pacing module in communication with the adaptive frame removal module.
According to another embodiment, the present invention is directed to a method for adaptively modifying a bit rate of a synchronized video and audio stream in real-time, including analyzing bits of the stream to detect encoded video frames in a video layer of the stream, removing certain of the encoded video frames from the stream based on available network bandwidth, and packetizing the stream.
The present invention represents an advancement over the prior art in that it provides a manner in which to adapt the actual bandwidth requirements of an encoded movie data stream to the current conditions of the packet-switched network over which the encoded movie is being distributed. The present invention represents a further advancement over the prior art in that it operates in real-time because it does not require demultiplexing and/or decoding of the data stream. In addition, the present invention provides an advantage over the prior art because it modifies the data stream without corruption and without destroying the synchronization information. These and other benefits of the present invention will be apparent from the detailed description hereinbelow.
USPTO Backgrounds
USPTO Backgrounds