Conventionally, multimedia such as video and audio has been delivered using analog delivery mediums such as NTSC (National Television System Committee) signals, and has been stored using analog storage mediums such as video cassette recorders. Thus, a significant part of the electronic components in a display device are dedicated to analog receiving hardware. If the display device has a digital output, additional electronic components are needed to convert the analog signal to a digital signal. With the advent of digital delivery mediums, such as ATSC (Advanced Television Systems Committee) signals, and of digital storage mediums such as DVDs, multimedia can be delivered and stored using pure digital signals. These digital signals typically contain compressed frames of video.
Digital media technology enables a greater variety of features to be delivered to users to enhance their multimedia experience. One particularly useful feature in conventional media systems involves layering graphics planes on top of an uncompressed video signal. A graphics overlay system may, for example layer still images, text, or animations on top of an input media stream. Applications of graphic overlay include various On-Screen Display (OSD) features such as navigation menus, setup screens, alerts, program information, digital watermarks, or other graphics layered on top of an uncompressed video signal without interrupting the media stream.
However, conventional graphics overlay systems are not well adapted for use with compressed digital media. In a conventional graphics overlay system, a series of blend operations is performed on stored uncompressed video frames with each successive blend operation layering an additional graphics plane. Conventionally, the blend operation uses a scan-line process by traversing and blending the video and graphics planes one line at a time. A conventional scan-line graphics overlay system is well adapted for use with conventional analog or uncompressed media in which video is sent to a display one line at a time. However, the conventional system is poorly suited for use with compressed digital media that do not follow a scan-line process but instead operate on a macroblock basis.
FIG. 1 illustrates a conventional graphic overlay system. Video input 104 comprises a compressed video stream. An input is coupled to a decoder 108 and receives the compressed video input 104. The decoder 108 is configured to decode the input video stream. Each frame of the video input 104 is fully decoded and stored in video memory 112 before the blend operations occur. Graphics planes 124 comprise a number of graphics layers (e.g. layers 132a-c) to be layered on video input 104. Graphics planes 124 may be stored in a graphics memory 128. Graphics memory 128 and video memory 112 are coupled to a series of blenders 116a-116c. Blenders 116a-c blend each layer 132a-c with the video stream in successive blend operations. Thus, conventionally, an additional blender 116 is used for each additional graphics plane to be layered on top of the video input 104. After the signal passes through all the blenders 116, a video output 120 is produced.
Conventional graphics overlay systems are inefficient for overlaying graphics on compressed digital video. With a conventional graphics overlay system, the full frame of the video input 104 must first be fully decoded and stored in a memory 112 before any blend operations take place. Traditional decoders must first store the video pixels so that they can be reordered from the macroblock order used in transmitting and processing compressed video into the raster scan order used for display. This means the conventional graphics overlay process cannot operate on streaming compressed video. Further, significant memory is needed to store the fully decoded frames. Additionally, conventional graphics overlay devices require separate blender hardware 116 for each layer 132 of graphics to be layered on the video input 104. This results in significant increases in cost and physical size as the number of graphics planes increases.
Conventional graphics overlay systems are further inefficient for providing output to multiple playback devices. For example, conventional systems are unable to send multiple compressed video streams to multiple playback devices with a different graphic image or OSD overlayed onto each stream. Instead, each playback device would require its own dedicated graphics overlay processor, each of which can be significantly costly.
Therefore, the present state of the art lack a system for layering multiple graphics planes on a compressed video stream that offers better performance and provides more efficient use of hardware than conventional devices. Furthermore, the present state of the art lacks a graphics overlay system that can efficiently provide output to multiple users each having a different playback device.
USPTO Backgrounds
USPTO Backgrounds