1. Field of the Invention
This invention relates generally to computer graphics and, more particularly, to user-defined shaders that implement graphics operations.
2. Description of the Related Art
Ever since 3D computer graphics evolved beyond wireframe rendering, shading has been a principal area of research and development. In the early days, shading primarily concerned processes by which pixel colors were applied to a surface. These days, the terms shading and shader are much broader and generally refer to any types of 3D graphics operation. Code which implements such graphics operations is commonly referred to as a shader. Examples of graphics operations that can be implemented by shaders include coordinate transformation, lighting, and determining the pixel colors across a surface. Shaders can also be used to produce geometric effects, such as skeletal animation, particle systems, or other dynamics such as textile modeling. Shaders are widely used for simulating the reflectance properties of surfaces, ranging from simple shaders describing a pattern on a surface to more sophisticated shaders modeling human skin, granite, velvet, etc. Shaders can also be used to simulate the optics in a camera lens through which a scene is viewed or to simulate the illumination properties of lights in a scene. Other examples will be apparent.
In 1988, Pixar's Renderman renderer became available. Renderman was the first widely used rendering application that supported programmable shading, although the technique was introduced commercially by Pixar with their Chap Reyes rendering system in 1986 and academically by Robert L. Cook in 1984 (“Shade Trees”, Robert L. Cook, Computer Graphics Siggraph 1984 proceedings). Prior to programmable shading, a user of a graphics system (e.g., an applications developer) was limited to a predefined set of shading operations, which shall be referred to as “standard operations.” All graphics had to be rendered using only the standard operations. If an effect was not supported by the standard operations, then the user either had to skip the effect or, if the effect was important enough, lobby the manufacturer of the graphics system to expand the set of standard operations to include the desired effect. In contrast, programmable shading allowed users to mathematically define shading functions using their own code. This resulted in a nearly infinite number of shading possibilities to simulate virtually every conceivable type of surface, lighting, atmosphere or other effect. Essentially, users could define their own shaders.
The shading techniques described above were typically first implemented as software running on general purpose computers. Such rendering software is generally used for off-line rendering, in which rendering times for each frame of a computer graphics movie can vary from seconds to days, depending on the processor performance and scene complexity. Later, as semiconductor performance increased, many shading techniques were implemented in hardware for real-time applications. In real-time applications, scenes must be rendered at interactive rates, which is usually somewhere between 10 and 100 Hz.
Due to the difficulty in meeting this performance requirement, advances in shading technology are implemented in off-line rendering systems significantly before they reach real-time renderingsystems. For example, an early implementation of real-time texture mapping occurred in the 1980's in General Electric's CompuScene III real time image generator. An early implementation of rudimentary real-time programmable shading was nVidia's Geforce3 accelerator, released in 2001. These dates are significantly later than the corresponding dates for off-line rendering systems.
Like their off-line rendering ancestors, prior to programmable shading, real-time graphics systems were based upon a predefined set of standard operations and a corresponding application programming interface (API). This predefined set of operations is also known as the fixed-function pipeline. It will also be referred to as the fixed-function mode for the graphics system. Examples of APIs that include a fixed function pipeline are OpenGL 1.1 and DirectX. Older APIs include IRISGL (SGI's API prior to OpenGL), Glide (by 3dfx), and PHIGS. The OpenGL specification describes a pipelined architecture for real-time 3D rendering. The pipeline includes stages for vertex processing, primitive processing, rasterization, texture mapping, and fragment processing. Each stage in the pipeline can implement a finite number of standard operations and the operations to be performed are described by states that are set by the user (including, for example, matrices, and lighting and material parameters).
For example, in the geometry processing stage (a combination of vertex processing and primitive assembly), the user might set state(s) to describe how texture coordinates are generated. Texture coordinates may, for example, be explicitly specified in source geometry, derived by means of a linear equation from the vertex positions of source geometry, transformed by a matrix, etc. The user sets the appropriate state(s) for the generation of texture coordinates and the graphics processor then executes the corresponding standard operation(s).
One important property of the standard operations is that they are typically “orthogonal.” Two graphics operations are orthogonal if the state of one operation does not affect the state of the other operation. For example, consider texture coordinate generation and texture coordinate transformation. The former describes how texture coordinates are initially generated; the latter describes a matrix transformation applied to the coordinates. These two operations are orthogonal because the transformation operation functions the same regardless of how the texture coordinates are initially generated, and vice versa.
One advantage of orthogonality for users is that it simplifies the use of the graphics system because the interplay between different graphics operations is reduced. This makes it easier to understand the graphics system and also makes incremental development possible. One disadvantage of orthogonality for manufacturers of graphics systems is that each additional graphics operation supported by the fixed function pipeline geometrically increases the number of combinations of possible states that the user may set.
Take the geometry processing stage as an example. Here, the addition of new graphics operations and the corresponding proliferation of states have led to the adoption of “fast paths.” Modern geometry processing stages are typically implemented using programmable processors that execute microcode. The microcode implements the standard operations of the geometry processing stage of the fixed function pipeline. It is fixed function because the user cannot easily alter the microcode (e.g., it may be preloaded by the graphics system manufacturer) and therefore can only perform the standard operations supported by the microcode. The microcode authors usually start by creating a “slow path,” which is an all-inclusive microprogram that is capable of handling every possible combination of states supported by the fixed function pipeline. This generalized microprogram is not optimized. For example, if the user disables texture coordinate transformation, rather than skipping this operation, the generalized microprogam typically would still perform the coordinate transformation but set the transformation matrix to the identity matrix so that no actual coordinate transformation occurred.
Because most applications use only a small subset of the possible combinations of states, the microcode authors often implement “fast path” microprograms for specific cases. For example, if flat-shaded wireframe rendering is used frequently in CAD applications, the authors may create an optimized microprogram to implement this combination of states more efficiently. Or if a popular computer game renders textured polygons with one diffuse light and fog enabled, the authors may create another optimized microprogram to implement this combination. The graphics driver typically chooses the appropriate fast path by analyzing the state settings made by the application. If no fast path is available, the generalized slow path is executed.
The programmable pipeline or programmable mode goes one step further. In the fixed function mode, the user sets states and, based on the states, a fast path microprogram is executed if one is available. In the programmable mode, the user supplies his own microprogram (i.e., a user-defined shader). The programmable pipeline simplifies the graphics system manufacturer's job because the user (e.g., an application developer) can create shaders optimized for his particular application and can also create shaders to implement graphics operations which are not supported by the fixed function pipeline. Furthermore, the user does this without affecting the fixed function pipeline or the corresponding graphics API. Early examples of the programmable pipeline include Direct3D Vertex Shaders (a.k.a. Vertex Programs in OpenGL) and Direct3D Pixel Shaders (a.k.a. Texture Shaders and Register Combiners in OpenGL). These allow the user to write shaders (vertex shaders and pixel shaders in the examples given above) that essentially bypass the API abstraction layer and operate directly with the underlying graphics hardware (or which are optimized to run on general CPUs if there is no direct hardware support).
While the programmable pipeline gives users the flexibility to create custom shaders, it comes at a price. FIG. 1A (prior art) is a functional diagram of a graphics system 150 with a fixed function mode 160 and a programmable mode 170. Typically, the programmable pipeline 170 and the fixed function pipeline 160 are mutually exclusive. Using the programmable pipeline 170 means that many of the standard operations of the fixed function pipeline 160 are not available. For example, when a Direct3D Vertex Shader is enabled, it completely replaces the vertex processing stage of the fixed function pipeline. Suppose a user simply wants to implement a new method for deriving texture coordinates from source geometry and uses the programmable pipeline to do so. By invoking the programmable pipeline for this one operation, the user can no longer take advantage of the texture matrix, geometry transformation, lighting, or any other standard vertex operations available from the fixed function pipeline. Rather, the user must supply all of these operations himself in additional user-defined shaders. In the case of Vertex/Pixel Shaders, some non-programmable functions of the fixed function pipeline, such as clipping and depth testing, remain when the programmable pipeline is invoked.
In other words, using shaders and the programmable pipeline shifts the burden of managing many of the features of the graphics pipeline from the graphics system manufacturer to the user. The problem of proliferating graphics operations and states now becomes the user's problem. As a result, there is a substantial barrier to entry to using shaders and there is a need for an approach which allows users to take advantage of the flexibility of the programmable pipeline while significantly reducing this barrier to entry.
USPTO Backgrounds
USPTO Backgrounds