Virtual auditory displays (including computer games, virtual reality systems or computer music workstations) create virtual worlds in which a virtual listener can hear sounds generated from sound sources within these worlds. In addition to reproducing sound as generated by the source, the computer also processes the source signal to simulate the effects of the virtual environment on the sound emitted by the source. In a first-person computer game, the player hears the sound that he/she would hear if he/she were located in the position of the virtual listener in the virtual world. One important environmental factor is reverberation, which refers to the reflections of the generated sound which bounce off objects in the environment. Reverberation can be characterized by measurable criteria, such as the reverberation time, which is a measure of the time it takes for the reflections to become imperceptible. Computer generated sounds without reverberation sound dead or dry.
Artificial reverberation algorithms are well known in the art and are described e.g. in Stautner, J., and Puckette, M. (1982). Designing Multi-Channel Reverberators. Computer Music Journal, Vol. 6, no. 1, Dattorro, J. (1997). Effect Design (Part 1: Reverberator and Other Filters; Part 2: Delay-Line Modulation and Chorus). Journal of the Audio Engineering Society, Vol. 45, no. 9–10, Jot, J.-M. (1997). Efficient Models for Reverberation and Distance Rendering in Computer Music and Virtual Audio Reality. Proceedings of the 1997 International Computer Music Conference. The implementation of these algorithms on digital signal processors is based on a network of digital delay lines which are connected together and to the input and output points of the algorithm by feed-forward or feedback connections. Rooms of different sizes and acoustical properties can be simulated by modifying the topology of the network (the number of delay lines and the connections between them), by varying the duration of the delays, or by adjusting the amplification or attenuation coefficients of multipliers and filters inserted on the feed-forward or feedback connections.
As depicted in FIG. 1, a typical model of reverberation breaks the reverberation effects into discrete time segments. The first signal that reaches the listener is the direct-path signal, which undergoes no reflections. Subsequently, a series of discrete “early” reflections are received during an initial period of the reverberation response. Finally, after a critical time, the exponentially decaying “late” reverberation is modeled statistically because of the combination and overlapping of the various reflections. The magnitudes of Reflections—delay and Reverb—delay are typically dependent on the size of the room and on the position of the source and the listener in the room. As illustrated in FIG. 2a, the early reflections and the late reverberation are often generated by two separate processing modules whose output signals are combined to produce the output of the reverberation processor. Examples of an early reflection module and a late reverberation module are shown on FIGS. 2b and 2c, respectively. The lengths of delay lines comprising these modules can be made smaller or larger according to the size of the virtual room.
Reverberation processors of the type described above are commonly used for the production of music and soundtracks in recording studios. In these applications, it is not common to produce drastic changes in reverberation characteristics while the sound is playing. Noticeable drop-offs and other artifacts in the output signal of the processor will occur, for instance, when the user loads a different reverberation “program” or adjusts the room size parameter (which may involve changing the network structure or modifying delay lengths). However, such artifacts are not acceptable in interactive audio applications. In immersive 3D games or simulation systems, for instance, different reverberation settings may be associated with different rooms or environments composing a virtual 3D world in which the virtual listener is allowed to travel. Consequently, in these systems, the reverberation processor must be able to change settings while creating a minimum of disruptive or distracting audible artifacts.
Artifacts due to dynamic changes in reverberation settings can be avoided by using two reverberation processors set to simulate different room acoustics and cross-fading from one processor to the other (at their input or at their output). However, it is generally more advantageous to use a single reverberation processor and modify its parameters in order to produce the desired change in room acoustics. The coefficients of multipliers and filters comprising a reverberation processor are easily changed, without noticeable artifacts, by ramping their values to new values over a short time. This avoids the introduction of sudden discontinuities in the audio signal waveform, audible as pops or clicks. For the same reason, it is necessary to avoid sudden changes in the duration of the delay lines.
Methods for implementing continuously variable delays are well known in the art and are described e.g. in Laakso, T. I. et al. (1996). Splitting the Unit Delay—Tools for Fractional Delay Filter Design. IEEE Signal Processing Magazine, Vol. 13, no. 1. However, these methods involve digital audio interpolators, adding significant computational complexity to each delay line. Furthermore, when an interpolator is used, a large variation in a delay length can produce a noticeable change in the pitch of the delayed signal, which may result in an audible artifact. Another technique for implementing variable delays is described in Van Duyne, S. A. (1997). A Lossless, Click-Free, Pitchbend-able Delay Line Loop Interpolation Scheme. Proceedings of the 1997 International Computer Music Conference and illustrated in FIG. 2d. In this technique, a delay change is realized by cross-fading between two signals read from two different locations (or “taps”) in the delay line's memory. These two taps are provided by two read pointers whose locations in the memory correspond to the original delay value and the final delay value. This method moves the read pointer to a new location in the delay memory without causing a drop-off, a discontinuity or a pitch alteration in the delayed audio signal. However, it causes a temporary timbre alteration known as “comb filter effect”.
In order to provide a wider range of variation in simulated room acoustics, it is common to allow control for the echo density of the reverberation decay, suggesting more or less diffusing room walls. For instance, the reverberation algorithm described in Dattorro, supra allows to control diffusion by adjusting the feedback coefficients of a set of all-pass filters in the reverberation network. Another kind of common musical effect, also described e.g. in Dattorro supra, is the echo effect, which can be obtained simply by a single delay line with feedback. A cyclic echo can sometimes be obtained with existing reverberation algorithms as a side effect—usually unwanted—for particular settings of a reverberation processor's parameters. However, reverberators such as described in the above references do not provide parameters for controlling explicitly and intuitively aspects of an echo effect embedded in the reverberation decay. Such control would be useful to simulate larger rooms or semi-open environments such as a courtyard.
USPTO Backgrounds
USPTO Backgrounds