In this series, we are taking an extensive look at past, current, and emerging reverberation techniques and reviewing them from an immersive and spatial perspective. In the previous articles, we covered the reasons why immersive reverberations are so challenging in virtual reality and some of the techniques behind classic reverberation algorithms. In this article, we review some of the current trends in acoustic simulations that can be used to create spatial reverberation effects.

 

Acoustics simulation of sound propagation on a metal plate

As we saw in the previous articles, although classic reverberation techniques are still very relevant today, we need to start looking at ways to make them more immersive. Just as reverberation techniques have evolved in the past to new production needs, virtual reality is currently pushing the limits on how we want to listen to reverberant virtual spaces.

Thanks to binaural algorithms, we can now perceive the direction of incoming sounds better then ever on headphones. While binaural algorithms will continue to improve in the future, refining the proximity effect and offering  personalized Head Related Transfer Function (HRTF) filters, we also need to ensure that reverberations can carry on these spatial cues well after the initial direct sounds. While classic reverberation algorithms are able to efficiently increase the density of echoes over time, in the process, we lose the interactivity that would be crucial to convey spatial cues. This is especially true of early reflections, which carry important perceptual information. To achieve this goal, we need to start using the geometry information from the virtual spaces and inform simulation algorithms.

Using headphones makes it easier to render spatial audio effects as the listener is always located in the center of the speakers[1]

As we look beyond traditional reverberation, we will examine various methods for auralization of virtual soundscapes. In other words, listening to virtual reverberating spaces. Auralization of virtual acoustics is a research field that aims to simulate sound propagation by borrowing notions from physics and graphics. Indeed, to gain more realistic simulations that can carry spatial information, we need to take another look at how sound propagates in an enclosed space and how we can simulate it.

Spatial sound is also needed for multi-channel setups, such as the SATosphere in Montreal which has 157 speakers [2]

Sound propagation is an amazingly complex phenomenon. Increasing the accuracy of the simulation will mean increasing the amount of computation required. Therefore, it’s important to consider where these methods excel, what their shortcomings are, and what perceptual ramifications can result from each approach.

The propagation of sound

The energy released by a rapid vibration, that we perceive as sound, travels through different materials as a mechanical wave. This kinetic energy which originates from the original vibration displacement, the vocal cords for instance, is passed onto the different particles of a surrounding medium, such as air. As sound itself never really moves, it merely passes its kinetic energy from particle to particle, and then gradually dissipates based on the viscosity of the material it is passing the energy to.

A denser material will allow a faster propagation because particles would be packed closer to each other, while a heavier material will dissipate more energy because it requires more to get the particles moving. Another property, called acoustics impedance, can describe how well different frequencies transmit onto a specific material. The energy that isn’t transmitted from air onto a wall material will be reflected back and will contribute to the reverberation of a room. This explains why a material with low density, such as a fibrous material, can still be good at reducing reverberation in a recording studio. The exterior texture of the material will also guide the direction of the reflections. It is important to note that all of these phenomena will vary depending on the wavelength of a sound, meaning these are frequency dependent phenomena. While a long wavelength is perceived as a low frequency sound, a short wavelength is perceived as a high frequency sound.

The easiest way to illustrate the frequency dependant nature of sound is by considering another key element of sound propagation, diffraction. Through diffraction, known as obstruction in game audio, a sound wave can bend around obstacles and openings to reach a listener without a direct line of sight to the emitting source. The reason a low pass filter is usually used to simulate obstruction is due to the way diffraction behaves in real life. When a wavelength smaller than an obstacle comes in contact with an obstacle, the sound will be reflected. However, when the wavelength is longer than the obstacle, it will go around it. So, if you imagine those two frequencies combined in one signal, traveling towards the same obstacle, a listener behind that wall will only hear the lower frequency (longer wavelength). This effect gets trickier with more complex shapes. For instance, the diffraction of sound around our head is so complex that we need to record it using an in-ear microphone and an impulse sound. This is how HRTF filters are captured to create binaural effects. Therefore, the size, shape, density, acoustics impedance, and texture are all material properties that impact how sound, at different frequencies, will travel through a space.

 

A short wavelength tends to reflect from an obstacle, while a long wavelength will diffract around it.

In contrast, light propagates through an entirely different process. A form of electromagnetic waves, light can travel through vacuum. In fact, light requires vacuum to propagate. This means it will go in between the particles of a material. Therefore, a denser material will slow down the propagation and can even block it. Furthermore, the wavelength of light we can see is much smaller than the wavelength of sound we can hear. Indeed, our eyes are able to capture light waves between 400 and 700 nanometers in length, as opposed to sound waves that need to be between 17 millimeters and 17 meters for us to hear. Moreover, sound will travel much slower, at about 343 meters per seconds in air, while light can travels at almost 300 million meters per seconds. This explains why we need to worry so much about the delay of reverberant sounds. All of these differences illustrate why light and sound waves will interact so differently with various materials and obstacles, especially diffraction. That being said, diffraction isn’t so prominent in an enclosed space, meaning that light propagation is actually a great place to start the approximation of sound waves.

Ray-based methods

Ray-based acoustics are sound propagation simulation techniques that take cues on how light travels. In this approach, sound travels in a straight line and, with a little linear algebra, you can easily calculate the direction of sound reflections based on the position of various walls. Delay can be added to these models to take into account the slow moving nature of sound, and filters can simulate how walls absorb different frequencies. Usually performed using simplified geometry, ray-based methods offer fast and fairly accurate ways of simulating sound propagation of early reflections at higher frequencies. Indeed, since rooms tend to be more complex in real life, it will reflect on more surfaces, thus creating a richer density of echoes. This is the reason why late reverberations are not well reproduced with these techniques. Also, when a space becomes more geometrically complex, diffraction of low frequency waves becomes a much more prominent factor. In this case, provided we know the incoming direction of different reflection rays, this can be simulated independently.

The two main variants of ray-based methods are the image source method and ray-tracing. Image sources will calculate the reflection paths of a sound from each wall. The process can continue to include higher order reflections when a reflection itself is reflected onto another wall. In ray-tracing, rays are sent in various directions and reflected when an obstacle is reached. Multiple new rays, going into different directions, are generated following each reflection from a surface. Random directions can be chosen to simulate sound being scattered by the wall’s texture or a corner, for instance [4]. This approach is useful when more complex geometry is used but will be require more computation. In both cases, it is possible to limit the depth of the algorithms to favor early reflection. Since we know the exact position of each ray, based on the position of a sound source and a listener, ray-based methods allow us to interactively position a sound, both in space and in time. To simulate the distance traveled by a sound, a delay is introduced based on the speed of sound in air.

The following video is courtesy of Brian Hamilton [4], member of the Acoustics and Audio group at the University of Edinburgh. We can see the propagation of a sound source over time using a ray-based method.