Mixing audio for VR & AR games and experiences can be tiresome. In this blog, we'll provide the first insights into Dear Reality's upcoming solution, dearVR SPATIAL CONNECT for Wwise, which facilitates gesture control of Wwise in game engines like Unity.

About Dear Reality

At Dear Reality, we are working on immersive audio solutions to make spatial audio mixing less technical and more intuitive for mixing engineers, sound designers, broadcasters, game audio designers, and musicians worldwide. With our spatializer and room virtualization DAW plugins, we defined workflows to easily switch between different spatial formats like binaural audio, Ambisonics, and multichannel audio. Instead of spending their time with endless technical setups, our users can focus on what's important: creating a great-sounding mix.

With dearVR SPATIAL CONNECT for Wwise, we are reaching the same goal of mixing audio for VR/AR games and experiences in Wwise and Unity. In this article, we want to present a workflow that speeds up the process and makes it way more intuitive.

Existing production workflow

Imagine being in a mixing studio where your loudspeaker setup and mixing console are installed in separate rooms. You must run to the first room to listen to the mix on your loudspeakers whenever you want to check the changes you made before on the mixing desk in the second room. We felt pretty similar when we were working on spatial audio mixes for VR experiences, as we needed to constantly switch between making changes to the mix on a desktop PC and checking the results in VR. Going back and forth between VR and a regular desktop PC became an exhausting routine. After some hours of the back and forth between the mouse and the Oculus Touch controllers, the solution to the problem became quite obvious. We came up with the idea of dearVR SPATIAL CONNECT, the linear predecessor of our latest tool. We needed some way of controlling the mixer in VR!

Improving productivity

From the feedback of our users and partner game studios, we quickly learned that DAW control is not enough. We got a lot of requests to bring our products to middleware, and especially Wwise, as it solves most of the needs for VR Audio Design except the live control within VR. With dearVR SPATIAL CONNECT for Wwise, we want to bring our workflow to Wwise to reduce context switching and make it possible to work on spatial audio in the immersive environment itself intuitively. The project is technically supported by Audiokinetic, whose development team added certain features into Wwise that allowed for this new workflow. It provides in-headset control of all main features of your Wwise sessions and gesture-controlled audio mixing to enhance the scene's perception.

Direct access in VR

dearVR SPATIAL CONNECT for Wwise offers direct access to the most-used features. After importing the package to the project, a three-part mixing console can be loaded into the working scene (through a customized shortcut on the VR controller). It enables you to control the main parameters of your mix and is synced with Wwise. And the best thing: you can walk around in your scene, and the console automatically follows you. However, if you want to keep it in a fixed place, you can also detach it.

Enhanced visualization and selection

In the scene, each audio event is visualized by a green sphere and a label attached to the game object it originated from, making it easy to spot where a sound is coming from. Intuitively select a game object through the ray to control its parameters.

Recent event panel

All elements in the panel can be controlled with a small sphere attached to the virtual hands. You can also select an event by directing and clicking the virtual laser on the event of interest or by selecting it in the recent events panel, which lists all recently played events. You can also list and select the audio objects of the selected event. 

Audio object panel

In the audio object panel, you can access the distance attenuation graph. Adjusting the points on the graph feels like using a vast touch screen, providing precise control and intuitive handling. Hearing and seeing the object while working on its graph offers a much better understanding of the distance. The maximum distance can also be configured using a slider. 

Audio mixer

The mixer comes with volume faders and mute and solo buttons for all audio objects of the selected mixing session. You can select different mixing sessions via a dropdown menu. To get one of the audio objects in the mixer session into focus of the audio object panel for detailed adjustments, just select it by clicking on its name in the mixer. 

Summary

Using dearVR SPATIAL CONNECT for Wwise, game audio designers can easily focus on hard-to-find, problematic sounds and understand the spatial audio session in VR & AR. Dear Reality's application removes all the vagueness of mixing sounds while not simultaneously hearing the results in the final VR & AR environment. In conclusion, the upcoming workflow enhances overall productivity by directly working in the final format. No more mixing with the headset on your forehead! Mixing should be done where the results are heard – in VR & AR.

Contact the Dear Reality team if you want to partner up, have questions, or just want to stay updated about relevant developments of dearVR SPATIAL CONNECT for Wwise. Simply sign up for the priority list here.