Using Ambisonics for Dynamic Ambiences

사운드 디자인 / 공간 음향 / Wwise에 대한 팁과 도구

Rotating Ambisonics

As we have seen in Ambisonics as an Intermediate Spatial Representation (for VR), ambisonics is a multichannel audio format that can favorably represent the spatiality of an audio mix. An ambisonic audio file/mix/signal can thus be regarded as a sound field, a mixture of different sound sources coming from different directions and arriving at a listener's ears with different angles of incidence. 

Rotation by matrixing

One useful property of the ambisonic representation is that it can be rotated very easily, by cleverly manipulating the matrix of mixing gains. In terms of resources (CPU and memory), this is equivalent to calculating panning gains; and, because this is done at every audio frame (~10-20 ms) instead of at every audio sample (~2 µs), it is orders of magnitude less demanding than filtering or even mixing (adding) audio signals.

After having rotated an ambisonic signal, you get a new ambisonic signal where the apparent angles of incidence of all its constituents have been rotated accordingly. 

Ambisonics rotation in Wwise

Wwise implements rotation of ambisonic sound fields automatically. The rotation corresponds to the relative orientation of the associated game object and its listener. To enable this magic, you need to set the ambisonic sound’s 3D Spatialization to Position + Orientation and route it to an ambisonic bus, as depicted in the screenshots below. Also, you need to make its Spread equal to 100% by adding an appropriate Attenuation Shareset, otherwise it will collapse into a mono point source because Spread is 0 by default. The game object and listener orientations are driven by the game through the Wwise API.

ambi-img1

(Fig 1-a)

bus

(Fig 1-b)

3-rotation-routing

(Fig 1-c)

4-Image_Pasted_at_2016-9-1_15-43

(Fig1-d)

Figure 1 - Positioning and routing required for enabling rotation of ambisonics in Wwise. Notice the 3D and Game-Defined options selected in (a). The asset in question is routed to a bus with an ambisonic channel configuration. (b) shows how to set a bus to ambisonics. The routing can be observed in the Profiler view (c). The actual rotation is given by the relative game object and listener orientations driven by the game engine, here in Unity (d). 

The following audio example illustrates ambisonic rotation. You should listen to it with headphones. Audio sample 1 was produced by playing an ambisonic file and converting it to binaural using the Auro Headphone plug-in in Wwise. In order to produce Audio sample 2, we simply changed the orientation of the listener (using a game engine) to point to the left, while the sound field's object orientation points to the front. The listener is rotated counterclockwise (towards the left), and this is why you hear the sound field rotated clockwise.

Audio sample 1 

Audio sample 2

 

Ambiences Using Ambisonics Assets

The ease with which the channels of an ambisonics signal/bed/stream can be manipulated in order to rotate the sound field it represents makes it great for implementing dynamic ambiances in games. You need to play them back on game objects that are constant in orientation and point towards a specific reference. For example, if there is an auditory element with clear directionality in a recording, which maps to a visual element in the game, you will want to orient the game objects so that they align. When the Listener navigates in the space and changes orientation, the ambisonics sound field is automatically rotated as it is being mixed into the bus. Consequently, the sound field is coherent with the visuals.

Using recorded material

In Wwise, you can import ambisonics recordings that you have captured using an appropriate coincident microphone (such as http://www.core-sound.com/TetraMic/TetraMic-small3.jpg) after converting them to B-format (conventional 1st-order ambisonics). Most microphones come with their own software to transform the raw microphone signal, typically in A-format, to B-format**.

Here is an example of rotation done in Wwise with a field recording. The original audio, Audio sample 3 has been recorded using an SPS-200 microphone, converted to B-Format, and imported in Wwise. The most noticeable auditory event is the car horn, located slightly to the left in the original sample. To produce Audio sample 4, we turned the listener 90 degrees towards the left (as we did in the previous example), our next step was to play back and record the output. The car horn moved mostly to the right and all other sound sources rotated accordingly.

 

Audio sample 3

Audio sample 4

 

"Bouncing" Ambiences

It is possible to generate ambisonic sound files from an interactive session in Wwise using the Wwise Recorder plug-in. For example, you may create a complex ambience in the Wwise authoring tool by using a plurality of mono sources placed in space, mixing them in an ambisonics signal, and then recording that ambisonics signal using the Wwise Recorder plug-in. The next step would be to reimport the recorded ambisonics asset into your Wwise project, and use it as a rotatable 3D ambience in your game, exactly as if it were recorded material. 

This workflow is similar to the "bouncing" or "freezing" option typically available in linear DAWs, such as Nuendo, Reaper, and ProTools. One motivation for bouncing an ambience to disk instead of keeping all its individual components for simultaneous playback is to save runtime resources (CPU and memory). This can be achieved by saving disk space and playing one multichannel file instead of many mono files at the same time.

Bouncing ambiences in Wwise

  1. Create a bus and set its channel configuration to Ambisonics (any order).
  2. Insert a Wwise Recorder plug-in on that bus. Since it is an ambisonics bus, the audio will be written to file in the ambisonics format. 
  3. Route all your individual mono sources (or their common parent) to this bus.
  4. Place each source in space using the Automation Editor in either Emitter with Automation or Listener with Automation modes.
  5. Trigger them at opportune times using a Soundcaster Session, an Event, a Random Container, or any combination thereof. When all sounds stop playing, because they are finished or because they are stopped, the audio will be "bounced" to disk. You may import it in your project directly by clicking the Import in Wwise... button on the Recorder plug-in.
  6. Inspect this new asset and set its positioning type to 3D. When played from the game, it will be rotated as described previously.

 

5-bounce-profiler-and-structure.png

(Fig 2-a)

 

6-user-defined_positioning.png

(Fig 2-b)

 

7-Recorder.png

(Fig 2-c)

Figure 2 - The Ambisonics City Ambience Blend Container of the Wwise Sample Project being recorded into a single ambisonics file. The Project Explorer view on the left of screenshot (Fig 2-a) shows the ambience made of the recording mentioned above (Ex1 - Ambisonic City 3D Ambiance), superimposed with a large number of individual sounds, and mixed and played back using various behaviors specified in the Random Containers. Screenshot (Fig 2-b) shows an example of how an asset (or group of assets) has been placed in space using the 3D User-Defined Position Editor. Screenshot (Fig 2-c) shows the settings of the Wwise Recorder plug-in. More details on this plug-in

 

As explained in a previous blog, Ambisonics as an Intermediate Spatial Representation (for VR), you may record up to 5th order ambisonics for improved spatial precision, but, the increase in the required number of channels (9, 16, 25 and 36 channels for 2nd, 3rd, 4th and 5th order respectively) and, consequently, memory demands, may outweigh such improvements. Also, while recorded ambisonics retains the relative arrival direction of its individual components, it does not allow modifying their individual properties–such as their relative volume and filtering–afterwards***.

 

* As long as the ambisonics format is "full-sphere".

** Wwise accepts AMB files in FuMa convention, identified by a proper GUID in their headers. See more

*** Some signal processing techniques allow one to do such things. However, in our context they are fairly irrelevant since we already have access to the individual components prior to mixing them.  

Figure 2 - The Ambisonics City Ambience Blend Container of the Wwise Sample Project being recorded into a single ambisonics file. The Project Explorer view on the left of screenshot (Fig 2-a) shows the ambience made of the recording mentioned above (Ex1 - Ambisonic City 3D Ambiance), superimposed with a large number of individual sounds, and mixed and played back using various behaviors specified in the Random Containers. Screenshot (Fig 2-b) shows an example of how an asset (or group of assets) has been placed in space using the 3D User-Defined Position Editor. Screenshot (Fig 2-c) shows the settings of the Wwise Recorder plug-in. More details on this plug-in. 

-Screenshot Figure 1(d): Unity 3D 

-Recorded 3D Ambience Sample, and modified version: 

Ambisonic B-Format ambiance from A-Format SPS200 microphone recording.
Audiokinetic Inc. All rights reserved.
Audio content provided by Studio Lab SONORUM
(c) 2016. www.sonorum.ca

The Audio Content contained in this file is the property of Studio Lab SONORUM. Your sole right with regard to this Audio Content is to listen to the Audio Content as part of this Sample Project. This Audio Content cannot be used or modified for a commercial purpose nor for public demonstration.

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Louis-Xavier Buffoni leads the research team at Audiokinetic and has been focusing on spatial audio, sound synthesis, audio coding and machine learning.

 @xbuffoni

댓글

David Eisler

September 20, 2016 at 02:52 pm

Wow. This seems really groundbreaking, or at least awesome

댓글 달기

이메일 주소는 공개되지 않습니다.

다른 글

사운드 디자이너가 PureData + Heavy를 사용하여 DSP 플러그인을 개발하는 법 - 제 1부

많은 사운드 디자이너들이 오디오 플러그인 개발을 오디오 프로그래머의 '흑마법' 영역이라 생각합니다. 보통 코딩 기술뿐만 아니라 수학, 물리학, 디지털 신호 처리 등 사운드...

17.11.2020 - 작성자: 천종 호우 (Chenzhong Hou)

Wwise로 도플러 효과 제작하기

도플러(Doppler) 효과는 음파를 기준으로 관찰자가 움직임에 따라 파동의 주파수가 변하는 것입니다. 이 물리적인 현상은 모든 종류의 파동 전달에서 일어나며 음파도 예외가...

30.3.2022 - 작성자: 쒸 웨이 (Xu Wei, 徐巍)

ReaWwise 개발 | 제 1부 - 사전 제작

이제 ReaWwise가 출시되었으니 이 REAPER 확장을 제작한 과정에 대해 공유할 좋은 기회가 왔다고 생각했습니다. 두 부분으로 구성된 이 블로그 시리즈의 첫 번째 부분에서는...

6.10.2022 - 작성자: 베르나르 로드리그 (Bernard Rodrigue)

노 스트레이트 로드(No Straight Roads)의 음악적 게임 세계 설계하기

안녕하세요, 게임 오디오 여러분들! 저희가 Wwise와 Unreal Engine을 사용해서 '노 스트레이트 로드(No Straight Roads, NSR)'의 극도의 스타일링을...

29.3.2023 - 작성자: Imba Interactive (임바 인터랙티브)

올바른 코덱 선택에 대한 안내

게임 오디오에서는 항상 오디오 파일을 압축해야 했습니다. 우리가 꿈꾸는 그대로의 오디오 환경을 모두 압축되지 않은 오디오 샘플로 두기에는 여전히 디스크 공간이나 메모리가 부족하다는...

14.5.2024 - 작성자: 마튜 장 (Mathieu Jean)

AudioLink로 떠나는 여행

지난 10월 게임사운드콘(GameSoundCon)에서 저는 호텔 근처 고급 샌드위치 가게에서 데미안(Damian)과 점심을 먹고 있었습니다. 예상하셨겠지만 저희는 오디오 기술에...

10.6.2024 - 작성자: 피터 "pdx" 드레셔 (Peter "pdx" Drescher)

다른 글

사운드 디자이너가 PureData + Heavy를 사용하여 DSP 플러그인을 개발하는 법 - 제 1부

많은 사운드 디자이너들이 오디오 플러그인 개발을 오디오 프로그래머의 '흑마법' 영역이라 생각합니다. 보통 코딩 기술뿐만 아니라 수학, 물리학, 디지털 신호 처리 등 사운드...

Wwise로 도플러 효과 제작하기

도플러(Doppler) 효과는 음파를 기준으로 관찰자가 움직임에 따라 파동의 주파수가 변하는 것입니다. 이 물리적인 현상은 모든 종류의 파동 전달에서 일어나며 음파도 예외가...

ReaWwise 개발 | 제 1부 - 사전 제작

이제 ReaWwise가 출시되었으니 이 REAPER 확장을 제작한 과정에 대해 공유할 좋은 기회가 왔다고 생각했습니다. 두 부분으로 구성된 이 블로그 시리즈의 첫 번째 부분에서는...