menu
 

Why VR will put Game Audio in the Spotlight

게임 오디오 / 공간 음향 / VR 체험

sense of presence in VR

The sense of Presence

The word that is on everyone’s lips when describing VR (Virtual Reality) is  “immersion”.  A cool word indeed. To achieve this ultimate goal however, developers need to simulate the perception of being physically present in a virtual environment, and we call this the sense of presence“Presence is a psychological state or subjective perception in which even though part or all of an individual's current experience is generated by and/or filtered through human-made technology, part or all of the individual's perception fails to accurately acknowledge the role of the technology in the experience.”  (The International Society of Presence Research - ISPR)

Creating this level of true immersion is a real challenge. While graphics already made tremendous improvements in the past few years, audio still has a long way to go in regards to reaching its true potential of playing a major role in this new and still evolving field of VR.

Spatial immersion and multi-sensory uses in VR gaming

In console and PC games, immersion can be separated into three categories: narrative immersion (players being invested in a story), tactical immersion (players performing tactile operations that involve skills), and strategic immersion (more cerebral, associated with mental challenges).  In VR however, a fourth category is added: spatial immersion, which convinces the player the artificial world surrounding them is real.  Among the former three categories, spatial immersion is also of great importance when it comes to triggering the sense of presence and delivering a believable and immersive environment.

To position ourselves physiologically, our visual system is used to help guide our bodies within our environment, our ears provide crucial information about our surroundings, and smell provides complimentary surrounding information.

Audio goes beyond vision in the sense that it is not limited to our field of view, but rather provides a 3D-audio feedback (including 360-degree, height and depth) information about our surrounding environment.

VR immersion via audio

In order to create realistic spatial immersion, ideally, we would want to be able to use all of our five senses and have full sensory processing (process of using our bodies effectively within an environment). The problem at the current stage in VR development is that we are unable to utilize all five senses. Sight and hearing are used as the primary senses, touch is not used very effectively, and taste and smell are not integrated at all at this point. With that in mind, audio is perhaps the most powerful medium we can develop in order to “hack” the user’s brain and create a complete 3D-audio experience that offers spatial immersion, as it gives the representation of a realistic space by providing virtual sources (which can’t be necessarily seen in the scene or action) rendered through a simulated acoustic environment.

3D-audio experience to create spatial immersion

The first step to enhancing spatial immersion through audio realism is audio positioning. A pattern emerging from recent studies (Skalski & Whitbred, 2010) showed that better sound positioning in games (surround VS stereo) had a much higher impact on the player’s spatial immersion and enjoyment than visual quality (HD VS standard).

The brain uses subtle differences in loudness, tone and timing between our two ears to localize sound sources in three-dimensional positions (horizontal, vertical, and distance). In VR, viewers are literally in the middle of a scene, so the process of determining the location of sound sources in a simulated acoustic environment is the most important audio improvement towards creating natural listening experiences.

 binaural

Early in media history, we discovered that the human ears pick up audio in three dimensions and the first binaural recording goes back to 1881.

“Binaural recording is a method of recording sound that uses two microphones, arranged with the intent to create a 3-D stereo sound sensation for the listener of actually being in the room with the performers or instruments. This effect is often created using a technique known as "dummy head recording", wherein a mannequin head is outfitted with a microphone in each ear.” (Wikipedia, “Binaural Recording”)

In games, 3D-audio should be created with real-time processing since we can’t anticipate where sound objects will be located corresponding to the player's location and head movements.  So, about a century after its invention, binaural recording is now being revisited with binaural processing, and engineers and investors are working hard to develop the best binaural algorithms suitable for VR.

Indeed, because of its nature in regards to the required use of stereo headphones rather than speakers to replicate 3D positioning (necessary in VR to isolate the player from the outside world and have audio playback that matches head/ears movements relative to the field of vision), binaural has been logically chosen by the professionals as the new standard for 3D-audio processing instead of the usual surround techniques (limited to horizontal and distance audio positioning).

Conclusion

Many audio challenges, considerations, and needs are being exposed as we work with VR.  By exploring these further, we will be able to push the boundaries of the VR and gaming industry altogether.  And, although the gaming industry has been improving visual quality in the last generation of consoles, with the VR development on a rapid rise, the mindset within the VR industry is quick focusing on audio as the VR community is realizing that better audio realism will increase the player’s enjoyment and engagement levels, elevate the state of immersion, and ultimately, the sense of presence.

3D-audio positioning is just one of the areas where audio improvements are being pursued as a direct result of VR developments.  Audio working tools are also in continuous development with new technologies catering to VR needs and its uniqueness when it comes to the mixing techniques required, in order to enhance VR experiences.

Audio is the next medium for the industry to explore and focus on, and VR is certainly helping put audio the spotlight!

 

 
 
 

Alex Riviere

Audio Director

CCP Games

Alex Riviere

Audio Director

CCP Games

Audio Director with 10+ years of professional experience in audio across multiple industries. Pioneer Audio Director / Audio Lead for VR Games.

 @CCP_alexriviere

댓글

댓글 달기

이메일 주소는 공개되지 않습니다.

다른 글

이미지 기반 파라미터를 이용한 오픈 월드 앰비언트 디자인

Blend Container는 강력한 시퀀싱 및 믹싱 도구가 될 수 있습니다. 단순히 그것의 기능을 배우는 것이 게임 사운드 디자이너의 생각에 온갖 종류의 새로운 아이디어를...

13.3.2020 - 작성자: 톰 토디아 (TOM TODIA)

Jurassic World: VR Expedition (쥬라기 월드: VR 탐험)

Jurassic World: VR Expedition은 위치 기반의 경험을 제공하는 인터렉티브 시네마틱 가상 현실(VR)로서, 플레이어들을 시각적으로 아주 아름다운 이슬라 누블라의...

18.3.2020 - 작성자: Hexany Audio (헥사니 오디오)

파이널 판타지 VII 리메이크의 사운드

수많은 사람들의 사랑을 받는 파이널 판타지 VII의 음악은 새로운 세대의 팬들이 경험할 수 있도록 수 년에 걸쳐 새롭게 재고되어 왔습니다. 최근 Square Enix(스퀘어...

4.5.2021 - 작성자: SQUARE ENIX (스퀘어 에닉스)

Wwise로 도플러 효과 제작하기

도플러(Doppler) 효과는 음파를 기준으로 관찰자가 움직임에 따라 파동의 주파수가 변하는 것입니다. 이 물리적인 현상은 모든 종류의 파동 전달에서 일어나며 음파도 예외가...

30.3.2022 - 작성자: 쒸 웨이 (Xu Wei, 徐巍)

실용적인 구현: Wwise를 활용하여 세인트 로우(2022)에서 실제 라디오 재구현하기

세인츠 로우(Saints Row, 2022)의 사전 제작 단계에서 시스템을 처음 설계할 때 오디오 팀은 여러 시스템의 작동 방식을 실제 세계에 맞추기로 결정했습니다. 이는 주로 팀...

15.1.2025 - 작성자: 브렌든 엘리스 (Brendon Ellis)

Wwise로 게임 마스터링하기 | 제 1부: 게임 오디오 마스터링 접근 방법

게임은 다양한 구성에서 뛰어난 청각적 경험을 제공해야 합니다. 이것은 헤드폰의 3D Audio가 될 수도 있고, 스테레오 TV 스피커 혹은 광대한 7.1.4 Atmos 시스템이 될...

19.3.2025 - 작성자: 로익 쿠티에(Loïc Couthier) & 단제리 스켐브리(Danjeli Schembri)

다른 글

이미지 기반 파라미터를 이용한 오픈 월드 앰비언트 디자인

Blend Container는 강력한 시퀀싱 및 믹싱 도구가 될 수 있습니다. 단순히 그것의 기능을 배우는 것이 게임 사운드 디자이너의 생각에 온갖 종류의 새로운 아이디어를...

Jurassic World: VR Expedition (쥬라기 월드: VR 탐험)

Jurassic World: VR Expedition은 위치 기반의 경험을 제공하는 인터렉티브 시네마틱 가상 현실(VR)로서, 플레이어들을 시각적으로 아주 아름다운 이슬라 누블라의...

파이널 판타지 VII 리메이크의 사운드

수많은 사람들의 사랑을 받는 파이널 판타지 VII의 음악은 새로운 세대의 팬들이 경험할 수 있도록 수 년에 걸쳐 새롭게 재고되어 왔습니다. 최근 Square Enix(스퀘어...