In some of my own games we tried to implement a lot of cool audio features, linked a lot of things from the game to the sound design in an attempt to create cool audio moments or, as mentioned in Part 1 of this blog, lighten the load on the graphic saturation. At the same time, we wanted to be careful about not creating a scenario where a hearing player or the player with the sound turned on would have a significant advantage.

EVE Online is a great example from my own work, where we tried to put as much information into the sounds of guns and impacts, which made it possible to listen to a battle and theoretically hear which gun and which ammunition was used, or what type of shield the opponent's spaceship carried. Basically, we added very direct and current informants using audio; however, all that information could still be accessed by right-clicking the opponent's ship, or by looking up various forums, forms, plans, Excel sheets, or websites carrying the information.

So, we were not trying to change or replace how information would be relayed to the player outside of audio, we were simply trying to connect our sound design to what was already known by the game. That being said, EVE Online was a really heavy load on the audio engine, so spending 10 voices on one gunshot and one impact because of layering was simply not possible in a 1000 vs. 1000 players PvP battle, especially since they would all fire at the same time. (12) 

During the early development of EVE Valkyrie and some of the prototypes we had, audio informants were a great deal of the sounds taken into consideration, especially because EVE Valkyrie is a VR game and we had to attempt to create the most 'real person' soundscape possible, which was very different from the 'first person' sound design we had done in the past for EVE Online. The sounds signaling where your opponent was or where the action was happening, such as an incoming missile or the direction of that missile, all had to be taken into consideration. We had to think about how each sound would benefit the gameplay or if it actually ruined the gameplay. 

Below are a few Wwise screenshots to demonstrate various examples of applying informant audio within your sound design. 

 

Simple Music Switch showing a simple hierarchy playing different loops depending on Switch value.

 

 

Simple RTPC Controlling the above Switch. This could be the same system as used in EVE Online, controlling which music to play relative to the amount of active players in the current star system.

 

Simple RTPC controlling volume and pitch of a sound. In this case it could be a constantly playing sound that simply warns the player of enemy presence. Controlled by an RTPC relative to the value of NPC Distance to Player.

 

 

A simple Switch illustrating a very basic Switch in the Wwise hierarchy to easily switch between types of bullet impact sounds depending on health status.

 

RTPC Impact relative to the above. Shows that a value as simple as player health can be used to determine which branch of the Switch to play. This can be used to 'gamify' the audio so that the specific sound played back will be the players only indicator of their current health value.

From the 'real' world

One of the prototypes from EVE Valkyrie required that we create UI sounds that would not only fit the information on the screen and in the VR Cockpit, but UI sounds that could be useful without looking at that specific warning sign blinking on the ship dashboard. This led to some very interesting research on how sounds are used in actual airplanes; warning sounds and voice warnings, playing at the same time, separate times, or perhaps at the same time, but with different use of spectrum so they could be interpreted as two warnings even if they would play at the same time.

In audio ecology there is a term called the acoustic niche, which describes the actual use of sounds in the real world by animals. Animals are like humans, designed to warn with noise and be warned by listening to noise or by providing mating calls (hence the reason why our voices have a very dominant 2-4.000 Hz range, and oddly enough our ears naturally boost those frequencies specifically). The acoustic niche describes how some animals in certain forests time their calls, or when in sync with each other, stick to one area of the frequency spectrum while other animals use another, so that it is always possible to tell each warning or mating call from another even when they call at the same time. Of course, there is more to it than that, but this is a good example to demonstrate how sound in the real world is constantly used as an informant.

Blind people and even people with perfect hearing, use sounds all the time to orientate themselves. A great example is the use of click sounds at traffic lights indicating whether or not it is a red or green light. Today, where almost every pedestrian is looking down at their phone rather than watch where they are going, this click sound has perhaps become more useful than ever. Similar to the registration model mentioned in Part 1 of this blog, the traffic light registers if the light is green or red (true or false, yes or no, etc.) and provides any listener with the correct response to that value. 

Audio Weenies 

In level design, we often use the following term: weenies (some say landmarks, same but different). The Castle in the middle of Disneyland is a great example of a landmark. You can see it almost always, regardless of where you are at in the park, and you can easily navigate around the park by knowing where it is. (13)  Weenies are used to describe objects that are meant to grab the players attention, show the player where they are meant to go, or emphasize objects that they are meant to physically grab in the game. While this can be done by changing the color on the ledges that can be climbed, for example, or pointing a light in a specific direction, such solutions are related to color guiding and level design, which is not my area of expertise. 

I tried a lot to use the term weenies in my former sound design and determine how I could create a sound weenie to help players navigate around the level or even subconsciously create a point of interest which can make the player go in a specific direction or make them believe that they made a conscious choice where in fact it would be us guiding them the whole time.  This led me to a lot of interesting studies on level design and various ways and tools that can be used to grab the attention of players,  make navigation easier for them, and guide and influence them without them realizing. 

Adding a sound to a level which emits from a point of interest can be an easy way to create such a sound weenie or an audio informant for direction or location. Imagine five doors, the one that you are meant to open is the one which has some noise coming from it—this sound can be described as the leitmotif of the point of interest in this specific game object or mechanic. There are several terms which all overlap when it comes to what such a sound may be, including iconic sounds, earcon sounds, wordless communication, leitmotif, and more. 

A very old example which I have always used when describing informant audio is the sound of footsteps. Regardless of how you technically produce the footsteps, be it by blending shoe and material, by randomization, via white noise procedural generation, or something else, all these steps can be an important audio informant, an earcon of what you are stepping on, or a leitmotif of specific materials.

The sound of a step on a hard solid floorboard is different from that on a hard, yet hollow floorboard, or such a floorboard with a hollow space behind it, underneath it, and so on. Signaling a player who has to find where the treasure is buried in a game as they walk on a wooden surface can easily be done by having a massive arrow pointing to the spot of it's burial, or a UI element which pops up saying "you are standing on the treasure" or perhaps "press X to find treasure". But, we could also simply have the floorboard sound solid, except for where the treasure is buried. If it sounded hollow at that specific spot and we explained to the player earlier in the process, "you must press X at the right spot" to find the treasure, we can then leave it to the sound design to explain where the treasure is. This is a really good example of how informant audio can replace the use of a UI element to simplify or even dumb down the game play to make sure that no player would miss where the treasure is hidden.

If you look up any DIY website on how to find hollow spots in walls or floors when drilling holes to hang up your paintings or lamps, you'll find endless amounts of information on how to use sound as a mechanical informant.

(14)

As mentioned in Part 1 of this blog, when we are working with something interactive, the constant conversation is vital because, as we very well know, interactivity means something with the ability to constantly change, but the change is also important to the player and may require specific actions or reactions for the game to continue or for the player to benefit from the situation at hand. Therefore, while much like leitmotifs in linear media, audio weenies in interactive media and games are leitmotifs but for the game mechanics, objects, and so on. We also have to ensure that almost every element is part of the interactive art installation (yes, video games are art), so that it makes sense within the constant conversation. Audio weenies are key when it comes to creating an informant video game soundscape, and we should remember that informant audio should only be adding to your experience, taking anything away from the overall experience of the game or its overall aesthetics. 

 

Conclusion

To sum things up, using sounds to inform and guide humans is a natural part of our lives and we should definitely be taking this into consideration when building a game, a level, or a mechanic in a game.  Sounds can create a better experience for your player and can be key when it comes to how your game is perceived as a whole.

Using sound more often as an informant, or even as a replacement for some UI or graphical elements, does not mean that sound is now the ultimate way of providing information or player guidance, and it should definitely not take away from the other elements. We should just remember that sound can also provide information and prevent graphical saturation in a game. Just like sound can be filtered out naturally by the psychoacoustic phenomena known as the cocktail party effect, I believe that graphics can too. And, from my personal experience, I've see that excessive use of graphical elements placed all over the screen at one time can create a very confusing experience,  especially if that is the only method used to inform players about a situation. 

Additionally, I would argue that using informant audio will help create a better connection between the player and the game, creating greater immersion and a better overall experience for the player.  Here's a final story to illustrate this. I was at a circus play in Copenhagen. The speakers were set up so poorly that it was impossible to determine the actual source of any performer on stage. The speakers, placed behind the action, were the unique source of audio and were so far apart that, even from my almost center seat, I was only able to hear the 'right side' of things. This left me with a feeling of being disconnected from the entire play and there was no way to tell if it was actually a performer on stage or just a recording being played from that speaker. Beyond the lack of a stereo experience, the whole thing felt off.

As a sound designer I knew that what was most likely making me feel disconnected was the speaker placement; but, most likely, everybody around me didn't (sorry if one of you also knew). That became apparent during the break when both my partner and the people next to us said that they felt disconnected and that the show was boring. It wasn't, the visuals were great and the artistic and acrobatic performances were fine. After the show, one guy actually said to the person next to him 'I didn't get it … hmm.. 3 out of 5 stars'. I am pretty sure that if the audio had been set up correctly, the connection between the performers and audience would not have been broken, and we would all have had a great time, perhaps leaving with a 4 or even 5 out of 5 star assessment and memory.

One of the most problematic things when working with informant audio is often the lack of communication or understanding between departments in game development. Surely a sound design department can be really good at creating an aesthetically pleasing soundscape, and game and level designers be really good at creating awesome game mechanics and easy to navigate levels; but, I believe that in combination the results can be so much better.

If making sure that sound works correctly can benefit the connection made between the game and the player (just like in this theater and audience example), then this can make the difference between a 3, 4, or 5 star review.  So, why not make sure that audio does play a part in connecting the player to the game, and put some faith in sound design as a game defining tool and art. It might make a difference for your metacritic score.

 

 

 

On Game Audio Informants and Game Audio Gamefication : Game Audio Talk: L.A. Noire

 

Informant Diegesis in Video games was my Ms. Thesis at Aarhus University.

Sound is important in all media; this thesis focuses on the use of sound as a method of grabbing the attention of the player and guiding them through the game. Simply put, using sound as an informant to support game design or create its own piece in the game design puzzle.

 

12.Minor nightmare when limiting voices and figuring out the hierarchy of importance!

13.For more and better information on level design, weenies, landmarks, narrative in levels and their use visit: https://www.patreon.com/GameMakersToolkit or https://www.youtube.com/LevelDesignLobby for some great podcasts on the subject as well. That’s where I found some of the best descriptions around.

14. Top down view of a floor with color coding tiles. Step on a red tile responds with a solid wood sound and blue results in a hollow wood sound - and we have ourselves an audio informant on where to find our treasure. Notice my more than awesome and rad skills at MS-Paint!