This article was written following the second annual Step Up Your Sound Game Jam, an Android mobile game jam where teams developed one-of-a-kind gaming experiences using Dolby Atmos with Wwise. This is a 5-part series; in this second blog, the Nekologic team dives into their game Menura.

A journey in birdsong

“The island is as intriguing as I imagined, there are so many birds here to find and catalog! Their calls are stranger than anything I've ever heard. There is much to explore!

It will be morning soon, and I cannot imagine the glory of the dawn chorus!”

In Menura (from Menura novaehollandiae or “Superb Lyrebird”), you play an ornithologist sent to a mysterious island to catalog the bird species that live there. Based on the childhood fascination with birdsong of one of our team members, we imagined a forest in which players could casually wander, surrounded by birdsong and the sounds of nature.

We are Nekologic, a game studio based in Kyoto, Japan, and it’s our pleasure to introduce the secret inner workings of our game, developed with Dolby Atmos and Wwise, for the Step Up Your Sound Jam 2023 with Playcrafting.

Beginning the adventure

Evoking a feeling

We started with a feeling that we wanted to evoke: a blend of relaxation, exploration and discovery. We felt we could use Dolby Atmos to create an immersive soundscape that could be experienced even without visuals. The bird theme naturally suggested a forest setting, but to retain a feeling of wonder–and to maintain an emphasis on the sonic aspect of the game–we decided to use an abstract visual language, describing the scene with color and movement. To emphasize the feeling of the natural world, we wanted everything to be organically generated, so the player could constantly discover new bird species, and the bird songs for each species would be recognizable yet never repetitive. The final piece of the puzzle was the passing of time, which would allow us to vary which birds appeared–and how they behaved–at different times of day.

Dawn in the forest

Gameplay design

We decided early on that a first-person perspective would allow the player the best chance to experience the feeling of moving through and looking around the forest. Each bird would be placed in 3D space–hiding in trees and bushes–and the spatial audio provided by Dolby Atmos would allow the player to pinpoint where the birds were hiding. Some of the trees are relatively tall, and since Dolby Atmos is particularly good at rendering the difference between sounds overhead and on the ground, this is a perfect scenario for a full 3D experience with a good depth of the mix. When the player is close enough, he or she would be able to tap on the screen to take a picture of the bird to place in their ornithologist’s notebook, along with a recording of the bird’s unique call.

There were a few technical challenges in executing this idea:

• How to use Dolby Atmos and Wwise to generate the birdsong
• Where the birds would be and when they would sing
• A system for the bird’s appearance in the logbook
• A relaxing and intriguing environment, varied in terrain and vegetation
• Controlling the randomness of all of the above

Generative Birdsong with Dolby Atmos and Wwise

Our idea was to create an infinite variety of distinct bird voices that not only imitated the sound of real-world birds but also went beyond, towards the supernatural. Each bird species has its own vocabulary of song elements consisting of notes, swoops, chirps, trills, warbles and other features. Individuals of that species then recombine these elements spontaneously when singing based on environmental factors.

The song elements are modeled as sequences of instructions passed into Wwise as gradually changing Real Time Parameter Controls (RTPCs) with different interpolation shapes and durations. Since real-world bird songs contain an incredible amount of detail in fast-paced modulations and timbral variations, we knew that we needed to model the majority of these modulations directly within Wwise to avoid overloading the Unity update loop. After some experiments with Wwise’s built-in synthesizer plugins–using simple sine oscillators and nested modulator signals of different shapes (see figure below)--we decided to use actual audio fragments of songs of different birds, each only a few milliseconds long, to provide a more realistic variety of timbres. We play these samples using looped Wwise sound sources equipped with LFO modulators, whose parameters (e.g. shapes, frequencies, depths, etc) are in turn controlled by RTPCs.

An early simple hand-drawn modulation curve for a finch-like sounding bird  |  Hear more at games.dolby.com

We then randomly generated RTPC ramps for each species based on constraints such as characteristic ranges for pitch, modulation frequency and depth, element durations, singing variety, and recombinations. Due to Dolby Atmos’s rendering clarity–especially of overhead sounds–this produced a satisfying soundscape where a large number of different bird species were distinguishable. In the current build, the settings of the constraints are still kept relatively tame. But for the next iteration, our model will generate much longer, wilder, and more complex songs.