Project Hiraeth is the world’s first video game score license to be fully adaptive, professionally mixed and mastered, and recorded with a live orchestra. It features over ninety minutes of original music, responds naturally to player choices, and comes with all the scripting needed to fully integrate into a game in under a week. This blog post explores why, and how, it was created.

A GOOD MORNING FOR A PARADIGM SHIFT

In the fall of 2017, I had coffee with Guy Whitmore.  Now, industry vets will know Guy without introduction. He’s the fedora-wearing, quirky man responsible for such marvels as the famed Peggle ‘cat choir’ and some of the oh-so-satisfying Xbox 1 UI ‘pops’ and ‘swooshes’ we all love. He’s a humble dude, but he’s got four GANG awards and the job of my dreams, so I – the young and inexperienced grad student - was listening intently. 

Guy and I were talking about the possibility of an internship at his studio, and we were both trying to feel each other out a bit. He sipped his black coffee and threw me a curveball:

“What” he said, “makes game music good to you?”

I sputtered and did the kind of pause that someone does when they’re trying to be eloquent but don’t belong in a conversation. Eventually I pulled something passable out: “I think that music is good if it’s well-crafted and captures the emotion that the developer’s need.”

“Of course!” He shot back, “but what about Wwise? What about technology?”

Audio middleware? What about it?

He continued: “we have all of the technology available to us to make game music every bit as responsive and engaging as film music.”

Wait. What?

“If we can make music that reacts in real time to player choices, avoids harsh transitions, and doesn’t loop over and over, that kind of thing – why don’t we? Why is that not expected from all composers? The technology is there, what’s holding us back?”

This was the first introduction I had to the seminal ideology behind Guy’s approach to music: that we, as composers and artists, have an obligation to use emerging technologies to push what we are doing with our work.

Guy has a word for it that I think is pretty cool: music design.

 

Music Design: a Quick and Dirty look

You probably guessed that I wound up getting the internship. In the summer of 2018, I shipped off to Seattle to follow Guy around for a few weeks. We hung sound panels, wrote music, ran a lot of miles, and drank hilariously inappropriate amounts of coffee.

As the internship wound down, I’d become thoroughly convinced of the ways of the Music Designer.

For the uninitiated, here’s a few of the truth’s that make-up a career in music design:

• The creative leads the technical.

When you begin writing, ask how it would sound and feel in a perfect world, with no barriers to technology. Consider scoring a gameplay capture video just like you would score a movie. Then, only after your music is gripping, responsive, and effective, ask how the technology can create what you envision.

• Audio Middleware is part of what it means to be a modern composer.

Gone are the days when our final submission was a linear audio file or stems. For a music designer, the final submission is at least a Wwise file. Music designers use middleware as a part of their compositional toolset, and view techniques like horizontal resequencing and vertical remixing as a part of the process, just like melody and good counterpoint.

• Video game music is the art music of our time.

No other compositional medium allows as much creative potential as writing music for games. We can control not only what our players hear, but how they hear it. Using variability and randomization, we create music systems, not just music compositions.

As Guy and I had a final cup of coffee before the end of my trip, I asked the obvious question: if music designers can consistently turn out better results, with more engaging music, why are developers not demanding their composers work this way, and why is every composer not biting at the bit to learn these emerging technologies?

Guy had a few things to say that I found profound, and I’ve learned a few of my own truths along the way:

• Not utilizing audio middleware as part of the composition process isn’t a ‘bad’ thing.

The last five years, top music awards have consistently gone to games that feature riveting scores that lack deep levels of integration. Clearly, we would never suggest that this approach is ‘sub-par’ – that’s just not true. Rather, we feel the depth that the average composer can reach if they include middleware in their creative process is remarkable worth-while.

• It’s easy to be intimidated by modern technologies, instead of inspired by them.

Let’s face it, Wwise is a big program. In spite of incredible resources like Youtube videos, the Wwise 201 and 301 certifications, and the Wwise Project Adventure, many composers are still worried about tackling a new way of thinking about their music.

• Game Developers often feel uncomfortable asking composers to integrate their own music into Wwise.

I don’t think I’m overstepping my bounds saying that communication between composers and game devs can often be... confusing. The fact is that both parties desperately need the services of the other, but a lot of the time don’t know how to communicate clearly with one another. Offering for a composer to do their own integration may be intended to be empowering but come off as belittling if the composer is unable or unwilling.

• Composers can be guilty of a ‘this is what I do, and I’ve always done it this way’ attitude.

It can be easy for composers to fall into the trap of thinking “I’ve always done it this way and I’m not going to change now”. Plus – if you’ve built a successful career sending in music stems and calling it a day, why change now? (Here’s the Music Designer’s answer, by the way: working with audio middleware yourself can result in better music, a more engaging game, and can actually shorten your production time!)

It’s at this point, as we were chin deep in our second-round latte’s, that gears began turning for us. I had discovered a real passion for Music Design and had already shared a strong desire to continue working together, and Guy... well.... he had a crazy, crazy idea.

 

The Crazy Idea

The idea was this: what if we could create a product to show composers that adaptive scores could be made quickly, cheaply, and professionally? To do this, Guy proposed, I could work for him during my final year of grad school, creating an adaptive score like nobody had done before: fully produced, integrated into Wwise, and orchestral, just without a game. If a new kid on the block with no industry experience to speak of can do it, after all, anybody can – right?

Guy and I made a deal: if I could do this, and make it work, he would seek out game developers who might be a good fit for the score, and would submit a proposal for a talk on the project at the Game Developers Conference.

So, fueled by a passion for adaptive audio and some truly criminal caffeine levels, I did. Over the course of the next year, I created and executed a plan for Project Hiraeth, and came out on the other end with something that I am very, very proud of. Ultimately, the purpose of this work was to do three things:

• Demonstrate that the composer as “music designer” - a creator who does all integration, production, and composing in-house - is a viable method for modern music-makers.
• To create a Wwise Template from which other adaptive scores can be created.
• To craft a deeply emotional and nuanced score that can be plugged into a thematically similar game in under a week.

The rest of this post will outline my process for creating Hiraeth and will conclude with a quick video demonstration.

 

Part 1, A: Overall Structure

My work on Project Hiraeth began after the culmination of my internship with Guy in August of 2018. I knew that all work on the project had to be completed before my graduation in May of 2019, so we had a production window of about nine months. I began by deciding specifically how the score should play out in its final form. In short, I wanted it to sound professional, evoke real emotional responses, and be as seamless as possible. Because I wasn’t writing for a specific game, but still wanted the score to be marketable, I chose a specific genre (gritty fantasy) and outlined 10 different music environments within that genre. These are the ten cues I proposed, along with quick write-ups about what I envisioned:

Project Hiraeth: Overall Structure

Each section includes a short text blurb designed to capture the overall sense of the score

• Environment 1 - The Gorgeous Over-World

Enigmatic exploration music that evolves with a customizable level of silence throughout. 

(Fully orchestral beauty cues seamlessly evolve from the score in key moments.)

• Environment 2 - The Mysterious Wood

A surreal harp and otherworldly winds bring a subtle uneasiness to nighttime exploration.

• Environment 3 - The Dark Underworld

An eerie choir, distorted brass, and mangled strings give a visceral and terrifying underscore to the game’s most evil settings.

• Environment 4 - The War-Torn Landscape

A solo trumpeter plays mournful laments over a dark string bed, hinting at the forlorn past of a tormented land.

• Dramatic 1 - Dark Machinations*

When malicious forces make their play, twisted percussion and chaotic horns set the perfect tone.

• Dramatic 2 - Rally the Troops*

Stabbing brass, booming percussion, and driving strings are the perfect fit when the hero rises to strike down his nemesis.

• Combat 1 - Bring the Mountain Down*

When swords are crossed, these pulsing violas pair with visceral synths, risers, and chugging basses. This is battle music for when the thick of the fight is underway.

(Transitions in under two seconds to in-game cinematics, with three different cues for emotional, tense, or violent scenes.)

• Combat 2 - Men of Iron*  

More subtle than Combat 1, but still with a hefty punch, this music is the perfect fit when a single warrior fights alone against all odds.

(Transitions in under two seconds to in-game cinematics, with three different cues for emotional, tense, or violent scenes.)

(Pairs seamlessly with Dramatic 2.)

• Menu Music - The Hero's Rest

The hero journeys through a gentle island in the mist, where he searches for deep secrets of a forgotten past. 

• Main Theme - Overture to Hiraeth

The cornerstone of the entire score. World class performances by elite soloists and a master sound engineer are only a few of the secrets hidden in the theme.

*Contains multiple intensity levels, each with its own independent variations.

 

Part 1, B: The Project Timeline

I tend to be organized to a fault, so creating a clear timeline for my production seemed a logical next step after settling on an outline for the ten music environments. To set out my goals clearly in a tweakable format, I used trello.com: a remarkably powerful service for organization and project management. In the picture below, you can see an early series of goals for the first months in the project:

 

Before I wrote a note, clear goals were laid out like you see above for every week of the coming year. I also:

• Color coordinated benchmarks based on topic.
• Included a number from 1 – 4 in each week’s heading to indicate how time intensive my work that week would be.
• Shared this overview sheet with Guy so he would know that I did, in fact, have a plan.

In short, production on the project was broken down as follows:

• August – November 1^st: Final deliverable is 100% of the game’s music integrated into Wwise as an electronic mock-up.
• November 1^st – November 30^th: Create a physical score and parts, contract an orchestra, record. Final deliverable is 100% of the score as live recorded WAV files in stems.
• December 1^st – February 1^st: Combine live instruments with MIDI, mix, and master all stems. Final deliverable is a full suite of independent stems for each instrument or section to maximize randomization in Wwise as much as possible, while also bearing in mind final file size.
• February 1^st – March: Integrate final stems into Wwise, troubleshoot, showcase at the Game Developer’s Conference.

Of course, I had the benefit of biweekly Skype updates with Guy throughout the duration of the project.

 

Part 2: Composing

I didn’t know when I started writing for Hiraeth that I was going to write ninety minutes of music. That quantity emerged organically, as I worked within the confines of strict weekly guidelines. I chose to frontload the writing into September, because October would be filled with graduate school, a hefty teaching load, and my own film scoring business. Typically, weeks broke down like this:

• Week 1: Create a sketch for Environment 1.
• Week 2: Flesh out my sketch for Environment 1 while beginning Environment 2.
• Week 3: Use Environment 1 stems in Wwise to recompose and deepen the music. Flesh out Environment 2. Begin Environment 3.
• Etc...

Ultimately, I wrote about 20 minutes of music a week, which I would whittle down to 10 or so minutes of really evocative material. Wwise was a critical part of my composition process, although my sketching took place in a DAW.

For a cue to be “done”, it had to be:

• Implemented into Wwise as stems.
• Functional using the “nested states” system that I decided to work with for middleware implementation. (this blog will not go into detail on this topic, although you can get the gist of it in the video in the final section).
• Approved by Guy.

Writing the combat cues took a bit of a different approach. For Combat 1, I chose a 10-minute combat clip from the most in-depth battle system I could find in terms of cinematography: God of War. I then re-scored that clip like a movie and asked how I could make a music system for a game that didn’t exist work with that level of interactivity. Ultimately, combat cues took the following form:

• 3 – 4 levels of intensity, which switch back and forth based on enemy proximity, quantity, and player health.
• 3 “Cinematic” music segments (“Emotional”, “Intense”, and “Beatdown”) that can be transitioned to in under two seconds.
• Multiple layers of vertical remixing to create the illusion of diversity and avoid listener fatigue.

At the end of the composing section of the project, I had a fully functional Wwise demo made up of mock-ups. This demo was showcased in its early form at the end of Guy’s presentation at the Wwise Interactive Music Symposium on November 6^th, 2018.

 

Part 3: Copying and Recording

Early on in the process, I reached out to two contractors’ in Rochester, NY (where I was studying at the Eastman School) to see about securing a 40-piece orchestra to record Project Hiraeth. This was an intimidating process, for several reasons:

• The music I was writing wasn’t easy.
• I was committed to doing this right – if I’m recording live, it’s got to be good.
• I had no budget for any of Hiraeth, which meant every musician, audio engineer, and contractor had to be willing to work only for backend money.

The two contractors I worked with were Max Berlin and Grant O’Brien, and let me tell you, they absolutely made it happen. Max and Grant were able to find 40 of the best players in Rochester (a thriving arts community) to come in and record. Many of them were passionate gamers, who shared our passion for Hiraeth. Our final session instrumentation was as follows:

 

Because I was a graduate student at Eastman, I was able to book 10 hours of studio time with an Eastman engineer free of charge.

It didn’t take me a long time to realize that printing and copying music for 90 minutes of music in two weeks wasn’t something I could do alone. At this point, I hired composer Shoghi Hayes as my assistant on the project. Shoghi was invaluable in prepping for the session, and also acted as my ears in the booth.

The final recording session took place on November 17^th, from 3 – 10 PM. Ultimately, I settled on a pretty tight schedule for the session:

3:00 – 5:15    Strings

5:15 – 6:15    Dinner

6:15 – 7:30:   Brass

7:40 – 9:00:   Both flute soloists, together

9:00 – 10:00: Oboe/English Horn soloist

The violin, viola, and trumpet solos were recorded separately, the following week.

 

Figure 1: Prepping to Record Strings