About Wwise 2022.1 Unreal Integration

This version is a major milestone in the Unreal Integration for Wwise. Much like Unreal Engine 5 bringing the engine to a new realm, Audiokinetic’s Wwise Integration for Unreal pushes the boundaries of what is possible with both tools working in harmony.

The changes span more than one year of hard development work, as well as many discussions with developers working with Wwise and Unreal. Many lessons have been learned during the incremental optimizations and feature inclusions throughout development over the past few years. Our ambitious goal is to support every possible workflow, where every game's requirements are met and work without customization. This release marks tremendous steps towards that goal as you will see in the following sections. We are very thankful to Wwise users for making their joys, hurdles, and frustrations known to us. The voice of the community has helped drive priorities and grow the integration towards the needs of the many.  

Changes Overview

Priorities

• Stability
• Extensibility and Code legibility
• Simplified Workflow
• Wwise Authoring Is Our Design Tool
• Testable

Foundational to the Wwise Integrations is the idea of exposing the Sound Engine as a native part of the development engine. Our code is now pivotal to an efficient development, packaging, and playback of audio assets for multiple products, that are constantly pushing forward with extraordinary ideas. Since our first Wwise Unreal Integration release, the priorities have shifted around due to new technologies becoming available, and the developed projects scaling to an incredible size. From a feature requirement, such as VR being possible, to a project complexity paradigm shift, such as migrating a Legacy workflow towards Event-Based Packaging, each Wwise release improves on its predecessor in large strides. For Wwise 2022.1, we took time to reflect on what is truly important to each of our users at this point in time, and how we can improve the state of Wwise Integrations, past the addition of compatibility and features. This priority introspection led to multiple important realizations on our current code base.

Stability: External Sources and Event-Based Packaging

We started the process of improving our integrations by disabling complex workflows that are not always important to every developer. Many of these caused major stability issues and some also could conflict with other equally important developer workflows.

A perfect example is the External Sources system. In the past years, we have tried to implement one system that would work for every product and attempted to add all of the possible ways to connect one or multiple Media files to one or multiple External Sources. We also allowed those assets to have a controlled lifetime in addition to handling the packaging of such assets as part of the system.

The reality is that there are as many uses of External Sources as there are products created by Wwise users. What we have found is that it is, in fact, a custom system that will be handled differently depending on the developer and project needs. While some developers are fine with the system that has been provided, many thought it included unnecessary complexity and made bugs difficult to track in their implementations. To counterbalance this, many developers have taken to actively modifying our compulsory piece of code to better suit their special cases.

Another example of scaling complexity has been Event-Based Packaging. This system has grown since its release in 2019.2 to a point where we felt that we were in a fight with the Unreal Engine and Wwise Authoring in order to arrive at the current iteration. While we have gained stability with the feature throughout its development over the course of 2019.2 and 2021.1, the system is more complex than intended. While most of the developers who adopted Event-Based Packaging in their workflows have seen success and are happy with the implementation, some teams’ workflows can be exacerbated by unyielding aspects of the system. The hard fact about Event-Based Packaging, as introduced in Wwise 2019.2, is that we learned through experience that there are as many ways to approach asset management as there are games being developed. Our intention has always been to support every project that leverages Wwise by applying simple, well-written code. However, Event-Based Packaging has evolved with complex code making it impossible to test every aspect of it across all platforms. Even worse, because Unreal Engine’s possibilities are truly endless, varied, and complex on their own, the freedom we built into the system made it possible to create a scenario where a  particular group of actions could make the entire system crumble. This complexity also meant that developers trying to use it are sometimes left with no other choice than to ask Audiokinetic Support when they encounter issues.

However, we continue to believe that the promise of “One bank per event” is truly a game changer for asset management and is a necessary step for developing a modern high-complexity product. It became clear that we could not solve all of the issues raised by the introduction of Event-Based Packaging and so, for Wwise 2022.1 we have taken special care to provide a solution that will support the majority of workflows while supporting developers with the ability to migrate and extend their asset management solutions. 

The results:

External Sources are now optional. Although we provide a basic implementation of External Sources as part of our Integrations Demo, developers are now free to implement their own version of External Sources in a way that suits their needs. This includes: discoverability, packaged information, media discovery, and packaging.

The Legacy workflow and the Event-Based Packaging workflow are merged together. Based on developer feedback and our own experience, we merged the best parts of both workflows, as well as the more modern and concise Wwise Unity Addressables package workflow. Instead of having to support multiple clashing systems, we now have only one extensible system.

Extensibility and Code Legibility

Historically, the Wwise Unreal Integration started its life as an example of how to plug the Wwise Sound Engine into an Unreal project. Then, the Unreal Integration started providing a default workflow (what is referred to as the Legacy Workflow in Wwise 2019.2 and Wwise 2021.1) in response to Unreal Engine’s limitations when packaging files alongside a game. 

Until Wwise 2019.2, the Integration code could be defined as four different parts: 

• Wwise Sound Engine
• Unreal Engine
• Wwise Unreal Integration
• Game Code 

Whenever the Wwise Unreal Integration code didn’t fit with a game’s integration or expected behavior, the answer was: “You can fix it yourself; the provided code is only an example.” While there used to be small pockets of hidden complexity, the code was still considered relatively simple to understand.

Starting in Wwise 2019.2 with Event-Based Packaging, the Wwise Unreal Integration code started to greatly expand in complexity. Things like: processes running in Unreal Editor, connecting over the network to Wwise Authoring, multiple workflows, workflow complexity, memory management, asset management and handling, asset copying, and extensive migration steps escalated the amount of work necessary to “simply” integrate Wwise with Unreal. At that point, the Wwise Unreal Integration could not realistically be considered a simple bridge and a mere example.

In hindsight, we can see that this led to a fundamental shift in the mentality of Wwise users. Instead of presenting a simple solution and having developers work through their unique pipeline-specific issues by themselves, they started asking for Support immediately when beginning their Wwise Integration with Unreal. During this time we worked with many talented developers to support their workflows due to the increased complexity. Internally, we began to see the challenges that had been delivered in an attempt to work within the bounds of what existed at the time.

Instead of relying on the four established pillars, we needed to add a fifth one: User Extensions to the Wwise Unreal Integration. Not truly a part of the Game Code itself, but more about a way to enable motivated teams to modify and extend the way the Wwise Unreal Integration works at a very low level without changing a single line of code within it. For this to be possible, we also needed to focus on code legibility, beginning with the Unreal Coding Standards, but also to create clear and concise building blocks while splitting things into logical parts. Finally, we started looking at our code through a different set of lenses: how can we explain any given element in documentation? Not only should the code be self-explanatory, but the code should be described simply.

Although changing a decade of code used by thousands of external projects is not something that’s to be easily tackled, at that point, it became necessary. So with Wwise 2022.1, we are starting the revitalization process with very few user-facing code changes. The assets definition is mostly changed internally. A few functions have been removed in what is currently called the Audio Device, which had become a list of half the operations accessible to developers in the past years. From there, we started writing the low-level code in user-modifiable modules that provide accessibility for change and future extensibility. 

With these humble first steps, here is a summary of the functionality provided across the different areas of the Wwise Unreal Integration:

There are now multiple low-level modules accessible to the developers, each extensible (from the lowest level):

• Wwise Sound Engine: Low-level access to most SoundEngine API operations. It used to be impossible to access code outside AkAudio, as the API needed to be accessed in the same dynamic library. With this module, it’s now possible to access it directly and securely.
• File Handler: Run-time file bridging between Unreal Engine and the Wwise Sound Engine. Tries to keep both parties happy. Includes the Wwise streaming I/O Hook, SoundBank loading manager, and the Media loading manager. Also includes the External Sources manager stub that needs to be implemented by a game wishing to use External Sources.
• Resource Loader: For every Wwise asset type, there is now a Load Data structure that contains what is required for that particular asset to load. When requested by a higher-level component, the Resource Loader module enumerates what is required to the File Handling module, and back on unloading. Likewise, it handles the optional Switch Container optimizations.
• Project Database: Handles loading a Wwise project’s information for use in the Unreal Editor. Contains a list of every single available data point from the Wwise project, such as Events, a list of all the Aux Busses, Switches, and States.
• Resource Cooker: Retrieves asset information to create the aforementioned Load Data structures, and stages the Wwise Generated SoundBanks binary files during packaging, so they are available to the Resource Loader and File Handler.

These modules are added alongside the existing AkAudio and AudiokineticTools modules inside the Wwise Plug-in.

Our integrations’ code has now gone through multiple approaches towards empowering developers to make comprehensive decisions when integrating Wwise with game engines. From the initial examples to code that was infinitely user-modifiable, then to a system limited to a single workflow, in addition to enabling the many user-requested workflows requested by the user community.

Simplified Workflow: SSOT and Packaging

For Wwise 2022.1’s Unreal Integration, we want to provide developers and Sound Designers with a clear path towards a well-defined integration so they can be efficient in their development process. By being stable, extensible, and having more legible code, we hope to empower developers to push the boundaries as they see fit for their unique pipelines and workflows without overcomplicating the choices.

Wwise Unreal Integration Background

In Wwise 2019.2 and 2021.1, the Wwise assets were located in different places. First, in the Wwise project itself, but then, duplicated as Unreal Assets in a protected Content folder. Changes between Wwise and Unreal are also synchronized using the Wwise Authoring API (WAAPI) or parsed from Wwise Work Units, creating a dependency on accessing Wwise project files, as not all of the information was accessible elsewhere.

This multiplication of diverging sources can cause issues, especially when changes appear while the Unreal Editor is closed. The prioritization and synchronization of those assets at load time add unacceptable slowdowns and uncertainty during production and while editing; the simple act of renaming one root folder in Wwise can issue thousands of renaming operations across engines, each costing a few seconds.

In practice, many appreciate the convenience of this methodology for smaller projects; however, for larger projects, it is a high price to pay for “simplicity”.

When cooking those assets to a final product, the first iteration of the Wwise Unreal Integration required users to add a folder to always cook that would contain all of the SoundBank and Media files. Unfortunately, that folder also contained supplemental metadata files that would increase the size of the game, as well as potentially make it easier to hack by malicious users.

Since Wwise 2019.2, unless using the Legacy workflow, those Wwise assets would be bundled in mandatory Unreal assets as binary bulk data. Even if this method is functional, the principle requires a 1:1 Unreal asset to Media or Bank assets. We are also at the mercy of Unreal and its asset lifetime cycle, which is unforgiving.

Single Source of Truth (SSOT)

Wwise assets and project databases are in one place: The Generated SoundBanks folder. This is now our Single Source Of Truth (SSOT). There are generated binary files, as well as generated JSON files containing all the necessary information to understand the Wwise project and its output. The Unreal assets are now optional, and only contain identifying data to know which Wwise assets are being referenced.

Wwise Picker is now linked to the Generated SoundBanks folder, not the Wwise Project. What is played in the Unreal Editor, in the Legacy Workflow, used to be located in the Generated SoundBanks folder or within UObjects as part of the Event-Based Packaging workflow. What was shown through the Wwise Picker under these circumstances was yet another source of half-truths: what was previously saved in the Wwise project. When retrieving information from Source Control, chances are good that it is equivalent to what is in the Generated SoundBanks folder, but there’s been no mechanism to confirm this assumption. Now, the Wwise project is completely ignored by the integration, only the Generated SoundBanks folder is used for the Wwise Picker.

WAAPI Picker is now only a viewport to the Wwise project. The assets and media generated by Wwise to the Generated SoundBanks folder now represent the “Truth” for a project. These assets are what will get played and shown as part of the Unreal integration. However, for convenience, it’s still possible to use the WAAPI Picker in the Unreal Editor to handle operations to and from the Wwise project in Authoring. While this is never the Truth, it is a convenience. To promote a Wwise Authoring change to the editor, it must be generated.

Sound Designers know about Wwise, not every single developer must know about it. While the convenience of having a full Wwise Authoring installation, as well as the entire Wwise project, accessible at all times is nice, not every developer on a project needs to know about or install Wwise. As such, our Wwise 2022.1 workflow is optimized so that anyone can develop as part of an Unreal project that includes Wwise without having either Wwise Authoring or the full Wwise project for the game (including packaging for any given platform). Now, only the Wwise Unreal Integration (with its Sound Engine SDK counterpart), and the Generated SoundBanks folder, which can be conveniently moved, are required. This also means that the Generated SoundBanks folder doesn’t have to reside in the Content folder of the Unreal project anymore, and can be moved at leisure.

Wwise binary assets are tree-shaken and only the necessary files are copied to the packaged project. For Wwise 2022.1, we are returning to the Legacy system, where assets are copied as loose files for runtime consumption. This system is the best as we are not bound to Unreal assets’ lifetime. However, we also learned from the Event-Based Packaging system by staging the used files depending on their Unreal Assets usage. The plug-in uses the Ak Components to determine what is being used for the final staging, making it seamless for products using regular Unreal assets. Finally, since it’s now possible to override those operations, a developer can handle those files differently if needed, including the ability to have special operations to load files differently or have multiple brews of the files, under their own control, without rewriting the entire Integration code.

Wwise Authoring Is Our Design Tool

For the past years, the Unreal integration proposed a workflow that basically bypassed the SoundBank Generation inside of Wwise Authoring. There are still today multiple good reasons why this method may be preferable for a project. While Authoring was initially developed to provide files through a folder containing monolithic SoundBanks, the introduction of faster storage access and migration from disk drives to solid-state now supports better randomized access to media in general. While providing project information through SoundBanks is still the most efficient way to handle a complex Wwise project, the industry in general has started requesting independent on-demand assets as part of its pipelines. Our first foray into this world was with Event-Based Packaging and further separated Authoring from the users of Unreal.

However, the concept and ideal of having a separate tool specialized for Sound Designers remain valid for efficiency, but also simplicity, since it allows a game’s audio to be developed fully in parallel with the game itself, much like a 3D artist would use a separate tool to draw and maintain their 3D assets. The benefits of this concept had been eroded across multiple versions and required rethinking and reprioritization.

With Wwise 2022.1, we now provide Auto Defined SoundBanks directly inside Wwise Authoring. This option allows some or all of the Wwise Events to be defined in their own independent SoundBank. This effectively brings the power of Event-Based Packaging to any custom Integration, as well as Unreal, and simplifies the granular management of Event-based Soundbanks and their media for any game engine.

This fundamental change caused a regression in our Unreal workflow, essentially removing the previous workflows and returning to the concept that Sound Designers use Wwise Authoring to manage the SoundBanks.

We are now targeting two previously-forgotten developer types:

• A developer not using Wwise at all (Game Play, AI, GFX, 3D);
• A Sound Designer using Wwise exclusively, and doesn’t need Unreal Editor at all.

This returns the responsibility to Sound Designers to manage asset sizes and memory usage within Wwise, which was missing from previous versions of the Wwise Unreal Integration. Since the SoundBank generation information panel is now accessible, it’s much easier to follow up with a game’s resource requirements, especially for smaller platforms.

The last remaining workflow is focused on Technical Sound Designers, who happily embraced Event-Based Packaging for its successes, and works interchangeably between both the Unreal Editor and Wwise Authoring during development. For folks splitting their time between Wwise and Unreal, the new workflow will bring two things: speed and consistency. We’ve removed the need to constantly keep things in sync between Wwise and Unreal and now know exactly when synchronization is required. This makes it possible to work without worrying about causing an operation that requires hours of delay (such as the aforementioned root folder renaming).

No workflow difference between developing audio for Unreal and developing audio for other games or middleware. Now, a Sound Designer can concentrate on having the best audio experience for games directly in Wwise Authoring. There are no hidden features or special ways to develop or manage assets, just because a game is being developed with the Unreal Engine.

The Wwise Project folder is not necessary for working in Unreal Editor. Because SoundBanks can be generated directly in a separate folder, it’s now the only requirement for having audio in and bundled for a final game. As long as the SoundBanks are properly generated from Wwise, nothing more is required to package a game.

No more night-time Commandlet maintenance on an Unreal project. This used to be necessary for some teams since Unreal’s Asset changes were sometimes massive after changes made by Sound Designers. Now, SoundBank Generation can be done either: directly by the Sound Designer on their machine, or it can be automated on a server, where only the Generated SoundBanks folder will change. Likewise, there is now less of an incentive to commit the generated Cache folder of a Wwise project to source control for sharing across the team, since Sound Designers are now in control of the output of the Wwise project exclusively.

Testable

Both Wwise and Unreal Engine allow for a tremendous amount of customization. While it’s true that at Audiokinetic, we have a dedicated, experienced, and well-versed Quality Assurance team to make sure our product is of the highest quality, there is no way to ensure that every path that developers wish to take with Wwise has been tested. While most software may arrive with one thousand audio assets, there are a few with over a million. Added to this, some cannot play everything due to platform limitations, others include a dozen (or more) languages, and still more pushing the envelope on streaming media; another one has a single Event with ten thousand pieces of optional audio; another that dynamically streams maps and another one that uses that elusive Unreal feature no one else uses. Not to mention the awesome teams who extend and modify our code and Unreal’s code to better suit their very particular needs. Supporting a particular Unreal version is not even the norm, as bigger games often create their own custom Unreal Engine, the result of multiple versions being merged together with custom code by hand.

While we are now trying to focus on a tried-and-tested middle ground, we know a team downloading the Wwise Unreal Integration for their unique production may encounter edge-cases we could never imagine, much less test against.

This first version of Wwise 2022.1 Unreal Integration is simultaneously a step backward and a huge leap forward. While we all know early optimization is the enemy of proper code, we are keeping sane optimization levels that will make most games happy: everything that has been burdened in the past by complex code paths has been simplified. Logs have been added, code quality has been improved.

Tools and Frameworks

We also started developing basic tools and frameworks to make the entire Wwise Unreal integration more testable. While not everything will be available immediately, our commitment to a fully testable Wwise Unreal Integration will continue to stretch out for multiple years. The first step in this direction is the development of Gyms. 

This external project has multiple goals:

• A product that contains a Wwise project, an Unreal project, and a Unity project.
• Has multiple small maps explaining a single concept for each map, easily and efficiently. Examples our users can follow.
• For each small map, there are linked pieces of code to unit test the integration, both positively and negatively, alongside edge cases. All of this by using the middleware’s preferred unit testing mechanism.
• Accessible to external developers. So that customized flows can be tested by developers and repro cases can be shared as a map for further iteration.

Our plan is for Gyms to be accessible through a third-party repository, and will be maintained starting with Wwise 2022.1.

There are many other scheduled improvements on testing, including the lower-level module’s extensibility design itself. We hope to make these unit test modules available beginning with the next major version so that developers can test out their own engine modifications using easily understood mechanisms.

Conclusion

Wwise is not the same as it used to be ten years ago. Unreal Engine also migrated by leaps and bounds during those past years. Keeping up doesn’t mean merely keeping compatibility and adding up some features and functionalities. It also means having a product enjoyed by the developers using it. For many years, the teams using the Wwise Integrations asked us to empower them in creating the best final product in performance, stability, and features.

In this first article, we covered the basics of our mentality shift, as well as the heavy requirements that shift entailed. By asking “what are our priorities, and how can we keep true to these priorities?”, we started a process that changed the way we see the Integrations inside Audiokinetic in general. That meant both adapting Wwise to Unreal and rebuilding a strong backbone from scratch.

Today, with Wwise Integrations 2022.1, we are laying the foundations of that new, modern system that will make it easier for both developers and sound designers to create their next success story. This stable foundation is modern enough to tackle many of the future challenges our users will face while keeping a lot of compatibility with previous products.