Blog homepage

How Sound Designers Use PureData + Heavy to Develop DSP Plug-ins - Part 1

Sound Design / Wwise Tips & Tools

 

When it comes to the development of audio plug-ins, many sound designers think of it as a “black magic” that is exclusive to audio programmers because it usually requires coding skills as well as good knowledge in math, physics and digital signal processing, skills not absolutely essential for sound designers. 

When sound designers hear of DSP effects, not available in Wwise, from someone else’s projects, they are probably able to understand how they work very quickly, and create technical prototypes with their commonly-used tools to reproduce the same features. However, due to either performance limitations or the difficulty for the output of such tools to be deployed as plug-ins in the authoring tool or engine, most ideas stop at the prototype stage and fail to benefit real games.

Luckily, with PureData and Heavy, sound designers now have the opportunity to independently build proper effects or source plug-ins that can be used in the authoring tool and run in real games.

What is PureData?

image001

PureData (PD) is a visual programming language, developed by Miller Puckette in the 1990s, used to create interactive music and multimedia works. PD is an open-source project with a large developer base working on new extensions. And you can now even load VST plug-ins in it!

The file containing the PD code is called a Patch. Also, PD supports SubPatches. Users can create complex PD projects with modular units of code.

Although, this is not a PD starter tutorial. For those who want to know more, please go to https://puredata.info/.

 

A PD version of “Hello World” (For readers in China, click here to watch the video)

 

As you can see from the above video, PD is used in much the same way as a modular synthesizer: building a signal system by connecting different objects and interacting in real-time with it by adjusting parameters.

PD can be considered a “blueprint environment” for sound designers in which to create technical prototypes with their commonly-used tools, then generate plug-ins with a proper compiler.

image004Just like making interesting things with your building blocks

 

What is Heavy?

Heavy is a PD Patch compiler, developed by Enzien Audio, that can convert the PD Patches created by sound designers into C/C++ code. This allows sound designers to work in the PD environment to generate code that can run on the target platform without having to write them manually.

Heavy can be considered as an “automatic workshop” that can produce things with the “blueprint” created by sound designers.

image006

So, what can we do with the product?

What can we do with the plug-ins created using PD + Heavy?

1. Perform real-time DSP operations based on the sound designers’ design in the PD Patch;

2. Provide support for the input and output of audio. If the source Patch contains an adc~ object, it will generate an effect plug-in. If not, it will generate a source plug-in (such as Wwise SoundSeed Wind, Wwise SynthOne).

3. Provide support for the input and output of control parameters. It can either be controlled by the Game Parameters from the Wwise Sound Engine or send data to them, to be used as a LFO, for example, among other things.

4. Provide support for the input and output of UI events (Unity only).

Now that we know what sound designers can do with these tools, let’s try to produce something with them.

Sample Divider

 

image008Just like an audio version of rasterized effects

Sample Divider effects are usually included in a distortion plug-in like Bitcrusher. I’d like to reproduce such features with PD + Heavy.

Why we would want to attemp this:

1. Wwise doesn’t have a Sample Divider plug-in;

2. Sample Divider is actually very useful, especially when Cyberpunk becomes popular again (let’s say you want to get the similar malfunction effects that are frequently occurred in Nier: Automata and Titan Fall 2, how can you do it without Sample Divider?).

Worry-free DSP Processing

Before creating a “blueprint” using the Patch file, we need to understand how Sample Divider works.

Sample Divider resamples the input signal with a lower frequency than its sample rate, then distorts the original signal with lower output quality.

The following image illustrates how the Sample and Hold (S/H) module works (retrieved from The Theory and Technique of Electronic Music, by Miller Puckette).

image009Input Signal
Sample Clock
Sample Output

S/H Module

In the image shown above, the sine wave signal is sent to the IN point of the S/H module, while the sawtooth wave signal is sent to the OUT point to trigger the Sample & Hold operations. When the sawtooth wave values drops down to 0, it will trigger the S/H object to resample the input signal while keeping the sampled values unchanged for the remaining duration. In this way, the output signal is more similar to the input signal when the sawtooth wave frequency comes close to the sample rate of the input signal. Otherwise, there will be more distortion.

Here is a demo video that shows how the S/H module works in VCVRack (similar to a physical synthesizer).

Using the SCOPE to draw terraces (For readers in China, click here to watch the video)

Creating a Blueprint

Now that we know how the S/H module works, let’s create a “blueprint” using the Patch file.

In order to compile the Patch file properly with Heavy, we can only use supported PD objects, otherwise there will be compilation errors. PD objects not supported by Heavy will be listed in Part 2.

To expose parameters to the target platform (that would be Wwise in this case), we need to declare them in the following manner:

  • In PD, press Ctrl+1 to create an object, then type “r SampleRateDivisionL @hv_param 4 24000 24000”.
  • The "r" indicates that the object receives information from outside PD.
  • SampleRateDivisionL will display parameter names in the plug-in UI.
  • "@hv_param" indicates that the object will be compiled into an interface parameter by Heavy.
  • 4, 24000 and 24000 are the minimum, maximum and default parameter values.

image014

r indicates that the object receives parameter values from the outside
(mandatory) @hv-param indicates that the input/output values are parameters, @hv_event indicates that they are events
SampleRateDivisionL will show parameter names in the plug-in UI within Wwise
Min Value
Max Value
Default Value

1. In order to process the input signal independently on the left and right channels and increase the asymmetry, I’m going to create a parameter for each channel.

  • phasor~ object is the sawtooth oscillator used as a sample clock;
  • samphold~ object is the S/H module itself;
  • dac~ object is the target where the audio signal will be output.

image015

Note: In the Patch, adc~ represents the input signal. If the source Patch contains an adc~ object, it will generate an effect plug-in; if not, it will generate a source plug-in.

2. Let’s see how the Patch code works.

 

Note: I added an hslider object to control the sample rate. It will be useless after you generate the plug-in, so please remember to remove it.

3. Save the Patch file to J:\HeavyAudioPlayground\bitcrusher.pd.

image019

We've now created a “blueprint”.  I will show you how to build an “automatic workshop” with the Heavy compiler, and then use it to generate a plug-in and deploy it in our game, in Part 2 of this blog. Stay tuned!

 

This blog has been translated from Chinese to English.
 

 

Chenzhong Hou (侯晨钟)

Senior Technical Sound Designer

Chenzhong Hou (侯晨钟)

Senior Technical Sound Designer

Chenzhong Hou is a sound designer, composer and music tech researcher. He used to work as a Product Expert for Greater China at Audiokinetic. Prior to that, he worked at Ubisoft Shanghai and was involved in the sound design of Just Dance and Far Cry series.

 @houchenzhong

Comments

michal Kulig

October 11, 2019 at 11:29 am

Amazing work! Does it mean you'll show us how to deploy in Unity and Android app bulds? That would be very useful !

Leave a Reply

Your email address will not be published.

More articles

Planet Coaster - Crowd Management : Using Data to Generate Dynamic Crowd Audio (PART 1)

Part 1. Scaling Ambition Part 2. The Crowd Soundbox System Part 3. Additional Layers Planet...

27.6.2017 - By Planet Coaster

12 Days of Interactive Foley Design Using Wwise and Unity

This could be both the dream and nightmare of a Foley artist: walking a character whose look,...

1.5.2019 - By Pierre-Marie Blind

Making Interactive Music in Real Life with Wwise

I decided to dive deeper into interactive audio some time ago. I wanted to create a project related...

25.2.2021 - By Ressa Schwarzwald

Building a Car Racing System Using Wwise

Growing up, I was very keen on car racing games; from playing Namco’s R4: Ridge Racer Type 4 on PS1...

10.9.2021 - By Johnson Jiang (江山)

Teaching Wwise to the Blind

Teaching game audio to distance learners isn’t easy. Teaching game audio to distance learners...

14.10.2021 - By Eric Berzins

Star Wars Jedi: Survivor | Q&A With the Audio Team

Star Wars Jedi: Survivor is a third-person, galaxy-spanning, action-adventure game from Respawn...

2.5.2024 - By Respawn Entertainment

More articles

Planet Coaster - Crowd Management : Using Data to Generate Dynamic Crowd Audio (PART 1)

Part 1. Scaling Ambition Part 2. The Crowd Soundbox System Part 3. Additional Layers Planet...

12 Days of Interactive Foley Design Using Wwise and Unity

This could be both the dream and nightmare of a Foley artist: walking a character whose look,...

Making Interactive Music in Real Life with Wwise

I decided to dive deeper into interactive audio some time ago. I wanted to create a project related...