|
Wwise SDK 2022.1.17
|
 | Warning: Object Processors are supersets of Effect Plug-Ins in that they can implement everything an Effect Plug-In can and offer additional functionality. However, they are more complicated to write. If you don't need to process multiple Audio Objects at once, and don't need to know about the object that is processed but only its audio signal, then you should write an Effect Plug-In instead. See Implementing an Effect Plug-in Interface. |
Introduction
Object Processors are similar to Effect Plug-Ins (see Implementing an Effect Plug-in Interface) in that they are inserted in one of the slots of the Effect tab of any Wwise Object. The difference with Effect Plug-Ins is apparent when used with Audio Objects. In essence, Effect Plug-Ins are agnostic with respect to Audio Objects (AkAudioObject) and only process audio signals, while Object Processors are aware of all Audio Objects passing through a bus, process them all at once and can access their metadata.
While effect plug-ins receive a single AkAudioBuffer, Object Processors receive an instance of AkAudioObjects that provides:
- An array of AkAudioBuffer instances representing the Audio Objects' signal.
- An array of AkAudioObject instances representing the Audio Objects' metadata.
These concepts are explained in the next section.
Audio Objects: Concepts
An Audio Object contains an audio signal, which may be mono or multi-channel. In Wwise, Audio Objects are most easily observed when a bus's configuration is set to Audio Objects. These busses are referred to as Audio Object busses. Instead of mixing its inputs into a single buffer, multi-channel or not, an Audio Object bus gathers Audio Objects and retains their metadata. An Audio Object bus supports a dynamic number of Audio Objects, while a non-Audio Object bus may be seen as a bus that only supports a single Audio Object at any one time.
Audio Objects' metadata essentially consists of positioning information, as well as an array of custom metadata, which are themselves plug-ins and may be used by Object Processor plug-ins. Refer to Audio Object Metadata for more details.
Implementing an Object Processor
When mounted on an Audio Object bus, an Effect Plug-In will be instantiated as many times as there are Audio Objects, and the lifetime of each instance corresponds to the lifetime of the Audio Object to which it is assigned. Indeed, a single instance cannot be reused to process each Audio Object in turn, because the Effect may be maintaining a state that is required to guarantee continuity of the audio with the following frame.
On the other hand, an Object Processor is instantiated once, regardless of the number of Audio Objects, and processes all these objects at once.
Object Processors implement AK::IAkInPlaceObjectPlugin or AK::IAkOutOfPlaceObjectPlugin, depending on whether they are processing in-place or out-of-place. Plug-ins declare that they are Object Processors by returning AkPluginInfo::bCanProcessObjects set to true from AK::IAkPlugin::GetPluginInfo. They declare that they are in-place or out-of-place by setting AkPluginInfo::bIsInPlace accordingly.
| Warning: Out-of-place Object Processors are only supported on busses. As is the case with Effect Plug-Ins on busses, they cannot change the rate at which they consume and produce data (AkPluginInfo::bCanChangeRate cannot be true). They can however change the output objects and their channel configuration. |
Only the functions specific to these interfaces are covered here. Refer to Creating Sound Engine Plug-ins, for information about interface components shared with other plug-in types (AK::IAkPlugin interface), and Implementing an Effect Plug-in Interface for information about the functionality shared with Effect Plug-Ins, such as initialization, bypassing, and monitoring.
Implementing an In-Place Object Processor
In-place Object Processors receive an array of Audio Objects' buffers and metadata (AkAudioBuffer and AkAudioObject, respectively) in AK::IAkInPlaceObjectPlugin::Execute. They are able to read and modify the audio signal and metadata of all Audio Objects, but they cannot create or remove Audio Objects, or change their channel configuration.
Note that when in-place Object Processors are inserted as Effects on objects in the Actor-Mixer or Interactive Music Hierarchies, as opposed to on busses, Object Metadata will be invalid during Execute, and any modifications to the Metadata will not be preserved. An Object Processor can check for this by determining if Object Keys are equal to AK_INVALID_AUDIO_OBJECT_ID or not, to either gracefully handle this scenario, or flag an error.
The Compressor is an example of an in-place Object Processor. It needs to be an Object Processor, because its algorithm depends on knowing the audio signal of all Audio Objects at once, but it does not modify the list of Audio Objects. It simply modifies the audio signal of the objects that pass through.
The Compressor is of course capable of processing a single object, that is, it works on traditional channel-based busses. It thus supersedes its old Effect Plug-In realization.
Handling Tails in In-Place Object Processors
Like Effect plug-ins, in-place Object Processors can handle tails by changing the AkAudioBuffer::eState field from AK_NoMoreData to AK_DataReady for the desired number of frames. Refer to Implementing Effect Plug-in Tails for more details. It is important to note however that Object Processors need to handle tails of all their Audio Objects independently. They would thus need to keep track of each object.
 | Caution: Do not store the addresses of audio objects passed to Execute. They can change from frame to frame. The preferred way of identifying an object is to use the AkAudioObject::key field. |
Implementing an Out-of-Place Object Processor
Out-of-place Object Processors manage distinct sets of Audio Objects at their input and output. The input Audio Objects depend on the host bus, while the output Audio Objects are created explicitly by the plug-in, using one of two methods discussed in the following sections. All these objects are passed to the plug-in at every frame via AK::IAkOutOfPlaceObjectPlugin::Execute.
Out-of-Place Object Processors and Bus Configuration
The channel configuration of the output Audio Objects is decided by the plug-in.
Additionally, a non-Audio Object (or "single-object") bus is essentially a special case of the general Audio Object bus, but with only one Audio Object. Since an Object Processor can receive and produce any number of Audio Objects indifferently, it is thus capable of making a single-object bus output a dynamic number of Audio Objects, and vice-versa, making an Audio Object bus output a single Audio Object as if it were a traditional mixing bus.
 | Note: When inserted on a non-Audio Object bus, an Object Processor receives the actual channel configuration of that bus in AK::IAkEffectPlugin::Init. Since Object Processors are supersets of Effect Plug-Ins, you should strive to make yours work seamlessly, whether it is inserted on an Audio Object or non-Audio Object bus. That is, unless it is a user error to insert it on a non-Audio Object bus. For example, it does not make much sense for users to insert the Software Binauralizer plug-in on a non-Audio Object bus, because the downmixed audio would not carry any useful positioning information. In this case, it is better to let your users know that they probably made a mistake. |
Handling Tails and Object Lifetime in Out-of-Place Object Processors
The sound engine resets all input and output Audio Objects' AkAudioBuffer::eState to AK_NoMoreData at the beginning of every frame. Any Audio Object whose AkAudioBuffer::eState remains set to AK_NoMoreData after processing is destroyed. Object Processors are not destroyed until all input and output objects have been destroyed. Thus, in order for an out-of-place Object Processor to continue producing audio when it no longer has any input Audio Objects, it simply needs to keep one or multiple output Audio Objects alive by setting their state to AK_DataReady.
| Caution: Use caution if you keep track of objects within your Object Processor. Make sure not to refer to an object after it has been destroyed by the sound engine. |
| Note: Input objects will be kept alive if you set their AkAudioBuffer::eState to AK_DataReady, but this should be avoided since it is not necessary to produce tails. |
| Note: An out-of-place Object Processor that outputs zero Audio Object outputs silence. |
Out-of-Place Object Processing Examples
Let's explore out-of-place object processing with the three following canonical examples.
Software Binauralizer
A software binauralizer may be implemented as an out-of-place Object Processor that takes in multiple Audio Objects, and outputs a single Audio Object with a stereo channel configuration. Such a plug-in should be mounted on an Audio Object bus, but it will effectively make this bus output a single stereo signal.
A convenient way to create the one and only stereo output object is via the handshaking method of AK::IAkEffectPlugin::Init.
Then, in Execute:
3D Panner
A 3D Panner can be implemented such that it works like the Software Binauralizer described above. However, one elegant way to implement it is by having it instantiate a set of output Audio Objects that each correspond to a spatialized virtual microphone. By doing so, these Audio Objects' signals will be panned by a bus or device downstream, which will make the best use out of the positioning metadata of these virtual microphones.
In AK::IAkEffectPlugin::Init, instead of returning a non-object configuration, create output objects explicitly using AK::IAkEffectPluginContext::CreateOutputObjects.
In the above example, you may wonder why (-0.707f, 0.f, 0.707f) represents the front left. See On 3D Transformations for details.
Particle Generator
For each of its input Audio Objects, a Particle Generator would create N output Audio Objects randomly positioned around the position of their corresponding object. This type of Object Processor cannot create objects in Init, it needs to create them dynamically in Execute, keep track of them and keep track of the input objects. When an input object's state is AK_NoMoreData, the state of the corresponding output objects should be set to AK_NoMoreData as well. This ensures that they get garbage collected by the sound engine.
| Caution: If an out-of-place object processor calls AK::IAkEffectPluginContext::CreateOutputObjects from within Execute, it cannot reliably access the output objects passed in out_objects. In that case it must use AK::IAkEffectPluginContext::GetOutputObjects. |
Assigning a Name to an Audio Object
In the authoring tool's Audio Object Profiler, Audio Objects are given the name of the Wwise Objects from which they were instigated. Output objects of an out-of-place Object Processor are therefore all named with the name of the hosting bus. In order to make profiling easier, it is recommended to use AkAudioObject::SetName in order to provide a name to output objects, when it makes sense to do so.
For example, when creating objects in the 3D Panner above, we could have named them like this:
Audio Object Metadata
The AkAudioObject struct encompasses all the Audio Object metadata that travels along Audio Objects' audio buffers throughout the object pipeline. It can be split in three categories:
- Identification: AkAudioObject::key, AkAudioObject::instigatorID and AkAudioObject::objectName. They must never be written to by an Object Processor, except objectName which out-of-place Object Processors may set using AkAudioObject::SetName.
- Positioning information: AkAudioObject::positioning, discussed below in Positioning Metadata.
- Cumulative Gain: AkAudioObject::cumulativeGain, discussed below in Cumulative Gain Metadata.
- Custom metadata plug-ins: AkAudioObject::arCustomMetadata, discussed below in Custom Metadata Plug-Ins.
Positioning Metadata
An Audio Object carries in AkAudioObject::positioning the positioning data of the sound from which it was instigated. AkAudioObject::positioning.behavioral holds all the relevant positioning settings that can be found on Wwise Objects. For example, if the sound uses speaker panning, AkAudioObject::positioning.behavioral.panType will be set to one of the panner types, and panLR, panBF and panDU will correspond to the position of the panner.
If the spatialization mode is 3D (AK_SpatializationMode_PositionOnly or AK_SpatializationMode_PositionAndOrientation), AkAudioObject::positioning.threeD contains all the data that relates to the 3D position:
AkAudioObject::positioning.threeD.xformwill typically inherit the position of the associated game object, although it can be modified or overriden for each sound depending on its 3D positioning settings, such as 3D automation.AkAudioObject::positioning.threeD.spreadandAkAudioObject::positioning.threeD.focusare usually calculated from the Attenuation Curves.
Cumulative Gain Metadata
An Audio Object carries with it the cumulative gain that has been applied upstream, such as the setting of a source's Volume, or changes in gain from busses and connections between busses. In a simple scenario, with no Effects or Object Processors, this means the gain of an Audio Object isn't applied to its audio signal until it's finally mixed down into a speaker bed, or sent to the system output as an Audio Object. This results in an audio mix that changes smoothly by avoiding ramping gains to an Audio Object's audio signal multiple times in a frame, especially when Audio Objects are created and destroyed due to Game Objects adding or removing positions.
Support for this metadata is optional for Object Processors and enabled by setting IAkPluginInfo::bUsesGainAttribute to true in the Object Processor's implementation of IAkPlugin::GetPluginInfo. If bUsesGainAttribute is left to false, then all audio buffers passed to Execute will have the cumulative gain of the Audio Object applied before execution, and the gain passed to the Object Processor will be neutral. However, if bUsesGainAttribute is set to true, then the audio buffers will not be modified, and the cumulative gain may be a non-unit value. This allows the Object Processor to acknowledge the gain as needed, and also modify the cumulative gain of the Audio Object as desired.
If the cumulative gain of the Audio Object is to be modified, take note that the value is an AkRamp, and continuity of the fNext value on one frame to the fPrev value on the next frame is not automatically handled by the sound engine. That is, if an Object Processor intends to modify fNext on one frame, then the same modification must be applied to fPrev on the next frame. If this is not managed properly, there may be audio glitches, or discontinuities in the audio signal, when some other part of the audio pipeline has to consume the Audio Object's gain.
On 3D Transformations
If the spatialization mode AkAudioObject::positioning.behavioral.spatMode is 3D (AK_SpatializationMode_PositionOnly or AK_SpatializationMode_PositionAndOrientation), the 3D position is transformed (translated and rotated) such that it is relative to the game object (listener) associated to the bus in which it travels. For example, a sound may be positioned at (2, 0, 0); when it is processed by an Audio Object bus associated to a listener at (10, 0, 0), the resulting Audio Object will be positioned at (-8, 0, 0). Object Processors on this bus will "see" the Audio Object as being located at (-8, 0, 0).
The coordinate system in Wwise is left-handed and the default orientation's front vector points towards Z and the top vector points towards Y, as defined in AkCommonDefs.h.
With the Software Binauralizer, for example, input objects have been rotated prior to reaching the plug-in, such that positive Z is to the front of the listener, positive X is to its right, and so on. This is also why the service AK::IAkMixerPluginContext::ComputePositioning used in that example does not take the listener's orientation: it assumes default orientation.
In the 3D Panner example above, (-0.707, 0, 0.707) thus points at 45 degrees towards the front left of the game object associated to the hosting bus.
Custom Metadata Plug-Ins
Object Processors are able to access custom metadata attached to Audio Objects.
Custom metadata is a type of plug-in that only consists of a set of parameters. In the authoring, metadata plug-ins can be added on any Wwise Object, and they support ShareSets. In the sound engine, they implement the AK::IAkPluginParam interface.
At every frame, Audio Objects gather all the metadata plug-ins attached to the Sound from which they were instigated, if applicable, and adds them to their AkAudioObject::arCustomMetadata array. Then, they gather all the metadata plug-ins attached to every bus they visit. Object Processors, in-place or out-of-place, can read this array. Of course, they need to know about the plug-in in order to interpret its content.
The implementor of an Object Processor can write one or multiple companion metadata plugins that users can add to Wwise Objects.
For example, imagine that in the Software Binauralizer described previously (see Software Binauralizer), you want to support a passthrough mode, so that certain sounds don't undergo HRTF filtering. You would create a companion metadata plug-in ObjectBinauralizerMetadata with a boolean property called Passthrough. Your users would be able to add this plug-in to any Wwise Object they want to opt out of HRTF. Then, in your Object Processor's Execute():
| Note: Since Audio Objects gather metadata plug-ins from each visited Wwise Object, it is possible that you find several instances of the same plug-in type on a single Audio Object. It is up to you to determine what the policy is when that happens, and inform your users. |
Was this page helpful?
Need Support?
Questions? Problems? Need more info? Contact us, and we can help!
Visit our Support pageTell us about your project. We're here to help.
Register your project and we'll help you get started with no strings attached!
Get started with Wwise