Wwise Unreal Integration Documentation

The Wwise Audio Input Plug-in

The Unreal integration provides a way of providing audio input to Wwise, via the Wwise Audio Input plug-in. See Audio Input Source Plug-in from the Wwise SDK documentation. In order to provide audio input to Wwise, classes should inherit from AkAudioInputComponent.

AkAudioInputComponent

AkAudioInputComponent is derived from AkComponent. It is a specialized AkComponent that can be used to provide audio input to Wwise. There are two key functions that need to be implemented:

/* The audio callback. This will be called continuously by the Wwise sound engine, 
*   and is used to provide the sound engine with audio samples. */

virtual bool FillSamplesBuffer(uint32 NumChannels, uint32 NumSamples, float** BufferToFill);
/* This callback is used to provide the Wwise sound engine with the required audio format. */

virtual void GetChannelConfig(AkAudioFormat& AudioFormat);

It also has one Blueprint function, Post Associated Audio Input Event, which posts the component's AkAudioEvent to Wwise along with associated AudioSamples callback and AudioFormat callback, using this component as the game object source.

Custom Audio Input behavior

Custom classes can be written that derive from AkAudioInputComponent in order to implement custom audio input behavior. The UAkVoiceInputComponent.h and UAkVoiceInputComponent.cpp examples below show a class that takes microphone input and pipes it to the Wwise sound engine.

In order to use these files in a C++ Unreal project, there is some initial setup to do. Firstly, both the AkAudio and the Unreal Voice modules must be linked, by adding them to the PublicDependencyModuleNames in the project's Build.cs file, for example:

public class MyModule : ModuleRules
{
    public MyModule(ReadOnlyTargetRules Target) : base(Target)
    {
        PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore", "AkAudio", "Voice" });
        
        // Other settings
    }
}

Next, the bEnabled flag in the Voice module must be set to true, by adding the following lines to the DefaultEngine.ini file:

[Voice]

bEnabled=true

With this initial setup complete, the following class, which demonstrates custom audio input behavior by taking microphone input and sending it to Wwise, can be added.

Note that this code is only used here to provide a quick simple example. It is NOT intended for use in shipped games!

AkVoiceInputComponent.h:

#pragma once
 
#include "CoreMinimal.h"
#include "AkAudioInputComponent.h"
#include "Voice.h"
#include "AkVoiceInputComponent.generated.h"
 
/*

 
*/
UCLASS(ClassGroup = Audiokinetic, BlueprintType, hidecategories = (Transform, Rendering, Mobility, LOD, Component, Activation), meta = (BlueprintSpawnableComponent))

class WWISEDEMOGAME_API UAkVoiceInputComponent : public UAkAudioInputComponent
{
    GENERATED_BODY()
        UAkVoiceInputComponent(const class FObjectInitializer& ObjectInitializer);
 
    virtual void TickComponent(float DeltaTime, enum ELevelTick TickType, FActorComponentTickFunction *ThisTickFunction) override;
 
protected:
    /* This is called after the GameObject that owns this component is unregistered from the Wwise sound engine. */
    virtual void PostUnregisterGameObject() override;
 
    /* The audio callback. This will be called continuously by the Wwise sound engine
    and is used to provide the sound engine with audio samples. */
    virtual bool FillSamplesBuffer(uint32 NumChannels, uint32 NumSamples, float** BufferToFill) override;
    /* This callback is used to provide the Wwise sound engine with the required audio format. */
    virtual void GetChannelConfig(AkAudioFormat& AudioFormat) override;
    /* The Unreal IVoiceCapture, used for accessing microphone input. */
    TSharedPtr<IVoiceCapture> VoiceCapture;
    /* This array is reset and refilled every time a new buffer comes in from the voice capture. */
    TArray<uint8> IncomingRawVoiceData;
    /* This array holds all of the previously collected audio data from the voice capture.
    It is written to when processing available microphone data,
    and read from (and shrunk) when passing data to the Wwise engine. 
    Shrinking a buffer in the audio callback is not advisable and this should NOT be used in a shipped game!*/
    TArray<uint8> CollectedRawVoiceData;
    /* This flag is used to protect against writing microphone data to the collected data buffer while it is being read from. */
    FThreadSafeBool bIsReadingVoiceData = false;
};

AkVoiceInputComponent.cpp:

#include "AkVoiceInputComponent.h"
 
UAkVoiceInputComponent::UAkVoiceInputComponent(const class FObjectInitializer& ObjectInitializer) :
    UAkAudioInputComponent(ObjectInitializer)
{
    CollectedRawVoiceData.Reset();
    VoiceCapture = FVoiceModule::Get().CreateVoiceCapture();
}
 

void UAkVoiceInputComponent::TickComponent(float DeltaTime, enum ELevelTick TickType, FActorComponentTickFunction *ThisTickFunction)
{
    Super::TickComponent(DeltaTime, TickType, ThisTickFunction);
 
    if (!VoiceCapture.IsValid())
    {
        return;
    }
 
    uint32 NumAvailableVoiceCaptureBytes = 0;
    EVoiceCaptureState::Type CaptureState = VoiceCapture->GetCaptureState(NumAvailableVoiceCaptureBytes);
 
    /* IVoiceCapture updates its EVoiceCaptureState on every tick.
    We therefore know, between ticks, the state will tell us if there is actually new data to collect. */
    if (CaptureState == EVoiceCaptureState::Ok && NumAvailableVoiceCaptureBytes > 0)
    {
        uint32 NumVoiceCaptureBytesReturned = 0;
        IncomingRawVoiceData.Reset((int32)NumAvailableVoiceCaptureBytes);
        IncomingRawVoiceData.AddDefaulted(NumAvailableVoiceCaptureBytes);
        uint64 SampleCounter = 0;
        VoiceCapture->GetVoiceData(IncomingRawVoiceData.GetData(), NumAvailableVoiceCaptureBytes, NumVoiceCaptureBytesReturned, SampleCounter);
        if (NumVoiceCaptureBytesReturned > 0)
        {
            /* Spin while data is being read from the collected buffer */
            while (bIsReadingVoiceData) {}
            CollectedRawVoiceData.Append(IncomingRawVoiceData);
        }
    }
}
 

bool UAkVoiceInputComponent::FillSamplesBuffer(uint32 NumChannels, uint32 NumSamples, float** BufferToFill)
{
    if (!VoiceCapture.IsValid())
    {
        return false;
    }
 
    const uint8 NumBytesPerSample = 2;
    const uint32 NumRequiredBytesPerChannel = NumSamples * NumBytesPerSample;
    const uint32 NumRequiredBytes = NumRequiredBytesPerChannel * NumChannels;
 
    int16 VoiceSample = 0;
    uint32 RawChannelIndex = 0;
    uint32 RawSampleIndex = 0;
 
    bIsReadingVoiceData = true;
    const int32 NumSamplesAvailable = CollectedRawVoiceData.Num() / NumBytesPerSample;
    const uint32 BufferSlack = (uint32)FMath::Max(0, (int32)(NumSamples * NumChannels) - NumSamplesAvailable);
 
    for (uint32 c = 0; c < NumChannels; ++c)
    {
        RawChannelIndex = c * NumRequiredBytesPerChannel;
        for (uint32 s = 0; s < NumSamples; ++s)
        {
            if (s >= (NumSamples - BufferSlack) / NumChannels)
            {
                /* In the case where we received less data from the voice capture than is required by the Wwise engine,
                pad the missing samples with zeros. */
                BufferToFill[c][s] = 0.0f;
            }
            else
            {
                /* Convert the incoming microphone audio data to signed floating point. */
                uint32 RawSampleDataMSBIndex = s * 2 + 1;
                uint32 RawSampleDataLSBIndex = s * 2;
                VoiceSample = (CollectedRawVoiceData[RawSampleDataMSBIndex] << 8) | CollectedRawVoiceData[RawSampleDataLSBIndex];
                BufferToFill[c][s] = VoiceSample / (float)INT16_MAX;
            }
 
        }
    }
 
    const int32 NumBytesRead = (NumSamples - BufferSlack) * NumBytesPerSample;
    /* NOTE: Shrinking a buffer in the audio callback is not advisable. This should NOT be used in a shipped game! */
    CollectedRawVoiceData.RemoveAt(0, NumBytesRead);
    bIsReadingVoiceData = false;
    return true;
}
 

void UAkVoiceInputComponent::GetChannelConfig(AkAudioFormat& AudioFormat)
{
    const int sampleRate = 16000;
    AudioFormat.uSampleRate = sampleRate;
 
    AudioFormat.channelConfig.SetStandard(AK_SPEAKER_SETUP_MONO);
    if (VoiceCapture.IsValid())
    {
        /* Pass in an empty device name to use default device. */
        if (!VoiceCapture->Init(FString(""), AudioFormat.uSampleRate, AudioFormat.channelConfig.uNumChannels))
        {
            UE_LOG(LogTemp, Error, TEXT("Failed to initialize device for voice input!"));
            return;
        }
        VoiceCapture->Start();
    }
}
 

void UAkVoiceInputComponent::PostUnregisterGameObject()
{
    Super::PostUnregisterGameObject();
    if (VoiceCapture.IsValid())
    {
        VoiceCapture->Stop();
        VoiceCapture->Shutdown();
    }
}

With this class added to an Unreal project, a custom Blueprint class can be created which has an AkVoiceInputComponent, and the Post Associated Audio Input Event Blueprint function (from base class AkAudioInputComponent) can be called in order to start sending microphone data to Wwise. The image below shows part of the Blueprint for a custom Blueprint class, based on Actor, that has an AkVoiceInputComponent called AkVoiceInput.

Was this page helpful?

Need Support?

Questions? Problems? Need more info? Contact us, and we can help!

Visit our Support page

Tell us about your project. We're here to help.

Get started with Wwise