The Unreal integration provides audio input to Wwise through the Wwise Audio Input Source Plug-in. In order to provide audio input to Wwise, classes must inherit from AkAudioInputComponent.

AkAudioInputComponent

AkAudioInputComponent is derived from AkComponent. It is a specialized AkComponent that you can use to provide audio input to Wwise. To do so, you must implement two key functions:

AkAudioInputComponent also has one Blueprint function, Post Associated Audio Input Event, which posts the component's AkAudioEvent to Wwise along with the associated AudioSamples callback and AudioFormat callback, using this component as the game object source.

Customizing Audio Input Behavior

You can write custom classes that derive from AkAudioInputComponent in order to implement custom audio input behavior. The following UAkVoiceInputComponent.h and UAkVoiceInputComponent.cpp examples show a class that takes microphone input and pipes it to the Wwise Sound Engine.

Before you can use these files in a C++ Unreal project, you must perform some initial setup. First, add the AkAudio and the Unreal Voice modules to the PublicDependencyModuleNames in the project's Build.cs file. For example:

Next, add the following lines to the DefaultEngine.ini file:

After these setup steps are complete, add the desired custom behavior. In the following example, a class takes microphone input and sends it to Wwise.

Note: The sample code in this section is not intended for use in shipped games. It is only a short example of how to implement custom behavior. Here is an example of a customized AkVoiceInputComponent.h file: #pragma once #include "CoreMinimal.h" #include "AkAudioInputComponent.h" #include "Voice.h" #include "AkVoiceInputComponent.generated.h" /* */ UCLASS(ClassGroup = Audiokinetic, BlueprintType, hidecategories = (Transform, Rendering, Mobility, LOD, Component, Activation), meta = (BlueprintSpawnableComponent)) class WWISEDEMOGAME_API UAkVoiceInputComponent : public UAkAudioInputComponent { GENERATED_BODY() UAkVoiceInputComponent(const class FObjectInitializer& ObjectInitializer); virtual void TickComponent(float DeltaTime, enum ELevelTick TickType, FActorComponentTickFunction *ThisTickFunction) override; protected: /* This is called after the GameObject that owns this component is unregistered from the Wwise sound engine. */ virtual void PostUnregisterGameObject() override; /* The audio callback. This will be called continuously by the Wwise sound engine and is used to provide the sound engine with audio samples. */ virtual bool FillSamplesBuffer(uint32 NumChannels, uint32 NumSamples, float** BufferToFill) override; /* This callback is used to provide the Wwise sound engine with the required audio format. */ virtual void GetChannelConfig(AkAudioFormat& AudioFormat) override; /* The Unreal IVoiceCapture, used for accessing microphone input. */ TSharedPtr<IVoiceCapture> VoiceCapture; /* This array is reset and refilled every time a new buffer comes in from the voice capture. */ TArray<uint8> IncomingRawVoiceData; /* This array holds all of the previously collected audio data from the voice capture. It is written to when processing available microphone data, and read from (and shrunk) when passing data to the Wwise engine. Shrinking a buffer in the audio callback is not advisable and this should NOT be used in a shipped game!*/ TArray<uint8> CollectedRawVoiceData; /* This flag is used to protect against writing microphone data to the collected data buffer while it is being read from. */ FThreadSafeBool bIsReadingVoiceData = false; }; Here is an example of a customized AkVoiceInputComponent.cpp file: #include "AkVoiceInputComponent.h" UAkVoiceInputComponent::UAkVoiceInputComponent(const class FObjectInitializer& ObjectInitializer) : UAkAudioInputComponent(ObjectInitializer) { CollectedRawVoiceData.Reset(); VoiceCapture = FVoiceModule::Get().CreateVoiceCapture(); } void UAkVoiceInputComponent::TickComponent(float DeltaTime, enum ELevelTick TickType, FActorComponentTickFunction *ThisTickFunction) { Super::TickComponent(DeltaTime, TickType, ThisTickFunction); if (!VoiceCapture.IsValid()) { return; } uint32 NumAvailableVoiceCaptureBytes = 0; EVoiceCaptureState::Type CaptureState = VoiceCapture->GetCaptureState(NumAvailableVoiceCaptureBytes); /* IVoiceCapture updates its EVoiceCaptureState on every tick. We therefore know, between ticks, the state will tell us if there is actually new data to collect. */ if (CaptureState == EVoiceCaptureState::Ok && NumAvailableVoiceCaptureBytes > 0) { uint32 NumVoiceCaptureBytesReturned = 0; IncomingRawVoiceData.Reset((int32)NumAvailableVoiceCaptureBytes); IncomingRawVoiceData.AddDefaulted(NumAvailableVoiceCaptureBytes); uint64 SampleCounter = 0; VoiceCapture->GetVoiceData(IncomingRawVoiceData.GetData(), NumAvailableVoiceCaptureBytes, NumVoiceCaptureBytesReturned, SampleCounter); if (NumVoiceCaptureBytesReturned > 0) { /* Spin while data is being read from the collected buffer */ while (bIsReadingVoiceData) {} CollectedRawVoiceData.Append(IncomingRawVoiceData); } } } bool UAkVoiceInputComponent::FillSamplesBuffer(uint32 NumChannels, uint32 NumSamples, float** BufferToFill) { if (!VoiceCapture.IsValid()) { return false; } const uint8 NumBytesPerSample = 2; const uint32 NumRequiredBytesPerChannel = NumSamples * NumBytesPerSample; const uint32 NumRequiredBytes = NumRequiredBytesPerChannel * NumChannels; int16 VoiceSample = 0; uint32 RawChannelIndex = 0; uint32 RawSampleIndex = 0; bIsReadingVoiceData = true; const int32 NumSamplesAvailable = CollectedRawVoiceData.Num() / NumBytesPerSample; const uint32 BufferSlack = (uint32)FMath::Max(0, (int32)(NumSamples * NumChannels) - NumSamplesAvailable); for (uint32 c = 0; c < NumChannels; ++c) { RawChannelIndex = c * NumRequiredBytesPerChannel; for (uint32 s = 0; s < NumSamples; ++s) { if (s >= (NumSamples - BufferSlack) / NumChannels) { /* In the case where we received less data from the voice capture than is required by the Wwise engine, pad the missing samples with zeros. */ BufferToFill[c][s] = 0.0f; } else { /* Convert the incoming microphone audio data to signed floating point. */ uint32 RawSampleDataMSBIndex = s * 2 + 1; uint32 RawSampleDataLSBIndex = s * 2; VoiceSample = (CollectedRawVoiceData[RawSampleDataMSBIndex] << 8) | CollectedRawVoiceData[RawSampleDataLSBIndex]; BufferToFill[c][s] = VoiceSample / (float)INT16_MAX; } } } const int32 NumBytesRead = (NumSamples - BufferSlack) * NumBytesPerSample; /* NOTE: Shrinking a buffer in the audio callback is not advisable. This should NOT be used in a shipped game! */ CollectedRawVoiceData.RemoveAt(0, NumBytesRead); bIsReadingVoiceData = false; return true; } void UAkVoiceInputComponent::GetChannelConfig(AkAudioFormat& AudioFormat) { const int sampleRate = 16000; AudioFormat.uSampleRate = sampleRate; AudioFormat.channelConfig.SetStandard(AK_SPEAKER_SETUP_MONO); if (VoiceCapture.IsValid()) { /* Pass in an empty device name to use default device. */ if (!VoiceCapture->Init(FString(""), AudioFormat.uSampleRate, AudioFormat.channelConfig.uNumChannels)) { UE_LOG(LogTemp, Error, TEXT("Failed to initialize device for voice input!")); return; } VoiceCapture->Start(); } } void UAkVoiceInputComponent::PostUnregisterGameObject() { Super::PostUnregisterGameObject(); if (VoiceCapture.IsValid()) { VoiceCapture->Stop(); VoiceCapture->Shutdown(); } } After you add the class to an Unreal project, you can create a custom Blueprint class that has an AkVoiceInputComponent, and can call the Post Associated Audio Input Event Blueprint function (from base class AkAudioInputComponent) to start sending microphone data to Wwise. The following image shows part of the Blueprint for a custom Blueprint class, based on Actor, that has an AkVoiceInputComponent called AkVoiceInput.