Our new developer documentation is now available. Please check it out!

FMOD and ODIN in Unreal

Integrating ODIN Voice Chat with the FMOD Audio Solution in Unreal.

FMOD and ODIN

Introduction

Welcome to this guide on integrating the ODIN Voice Chat Plugin with the FMOD Audio Solution in Unreal. The code used in this guide is available on the ODIN-FMOD Sample Project GitHub Repository.

What You’ll Learn:

Two approaches to add spatialized voice chat to your Unreal project using FMOD
Properly set up ODIN in Unreal when using FMOD as audio solution
Deal with limitations and potential pitfalls

Getting Started

To follow this guide, you’ll need to have some prerequisites:

Basic knowledge of Unreal as well as FMOD
The FMOD Plugin for Unreal, which you can get here
The ODIN Voice Chat Plugin, available here

To set up FMOD in your project, please follow FMOD’s in-depth integration-tutorial. You can find the tutorial here.

To set up the ODIN Voice Chat Plugin, please take a look at our Getting-Started guide, which you can find here:

Begin ODIN Getting Started Guide

This guide will show you how to access the Odin Media Stream and copy it to the Audio Channels of FMOD in order to pass it to the FMOD Audio Engine. This means, we will only cover the receiver-side of the communication - the sender just uses Unreal’s Audio Capture Module and thus is handled no different than any other implementation of Odin in Unreal.

Sample Project

You can find a sample project in a repository in our GitHub account. Feel free to download it and set it up in order to view a working integration of this class in a small sample project. This sample is based on the result of the second video of the Odin tutorial Series.

There are two branches in the sample project - the main branch writes the voice chat audio data directly to the master channel group of fmod by using a dynamically declared DSP plugin. The programmer sound branch uses the Audio Input Plugin by FMOD through the use of a programmer sound. The former is preferred because of its lower delay in processing.

DSP Plugin

UOdinFmodAdapter

The UOdinFmodAdapter class is an essential part of the FMOD integration. It replaces the default ODIN UOdinSynthComponent component, taking over the voice output responsibilities by using FMOD. This script is crucial for receiving voice chat data from the ODIN servers. Additionally it adds the built-in Pan plugin from FMOD, to show an example of how to add effects to it.

The header can be found here and the source file is located here

The UOdinFmodAdapter inherits from Unreal Engine’s UActorComponent.

You can either follow the Usage setup to drop the UOdinFmodAdapter directly into your project, or take a look at how it works to adjust the functionality to your requirements.

This is the header:

#pragma once

#include "CoreMinimal.h"
#include "FMODAudioComponent.h"
#include "fmod_studio.hpp"
#include "Components/ActorComponent.h"
#include "OdinFmodAdapter.generated.h"

class OdinMediaSoundGenerator;
class UOdinPlaybackMedia;

UENUM(BlueprintType)
enum class EFmodDspPan3dRolloffType : uint8 {
	FMOD_DSP_PAN_3D_ROLLOFF_LINEARSQUARED UMETA(DisplayName = "Linear Squared"),
	FMOD_DSP_PAN_3D_ROLLOFF_LINEAR UMETA(DisplayName = "Linear"),
	FMOD_DSP_PAN_3D_ROLLOFF_INVERSE UMETA(DisplayName = "Inverse"),
	FMOD_DSP_PAN_3D_ROLLOFF_INVERSETAPERED UMETA(DisplayName = "Inverse Tapered"),
	FMOD_DSP_PAN_3D_ROLLOFF_CUSTOM UMETA(DisplayName = "Custom")
};

UENUM(BlueprintType)
enum class EFmodDspPan3dExtentMode : uint8 {
	FMOD_DSP_PAN_3D_EXTENT_MODE_AUTO UMETA(DisplayName = "Auto"),
	FMOD_DSP_PAN_3D_EXTENT_MODE_USER UMETA(DisplayName = "User"),
	FMOD_DSP_PAN_3D_EXTENT_MODE_OFF UMETA(DisplayName = "Off")
};

UCLASS( ClassGroup=(Custom), meta=(BlueprintSpawnableComponent) )
class ODINTESTPROJECT_API UOdinFmodAdapter : public USceneComponent
{
	GENERATED_BODY()

public:
	// Sets default values for this component's properties
	void BeginPlay() override;

	void DestroyComponent(bool bPromoteChildren) override;

	UOdinFmodAdapter();

	virtual void TickComponent(float DeltaTime, ELevelTick TickType, FActorComponentTickFunction* ThisTickFunction) override;

	UFUNCTION(BlueprintCallable, Category = "Odin|Sound")
	void AssignOdinMedia(UPARAM(ref) UOdinPlaybackMedia*& Media);

	FMOD_RESULT dspreadcallback(FMOD_DSP_STATE* dsp_state, float* data, unsigned int datalen, int inchannels);

	// Object Spatializer Parameters

	UPROPERTY(EditAnywhere, BlueprintReadOnly)
	EFmodDspPan3dRolloffType RolloffType = EFmodDspPan3dRolloffType::FMOD_DSP_PAN_3D_ROLLOFF_LINEARSQUARED;

	UPROPERTY(EditAnywhere, BlueprintReadOnly, meta = (ClampMin = "0.0", UIMin = "0.0"))
	float MinimumDistance = 1.0f;

	UPROPERTY(EditAnywhere, BlueprintReadOnly, meta = (ClampMin = "0.0", UIMin = "0.0"))
	float MaximumDistance = 20.0f;

	UPROPERTY(EditAnywhere, BlueprintReadOnly)
	EFmodDspPan3dExtentMode ExtentMode = EFmodDspPan3dExtentMode::FMOD_DSP_PAN_3D_EXTENT_MODE_AUTO;

	UPROPERTY(EditAnywhere, BlueprintReadOnly, meta = (ClampMin = "0.0", UIMin = "0.0"))
	float SoundSize = 0.0f;

	UPROPERTY(EditAnywhere, BlueprintReadOnly, meta = (ClampMin = "0.0", ClampMax = "360.0", UIMin = "0.0", UIMax = "360.0"))
	float MinimumExtent = 0.0f;

	UPROPERTY(EditAnywhere, BlueprintReadOnly, meta = (ClampMin = "0.0", ClampMax = "1.0", UIMin = "0.0", UIMax = "1.0"))
	float OutputGain = 0.0f;

	UFUNCTION(BlueprintCallable, Category = "Odin|Sound")
	void SetAttenuation(EFmodDspPan3dRolloffType InRolloffType, float InMinimumDistance, float InMaximumDistance, EFmodDspPan3dExtentMode InExtentMode, float InSoundSize, float InMinimumExtent, float InOutputGain);

	static FMOD_RESULT OdinDSPReadCallback(FMOD_DSP_STATE* dsp_state, float* inbuffer, float* outbuffer, unsigned int length, int inchannels, int* outchannels);

	FMOD::DSP* dsp_objectpan;
	FMOD::DSP* mOdinDSP = nullptr;

protected:

	UPROPERTY(BlueprintReadOnly, Category = "Odin|Sound")
	UOdinPlaybackMedia* PlaybackMedia = nullptr;
	TSharedPtr<OdinMediaSoundGenerator, ESPMode::ThreadSafe> SoundGenerator;

	FMOD::ChannelGroup* group;

	void Update3DPosition();

	void UpdateAttenSettings();

	FMOD_VECTOR ConvertUnrealToFmodVector(FVector in, float scale = 1.0f);
};

And this is the source file of the class:

// Fill out your copyright notice in the Description page of Project Settings.

#include "OdinFmodAdapter.h"
#include "fmod_studio.hpp"
#include "FMODStudioModule.h"
#include "odin.h"
#include "OdinFunctionLibrary.h"
#include "OdinMediaSoundGenerator.h"
#include "OdinPlaybackMedia.h"
#include <Kismet/KismetMathLibrary.h>

void UOdinFmodAdapter::AssignOdinMedia(UOdinPlaybackMedia*& Media)
{
	if (nullptr == Media)
		return;

	this->SoundGenerator = MakeShared<OdinMediaSoundGenerator, ESPMode::ThreadSafe>();
	this->PlaybackMedia = Media;

	SoundGenerator->SetOdinStream(Media->GetMediaHandle());
}

void UOdinFmodAdapter::SetAttenuation(EFmodDspPan3dRolloffType InRolloffType, float InMinimumDistance, float InMaximumDistance, EFmodDspPan3dExtentMode InExtentMode, float InSoundSize, float InMinimumExtent, float InOutputGain)
{
	this->RolloffType = InRolloffType;
	this->MinimumDistance = InMinimumDistance;
	this->MaximumDistance = InMaximumDistance;
	this->ExtentMode = InExtentMode;
	this->SoundSize = InSoundSize;
	this->MinimumExtent = InMinimumExtent;
	this->OutputGain = InOutputGain;

	UpdateAttenSettings();

	Update3DPosition();
}

FMOD_RESULT UOdinFmodAdapter::OdinDSPReadCallback(FMOD_DSP_STATE* dsp_state, float* inbuffer, float* outbuffer, unsigned int length, int inchannels, int* outchannels)
{
	void* userdata;

	dsp_state->functions->getuserdata(dsp_state, &userdata);

	*outchannels = 2;

	UOdinFmodAdapter* instance = reinterpret_cast<UOdinFmodAdapter*>(userdata);

	return instance->dspreadcallback(dsp_state, outbuffer, length, inchannels);
}

void UOdinFmodAdapter::Update3DPosition()
{
	FMOD_DSP_PARAMETER_3DATTRIBUTES_MULTI attrs = { 0 };

	FMOD_3D_ATTRIBUTES relattr = { 0 };

	FMOD_3D_ATTRIBUTES absattr = { 0 };

	if (GEngine == nullptr) return;

	UWorld* world = this->GetWorld();
	if (world == nullptr) return;

	ULocalPlayer* player = GEngine->GetGamePlayer(world, 0);
	if (player == nullptr) return;

	TObjectPtr<APlayerController> controller = player->PlayerController;
	if (controller == nullptr) return;

	AActor* listener = controller->GetPawn();
	if (listener == nullptr) return;

	relattr.position = ConvertUnrealToFmodVector(this->GetComponentLocation() - listener->GetActorLocation(), 0.01f);
	relattr.velocity = ConvertUnrealToFmodVector(this->GetComponentVelocity() - listener->GetVelocity(), 0.01f);
	relattr.forward = ConvertUnrealToFmodVector(UKismetMathLibrary::GetForwardVector(this->GetComponentRotation()));
	relattr.up = ConvertUnrealToFmodVector(UKismetMathLibrary::GetUpVector(this->GetComponentRotation()));

	absattr.position = ConvertUnrealToFmodVector(this->GetComponentLocation(), 0.01f);
	absattr.velocity = ConvertUnrealToFmodVector(this->GetComponentVelocity(), 0.01f);
	absattr.forward = ConvertUnrealToFmodVector(UKismetMathLibrary::GetForwardVector(this->GetComponentRotation()));
	absattr.up = ConvertUnrealToFmodVector(UKismetMathLibrary::GetUpVector(this->GetComponentRotation()));

	attrs.relative[0] = relattr;
	attrs.numlisteners = 1;
	attrs.absolute = absattr;

	dsp_pan->setParameterData(FMOD_DSP_PAN_3D_POSITION, &attrs, sizeof(FMOD_DSP_PARAMETER_3DATTRIBUTES_MULTI));

	group->set3DAttributes(&absattr.position, &absattr.velocity);

	FMOD::Studio::System* System = IFMODStudioModule::Get().GetStudioSystem(EFMODSystemContext::Runtime);
	FMOD::System* CoreSystem = nullptr;
	System->getCoreSystem(&CoreSystem);

	FMOD_VECTOR pos = ConvertUnrealToFmodVector(listener->GetActorLocation(), 0.01f);
	FMOD_VECTOR vel = ConvertUnrealToFmodVector(listener->GetVelocity(), 0.01f);
	FMOD_VECTOR forward = ConvertUnrealToFmodVector(listener->GetActorForwardVector());
	FMOD_VECTOR up = ConvertUnrealToFmodVector(listener->GetActorUpVector());

	CoreSystem->set3DListenerAttributes(0, &pos, &vel, &forward, &up);
	CoreSystem->update();
}

void UOdinFmodAdapter::UpdateAttenSettings()
{
	dsp_pan->setParameterInt(FMOD_DSP_PAN_3D_ROLLOFF, (int)RolloffType);
	dsp_pan->setParameterFloat(FMOD_DSP_PAN_3D_MIN_DISTANCE, MinimumDistance);
	dsp_pan->setParameterFloat(FMOD_DSP_PAN_3D_MAX_DISTANCE, MaximumDistance);
	dsp_pan->setParameterInt(FMOD_DSP_PAN_3D_EXTENT_MODE, (int)ExtentMode);
	dsp_pan->setParameterFloat(FMOD_DSP_PAN_3D_SOUND_SIZE, SoundSize);
	dsp_pan->setParameterFloat(FMOD_DSP_PAN_3D_MIN_EXTENT, MinimumExtent);
}

FMOD_VECTOR UOdinFmodAdapter::ConvertUnrealToFmodVector(FVector in, float scale)
{
	auto out = FMOD_VECTOR();
	out.x = in.Y * scale;
	out.y = in.Z * scale;
	out.z = in.X * scale;

	return out;
}

void UOdinFmodAdapter::BeginPlay()
{
	FMOD::Studio::System* System = IFMODStudioModule::Get().GetStudioSystem(EFMODSystemContext::Runtime);
	FMOD::System* CoreSystem = nullptr;
	System->getCoreSystem(&CoreSystem);

	FMOD_DSP_READ_CALLBACK mReadCallback = OdinDSPReadCallback;

	FMOD_DSP_DESCRIPTION desc = { 0 };
	desc.read = mReadCallback;
	desc.userdata = this;
	desc.numoutputbuffers = 1;

	FMOD_RESULT res = CoreSystem->createDSP(&desc, &mOdinDSP);

	if (res == FMOD_RESULT::FMOD_OK)
	{
		CoreSystem->getMasterChannelGroup(&group);

		if (group->addDSP(0, mOdinDSP) == FMOD_RESULT::FMOD_OK)
		{
			UE_LOG(LogTemp, Warning, TEXT("Added Odin DSP to channel group"));
		}
	}

	FMOD_RESULT result = CoreSystem->createDSPByType(FMOD_DSP_TYPE_PAN, &dsp_pan);

	

	if (result == FMOD_RESULT::FMOD_OK)
	{
		CoreSystem->getMasterChannelGroup(&group);

		if (group->addDSP(FMOD_CHANNELCONTROL_DSP_HEAD, dsp_pan) == FMOD_RESULT::FMOD_OK)
		{
			UE_LOG(LogTemp, Warning, TEXT("Added Object Spatializer DSP to channel group"));

			group->setMode(FMOD_3D | FMOD_3D_WORLDRELATIVE | FMOD_3D_LINEARROLLOFF);
		}
	}

	// Set Pan Output to Stereo
	dsp_pan->setParameterInt(FMOD_DSP_PAN_MODE, (int)FMOD_DSP_PAN_MODE_SURROUND);

	// Set Pan Mode to full 3D Positional
	dsp_pan->setParameterFloat(FMOD_DSP_PAN_3D_PAN_BLEND, 1.0f);

	UpdateAttenSettings();
	Update3DPosition();
}

void UOdinFmodAdapter::DestroyComponent(bool bPromoteChildren)
{
	auto result2 = group->removeDSP(dsp_pan);
	auto result4 = group->removeDSP(mOdinDSP);

	dsp_pan->release();
	mOdinDSP->release();

	Super::DestroyComponent(bPromoteChildren);
}

UOdinFmodAdapter::UOdinFmodAdapter()
{
	PrimaryComponentTick.bCanEverTick = true;
}

void UOdinFmodAdapter::TickComponent(float DeltaTime, ELevelTick TickType, FActorComponentTickFunction* ThisTickFunction)
{
	Super::TickComponent(DeltaTime, TickType, ThisTickFunction);

	Update3DPosition();
}

FMOD_RESULT UOdinFmodAdapter::dspreadcallback(FMOD_DSP_STATE* dsp_state, float* data, unsigned int datalen, int inchannels)
{
	if (!data)
		return FMOD_ERR_INVALID_PARAM;

	if (!SoundGenerator || !PlaybackMedia)
		return FMOD_OK;

	unsigned int requestedDataArrayLength = datalen * 2;

	const uint32 Result = SoundGenerator->OnGenerateAudio(data, (int32)requestedDataArrayLength);

	if (odin_is_error(Result))
	{
		FString ErrorString = UOdinFunctionLibrary::FormatError(Result, true);
		UE_LOG(LogTemp, Error, TEXT("UOdinFmodAdapter: Error during FillSamplesBuffer: %s"), *ErrorString);
		return FMOD_OK;
	}

	return FMOD_OK;
}

Remember to adjust the Build.cs file of your game module accordingly. We need to add dependencies to “Odin” obviously, but also “OdinLibrary” is needed for the call to odin_is_error(). From FMOD we need the “FMODStudio” Module in order to work with the FMOD Programmer Sounds. So all in all add these to your Public and Private Dependency Modules:

PublicDependencyModuleNames.AddRange(
	new string[]
	{
        "Core",
        "Odin",
		"OdinLibrary"
	}
	);

PrivateDependencyModuleNames.AddRange(
	new string[] 
	{ 
		"FMODStudio" 
	}
	);

Usage

The above class uses the FMOD DSP Plugin API to pass dynamically created Audio Data to the FMOD Engine. It does not need any steps in FMOD Studio and will simply work out of the box. All settings are done in the code.

Integrating the Input Component in your Unreal Project

In the next step we will now use the created Component to play back the incoming Odin Media Stream. Again you can find an example of this in the Odin Client Component of the sample project.

First replace the creation of an OdinSynthComponent that you have placed in the Odin Unreal Guide in your project with the new UOdinFmodAdapter.

In the OnMediaAdded event of your Odin implementation in your project you can then call the Assign Odin Media function that we have declared in the C++ class and pass it the reference to the incoming Media Stream.

Like with the OdinSynthComponent, you can also choose to place the UOdinFmodAdapter directly on the Player Character as a component and then reference it in your OnMediaAdded event handler. This way you do not have to create it in the Blueprint and it is easier to change its properties - e.g. its FMOD-specific (attenuation) settings.

You can see a sample of the Blueprint implementation below:

How it works

The above class uses the FMOD DSP Plugin API to pass dynamically created Audio Data to the FMOD Engine. It copies the incoming Audio Stream from Odin to the Master Group’s Channel Buffers of FMOD. This is done by creating an FMOD DSP Plugin and implementing a handler for the read event.

1. Setup

The setup of the UOdinFmodAdapter is done by passing it a reference to the incoming Odin Media Stream. In this guide we have done this via a Blueprint call, but the function can also be called from another C++ Class in your game module.

This function creates a new pointer to a new OdinMediaSoundGenerator and sets its OdinStream to the incoming Media’s handle.

void UOdinFmodAdapter::AssignOdinMedia(UOdinPlaybackMedia*& Media)
{
	if (nullptr == Media)
		return;

	this->SoundGenerator = MakeShared<OdinMediaSoundGenerator, ESPMode::ThreadSafe>();
	this->PlaybackMedia = Media;

	SoundGenerator->SetOdinStream(Media->GetMediaHandle());
}

Next, in the BeginPlay() function of the Component we want to create and initialize the FMOD DSP Plugin. We get references to the FMOD Studio’s Core System that is responsible for creating and playing back audio. With it, we can create the DSP for our custom component.

FMOD::Studio::System* System = IFMODStudioModule::Get().GetStudioSystem(EFMODSystemContext::Runtime);
FMOD::System* CoreSystem = nullptr;
System->getCoreSystem(&CoreSystem);

To do so, we create a new DSP Description and set its fields to what we need for our Odin playback:

FMOD_DSP_DESCRIPTION desc = { 0 };
desc.numoutputbuffers = 1;
desc.read = mReadCallback;
desc.userdata = this;

numoutputbuffers tells the FMOD Audio Engine how many buffers our plugin can handle. Since Odin uses an interleaved stereo signal, we need to set it to use a single buffer.

The last two fields in the example are needed to handle the FMOD event to fill the sound buffer:

The read field needs to point to a static function of your class. Since the function is static, but we want to access the correct Odin SoundGenerator (which is a different one for each object of the class), we need to also pass a pointer to the correct instance of this Actor Component. Since the read callback passes a reference to the created DSP Plugin, we can use its userdata field to pass any needed data. Here we just put a pointer to the ActorComponent that created the sound. Later we will use this to call the function on the correct instance, when the static OdinDSPReadCallback event handler is called.

Lastly we will create the DSP Plugin via the CoreSystem->createDSP function. We pass the pointers to our FMOD_DSP_DESCRIPTION and a pointer to the created DSP. The latter is set in the function so that we can manipulate it later.

If the creation succeeds, we retrieve the master channel group and add the DSP to it with group->addDSP(). We need to pass it the index, where we want to put it in the channel’s dsp chain and the pointer to the DSP itself.

FMOD_RESULT res = CoreSystem->createDSP(&desc, &mOdinDSP);

if (res == FMOD_RESULT::FMOD_OK)
{
	CoreSystem->getMasterChannelGroup(&group);

	if (group->addDSP(0, mOdinDSP) == FMOD_RESULT::FMOD_OK)
	{
		UE_LOG(LogTemp, Warning, TEXT("Added Odin DSP to channel group"));
	}
}

Lastly we also add a Pan DSP to enable simple spatialization. We create a DSP by Type and add it to the head of the Master Channel Group afterwards. Lastly we set its initial properties - in the sample project they are passed in through the Blueprint interface using instance variables of the component.

FMOD_RESULT result = CoreSystem->createDSPByType(FMOD_DSP_TYPE_PAN, &dsp_pan);



if (result == FMOD_RESULT::FMOD_OK)
{
	CoreSystem->getMasterChannelGroup(&group);

	if (group->addDSP(FMOD_CHANNELCONTROL_DSP_HEAD, dsp_pan) == FMOD_RESULT::FMOD_OK)
	{
		UE_LOG(LogTemp, Warning, TEXT("Added Object Spatializer DSP to channel group"));

		group->setMode(FMOD_3D | FMOD_3D_WORLDRELATIVE | FMOD_3D_LINEARROLLOFF);
	}
}

// Set Pan Output to Stereo
dsp_pan->setParameterInt(FMOD_DSP_PAN_MODE, (int)FMOD_DSP_PAN_MODE_SURROUND);

// Set Pan Mode to full 3D Positional
dsp_pan->setParameterFloat(FMOD_DSP_PAN_3D_PAN_BLEND, 1.0f);

UpdateAttenSettings();
Update3DPosition();

The UpdateAttenSettings()and Update3DPosition() functions are helper functions that set the corresponding properties of the Pan Plugin - they can be looked up in the project for more details - overall they read the instance variables and 3D Position of the component and pass them to the plugin, after converting positional and rotational vectors from Unreal to the FMOD coordinate system.

Warning

Disclaimer: This is a simplified implementation of the Pan Plugin by FMOD. You might want to consider e.g. using different Channel Groups for your Odin Peers to give them distinct effects. The implementation of such structures would exceed the scope of this guide, though.

2. Reading and Playing Back ODIN Audio Streams

The OdinDSPReadCallback function is called from the FMOD Stduio API whenever the playback requests more data for its buffer.

Since this function needs to be static, we need to get the userdata that we created the sound with and call the proper handler function on the instance that we reference here. We can call dsp_state->functions->getuserdata() to retrieve the given userdata. We just need to reinterpret it as a pointer to a UOdinFmodAdapter component and then we can call its instance function dspreadcallback with the given parameters from FMOD.

FMOD_RESULT UOdinFmodAdapter::OdinDSPReadCallback(FMOD_DSP_STATE* dsp_state, float* inbuffer, float* outbuffer, unsigned int length, int inchannels, int* outchannels)
{
	void* userdata;

	dsp_state->functions->getuserdata(dsp_state, &userdata);

	*outchannels = 2;

	UOdinFmodAdapter* instance = reinterpret_cast<UOdinFmodAdapter*>(userdata);

	return instance->dspreadcallback(dsp_state, outbuffer, length, inchannels);
}

In this function we first check if the component was set up properly and if the incoming parameters are valid.

if (!data)
	return FMOD_ERR_INVALID_PARAM;

if (!SoundGenerator || !PlaybackMedia)
	return FMOD_OK;

The datalen parameter indicates the number of samples per channel, so in order to pass it to Odin, we first need to calculate the total number of samples from it.

unsigned int requestedDataArrayLength = datalen * 2;

Now the UOdinFmodAdapter calls the OnGenerateAudio function of the OdinMediaSoundGenerator. The generated sound is copied into the data directly.

const uint32 Result = SoundGenerator->OnGenerateAudio(data, (int32)requestedDataArrayLength);

If any error occurs in that call, the function will return without copying anything. Lastly, if everything worked as expected we finally return FMOD_OK to let FMOD know it can now use the data buffer.

if (odin_is_error(Result))
{
	FString ErrorString = UOdinFunctionLibrary::FormatError(Result, true);
	UE_LOG(LogTemp, Error, TEXT("UOdinFmodAdapter: Error during FillSamplesBuffer: %s"), *ErrorString);
	return FMOD_OK;
}
return FMOD_OK;

Alternative Approach: Programmer Sound

Warning

Disclaimer: Be aware that the implementation shown here uses Programmer Sounds of the FMOD Engine. While this allows real-time audio data, a big disadvantage of this approach is an increased latency by ~500ms.

This second approach uses Programmer Sounds of FMOD. It is easier to implement in your project as it does not directly write to audio channels and thus reduces management effort. As a downside, it has a rather high latency. If you want to avoid that, you will need to go for the first approach. The code for this approach can be found on the corresponding branch.

UOdinFmodAdapter

The header can be found here and the source file is located here

The UOdinFmodAdapter inherits from Unreal Engine’s UActorComponent.

You can either follow the Usage setup to drop the UOdinFmodAdapter directly into your project, or take a look at how it works to adjust the functionality to your requirements.

This is the header:

#pragma once

#include "CoreMinimal.h"
#include "FMODAudioComponent.h"
#include "fmod_studio.hpp"
#include "Components/ActorComponent.h"
#include "OdinFmodAdapter.generated.h"

class OdinMediaSoundGenerator;
class UOdinPlaybackMedia;

UCLASS(BlueprintType, Blueprintable, meta = (BlueprintSpawnableComponent))
class ODINTESTPROJECT_API UOdinFmodAdapter : public UActorComponent
{
	GENERATED_BODY()

public:	
	// Sets default values for this component's properties
	UOdinFmodAdapter();

	UFUNCTION(BlueprintCallable, Category = "Odin|Sound")
	void AssignOdinMedia(UPARAM(ref) UOdinPlaybackMedia*& Media);

	FMOD_RESULT pcmreadcallback(FMOD_SOUND* inSound, void* data, unsigned int datalen);

	void DestroyComponent(bool bPromoteChildren) override;

protected:

	UPROPERTY(BlueprintReadOnly, Category = "Odin|Sound")
	UOdinPlaybackMedia* PlaybackMedia = nullptr;
	TSharedPtr<OdinMediaSoundGenerator, ESPMode::ThreadSafe> SoundGenerator;

	float* Buffer = nullptr;
	unsigned int BufferSize = 0;


	FMOD::Sound* Sound = nullptr;
};

And this is the source file of the class:

#include "OdinUnrealSample.h"
#include "fmod_studio.hpp"
#include "FMODStudioModule.h"
#include "odin.h"
#include "OdinFunctionLibrary.h"
#include "OdinMediaSoundGenerator.h"
#include "OdinPlaybackMedia.h"
#include "OdinFmodAdapter.h"

static FMOD_RESULT F_CALLBACK onodinpcmreadcallback(FMOD_SOUND* inSound, void* data, unsigned int datalen)
{
	FMOD::Sound* sound = (FMOD::Sound*)inSound;
	void* userdata;

	sound->getUserData(&userdata);

	UOdinFmodAdapter* instance = reinterpret_cast<UOdinFmodAdapter*>(userdata);

	return instance->pcmreadcallback(inSound, data, datalen);
}

void UOdinFmodAdapter::AssignOdinMedia(UOdinPlaybackMedia*& Media)
{
	if (nullptr == Media)
		return;

	this->SoundGenerator = MakeShared<OdinMediaSoundGenerator, ESPMode::ThreadSafe>();
	this->PlaybackMedia = Media;

	SoundGenerator->SetOdinStream(Media->GetMediaHandle());
}

UOdinFmodAdapter::UOdinFmodAdapter()
{
	PrimaryComponentTick.bCanEverTick = true;

	FMOD::Studio::System* System = IFMODStudioModule::Get().GetStudioSystem(EFMODSystemContext::Runtime);
	FMOD::System* CoreSystem = nullptr;
	System->getCoreSystem(&CoreSystem);

	FMOD_CREATESOUNDEXINFO SoundInfo = { 0 };
	SoundInfo.cbsize = sizeof(FMOD_CREATESOUNDEXINFO);
	SoundInfo.format = FMOD_SOUND_FORMAT_PCMFLOAT;
	SoundInfo.defaultfrequency = 48000;
	SoundInfo.numchannels = 2;
	SoundInfo.pcmreadcallback = onodinpcmreadcallback;
	SoundInfo.length = (unsigned int)(48000 * sizeof(float) * 2);
	SoundInfo.userdata = this;

	if (CoreSystem->createStream("", FMOD_OPENUSER | FMOD_LOOP_NORMAL, &SoundInfo, &Sound) == FMOD_OK)
	{
		FMOD::ChannelGroup* group;
		CoreSystem->getMasterChannelGroup(&group);
		FMOD::Channel* channel;
		CoreSystem->playSound(Sound, group, false, &channel);
	}
}

FMOD_RESULT UOdinFmodAdapter::pcmreadcallback(FMOD_SOUND* inSound, void* data, unsigned int datalen)
{
	if(!data)
		return FMOD_ERR_INVALID_PARAM;

	if (!SoundGenerator || !PlaybackMedia)
		return FMOD_OK;

	unsigned int requestedDataArrayLength = datalen / sizeof(float);

	if (this->BufferSize < requestedDataArrayLength)
	{
		if (nullptr != Buffer)
			delete Buffer;
		Buffer = new float[requestedDataArrayLength];

		BufferSize = requestedDataArrayLength;
	}

	const uint32 Result = SoundGenerator->OnGenerateAudio(Buffer, (int32)requestedDataArrayLength);

	if (odin_is_error(Result))
	{
		FString ErrorString = UOdinFunctionLibrary::FormatError(Result, true);
		UE_LOG(LogTemp, Error, TEXT("UOdinFmodAdapter: Error during FillSamplesBuffer: %s"), *ErrorString);
		return FMOD_OK;
	}

	memcpy(data, Buffer, datalen);

	return FMOD_OK;
}

void UOdinFmodAdapter::DestroyComponent(bool bPromoteChildren)
{
	Super::DestroyComponent(bPromoteChildren);
	if (nullptr != Buffer)
	{
		delete Buffer;
		Buffer = nullptr;
		BufferSize = 0;
	}
}

PublicDependencyModuleNames.AddRange(
	new string[]
	{
        "Core",
        "Odin",
		"OdinLibrary"
	}
	);

PrivateDependencyModuleNames.AddRange(
	new string[] 
	{ 
		"FMODStudio" 
	}
	);

Usage

The above class uses the FMOD Audio Input Plugin to pass dynamically created Audio Data to the FMOD Engine. It does not need any steps in FMOD Studio and will simply work out of the box. All settings are done in the code.

Integrating the Input Component in your Unreal Project

In the next step we will now use the created Component to play back the incoming Odin Media Stream. Again you can find an example of this in the Odin Client Component of the sample project.

First replace the creation of an OdinSynthComponent that you have placed in the Odin Unreal Guide in your project with the new UOdinFmodAdapter.

You can see a sample of the Blueprint implementation below:

How it works

The above class uses the FMOD Programmer Sound API to pass dynamically created Audio Data to the FMOD Engine. It copies the incoming Audio Stream from Odin to the Input Buffer of the Programmer Sound by FMOD. This is done by creating an FMOD Programmer Sound and implementing a handler for the pcmreadcallback event.

1. Setup

This function creates a new pointer to a new OdinMediaSoundGenerator and sets its OdinStream to the incoming Media’s handle.

void UOdinFmodAdapter::AssignOdinMedia(UOdinPlaybackMedia*& Media)
{
	if (nullptr == Media)
		return;

	this->SoundGenerator = MakeShared<OdinMediaSoundGenerator, ESPMode::ThreadSafe>();
	this->PlaybackMedia = Media;

	SoundGenerator->SetOdinStream(Media->GetMediaHandle());
}

Next, in the Constructor of the Component we want to create and initialize the FMOD Programmer Sound. We get references to the FMOD Studio’s Core System that is responsible for creating and playing back sounds. With it, we can create the sound for our custom component.

FMOD::Studio::System* System = IFMODStudioModule::Get().GetStudioSystem(EFMODSystemContext::Runtime);
FMOD::System* CoreSystem = nullptr;
System->getCoreSystem(&CoreSystem);

To do so, we create a new SoundInfo and set its fields to what we need for our Odin playback:

SoundInfo.cbsize = sizeof(FMOD_CREATESOUNDEXINFO);
SoundInfo.format = FMOD_SOUND_FORMAT_PCMFLOAT;
SoundInfo.defaultfrequency = 48000;
SoundInfo.numchannels = 2;
SoundInfo.length = (unsigned int)(48000 * sizeof(float) * 2);
SoundInfo.pcmreadcallback = onodinpcmreadcallback;
SoundInfo.userdata = this;

defaultfrequency and numchannels are Odin’s default frequencies but choose any format that you want as long as you also initialize Odin’s sound system with it. The length (in bytes) is not very important, in the example we just use the sample rate and multiply it by the number of bytes per sample and the number of channels to get an effective length of one second.

The last two fields in the example are needed to handle the FMOD event to fill the sound buffer:

The pcmreadcallback needs to point to a static function of your class. Since the function is static, but we want to access the correct Odin SoundGenerator (which is a different one for each object of the class), we need to also pass a pointer to the correct instance of this Actor Component. Since the pcmreadcallback passes a reference to the created FMODSound, we can use its userdata field to pass any needed data. Here we just put a pointer to the ActorComponent that created the sound. Later we will use this to call the function on the correct instance, when the static pcmreadcallback event handler is called.

Lastly we will create the Programmer Sound via the CoreSystem->createStream function. We don’t need to pass it a name, but need to set the FMOD_OPENUSER and FMOD_LOOP_NORMAL flags. The first one is needed to mark the sound as a Programmer Sound, meaning that FMOD will call the pcmreadcallback in order to fill its buffer - this is opposed to other Sounds, where you would pass e.g. a Sound file. The second flag marks the stream as looping, so that the buffer is filled indefinitely. Otherwise the Sound would stop playing, after the number of samples specified in the length field has been played. Additionally we pass the addresses to our SoundInfo and a pointer to the created FMODSound. The latter is set in the function so that we can manipulate it later.

If the creation succeeds, we call the CoreSystem->playSound() function with the just created sound and are good to go!

if (CoreSystem->createStream("", FMOD_OPENUSER | FMOD_LOOP_NORMAL, &SoundInfo, &Sound) == FMOD_OK)
{
	FMOD::ChannelGroup* group;
	CoreSystem->getMasterChannelGroup(&group);
	FMOD::Channel* channel;
	CoreSystem->playSound(Sound, group, false, &channel);
}

2. Reading and Playing Back ODIN Audio Streams

The onodinpcmreadcallback function is called from the FMOD Stduio API whenever the playback requests more data for its buffer.

Since this function needs to be static, we need to get the userdata that we created the sound with and call the proper handler function on the instance that we reference here. We cast the FMOD_SOUND object to an FMOD::Sound (which is exactly the same kind of object but has different functions). On the cast object we can call getUserData() to get the pointer. We just need to reinterpret it as a pointer to a UOdinFmodAdapter component and then we can call its instance function pcmreadcallback with the given parameters from FMOD.

static FMOD_RESULT F_CALLBACK onodinpcmreadcallback(FMOD_SOUND* inSound, void* data, unsigned int datalen)
{
	FMOD::Sound* sound = (FMOD::Sound*)inSound;
	void* userdata;

	sound->getUserData(&userdata);

	UOdinFmodAdapter* instance = reinterpret_cast<UOdinFmodAdapter*>(userdata);

	return instance->pcmreadcallback(inSound, data, datalen);
}

In this function we first check if the component was set up properly and if the incoming parameters are valid.

if (!data)
	return FMOD_ERR_INVALID_PARAM;

if (!SoundGenerator || !PlaybackMedia)
	return FMOD_OK;

The datalength parameter indicates the number of bytes instead of samples, so in order to pass it Odin, we first need to calculate the number of samples from it. Afterwards we create a buffer with the according size or re-create it if the last used one was not big enough. This is then saved in an instance variable Buffer.

unsigned int requestedDataArrayLength = datalen / sizeof(float);

if (this->BufferSize < requestedDataArrayLength)
{
	if (nullptr != Buffer)
		delete Buffer;
	Buffer = new float[requestedDataArrayLength];

	BufferSize = requestedDataArrayLength;
}

Now the UOdinFmodAdapter calls the OnGenerateAudio function of the OdinMediaSoundGenerator. The generated sound is copied into the Buffer.

const uint32 Result = SoundGenerator->OnGenerateAudio(Buffer, (int32)requestedDataArrayLength);

If any error occurs in that call, the function will return without copying anything.

if (odin_is_error(Result))
{
	FString ErrorString = UOdinFunctionLibrary::FormatError(Result, true);
	UE_LOG(LogTemp, Error, TEXT("UOdinFmodAdapter: Error during FillSamplesBuffer: %s"), *ErrorString);
	return FMOD_OK;
}

Lastly, if everything worked as expected we finally copy the data in the Buffer over to the data buffer to pass it to FMOD. The function returns FMOD_OK to let FMOD know it can now use the data buffer.

memcpy(data, Buffer, datalen);

return FMOD_OK;

Conclusion

This simple implementation of an Odin to FMOD adapter for Unreal is a good starting point to give you the control over the audio playback that you need for your Odin Integration in your project. Feel free to check out the sample project in our public GitHub and re-use or extend any code to fit your specific needs.

This is only a starting point of your Odin Integration with FMOD and the Unreal Engine. Feel free to check out any other learning resources and adapt the material like needed, e.g. create realistic or out of this world dynamic immersive experiences with FMOD Spatial Audio aka “proximity chat” or “positional audio”:

Developer Documentation

FMOD and ODIN in Unreal

Introduction#

Getting Started#

Sample Project#

DSP Plugin#

UOdinFmodAdapter#

Usage#

Integrating the Input Component in your Unreal Project#

How it works#

1. Setup#

2. Reading and Playing Back ODIN Audio Streams#

Alternative Approach: Programmer Sound#

UOdinFmodAdapter#

Usage#

Integrating the Input Component in your Unreal Project#

How it works#

1. Setup#

2. Reading and Playing Back ODIN Audio Streams#

Conclusion#

Introduction

Getting Started

Sample Project

DSP Plugin

UOdinFmodAdapter

Usage

Integrating the Input Component in your Unreal Project

How it works

1. Setup

2. Reading and Playing Back ODIN Audio Streams

Alternative Approach: Programmer Sound

UOdinFmodAdapter

Usage

Integrating the Input Component in your Unreal Project

How it works

1. Setup

2. Reading and Playing Back ODIN Audio Streams

Conclusion