So RT’s current audio engine sucks. Big surprise. How do we fix it.
Preamble: levels of audio engine features
When talking about the features an audio engine has, there are multiple “levels” to it:
The raw audio processing logic and capabilities. This is purely technical: DSPs effects, math, audio output, audio loading, streaming systems, that kind of stuff.
Designer-focused systems to control the “why, what, when” of audio playback. This is stuff like “this area should have a different reverberation environment”, “limit the amount of sounds of a type that can play at once”, and “actually select which sound effect to play here”.
This post is about level 1. Level 2 discussions will happen in another thread.
Current status
We are currently using OpenAL for audio playback. This is an extremely dated and limited API. On Windows and Linux we use OpenAL Soft, which is still maintained to this day, and has many features on top of the base OpenAL spec. On macOS we are currently using Apple’s proprietary implementation, which has way more limitations.[1]
At the moment, RT currently exposes the following audio features:
A single audio listener in 2D space, perceived as if the game is a plane in 3D space with a configurable Z offset. Can also have velocity applied for some calculations.
Audio distance attentuation is configurable, but linear clamped is used by SS14.
Loading of .ogg Vorbis and .wav PCM audio files.
Streamed audio buffers (exclusively used by MIDI right now, there’s no support for streaming audio decompression).
Some number[2] of concurrently playing audio sources, with the following basic parameters:
Loop mode
Global or positioned in 2D space (incl. velocity for Doppler)
Pitch adjust
Volume/gain adjust (I don’t think the volume math makes any sense)
Maximum audio distance.
Reference distance and rolloff factor for audio attenuation.
Occlusion amount (fixed low-pass filter).
A reverb effect can be applied after 2D panning. I don’t think it’s even compatible with the occlusion low-pass and nothing in RT uses it, but the parameters are all from OpenAL’s EAX Reverb effect:
Can I request two features if your reworking the audio engine:
When you change output device the game will change to the new default device (currently you have to reload the whole game)
Midi-files having a volume slider to help you volume level songs with the level saving to your local device allowing musicains to volume level there playlist (currently there is no volume control for midi’s)
Even without Apple’s OpenAL limitations, OpenAL still sucks. For example you can’t have custom DSP effects, so if we wanted to consider Steam Audio integration that’d be impossible without modifying OpenAL Soft.
Steam Audio is not a full audio engine. It handles spatialization (reverb and other effects for presenting audio in 3D space), but does not include many of the features we’d otherwise need.
In my opinion as a sound designer (I cannot comment on the code feasibility of it) the biggest, fastest thing we could do to make the game sound better is to completely revamp the game’s acoustics. Real-time DSP would be amazing but I’m guesstimating that implementing it would be quite difficult - and based on the fact the game currently doesn’t have it, it can probably wait a bit. Most effects that can be done with real-time DSP can be faked by baking it into the sound at export - something that isn’t feasible with acoustics (not in an immersive way, at least). On the other hand, I can guess that acoustics will have a bigger, more tangible effect for a similar amount of effort.
The biggest difference-maker when doing linear sound design (in my experience) is positional audio and positional acoustics IE reverb and occlusion.
SS13 used some hacks to add context-aware reverb to simulate different spaces - it has a very tangible impact on the game’s soundscape.
Acoustics applies to everything, while DSP can only benefit a more limited set of use-cases.
Considering all of that, I think that we absolutely need to look into the feasibility of using Steam Audio’s C API for Robust Toolbox’s audio spatialization, acoustics, reverb, etc. I cannot stress this enough - Steam Audio is the best audio software at those 3 tasks I have literally ever seen. It is genuine witchcraft and we may have won the lottery with it’s open-source release.
I’m still working on making a demo reel with linear editing to put tangible sounds to the theoretical stuff, but I have to guess that out of everything I recreate, acoustics will have the biggest impact.
I can’t say for certain if using Steam Audio will be worth the opportunity cost of what we’ll lose out on in, but I can guess that it’s absolutely worth determining the amount of work a Steam Audio implementation will take.
