Azul_Audio: Multithreaded Engine
Real-Time Multihreaded Architecture
Azul_Audio is a custom multithreaded audio engine built on top of Microsoft’s XAudio2 API. The library includes support for various audio-based features, including:
-
Music streaming using dedicated thread for on-demand loading
-
User side sound options for modifying and/or retrieving volume, pitch, pan, and duration
-
User-defined synchronous and asynchronous file loading
-
Handle system for cross-thread protection
-
User-defined callback functionality
-
Automatic Priority system restricting active sound calls and providing duration tracking
-
Mutex-protected Circular Queue system for cross-thread communication
There are 4 separate threads launched at the initialization of the Azul_Audio Engine:
Azul_Audio(primary): The main audio processing thread. All commands are first processed through the Azul_Audio thread and dispatched accordingly. This is the primary means of communication with the main (or calling) thread of the application.
Auxillary: A helper thread utilized only within the processing of Azul_Audio commands. Helps commands enter and exit the Azul_Audio thread quickly, thereby improving performance. Handles internal actions such as terminating sounds within the system.
FileSystem: Handles synchronous and asynchronous file loading within the system. Ensures that other thread performance is not negatively affected by slow file I/O operations.
FileStreamingSystem: Dedicated entirely to ensuring the uninterrupted behavior of music streaming within the application.
The following videos highlight the concurrent running of Azul_Audio within the Azul Game Engine. This is to demonstrate the processing of Azul_Audio’s dedicated threads in unaffecting the performance of the Azul_Engine (main) thread. The threads present within Azul_Audio are represented within the output window on the right-hand side of the videos, as well as the commands they are each uniquely executing. This is to give a demonstration of the primary cross-thread communication which occurs in real-time. All of the sound calls are triggered within the input system using keyboard presses. All demos were captured while running in Debug mode.
Streaming Functionality
The streaming functionality available in Azul_Audio was engineered for ease-of-use, with measures taken to mitigate performance hits. To accomplish this, a dedicated thread was established with the sole purpose of executing unique streaming commands. The process begins with a streaming sound call, which sends a command to the Azul_Audio thread initiating the creation of the requested audio data. This command informs an internal manager to pre-fill several small audio data structures holding buffers containing parts of the file. The size of each of these audio buffers is 200KB,which denotes a fairly insignificant amount of audio data. This limited size was intentional when engineering the system, as to quickly jump in and out of threads. By only reading in small segments of the file, a single streaming sound does not unequally consume the entirety of the File Streaming thread’s processing.
As seen in the video, the primary threads involved in the streaming process are Azul_Audio and File Streaming. Since Azul_Audio delegates work on behalf of the entire system, the streaming commands simply pass through, with the actual work being performed on the File Streaming thread. At any given time when a sound is currently streaming, three data buffers remain resonant in memory. When an XAudio2 OnBufferEnd callback occurs, the oldest buffer is deleted. Immediately proceeding on the OnStreamEnd callback, the newest buffer is then submitted to the XAudio2 Source Voice. Finally, a new command is then sent to the File Streaming thread to read in another buffer’s worth of audio data from the given file. This process resembles a self-consuming “snake-like” behavior (Ouroboros), ensuring the system only retains the required amount of data in memory. This proves immensely useful in games or other real-time applications which don’t require specific sound calls to exhibit absolute precision (ambient or background music for example).
Starting Azul_Audio
Azul_Audio utilizes a Singleton Design Pattern to ensure it is only instantiated once. Because of this, the initialization, processing, and running of Azul_Audio may be encapsulated in three basic static function calls:
Azul_Audio::Initialize(const int Active_Sounds_Allowed): “Initialize” needs to be called within the initialization phase of the calling application. The thread which makes the initialize call will be the one with which the primary Azul_Audio thread communicates. “Initialize” will ensure the proper creation of the XAudio2 engine, as well as launching dedicated audio threads to handle processing within the system. Active_Sounds_Allowed refers to the initialization of the Priority Table which restricts active sound calls within the system. If unspecified, the value will be default-initialized.
Azul_Audio::Update(const int Time): “Update” must be called within the “Update” function of any real-time application. “Update” will pass the duration of the application to the audio engine, as well as process any user-end callbacks which occur.
Azul_Audio::Uninitialize(): “Uninitialize” will ensure that all audio engine assets are properly destroyed. This includes draining the circular queues of each dedicated system and cleanly terminating their executing threads.
Running the Engine
Asynchronous: This will send a command to Azul_Audio to start the loading of the file, but will not block the calling thread. More importantly, asynchronous loading within Azul_Audio allows the user to attach an optional callback, which will be executed when the file is fully processed. Asynchronous file loading is also specified in a call to the load function:
Streaming: While synchronous and asynchronous file loading are engineered with optimal system performance in mind, streaming is available for ease-of-use. Streaming is achieved not through the Azul_Audio interface, but directly through an instance of the Azul_Sound class as such:
Playing and Modifying Sounds
The above video demonstrates all of the key features present within the engine pertaining to Azul_Sound objects. These objects not only permit the user to play sounds within the system, but also to retrieve and modify behavior in real-time. This is achieved through several methods which send commands directly to the Azul_Audio processing thread including: Start, Stop, Pause, ChangeVolume, ChangePan, ChangePitch and ChangeLoop. Additional methods are included for queueing certain sound characteristics such as: GetTime, GetVolume, GetPan and GetPitch. Here is an example of how some of these methods are called:
Regarding the use of these methods in the above video, the first section simply showcases the synchronous loading triggered by a key press within the input system. This blocks the calling thread to load in all of the files to be used in the demo.
The second section highlights the ability to retrieve the internal time of the sound call durations and slowly modify the Pan and Volume over time. The classical music demo demonstrates two separate channels playing as two distinct sound calls in real-time, thus giving the illusion of one song. The duration of these sound calls is queued and printed to the screen, with one being ended prematurely to demonstrate the separate channels.
The third section highlights the user callback functionality which is available on each sound call. In this particular case, a callback is attached to execute when the stream ends, and prints specific sound call information to the output window. Here is an example of how this would be achieved:
The fourth section showcases the asynchronous file loading ability in conjunction with the system callback. In this instance, the file loading is attached with a callback which will inform the user when the audio data is ready to play. The callback will subsequently create and play a sound using the recently loaded file, which is then later retrieved on the original (calling) thread and terminated. Here is an example of how the user would accomplish something similar to this within the engine:
