Mixin / Mixout Audio Processing Architecture#

The Mixin / Mixout subsystem (implemented in src/audio/mixin_mixout/) provides the decoupled, multi-stream audio mixing and distribution architecture in Sound Open Firmware.

In modern audio systems, mixing multiple concurrent streams (such as media playback, navigation alerts, voice calls, and notification chimes) while routing them to disparate output endpoints (main speakers, headphones, and Acoustic Echo Cancellation loopback references) demands a flexible, non-blocking architecture.

Rather than relying on a legacy monolithic mixer that forces rigid scheduling across pipelines, SOF decomposes audio mixing into paired, asynchronously coordinated components: Mixin and Mixout.

This guide provides a comprehensive, high-level architectural walkthrough of the Mixin / Mixout subsystem, its direct-to-sink zero-intermediate-buffer accumulation mechanics, asynchronous pending frame tracking, per-sink gain attenuation, SIMD vector saturation, and underrun telemetry without delving into low-level C code.

—

1. Decoupled Mixing Paradigm: Monolithic Mixer vs Mixin / Mixout #

Traditional audio DSP architectures implement audio mixing using a single, monolithic Multi-Input Single-Output (MISO) mixer component. While conceptually straightforward, the monolithic mixer introduces severe architectural bottlenecks in modern multi-rate, multi-core audio DSPs.

Limitations of the Legacy Monolithic Mixer #

In a monolithic mixer architecture (e.g., src/audio/mixer/):

Tightly Coupled Scheduling: All upstream source pipelines must be synchronously locked to the exact same scheduling clock tick and period (e.g., 1 ms). If one client pipeline executes at a different cadence (e.g., 4 ms or 10 ms), the mixer stalls or suffers from buffer starvation.
Core Affinity Bottlenecks: A monolithic mixer component resides on a single DSP core. Mixing streams originated from different cores requires complex cross-core synchronization and frequent inter-processor interrupts (IDC), creating memory bus contention.
Rigid Buffer Locking: All input streams compete for buffer access within a single component processing pass. If one audio application experiences jitter or pauses, the entire mixer can block, causing audible glitches across all other running streams.

The Decoupled Mixin / Mixout Solution #

SOF resolves these challenges by separating the mixing function into two complementary modules:

Mixin Component: Serves as the terminal output endpoint of individual client and application pipelines. It executes independently within its own pipeline scheduling domain, consuming source audio and mixing it directly into target mixout buffers.
Mixout Component: Serves as the origin and primary synchronization point of downstream output pipelines (e.g., post-processing, equalization, and hardware digital audio interfaces). It coordinates buffer availability, ensures all connected mixins have contributed their data, and commits mixed audio frames downstream.

Figure 56 Architectural Comparison: Monolithic Mixer vs Decoupled Mixin / Mixout Paradigm#

—

2. Fan-Out & Fan-In Routing Topologies #

Real-world audio systems require complex many-to-many audio routing: a single media stream may need to play simultaneously on internal speakers, external headphones, and an echo cancellation reference monitor, while the main speakers simultaneously combine media, navigation voice guidance, and notification sounds.

Multi-Sink Fan-Out (Mixin Capabilities)#

Each Mixin module supports up to 3 independent output queues (sinks) (IPC4_MIXIN_MODULE_MAX_OUTPUT_QUEUES):

Sink 0: Primary output path routed to the Main Speakers Mixout.
Sink 1: Secondary output path routed to the Headphone Mixout.
Sink 2: Reference loopback path routed to an Acoustic Echo Cancellation (AEC) Mixout for real-time acoustic echo suppression.

Multi-Source Fan-In (Mixout Capabilities)#

Each Mixout module accepts up to 8 concurrent input queues (sources) (IPC4_MIXOUT_MODULE_MAX_INPUT_QUEUES):

Collects and mixes up to 8 active Mixin streams simultaneously.
Each connected Mixin stream can operate with distinct channel counts, independent gain attenuation factors, and custom channel remapping matrices.

Figure 57 Multi-Stream Fan-In & Fan-Out Routing Matrix (3 Sinks per Mixin, 8 Sources per Mixout)#

—

3. Direct-to-Sink Zero-Intermediate-Buffer Mixing Mechanics #

In a conventional multi-component audio pipeline, interconnecting multiple producers to a single consumer typically requires dedicated First-In First-Out (FIFO) intermediate ring buffers between every connection.

The Cost of Intermediate Buffering #

If 8 Mixins were connected to 3 Mixouts using intermediate buffers:

The system would require $8 \times 3 = 24$ distinct intermediate circular buffers.
In SRAM-constrained DSPs, allocating 24 separate audio buffers (each several kilobytes) severely fragments and depletes high-performance memory.
Every audio sample would be copied twice: first from Mixin into the intermediate FIFO, and then from the FIFO into the downstream pipeline, doubling memory bus bandwidth and cache thrashing.

The SOF Direct-to-Sink Accumulation Solution #

Sound Open Firmware completely eliminates intermediate buffers between Mixins and Mixouts. Mixing is performed directly inside the Mixout sink buffer:

First Mixin Execution: When the first active Mixin executes (mixin_process()), it detects that the Mixout sink buffer has no data present (mixed_frames == 0). It acquires the Mixout sink buffer (sink_get_buffer()) and directly copies its source audio into the buffer.
Subsequent Mixin Executions: When subsequent connected Mixins execute within the same period, they detect that audio frames already exist in the Mixout sink buffer. Rather than overwriting, they read the existing samples, add their scaled source samples to the buffer (accumulating), and write the sum back into the Mixout sink buffer.
Mixout Commit: When all connected Mixins have completed their processing, mixout_process() simply commits the accumulated audio buffer downstream (sink_commit_buffer()).

Figure 58 Direct-to-Sink In-Place Accumulation Sequence without Intermediate Buffers#

By performing mixing directly in the destination buffer, SOF eliminates intermediate FIFO allocations entirely and cuts memory bus read/write traffic by 50%.

—

4. Asynchronous Scheduling & Pending Frames Tracking #

In a multi-pipeline audio DSP graph, different pipelines may execute under different scheduling conditions:

Mixin pipelines may run under the Low-Latency (LL) 1 ms timer domain, while the Mixout pipeline runs under a 4 ms or 10 ms DMA domain.
Mixin and Mixout may reside on different DSP cores, communicating across core boundaries.

The Pending Frames Mechanism (`struct pending_frames`)#

Because Mixins consume source audio during mixin_process(), but sink audio cannot be committed until mixout_process() runs, there is an inherent temporal phase offset between consumption and production:

For each connected Mixin $leftrightarrow$ Mixout pair, the Mixout maintains a struct pending_frames entry.
When a Mixin processes and writes frames into the Mixout buffer, it increments its pending_frames->frames counter.
When mixout_process() runs, it evaluates the pending frames across all connected active Mixins:

\[\text{Frames to Produce} = \min_{k \in \text{Active Mixins}} \left( \text{pending\_frames}_k \right)\]
After committing the data downstream, mixout_process() decrements the pending frame counters by the committed amount.

Figure 59 Asynchronous Pipeline Execution and Pending Frames State Machine#

Autonomous Silence Generation on Starvation #

A critical challenge in audio mixing occurs when one input stream pauses, ends, or encounters an underrun while other streams remain active:

If the system waited for the starved stream to produce data, the entire Mixout pipeline would stall, causing audio dropouts across all active streams.
SOF resolves this via autonomous silence injection: If an active Mixin has zero source frames available (source_avail_frames == 0), it invokes silence().
The Mixin automatically fills its unmixed portion of the Mixout buffer with zeros, allowing the other connected Mixins to mix normally and ensuring continuous, glitch-free audio playback.

—

5. Per-Sink Gain Attenuation & Channel Remapping Engine #

Each connection between a Mixin and a destination Mixout can have unique acoustic requirements. For example, navigation speech routed to the driver’s speaker may require full volume, while simultaneously being ducked by -12 dB when routed to passenger speakers.

IPC4 Mixer Mode Configuration (`struct ipc4_mixer_mode_sink_config`)#

SOF allows independent gain and channel matrix configuration on every Mixin output queue:

10-Bit Fractional Gain Attenuation (``gain``):
- Gain is expressed as a 16-bit integer ranging from 0x0 (silence) to 0x400 (1024, representing $1.0$ or $0\text{ dB}$ unity gain).
- Samples are scaled by multiplying by gain and right-shifting by 10 bits:
  
  \[y_n = \frac{x_n \times \text{gain}}{1024}\]
Flexible Channel Remapping (``output_channel_map``):
- A 32-bit bitfield where each 4-bit nibble corresponds to an output destination channel, storing the index of the source channel to copy.
- A nibble value of 0xF designates that the output channel should be left unmodified.
- Enables dynamic downmixing (e.g. Stereo L/R downmixed to Mono center), channel duplication, or surround channel routing per destination Mixout.

Figure 60 Independent Per-Sink Gain Attenuation & Channel Remapping Engine#

—

6. SIMD Vector Acceleration & Saturation Arithmetic #

Mixing audio streams requires summing digital samples across multiple channels and sources:

\[S_{\text{mixed}}[n] = \sum_{k=1}^{N} \left( x_k[n] \times G_k \right)\]

If multiple high-amplitude signals are summed simultaneously, the resulting values can easily exceed the container limits ($+32767$ for 16-bit, $+2^{31}-1$ for 32-bit). Numerical overflow causes catastrophic acoustic clipping.

Hardware Saturation Protection #

To prevent integer wraparound, SOF’s mixing kernels apply saturation arithmetic:

\[S_{\text{clamped}}[n] = \text{clamp}\left( S_{\text{mixed}}[n], \text{MIN\_VAL}, \text{MAX\_VAL} \right)\]

SIMD Vector Implementations #

To maximize throughput and minimize battery consumption, SOF provides architecture-specific SIMD implementations:

Generic Portable C (``mixin_mixout_generic.c``): Portable scalar C reference with 64-bit integer accumulators and clamping, running on ARM Cortex-M and RISC-V.
Cadence Xtensa HiFi 3 (``mixin_mixout_hifi3.c``): Vectorized 24-bit and 32-bit SIMD instructions with automated hardware saturation.
Cadence Xtensa HiFi 5 (``mixin_mixout_hifi5.c``): 8-way 32-bit vector engine processing eight audio samples per clock cycle, utilizing dual 128-bit memory load/store operations.

Figure 61 SIMD Vector Accumulation and Saturation Clamping Architecture#

—

7. Telemetry, Underrun Rate-Limiting & End-of-Stream (EOS)#

Because the Mixin is the terminal component of client pipelines, it is the primary observer of stream starvation and playback completion.

Underrun Detection & Rate-Limiting #

When a client application fails to supply audio frames on time, the Mixin detects buffer exhaustion (source_avail_frames == 0):

To alert the host operating system, the Mixin generates an asynchronous underrun notification (send_mixer_underrun_notif_msg()).
Notification Flooding Prevention: If an application remains starved for multiple seconds, sending notifications on every 1 ms frame tick would overwhelm the host IPC queue with thousands of messages.
SOF applies underrun rate-limiting (underrun_notification_period, defaulting to 10 periods): Underrun notifications are throttled, delivering timely diagnostics without flooding the host driver.

End-of-Stream (EOS) Delay Compensation #

When an audio track finishes playback, the pipeline enters the End-of-Stream (EOS) state:

Signaling EOS immediately when the last sample reaches the Mixin would cause the host driver to stop the sound card while audio samples are still traversing downstream buffers and the hardware Digital-to-Analog Converter (DAC).
SOF queries the physical latency of downstream components (pipeline_get_dai_comp_latency()).
The Mixin delays emitting the final EOS notification until the remaining samples have cleared all downstream FIFOs and physically exited the speakers, guaranteeing that audio tracks are never prematurely truncated.

Figure 62 Mixin Telemetry, Rate-Limited Underrun Reporting, and EOS Flushing Sequence#

—