Key Phrase Buffer (KPB) & Wake-on-Voice (WoV) Architecture#
Sound Open Firmware (SOF) provides an autonomous, low-power audio architecture designed to support Wake-on-Voice (WoV) and always-listening acoustic keyword activation. In modern mobile laptops, smart home hubs, automotive cockpits, and wearable devices, users expect immediate responsiveness to spoken wake phrases (such as “Hey Computer” or “OK Assistant”). However, keeping the host application processor and PCIe/USB interconnects continuously awake to analyze ambient microphone audio would consume several watts of power, draining portable batteries in a matter of hours.
To resolve this challenge, modern acoustic architectures offload keyword spotting and voice activity detection to an ultra-low-power Digital Signal Processor (DSP) running SOF. While the host CPU remains in deep system sleep (such as ACPI S0ix / Modern Standby, S3 suspend-to-RAM, or S4 hibernation) drawing only microamperes, the audio DSP operates in an autonomous, power-optimized D0ix state.
A critical engineering obstacle in always-listening architectures is The Pre-Roll Dilemma: acoustic keyword spotters—whether running neural networks via TensorFlow Lite for Microcontrollers (TFLM) or proprietary vendor models—require an integration window of 500 ms to 1500 ms of spoken phonemes before achieving statistical confidence to trigger a detection event. Furthermore, waking the host CPU, resuming platform power rails, re-initializing PCIe/SoundWire DMA controllers, and starting host user-space capture pipelines introduces an additional system resume latency of 1000 ms to 2000 ms. If microphone audio is not buffered during this multi-second interval, the opening syllables of the user’s command (“Hey Computer, what is the weather?”) are permanently lost before host recording begins.
The Key Phrase Buffer (KPB) component (src/audio/kpb.c, COMP_KPB, UUID D8218443-5FF3-4A4C-B388-6CFE07B9562E) solves this problem by maintaining a continuous circular ring buffer of incoming microphone audio. Operating as a specialized dual-sink streaming engine, KPB simultaneously provides a real-time low-latency stream to local on-DSP keyword spotters and maintains a multi-second history buffer. Upon a keyword detection event, KPB transitions into an accelerated draining engine that burst-transfers the pre-roll history to the host DMA buffer before seamlessly handing off to real-time audio capture without dropping a single acoustic frame.
Figure 154 SOF Wake-on-Voice (WoV) System Architecture: Host Sleep, DSP D0ix & Wake Sequence#
Principles of Low-Power Wake-on-Voice & The Pre-Roll Dilemma#
In modern computing platforms, acoustic energy efficiency is governed by the operational power consumption of different platform processing tiers:
Platform Power Tier |
Typical Power |
Wake Latency |
Active Audio Processing Capabilities |
|---|---|---|---|
Host Active (S0) |
10 W – 45 W |
0 ms (running) |
Full desktop OS, cloud streaming, complex large language models, high-resolution rendering. |
Host Modern Standby (S0ix) |
500 mW – 1.5 W |
500 ms – 1500 ms |
Host cores in deep C-states; PCIe, DRAM controllers, and display engines clock-gated. |
Host Suspend-to-RAM (S3) |
100 mW – 300 mW |
1000 ms – 2500 ms |
Host completely powered off except DRAM refresh logic; interconnects dormant. |
DSP Low-Power Mode (D0ix) |
3 mW – 12 mW |
< 1 ms |
Primary DSP core running at reduced clock frequency (e.g. 24 MHz – 38.4 MHz); autonomous DMIC audio capture, low-power Voice Activity Detection (VAD), and keyword spotters. |
The Pre-Roll Timing Equation#
To understand the necessity of historical buffering, consider the chronological progression of a voice activation sequence:
Acoustic Speech Commencement (\(t = t_0\)): The user begins uttering the activation phrase (“Hey Computer”).
Voice Activity Detection (\(t = t_0 + \Delta t_{\text{VAD}}\)): Energy-based or spectral VAD algorithms detect acoustic activity above background ambient noise (\(\approx 50\text{--}150\text{ ms}\)).
Keyword Model Inference Latency (\(t = t_0 + \Delta t_{\text{KWD}}\)): The acoustic keyword classifier integrates temporal audio frames over a multi-layer neural network or acoustic model. Because phonetic recognition requires sufficient acoustic context across syllables, confident detection occurs near the end of the phrase (\(\Delta t_{\text{KWD}} \approx 800\text{--}1500\text{ ms}\)).
Host Wakeup & Platform Rail Settlement (\(t = t_0 + \Delta t_{\text{KWD}} + \Delta t_{\text{wake}}\)): Upon keyword detection, the DSP asserts a platform interrupt (IPC or PCIe MSI). The host power management IC (PMIC) ramps platform voltage rails, DRAM exits self-refresh, the kernel resumes, and the ALSA audio driver invokes hardware parameters and stream prepare (\(\Delta t_{\text{wake}} \approx 800\text{--}2000\text{ ms}\)).
Host DMA Capture Activation (\(t = t_0 + \Delta t_{\text{total\_latency}}\)): The host application initiates reading from the ALSA capture device (e.g.
arecord).
The cumulative latency before the host application begins receiving audio data is:
If \(\Delta t_{\text{KWD}} = 1200\text{ ms}\) and \(\Delta t_{\text{wake}} = 1500\text{ ms}\), the total elapsed duration is \(2700\text{ ms}\). Without a circular buffer holding at least \(2.7\text{ seconds}\) of historical microphone data, the entire wake word and the initial segment of the user command would be completely lost.
The KPB component eliminates this data loss by continuously recording into a dedicated circular history buffer in DSP SRAM while the host is asleep. When the host resumes and initiates capture, KPB transfers this buffered historical speech into the host DMA buffer at accelerated speed before transitioning seamlessly to real-time audio.
KPB Component State Machine & Execution Lifecycle#
The KPB component is implemented as an audio processing module conforming to the SOF component driver interface. Internally, KPB maintains ten discrete states that govern its execution during audio streaming, buffer writing, trigger events, and draining.
State Enumeration |
Value |
Functional Role & Operational Behavior |
|---|---|---|
|
0 |
Initial unconfigured state prior to memory allocation and pipeline initialization. |
|
1 |
Ephemeral cleanup state entered when a reset interrupt interrupts an ongoing buffering or draining operation. |
|
2 |
Module instance allocated, driver private data initialized, and unique identifier (UUID) assigned. |
|
3 |
Validation of sampling rate (16 kHz), container width, channel count, and circular history buffer allocation during |
|
4 |
Normal listening mode. Incoming DMIC frames are copied to the internal history buffer and simultaneously forwarded to the active real-time selector sink (pin 0). |
|
5 |
Transient state entered within |
|
6 |
Triggered by client detection event. Locks state, calculates backward read pointer in history rings, and prepares asynchronous draining task. |
|
7 |
Asynchronous draining active. The background draining task reads historical audio from the ring buffer and copies it to the host sink at accelerated speed. |
|
8 |
Draining completed (“draining on demand”). History buffer is emptied, and incoming real-time audio is copied directly to the host capture sink without latency. |
|
9 |
Teardown requested via pipeline trigger stop or reset command. Halts background tasks and frees resources. |
Figure 155 KPB Component State Machine (10 Lifecycle States: Reset, Run, Buffering, Draining, and Host Copy)#
Lifecycle Transitions Walkthrough#
Initialization & Preparation: When the audio pipeline is configured via topology,
kpb_new()transitions the module toKPB_STATE_CREATED. Upon receiving the IPC hardware parameters and prepare commands,kpb_prepare()verifies that the sampling frequency is 16 kHz and allocates the circular history buffers in DSP internal SRAM, moving toKPB_STATE_PREPARING.Normal Listening (RUN & BUFFERING): Upon receiving
COMP_TRIGGER_START, the state transitions toKPB_STATE_RUN. Each time the pipeline period executes,kpb_copy()inspects the source DMIC buffer. Audio samples are copied to the active real-time selector sink (pin 0) if downstream components (the keyword spotter) are inCOMP_STATE_ACTIVE. Simultaneously, KPB temporarily entersKPB_STATE_BUFFERINGto append the incoming PCM frames to the circular history buffer before reverting toKPB_STATE_RUN.Keyword Trigger & Draining Initialization: When the keyword classifier identifies the activation phrase, it emits a notification event (
KPB_EVENT_BEGIN_DRAINING). KPB locks its private spinlock/mutex and entersKPB_STATE_INIT_DRAINING. The component calculates the historical read pointer offset corresponding to the requested pre-roll duration, locks available buffer headroom, pauses the real-time selector sink, and launches an asynchronous draining task.Accelerated Burst Draining: In
KPB_STATE_DRAINING, the draining task executes at an accelerated cadence (e.g. \(2\times\) to \(4\times\) real-time speed), reading from the historical read pointer and writing to the host sink buffer (pin 1). If new real-time microphone samples arrive during draining, they are buffered into the history ring while a running counter (buffered_while_draining) extends the total remaining draining requirement.Real-Time Handoff (HOST_COPY): Once the historical buffer is completely drained and all accumulated audio frames have been transferred, KPB transitions to
KPB_STATE_HOST_COPY. In this state, the circular history buffer is bypassed, and new incoming microphone frames are copied directly to the host capture sink in real time, guaranteeing zero-latency streaming to the host voice recognition application.
History Circular Ring Buffer Architecture & Mathematics#
The KPB storage engine is built around a chained linked list of circular history buffers:
In standard SOF configurations, the ring comprises two distinct buffers (KPB_NO_OF_HISTORY_BUFFERS = 2) managed by struct history_buffer:
struct history_buffer {
enum buffer_state state; /* KPB_BUFFER_FREE, KPB_BUFFER_FULL, KPB_BUFFER_OFF */
void *start_addr; /* Base memory address of buffer in DSP SRAM */
void *end_addr; /* Upper boundary address (start_addr + size) */
void *w_ptr; /* Current write pointer */
void *r_ptr; /* Current read pointer for draining */
struct history_buffer *next; /* Pointer to next ring segment */
struct history_buffer *prev; /* Pointer to previous ring segment */
};
Mathematical Buffer Sizing Equations#
The memory footprint of the KPB history buffer is determined by four platform configuration parameters:
Sampling frequency (\(f_s\), strictly 16,000 Hz for voice keyword processing).
Audio channel count (\(N_{\text{ch}}\), typically 2 to 6 channels).
Sample container width (\(W_{\text{container}}\), 16 bits or 32 bits).
Target historical buffer duration (\(T_{\text{buff}}\), in milliseconds).
The sample container size is defined as:
The required history buffer capacity \(S_{\text{buff}}\) in bytes is derived as:
Platform Target |
Channels (\(N_{\text{ch}}\)) |
Width (\(W_{\text{sample}}\)) |
History (\(T_{\text{buff}}\)) |
Total Allocated Memory |
|---|---|---|---|---|
Tiger Lake (TGL) |
2 (Stereo) |
16-bit |
3000 ms |
\(16 \times 2 \times 2 \times 3000 = 192{,}000\text{ bytes} \approx 187.5\text{ KB}\) |
Tiger Lake (TGL) |
4 (Quad) |
16-bit |
3000 ms |
\(16 \times 2 \times 4 \times 3000 = 384{,}000\text{ bytes} \approx 375.0\text{ KB}\) |
Generic CAVS / ACE |
2 (Stereo) |
16-bit |
2100 ms |
\(16 \times 2 \times 2 \times 2100 = 134{,}400\text{ bytes} \approx 131.25\text{ KB}\) |
Generic CAVS / ACE |
4 (Quad) |
32-bit |
2100 ms |
\(16 \times 4 \times 4 \times 2100 = 537{,}600\text{ bytes} \approx 525.0\text{ KB}\) |
Figure 156 Dual-Sink Buffer Architecture: Continuous Keyword Detector Feed vs Burst Draining Host Sink#
Pointer Mechanics & Overwrite Protection#
During normal listening (KPB_STATE_RUN), the write pointer (w_ptr) advances sequentially through the memory of the active buffer. When w_ptr reaches end_addr, the buffer state is flagged as KPB_BUFFER_FULL, the write pointer is reset to start_addr of the subsequent buffer (buff->next), and writing continues without disruption.
When a keyword trigger initiates draining of \(B_{\text{req}}\) bytes, the read pointer \(P_{\text{read}}\) must be positioned exactly \(B_{\text{req}}\) bytes behind the current write pointer \(P_{\text{write}}\) across the circular buffer boundaries:
To prevent newly arriving microphone audio from overwriting history samples that are staged for host draining, KPB dynamically clamps its writable headroom:
As the draining task reads and emits audio to the host sink, it increments kpb->hd.free, restoring writable memory space in exact synchrony with host consumption.
Figure 157 History Circular Ring Buffer Pointer Mechanics: Pre-Roll Window, Wrap Safety & Overwrite Protection#
Dual-Sink Architecture & Microphone Channel Selection#
The KPB component is architected with dual output pins (num_output_pins = 2):
Pin 0: Real-Time Selector Sink (``sel_sink``, ``REALTIME_PIN_ID``): This sink is dedicated to low-latency processing and feeds local on-DSP keyword detection engines (e.g. TFLM, MFCC feature extractors, or vendor detection algorithms). During normal system sleep, audio is delivered directly to Pin 0 on every pipeline period.
Pin 1: Host Draining Sink (``host_sink``): This sink connects to the host capture pipeline through downstream volume and copier components. During host sleep, Pin 1 remains inactive and paused. Upon a keyword activation event, Pin 1 receives the burst-drained pre-roll historical audio and subsequent live microphone speech.
Microphone Channel Selection (MicSelector)#
In modern platforms equipped with digital microphone arrays (such as 3-mic or 4-mic beamforming arrays with reference loopback channels), passing the full multi-channel stream to the keyword detector during low-power sleep would waste substantial memory bandwidth and DSP processing cycles.
To minimize energy consumption, KPB incorporates an integrated microphone channel selector (kpb_micselector_config, configured via IPC4 parameter KP_BUF_CLIENT_MIC_SELECT):
struct kpb_micselector_config {
uint32_t mask; /* Channel selection bitmask */
};
When kpb->num_of_sel_mic is configured (e.g. selecting channel 0 or channel 1 via bitmask 0x01 or 0x02), KPB automatically demultiplexes and extracts only the designated voice microphone channel when copying to the real-time sink (Pin 0). Meanwhile, the full multi-channel stream is preserved intact in the circular history buffer, ensuring that when the host wakes up, beamforming and multi-channel noise suppression algorithms have access to all physical microphone signals for high-fidelity speech recognition.
Accelerated Burst Draining & Dynamic Pace Adjustment#
When a keyword trigger initiates host streaming, transferring historical data at standard real-time speed (\(1\times\)) would be inadequate: if the host resumes 2 seconds after the trigger, draining 2 seconds of pre-roll at \(1\times\) speed would mean the host remains perpetually 2 seconds behind real-time audio.
To eliminate this lag, KPB executes an asynchronous Burst Draining Task (kpb_draining_task) scheduled via the SOF Earliest Deadline First (EDF) scheduler. The draining task empties the history buffer at a multiple of real-time speed before transitioning seamlessly into live streaming.
Figure 158 Accelerated Burst Draining Timeline & Dynamic Interval Adjustment (FMT vs Real-Time Hand-off)#
Synchronized Draining & Dynamic Pace Adjustment#
SOF supports two operational draining modes:
Unsynchronized (Unlimited) Draining: Audio samples are copied to the host sink buffer as fast as downstream memory and DMA allow, constrained only by available sink space.
Synchronized Draining (``sync_draining_mode``): Draining is paced to prevent overflowing host DMA ring buffers while remaining significantly faster than real-time consumption. The target interval is governed by:
\[I_{\text{drain}} = \frac{T_{\text{host\_period}}}{M_{\text{drain}}}\]where \(M_{\text{drain}} = \text{KPB\_DRAIN\_NUM\_OF\_PPL\_PERIODS\_AT\_ONCE} = 2\). Draining operates at double the normal pipeline period rate.
Dynamic Pace Regulation Algorithm#
Because host interrupt response and DMA scheduling exhibit jitter, KPB incorporates an adaptive pace controller (adjust_drain_interval) evaluated every 32 task iterations using 64-bit DSP wall-clock cycles (sof_cycle_get_64()):
If \(P_{\text{actual}} < P_{\text{optimal}}\) (draining is falling behind target pace), the drain interval is reduced:
Conversely, if \(P_{\text{actual}} > P_{\text{optimal}}\), the interval is lengthened proportionally, maintaining stable DMA buffer levels without underrun or overrun.
Fast Mode Task (FMT) Pipeline Infrastructure#
In complex audio graphs, intermediate components (such as Gain/Volume widgets or PCM Format Converters) may sit between KPB and the Host DMA Copier. Under standard scheduling, these intermediate modules execute only once per pipeline period (e.g. every 1 ms or 4 ms).
To prevent these intermediate modules from throttling burst draining, SOF implements the Fast Mode Task (FMT) framework (struct fast_mode_task, configured via IPC4 parameter KP_BUF_CFG_FM_MODULE). FMT registers downstream modules into an accelerated execution list, triggering their processing routines in direct synchronization with KPB burst cycles until pre-roll draining finishes.
Event Notification Framework: IPC3 Notifiers vs IPC4 AMS#
Communication between keyword spotters, client pipelines, and the KPB component differs across SOF IPC architectures:
Figure 159 Event Notification Architecture: IPC3 Notifier Dispatch vs IPC4 Asynchronous Message Service (AMS)#
IPC3 Notifier Implementation#
In IPC3 topologies, communication between the detection module and KPB relies on the internal core notifier system:
enum kpb_event {
KPB_EVENT_REGISTER_CLIENT = 0,
KPB_EVENT_UPDATE_PARAMS,
KPB_EVENT_BEGIN_DRAINING,
KPB_EVENT_STOP_DRAINING,
KPB_EVENT_UNREGISTER_CLIENT,
};
Clients (such as detect_test) register with KPB by passing KPB_EVENT_REGISTER_CLIENT along with their requested history draining window (drain_req, up to KPB_MAX_DRAINING_REQ = 2000 ms to 3000 ms). When the keyword model confirms an utterance match, it fires KPB_EVENT_BEGIN_DRAINING, causing KPB to calculate the historical read pointer and start the draining task.
IPC4 Asynchronous Message Service (AMS)#
Under IPC4, inter-module signaling leverages the Asynchronous Message Service (AMS) (CONFIG_AMS). Modules communicate via standardized large configuration parameters:
KP_BUF_CFG_FM_MODULE(Parameter ID 1): Configures the list of downstream modules participating in the Fast Mode Task during accelerated pre-roll draining.KP_BUF_CLIENT_MIC_SELECT(Parameter ID 11): Updates the real-time microphone channel selection mask without tearing down active audio pipelines.
Linux Driver & DAPM Control Sequencing#
On the Linux host, keyword detection pipelines are managed through ALSA Dynamic Audio Power Management (DAPM). Two intertwined pipelines are constructed:
Pipeline 8 (Host Capture Pipeline): DMIC \(\to\) Volume \(\to\) KPB \(\to\) Host Copier \(\to\) ALSA PCM capture device.
Pipeline 9 (Keyword Detect Pipeline): KPB Pin 0 \(\to\) Selector \(\to\) Detector Module \(\to\) Virtual Detector Sink.
Stream Control Action |
Host Pipeline (Pipe 8) |
Detector Pipeline (Pipe 9) |
Operational Hardware State |
|---|---|---|---|
1. HW Parameters |
|
|
DSP sets 16 kHz sampling, validates minimum host buffer (\(\ge 67200\text{ frames}\)). |
2. Trigger Start |
Host suspended |
Pipeline 9 Started |
DSP enters D0ix; KPB buffers incoming audio; Detector continuously scans. |
3. Keyword Detected |
Host resumes via IRQ |
Draining triggered |
KPB empties pre-roll history to host DMA; transitions to live copy. |
4. Capture Stop |
|
|
Host application finishes reading speech command; pipeline resets to listening state. |
End-to-End WoV System Pipeline & Topology 2 Wiring#
The integration of KPB within an end-to-end Sound Open Firmware audio graph is illustrated in Figure 201:
Figure 160 End-to-End WoV Audio Graph: DMIC Array, DC Blocker, KPB, Keyword Spotter & Host DMA Copier#
Topology 2 Widget Declaration#
In ALSA Topology 2 (tools/topology/topology2/include/components/kpb.conf), the KPB widget is declared as an effect class with one input pin and two output pins:
Class.Widget."kpb" {
DefineAttribute."index" {}
DefineAttribute."instance" {}
DefineAttribute."cpc" {
token_ref "comp.word"
}
<include/components/widget-common.conf>
attributes {
!constructor [
"index"
"instance"
]
!mandatory [
"no_pm"
"uuid"
]
!immutable [
"uuid"
]
unique "instance"
}
type "effect"
num_input_audio_formats 1
num_output_audio_formats 1
# UUID: D8218443-5FF3-4A4C-B388-6CFE07B9562E
uuid "43:84:21:d8:f3:5f:4c:4a:b3:88:6c:fe:07:b9:56:2e"
no_pm "true"
cpc 720000
num_input_pins 1
num_output_pins 2
}
Backend Pipeline Integration#
In tools/topology/topology2/include/pipelines/cavs/dai-kpb-be.conf, the KPB widget is instantiated downstream of the DAI copier:
Object.Widget.kpb."1" {
index $DRAINING_PIPELINE_ID
num_input_audio_formats 2
num_output_audio_formats 2
Object.Base.input_audio_format [
{
in_rate 16000
in_bit_depth 32
in_valid_bit_depth 32
}
{
in_rate 16000
in_channels 4
in_bit_depth 32
in_valid_bit_depth 32
in_ch_cfg $CHANNEL_CONFIG_3_POINT_1
}
]
}
Host Buffer Sizing Requirements & Best Practices#
Important
Host DMA Buffer Sizing: Platform resume from ACPI S0ix / Modern Standby requires between 1000 ms and 2000 ms under typical operating conditions. To ensure that pre-roll historical audio is not overwritten before the host application begins consuming samples, the ALSA capture buffer must be dimensioned adequately:
The host
buffer-sizemust be configured to at least 67,200 frames (\(\approx 4.2\text{ seconds}\) at 16 kHz).Host capture should be invoked with memory-mapped non-blocking I/O:
arecord -Dhw:0,8 -M -N -c 2 -f S16_LE -r 16000 --buffer-size=68000 capture.wav -vvv
Smaller buffer allocations will be rejected by the SOF firmware during the
hw_paramsvalidation stage with an-EINVALerror to prevent buffer overrun corruption.