Scheduler Architecture

The Scheduling Infrastructure in Sound Open Firmware (SOF) is the real-time engine responsible for orchestrating task execution across multi-core Digital Signal Processors (DSPs). Deeply integrated with the underlying Zephyr RTOS, SOF utilizes a multi-tiered scheduling model to satisfy contrasting computing demands: deterministic, sub-millisecond low-latency audio streaming alongside heavy, variable-duration algorithmic processing (such as Echo Cancellation, Beamforming, and Machine Learning inference).

This guide provides a high-level conceptual overview of the three scheduling domains, hardware interrupt triggers, Earliest Deadline First (EDF) mechanics, cycle budgeting, multi-core affinity, and power-saving tickless idle operation without focusing on low-level C code.

—

1. Multi-Tier Scheduling Architecture

Why Audio DSPs Require Multiple Scheduling Domains

Audio signal processing imposes conflicting real-time constraints on embedded DSP systems:

1. Deterministic Low-Latency Streaming: Hardware audio interfaces (I2S, SoundWire, HDA) and Host DMA controllers transfer audio in strict, repetitive time slots (typically 1 ms or sub-millisecond frames). Missing a single deadline causes audible glitches, buffer underruns, or audio dropouts.

2. Heavy, Variable-Duration Computation: Algorithms like Acoustic Echo Cancellation (AEC), multi-microphone beamforming, noise suppression, and neural network inference require millions of math operations. Processing times vary dynamically depending on acoustic convergence and input features.

A single flat scheduling model cannot satisfy both needs. If heavy compute algorithms ran synchronously on the audio interrupt tick, they would delay I/O transfers and cause buffer underruns. Conversely, if all audio transfers ran in standard cooperative OS threads, scheduling jitter would break strict timing guarantees.

To solve this, SOF implements a three-tier scheduling architecture on top of the Zephyr RTOS:

Figure 20 Multi-Tier Scheduling Architecture: Hardware Triggers to RTOS Execution

The Three Scheduling Domains

1. Low-Latency (LL) Domain:

   • Execution Model: Deterministic, synchronous task execution within a high-priority Zephyr domain thread pinned to each core.

   • Trigger: Hardware timer ticks (typically 1 ms or 10 ms) or hardware DMA completion interrupts.

   • Characteristics: Strict priority ordering, sub-millisecond execution deadlines, zero thread-context switching between internal tasks, and hard real-time guarantees.

   • Typical Workloads: Host DMA transfers, DAI endpoints (I2S, SoundWire), mixers, linear volume controls, and sample format conversions.

2. Data Processing (DP) Domain:

   • Execution Model: Asynchronous, multithreaded processing where each DP task runs inside its own dedicated Zephyr RTOS thread.

   • Trigger: Buffer threshold readiness (when sufficient input frames are present and downstream output space is available), evaluated at the end of each LL tick.

   • Characteristics: Employs Zephyr’s Earliest Deadline First (EDF) scheduler. Deadlines are calculated dynamically based on frame sizes and stream sample rates.

   • Typical Workloads: Acoustic Echo Cancellation (AEC), Time-Domain Fixed Beamforming (TDFB), noise suppression, parametric equalizers with large FIR filter taps, and TensorFlow Lite Micro (TFLM) neural networks.

3. Thread With Budget (TWB) Domain:

   • Execution Model: Sandboxed time-sliced execution using Zephyr thread time slicing and hardware cycle accounting.

   • Trigger: Periodic scheduling with an allocated cycle budget per tick.

   • Characteristics: Prevents CPU starvation. If a task exceeds its budgeted cycle quota before completing its chunk, the kernel invokes a callback that immediately demotes the thread to a background priority. The budget is replenished on the subsequent LL tick.

   • Typical Workloads: Background room acoustic calibration, non-critical telemetry, diagnostic probes, and low-priority algorithmic tasks.

—

2. Low-Latency (LL) Scheduler Domain

The Low-Latency scheduler is the real-time backbone of SOF. Designed for minimal latency and jitter, it bypasses generic OS thread context switching for its child tasks by multiplexing all low-latency work within a single, dedicated high-priority Zephyr thread pinned to each DSP core.

Domain Architecture & Execution Sequence

Figure 21 Low-Latency Domain Execution Flow: Hardware Interrupt to Task Dispatch

Key Design Characteristics

• Pinned Core Domains: Each active DSP core runs an independent LL domain thread (such as ll_thread0 on Core 0 and ll_thread1 on Core 1). This ensures that core-local audio processing never suffers from inter-core cache invalidation or cross-core spinlock contention.

• Synchronous Task Iteration: When the domain wakes up, it walks through all queued tasks in strict priority order. Because tasks are invoked via direct C function calls rather than thread yields, context-switching overhead is virtually zero.

• Deterministic Timing: Tasks within the LL domain must complete within a fraction of the period window (e.g., within 200 µs of a 1 ms tick), leaving sufficient DSP headroom for data processing threads and low-power sleep states.

• Post-Run Hook: At the completion of each LL task walk, the scheduler invokes a post-run hook. This hook triggers readiness evaluations for the Data Processing (DP) and Thread With Budget (TWB) domains.

LL Task State Machine

Tasks registered with the LL scheduler progress through an operational lifecycle:

Figure 22 Low-Latency Task Lifecycle and State Transitions

—

3. Data Processing (DP) Scheduler Domain

While the Low-Latency domain handles time-critical I/O movement, the Data Processing (DP) domain manages compute-heavy algorithms that require multi-millisecond or variable execution times.

Thread-Per-Task Architecture

Unlike the LL domain, which serializes all tasks within a single domain thread, each DP task executes inside its own dedicated Zephyr RTOS thread. This decoupling ensures that:

1. A slow or complex algorithm running on one stream cannot block or delay audio streaming on other pipelines.

2. The Zephyr kernel can preempt a running DP thread whenever an LL timer tick or hardware DMA interrupt arrives.

3. Compute tasks can be prioritized dynamically based on their actual consumption deadlines.

Readiness Evaluation & Earliest Deadline First (EDF)

The DP scheduling cycle is driven by data availability and buffer space rather than a rigid clock tick:

Figure 23 Data Processing (DP) Execution Workflow with Buffer Readiness and EDF

Dynamic Deadline Calculation

In real-time audio, a deadline is the point in time when the next buffer consumer (such as the speaker output DMA) will starve if new samples are not delivered.

The DP scheduler dynamically calculates the task’s absolute deadline timestamp based on:

\[\text{Deadline} = \text{Current Time} + \frac{\text{Frames in Buffer}}{\text{Sampling Frequency}} - \text{Safety Margin}\]

When the DP thread wakes up, it passes this absolute timestamp to Zephyr via k_thread_absolute_deadline_set(). Zephyr’s EDF kernel prioritizes threads whose deadlines are closest to expiring, automatically resolving scheduling conflicts between competing audio streams.

—

4. Thread With Budget (TWB) Scheduler Domain

The Thread With Budget (TWB) domain is designed for non-critical, intensive, or bursty computational tasks where starvation of the primary audio pipelines must be strictly prevented.

The Challenge of Unbounded Compute

Certain audio algorithms—such as acoustic space measurement, complex FIR filter calculation, or machine learning background training—can consume substantial DSP cycles. If a high-priority thread runs without restriction, it can starve lower-priority system tasks or monopolize the processor, preventing other pipelines from meeting their deadlines.

Time Slicing & Cycle Demotion Model

The TWB domain combines Zephyr RTOS time slicing with hardware cycle accounting:

Figure 24 Thread With Budget (TWB) Priority Demotion and Replenishment Cycle

How TWB Protects System Integrity

1. Cycle Quota Allocation: When a TWB task is scheduled, its budget is configured in OS ticks via k_thread_time_slice_set(). The runtime converts this into equivalent DSP hardware cycles.

2. Autonomous Kernel Demotion: If the task runs continuously and depletes its cycle quota before completing its current unit of work, the Zephyr kernel triggers scheduler_twb_task_cb(). This callback immediately lowers the thread’s priority to a background level.

3. Audio Chain Protection: In the background state, the demoted task can only run when all LL audio streaming tasks and DP algorithms have completed their work.

4. Periodic Priority Restoration: On the subsequent LL timer tick, the scheduler invokes scheduler_twb_ll_tick(). This resets the consumed cycle counter, restores the thread’s high priority, and re-enables its time slice.

—

5. Multi-Core Scheduling & Core Affinity

Modern Intel and partner audio DSPs feature multi-core architectures (Dual-Core, Quad-Core, or Octa-Core). Sound Open Firmware leverages multi-core processing by statically partitioning scheduling domains across physical cores.

Core Affinity Model

Figure 25 Multi-Core Scheduling Topology: Core 0 (I/O) and Core 1 (Compute Offload)

Principles of Multi-Core Distribution

1. Dedicated Core 0 for I/O & Host Communications: Core 0 typically manages all Host IPC messaging, DMA gateways, and low-latency audio hardware links. Pinning I/O tasks to Core 0 guarantees uninterrupted streaming and immediate host response times.

2. Compute Offload to Secondary Cores: Complex algorithms (AEC, beamforming, ML models) are assigned to secondary cores (Core 1, Core 2, Core 3). This shields real-time audio links on Core 0 from heavy algorithmic processing spikes.

3. Cross-Core Ring Buffers: Data movement between cores occurs through shared circular buffers located in shared DSP memory. Producers and consumers synchronize using cache-coherent read/write pointers.

4. Inter-Domain Communication (IDC): When Core 0 deposits audio frames into a shared cross-core buffer, it signals the destination core via hardware doorbell interrupts (IDC). This awakens the secondary core’s LL domain thread without polling or spinlocks.

—

6. Task Prioritization & Deadline Calculation

Task Priorities

Within each scheduling domain, tasks are assigned explicit priority values:

Priority Level	Domain	Typical Component Assignment
Critical / High	LL Domain	Hardware DAI copy, Host DMA gateway reader/writer, clock synchronization.
Medium	LL Domain / DP Domain	Real-time mixers, standard volume controls, sample rate converters.
Low / Dynamic	DP Domain (EDF)	Asynchronous filter updates, multi-frame acoustic echo cancellation, noise suppression.
Background	TWB Domain	Diagnostic trace DMA, room calibration estimation, power telemetry sampling.

Queue Processing & Tie-Breaking

When multiple tasks are scheduled within the same domain:

• In the LL domain, tasks are queued in a doubly linked list sorted strictly by priority. The domain thread executes higher-priority tasks first. If multiple tasks share the same priority, they are dispatched in FIFO (First-In, First-Out) arrival order.

• In the DP domain, the Zephyr kernel schedules runnable threads using their calculated absolute deadline timestamps. A thread processing a 1 ms frame with an impending 500 µs deadline automatically preempts a thread processing a 10 ms background frame whose deadline is 8 ms away.

—

7. Power Management & Tickless Idle

Power consumption is critical in modern laptops, smartphones, and embedded audio devices. The SOF scheduling architecture is explicitly designed to maximize the duration the DSP spends in ultra-low-power autonomous wait states.

The Active vs. Sleep Duty Cycle

Figure 26 Execution Timeline: Active Processing Window vs. Autonomous Low-Power Sleep

Tickless Idle Operation

When all audio streams are stopped or paused:

1. Timer Suppression: The SOF scheduler works with Zephyr’s tickless idle subsystem to suppress periodic hardware timer interrupts.

2. Autonomous Wait States: Rather than spinning or polling, the DSP core executes a Wait For Interrupt (WFI) instruction, allowing hardware power controllers to lower core voltage, gate DSP clocks, or enter autonomous D0ix states.

3. Interrupt-Only Wakeup: The DSP remains quiescent until a hardware event occurs—such as a new host IPC command, a wake-on-voice (WOV) sound detector trigger, or an external jack insertion interrupt.

—

8. Upstream Code References & Related Guides

For developers seeking low-level C implementation details, data structures, and function prototypes:

• Upstream Scheduler Specification: Consult the comprehensive scheduler architecture specification in the SOF repository at thesofproject/sof: src/schedule/README.md.

• Core Source Files:

  • src/schedule/schedule.c: Generic scheduler registration, task queuing, and API entry points.

  • src/schedule/zephyr_ll.c: Low-Latency scheduler engine and synchronous task dispatch loop.

  • src/schedule/zephyr_domain.c: Pinned domain thread initialization and core affinity management.

  • src/schedule/zephyr_dma_domain.c: DMA interrupt-driven scheduling domain.

  • src/schedule/zephyr_dp_schedule.c: Data Processing scheduler readiness evaluation and event signaling.

  • src/schedule/zephyr_dp_schedule_thread.c: Dedicated thread execution and EDF deadline management.

  • src/schedule/zephyr_twb_schedule.c: Thread With Budget cycle monitoring and priority demotion callbacks.

• Core Header Files:

  • src/include/sof/schedule/schedule.h: Core scheduler structures and task lifecycle definitions.

  • src/include/sof/schedule/ll_schedule.h: Low-Latency domain prototypes.

  • src/include/sof/schedule/dp_schedule.h: Data Processing readiness and thread structures.

  • src/include/sof/schedule/twb_schedule.h: Thread With Budget constants and time-slice interfaces.

Related Guides

• IPC Infrastructure (IPC3 & IPC4): Host-to-DSP messaging, hardware mailbox windows, and dynamic IPC4 compound commands.

• Audio Buffer Management: Lockless circular ring buffers, multi-tier DSP memory (SRAM/DRAM), and cache coherency.

• Pipeline Architecture: How audio pipelines interact with the scheduling domains to stream data.

• Module Framework Architecture: The standardized module interface executed by LL and DP scheduler tasks.

• Firmware Initialization & Boot Architecture: Boot flow, hardware mailbox FW Ready handshake, and Zephyr initialization.

• Hostless Embedded Firmware Architecture: Autonomous firmware pipelines and timer configurations on embedded targets.

• Unit Testing with Zephyr Ztest & Twister: Unit testing scheduler components and domain threads using Zephyr Ztest.