Media Processing Pipelines Overview

The Media Processing Pipelines (MPP) layer role is to enable SOF specific use cases that are not supported directly by Zephyr. The MPP is responsible for host communication, tasks scheduling, pipeline and component management.

@startuml allowmixing scale max 1024 width package "SOF" { package "Media Processing Pipelines layer" as MEDIA_PROCESSING_PIPELINES { package "MPP Scheduling" as MPP_SCHEDULING { component "LL Tasks" as LL_TASKS component "DP Tasks" as DP_TASKS DP_TASKS -[hidden]down- LL_TASKS } package "Communication" as COMMUNICATION { component "IPC Message Processing and common command definitions" as IPC_MESSAGE_PROCESSING component "Async Messaging" as ASYNC_MESSAGING IPC_MESSAGE_PROCESSING -[hidden]right- ASYNC_MESSAGING } package "Pipeline/Component Infrastructure" as PIPELINE_COMPONENT_INFRASTRUCTURE { component "Pipeline Management" as PIPELINE_MANAGEMENT component "Host/DAI Gateways" as HOST_DAI_GATEWAYS component "Processing Component Management" as PROCESSING_COMPONENT_MANAGEMENT PIPELINE_MANAGEMENT -[hidden]right- HOST_DAI_GATEWAYS HOST_DAI_GATEWAYS -[hidden]right- PROCESSING_COMPONENT_MANAGEMENT } COMMUNICATION -[hidden]down- PIPELINE_COMPONENT_INFRASTRUCTURE COMMUNICATION -[hidden]right- MPP_SCHEDULING } package "Zephyr" as ZEPHYR { interface "Zephyr Services, SoC HAL and Driver Interfaces" as SS component "SoC HAL" as SOC component "Drivers" as DRIVERS component "XTHAL" as XTHAL component "Services" as SERVICES SS -[hidden]down- SERVICES SERVICES -[hidden]right- SOC SOC -[hidden]right- DRIVERS DRIVERS -[hidden]right- XTHAL } MEDIA_PROCESSING_PIPELINES -[hidden]down- ZEPHYR PIPELINE_COMPONENT_INFRASTRUCTURE -[hidden]down- ZEPHYR } @enduml

Figure 44 Media Processing Pipelines Layer diagram

Services in Media Processing Pipelines Layer

Gateways

The gateways are a key element in SOF data exchange with host, external audio peripherals and internally between firmware components. They serve as an abstraction layer for multiple data protocols.

The typical audio stream (chain of pipelines) starts and ends with I/O gateway. I/O gateway represents a sink or a source interface that can be read or written via DMA operations on I/O FIFO (i.e. DMIC, SNDW, etc.) or directly via memory operations (i.e. memory buffers of IPC Gateway). Host Gateway is unique as it exposes interface endpoint to the Host driver.

The stream audio gateways are created as a part of Copier component configuration.

Examples of gateways:

  • DMIC Gateway,

  • SoundWire Gateway,

  • I2S Gateway,

  • HD/A Gateway,

  • IPC Gateway,

NOTE: Not all I/O gateways must be available in all configurations.

Pipeline Management

The Pipeline Management is a host IPC driven service that is used to:

  • create / delete pipeline

  • switch pipeline state

  • create pipeline processing tasks

  • allocate pipeline buffers memory

Processing Component Management

Processing Component Management is driven by host IPC requests. It is used to:

  • instantiate / delete components

  • configure components

  • bind / unbind components into processing paths

  • load components into ADSP memory

Asynchronous Messaging

Asynchronous Messaging Service (AMS) provides functionality to:

  • send asynchronous messages to firmware components or host,

  • broadcast messages to multiple consumers,

  • asynchronous message exchange between components running on different cores

MPP Scheduling

MPP Scheduling is dedicated to support Media Processing Pipelines services tasks scheduling. It exposes SOF specific interface that is implemented on top of Zephyr scheduling API.

MPP Scheduling features:

  • Low Latency tasks scheduling,

  • Data Processing tasks scheduling,

  • Tasks with budget scheduling