Build toolchains and SOF from sources

You may boot and test Sound Open Firmware on a target machine or VM. Current target Intel platforms include: Bay Trail, Cherry Trail, Haswell, Broadwell, Apollo Lake, Cannon Lake, Ice Lake, Jasper Lake, and Tiger Lake.

Support also exists for NXP i.MX8/i.MX8X/i.MX8M platforms.

The following steps describe how to install the SOF development environment on Ubuntu 16.04, 18.04, 18.10, and 20.04, and Fedora 36. They should work on Ubuntu 19.04, 19.10 and other Linux distributions with minor or no modifications.

Note

Building the toolchains from source might take several hours. We recommend that you use Docker to build SOF. For more information, see Build SOF with Docker.

Step 1. Set up the workspace directory

Point the $SOF_WORKSPACE environment variable to the directory in which you store all sof work.

The code examples assume $SOF_WORKSPACE is the top-level working directory. Clone all git repositories at the same directory level because some default configuration files refer to other clones using relative locations like ../sof/.

Make sure that $SOF_WORKSPACE has adequate disk space when building the toolchain. About 15GB is needed per toolchain. You can reclaim some of the disk space after building the toolchain.

SOF_WORKSPACE=~/work/sof
mkdir -p "$SOF_WORKSPACE"

Step 2. Set up build environment

Install package dependencies

Note

This guide uses Ubuntu/Fedora as an example but any modern distribution can be used for SOF development.

Due to continuous default package updates in distributions, SOF documentation may not include explicit instructions for possible missing tools and packages. When you encounter missing dependencies, refer to your distribution’s documentation on how to install them.

• For Fedora (tested with v36, other recent versions should work fine):

  sudo dnf group install "Development Tools" "C Development Tools and Libraries"
  sudo dnf install ncurses-devel gtk3-devel gettext-devel texinfo help2man \
    glibc-static libstdc++-static openssl-devel tree
  

• For Ubuntu 20.04:

  sudo apt install build-essential git autoconf flex bison texinfo help2man \
     gawk libtool-bin libncurses5 libncurses5-dev libssl-dev libgtk-3-dev \
          tree ninja-build gettext libasound2-dev
  

• For Ubuntu 18.10:

  sudo apt-get install build-essential git libgtk-3-dev libsdl1.2-dev \
     libspice-protocol-dev libspice-server-dev libusb-1.0-0-dev \
     libusbredirhost-dev libtool-bin acpica-tools valgrind texinfo \
     virt-manager qemu-kvm libvirt-daemon-system libvirt-clients virtinst \
     libfdt-dev libssl-dev pkg-config help2man gawk libncurses5 \
     libncurses5-dev
  

• For Ubuntu 16.04 and 18.04:

  sudo apt-get install build-essential git libgtk-3-dev libsdl1.2-dev \
     libspice-protocol-dev libspice-server-dev libusb-1.0-0-dev \
     libusbredirhost-dev libtool-bin iasl valgrind texinfo virt-manager \
     qemu-kvm libvirt-bin virtinst libfdt-dev libssl-dev pkg-config help2man \
     gawk libncurses5 libncurses5-dev
  

If you are using Ubuntu 16.04, the gcc version must be updated to gcc 7.3+ in order for the Advanced Linux Sound Architecture (ALSA) to build.

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install gcc-7 g++-7
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 70 --slave /usr/bin/g++ g++ /usr/bin/g++-7

Install CMake

If you use Ubuntu 18.04+ or Fedora you can install CMake with apt/dnf:

sudo apt-get install cmake # Ubuntu
sudo dnf install cmake # Fedora

For Ubuntu 16.04, CMake from apt is outdated and you must install CMake from sources. Refer to this short guide: https://cmake.org/install/.

Build alsa-lib and alsa-utils from source

This project requires some new features in https://git.alsa-project.org/?p=alsa-lib.git and https://git.alsa-project.org/?p=alsa-utils.git, so build the newest ALSA from source code.

Warning

Installing alsa-lib systemwide may break some audio applications. Only perform this if you know what you are doing. We recommend that you install it locally (under $HOME) or use Docker (see Build SOF with Docker.)

cd "$SOF_WORKSPACE"
git clone git://git.alsa-project.org/alsa-lib
cd alsa-lib
# To install alsa-lib systemwide
./gitcompile
# To install alsa-lib locally
./gitcompile --prefix=$HOME/local
sudo make install

(Optional) To enable alsabat’s frequency analysis, install the FFT library before you configure alsa-utils.

sudo apt-get install libfftw3-dev libfftw3-doc # Ubuntu
sudo dnf install fftw3-devel # Fedora

Clone, build, and install alsa-utils.

cd "$SOF_WORKSPACE"
git clone git://git.alsa-project.org/alsa-utils
cd alsa-utils
# To install alsa-utils systemwide
./gitcompile
# To install alsa-utils locally
./gitcompile --prefix=$HOME/local \
             --with-alsa-inc-prefix=$HOME/local/include \
             --with-alsa-prefix=$HOME/local/lib \
             --with-systemdsystemunitdir=$HOME/local/lib/systemd \
             --with-udev-rules-dir=$HOME/local/lib/udev
sudo make install

If you run into alsa-lib linking errors, try to re-build it with the libdir parameter.

cd ../alsa-lib
./gitcompile --prefix=/usr --libdir=/usr/lib/x86_64-linux-gnu/
sudo make install
cd ../alsa-utils
./gitcompile --prefix=/usr --with-curses=ncurses --disable-xmlto --disable-bat
sudo make install

Note

If the gitcompile script does not work, refer to the INSTALL file for manual build instructions.

Create or append to the LD_LIBRARY_PATH environment variable.

export LD_LIBRARY_PATH="${SOF_WORKSPACE}"/alsa-lib/src/.libs:$LD_LIBRARY_PATH

Step 3. Build toolchains from source

Build the xtensa cross-compilation toolchains with crosstool-ng for Intel Bay Trail, Cherry Trail, Haswell, Broadwell, Apollo Lake, Cannon Lake, Ice Lake, Jasper Lake, Tiger Lake platforms and NXP i.MX8/i.MX8X/i.MX8M platforms. Building the toolchains may take about an hour but only once and it removes the dependency on the Docker image.

For more details go to https://crosstool-ng.github.io/

crosstool-ng

Clone both repos and check out the sof-gcc10.2 and sof-gcc10x branch.

cd "$SOF_WORKSPACE"
git clone https://github.com/thesofproject/xtensa-overlay
git clone https://github.com/thesofproject/crosstool-ng
git -C xtensa-overlay/ checkout  sof-gcc10.2
git -C crosstool-ng/   checkout  sof-gcc10x

Build crosstool-ng and install it in its own source directory.

cd crosstool-ng/
./bootstrap
./configure --prefix=$(pwd)
make
make install

Toolchains

The config files provided refer to ../xtensa-overlay/ and point at different ./builds/xtensa-*-elf subdirectories. Copy the ones you want to .config and build the cross-compiler(s) for your target platform(s). Note that ./ct-ng build requires an network connection to download gcc components. While other steps take minutes at most, building all toolchains may last about an hour depending on your network connection and the performance of your system.

unset LD_LIBRARY_PATH

# byt = Bay Trail / Cherry Trail
# hsw = Haswell/Broadwell
# apl = Apollo Lake
# cnl = Cannon Lake, Ice Lake, Jasper Lake, and Tiger Lake
# imx = i.MX8/i.MX8X
# imx8m = i.MX8M

# Omit the toolchains you don't want to save (a lot of) time
time for i in byt hsw apl cnl imx imx8m; do
  cp config-$i-gcc10.2-gdb9 .config &&
     time ./ct-ng build || break
done

# ... or just build all toolchains
time for i in config*gcc10.2-gdb9; do
   cp "$i" .config && time ./ct-ng build || break
done

./ct-ng is a Linux kernel style Makefile; so the sample commands below can be used to fix some out of date config-*-gcc10.2-gdb9 file or find default values missing from it:

./ct-ng help
cp config-apl-gcc10.2-gdb9 .config
./ct-ng oldconfig V=1
diff -u config-apl-gcc10.2-gdb9 .config

“Install” toolchains in the expected location by linking from $SOF_WORKSPACE to them:

ls builds/
# xtensa-apl-elf  xtensa-byt-elf   xtensa-cnl-elf   xtensa-hsw-elf  xtensa-imx-elf  xtensa-imx8m-elf
cd "$SOF_WORKSPACE"
for i in crosstool-ng/builds/xtensa-*; do ln -s "$i"; done

Remove the temporary build files (~7GB per toolchain):

rm -rf $SOF_WORKSPACE/crosstool-ng/.build

Note

Haswell and Broadwell share the same toolchain: xtensa-hsw-elf

Bay Trail and Cherry Trail share the same toolchain: xtensa-byt-elf

Cannon Lake, Ice Lake, Jasper Lake and Tiger Lake share the same toolchain: xtensa-cnl-elf

i.MX8 and i.MX8X share the same toolchain: xtensa-imx-elf

Additional headers

To get some required headers, clone the following newlib repository and switch to the xtensa branch.

cd "$SOF_WORKSPACE"
git clone https://github.com/jcmvbkbc/newlib-xtensa
cd newlib-xtensa
git checkout -b xtensa origin/xtensa

Temporarily add toolchains to your PATH variable. This is not required when using the high-level, “every day” build scripts described in the next sections. It’s only required for this once-off newlib headers step or when invoking CMake manually. In other words, you don’t need to change your PATH permanently which would interfere with other, non-SOF work.

for i in "${SOF_WORKSPACE}"/xtensa-*-elf; do PATH="$PATH:$i"/bin; done

Build and install the newlib headers for each toolchain:

XTENSA_ROOT="${SOF_WORKSPACE}"/xtensa-root
cd "${SOF_WORKSPACE}"/newlib-xtensa
time for toolchain in ../xtensa-*-elf; do
   ./configure --target="${toolchain#../}" --prefix="$XTENSA_ROOT" &&
   make && make install || break;
   rm etc/config.cache
done
ls "$XTENSA_ROOT"
  => share  xtensa-apl-elf  xtensa-byt-elf  xtensa-cnl-elf  xtensa-hsw-elf ...

This should take a few minutes.

Note

--prefix= expects an absolute path. Define XTENSA_ROOT according to your environment.

The required headers are now in "$SOF_WORKSPACE"/xtensa-root, and cross-compilation toolchains for xtensa DSPs are set up.

Step 4. Build and sign firmware binaries

After the SOF environment is set up, clone the sof repo:

cd "$SOF_WORKSPACE"
git clone --recursive https://github.com/thesofproject/sof
cd sof

Copy the commented installer/sample-config.mk to installer/config.mk, then select a list of platforms and provide an optional target hostname in the latter file. Then run the installer:

make -C installer/ [ -j 4 ]

Adjust the -j 4 example to your number of CPU cores or remove it when the build fails.

This builds multiple platforms in parallel and deploys firmware and topologies to /lib/firmware/intel/ on the local or remote destination that you configured. It builds with the default platform configurations the first time and then switches to incremental builds which preserves any make menuconfig or other configuration changes you made. These two ways to build are described below, so read on if you need finer control on the build system and configuration. Otherwise you can skip the next two sections.

The installer also builds and deploys some user-space binaries from the sof/tools/ subdirectory.

Note

The installer is much faster than the lower level ./scripts/, on which it relies, because it does not delete the build directories every time it runs. However, some “big” configuration changes, such as switching to a different toolchain or some rare build failures, can leave the installer-builds/build_* directories in an inappropriate state. In such a case, just delete these directories and run the installer again.

rm -rf $SOF_WORKSPACE/sof/installer-builds
make -C installer/

Re-configure and rebuild from scratch

To rebuild Sound Open Firmware from scratch, the installer Makefile above relies on the https://github.com/thesofproject/sof/tree/master/scripts/xtensa-build-all.sh script. If you need finer control or to troubleshoot some build issue you can also use it directly. To build the firmware for all platforms:

cd "$SOF_WORKSPACE"/sof/
./scripts/xtensa-build-all.sh -a

Note

This script works only if the cross-compiler and xtensa-root are siblings in the same sof directory, as instructed above.

As of May 2021, you may specify one or more of the following platform arguments: byt, cht, bdw, hsw, apl, skl, kbl, cnl, sue, icl, jsl, tgl, tgl-h, imx8, imx8x, imx8m. Example:

./scripts/xtensa-build-all.sh byt
./scripts/xtensa-build-all.sh byt apl

For the latest platforms list and help message, run the script without any argument. You can also enable debug builds with -d, enable rom builds with -r and speed up the build with -j [n]

./scripts/xtensa-build-all.sh -d byt
./scripts/xtensa-build-all.sh -d -r apl
./scripts/xtensa-build-all.sh -d -r -j 4 apl

Note

The xtensa-build-all.sh script uses rimage to build the final firmware image. rimage uses by default a public key included in the sof repo for signing. However, if you need to use some other external key for signing you can specify the path to your key as environment variable before invoking the build:

export PRIVATE_KEY_OPTION=-DRIMAGE_PRIVATE_KEY=/path_to_key/private.pem

The same export mechanism should work also when building with Docker.

Incremental builds

This is a more detailed build guide for the sof repo. Unlike xtensa-build-all.sh, this doesn’t rebuild everything every time. The installer Makefile above relies on this for incremental builds.

Snippets below assume that your current directory is the root of the sof clone ("$SOF_WORKSPACE"/sof/).

CMake recommends out-of-tree builds. Among others, this lets you build different configurations/platforms in different build directories from the same source without starting from scratch.

Note

The -j argument tells make how many processes to use concurrently. Select a value that matches your build system.

for Bay Trail:

mkdir build_byt && cd build_byt
cmake -DTOOLCHAIN=xtensa-byt-elf -DROOT_DIR="$XTENSA_ROOT"/xtensa-byt-elf -DINIT_CONFIG=baytrail_defconfig ..
make help # lists all available targets
make bin -j4 VERBOSE=1

You can replace byt above with any other platform listed in the help output of the sof/scripts/xtensa-build-all.sh. Find the toolchain matching each platform in the same script or above.

Note

After the cmake step, you can customize your build with ‘make menuconfig’.

DEBUG and ROM options are available for the FW binary build. Enable them with ‘make menuconfig’.

mkdir build_cnl_custom && cd build_cnl_custom
cmake -DTOOLCHAIN=xtensa-cnl-elf -DROOT_DIR="$XTENSA_ROOT"/xtensa-cnl-elf -DINIT_CONFIG=cannonlake_defconfig ..
make menuconfig # select/deselect options and save
make bin -j4

Note

If you have Ninja installed, you can use it instead of Make. Just type cmake -GNinja … during the configuration step.

Firmware build results

The firmware binary files are located in build_<platform>/src/arch/xtensa/. The installer copies them to your target machine’s /lib/firmware/intel/sof folder.

sof-apl.ri  sof-bdw.ri  sof-byt.ri  sof-cht.ri  sof-cnl.ri  sof-hsw.ri

Step 5. Build topology and tools

You can probably skip this section if you use the firmware installer in the previous section.

One-step rebuild from scratch

Without any argument https://github.com/thesofproject/sof/tree/master/scripts/build-tools.sh builds the default CMake target “ALL” of https://github.com/thesofproject/sof/tree/master/tools/.

cd "$SOF_WORKSPACE"/sof/
./scripts/build-tools.sh

To see the list of options, run https://github.com/thesofproject/sof/tree/master/scripts/build-tools.sh with the -h option.

./scripts/build-tools.sh -h

Incremental build

cd "$SOF_WORKSPACE"/sof/tools/
mkdir build_tools && cd build_tools
cmake ..
make -j4

If your cmake --version is 3.13 or higher, you may prefer the new -B option:

cmake -B build_tools/
make  -C build_tools/ -j4 VERBOSE=1
rm -rf   build_tools/ # no need to change directory ever

Topology and tools build results

The topology files are located in the tools/build_tools/topology folder. The installer Makefile copies them to the target machine’s /lib/firmware/intel/sof-tplg/ folder.

The sof-logger tool is in the tools/build_tools/logger folder. The installer Makefile copies them to the target directory of your choice.

Step 6. Build Linux kernel

Sound Open Firmware uses the Linux kernel dev branch, and it must work with other dev branch firmware and topology. This short section shows how to build Debian kernel packages tested on Ubuntu in a small number of commands. Note that these commands rebuild everything from scratch every time which makes then unsuitably slow for development. If you need to make kernel code changes, ignore this and look at Set up a Ktest-based Environment, the README file of the kconfig repo, and the SOF Linux Driver Architecture.

1. Build the kernel with this branch.

   sudo apt-get install bison flex libelf-dev
   cd "$SOF_WORKSPACE"
   git clone https://github.com/thesofproject/linux
   cd linux
   git checkout topic/sof-dev
   make defconfig
   git clone https://github.com/thesofproject/kconfig
   scripts/kconfig/merge_config.sh .config ./kconfig/base-defconfig ./kconfig/sof-defconfig  ./kconfig/mach-driver-defconfig ./kconfig/hdaudio-codecs-defconfig
   

   Optionally, you can also run make menuconfig, navigate to Device Drivers > Sound card support > Advanced Linux Sound Architecture, and select the Prefer SOF driver over SST on BY/CHT platforms option.

2. Make the kernel deb package to install on the target machine.

   make deb-pkg -j 4
   

3. Copy the three resulting .deb files from $SOF_WORKSPACE to the target machine and install them.

   sudo dpkg -i /absolute/path/to/deb/file
   sudo apt-get install -f