SOF Documentation Generation¶
These instructions will walk you through generating the SOF Project’s documentation and publishing it to https://thesofproject.github.io. You can also use these instructions to generate the SOF documentation on your local system.
The SOF Project content is written using the reStructuredText markup language (.rst file extension) with Sphinx extensions, and processed using Sphinx to create a formatted standalone website. As a developer, you can view this content either in its raw form as .rst markup files, or you can generate the HTML content and view it with a web browser directly on your workstation.
The project’s documentation contains reStructuredText source files used to generate documentation found at the http://thesofproject.github.io website. All of the reStructuredText sources are found in the thesofproject/sof-docs repo.
The reStructuredText files are processed by the Sphinx documentation system, and make use of the breathe extension for including the doxygen-generated API material.
Set up documentation working folders¶
You must install git to set up the working folders:
For an Ubuntu development system use:
sudo apt-get install git
For a Fedora development system use:
sudo dnf install git
For a Windows development system, download and install Git manually from the https://git-scm.com/download/win website.
We use github.io for publishing the generated documentation. The recommended folder setup for documentation contributions and generation is as follows:
thesofproject/ sof/ sof-docs/
thesofproject folder is present because we use the
publishing area (
thesofproject.github.io) later in these steps. It’s
best if the
sof-docs folder is an ssh clone of your personal fork of the
upstream project repos (although https clones also work):
Use your browser to visit https://github.com/thesofproject and do a fork of the
sof-docsrepo to your personal GitHub account.)
At a command prompt, create the working folder and clone the sof-docs repository to your local computer (and if you have publishing rights, the thesofproject.github.io repo). If you don’t have publishing rights, can still generate the docs locally but not publish them:
cd ~ mkdir thesofproject && cd thesofproject git clone firstname.lastname@example.org:<github-username>/thesofproject/sof-docs.git
The documentation of the SOF source code generated by doxygen is referenced and included by the
sof-docs. Clone the
git clone email@example.com:thesofproject/sof.git # use next until merged back to master cd sof git checkout next cd ..
For the cloned local repos, tell git about the upstream repo:
cd sof-docs git remote add upstream firstname.lastname@example.org:thesofproject/sof-docs.git
If you haven’t done so already, be sure to configure git with your name and email address for the signed-off-by line in your commit messages:
git config --global user.name "David Developer" git config --global user.email "email@example.com"
Install documentation tools¶
Our documentation processing has been tested to run with:
- Python 3.6.3
- Doxygen version 1.8.13
- Sphinx version 1.7.5
- Breathe version 4.9.1
- docutils version 0.14
- sphinx_rtd_theme version 0.4.0
The SOF documentation makes use of additional Sphinx extensions used for creating drawings:
- sphinx.ext.graphviz (included with Sphinx)
The plantuml extension uses Java to render the uml drawing syntax into an image. You’ll need to have a Java runtime environment (JRE) installed when generating documentation.
Depending on your Linux version, install the following tools:
For Ubuntu use:
sudo apt-get install doxygen python3-pip python3-wheel make \ default-jre graphviz
For Fedora use:
sudo dnf install doxygen python3-pip python3-wheel make \ default-jre graphviz
For either Linux environment, install the remaining python-based tools:
cd ~/thesofproject/sof-docs pip3 install --user -r scripts/requirements.txt
For Windows, install the needed tools manually:
Python (3.7+) from https://www.python.org/downloads/
Python package installer (pip) from https://pip.pypa.io/en/stable/installing/
Doxygen from http://www.doxygen.nl/download.html
GraphViz from https://graphviz.gitlab.io/
CMake (3.10+) from https://cmake.org/install/
Make - if you do not already have make, install it using MSYS2 from https://www.msys2.org/. Use the following command:
pacman -S make
Make sure that installed executable files are in your path. If not, manually add the paths to the PATH variable.
For Linux and Windows, install the remaining python-based tools:
cd <path to the directory>\thesofproject\sof-docs pip3 install --user -r scripts\requirements.txt
You are ready to generate the documentation.
Documentation presentation theme¶
Sphinx supports easy customization of the generated documentation
appearance through the use of themes. Replace the theme files and do
make html and the output layout and style is changed.
read-the-docs theme is installed as part of the
requirements.txt list above.
Run documentation processors¶
The sof-docs directory contains all the .rst source files, extra tools, and Makefile for generating a local copy of the SOF technical documentation.
For Linux, compile the output by using the following commands:
cd ~/thesofproject/sof/doc cmake . make doc cd ~/thesofproject/sof-docs make html
cd <path to the directory>\thesofproject\sof\doc cmake -GNinja . ninja doc cd <path to the directory>\thesofproject\sof-docs make html
Depending on your development system, HTML content might take a few minutes to generate. When done, view the HTML output with
your browser, starting at
If you have merge rights to the
thesofproject repo called
thesofproject.github.io, you can update the public project documentation
found at https://thesofproject.github.io.
You must perform a one-time clone of the upstream repo (we publish directly to the upstream repo rather than to a personal forked copy):
cd ~/thesofproject git clone firstname.lastname@example.org:thesofproject/thesofproject.github.io.git
After you have verified that the generated HTML from
make html looks
good, you can push directly to the publishing site using this command:
This will delete everything in the publishing repo’s latest folder (in case the new version has deleted files) and push a copy of the newly-generated HTML content directly to the GitHub pages publishing repo. The public site at https://thesofproject.github.io will be updated within a few minutes so it’s best to verify the locally-generated html before publishing.
In some situations it is necessary to clean all the files and build from the very beginning. To do this, use the
make clean command.
In some cases, after you run
make html, the documentation processors might return the following errors:
Warning: sphinx_rtd_theme missing. Use pip to install it. Extension error: Could not import extension breathe (exception: No module named breathe) Makefile:36: recipe for target 'html' failed make: *** [html] Error 1
The issue could be related to the default policy on Debian-based Linux distributions (i.e. Ubuntu) that links Python commands to Python 2.7.x. You can verify this by entering the following steps:
python --version Python 2.7.15rc1 ll /usr/bin/python lrwxrwxrwx 1 root root 9 sie 29 07:36 /usr/bin/python -> python2.7*
The issue can be resolved by running a dedicated environment with the Python 3.x binary and include its own set of installed Python packages. Virtualization of the Python environment is recommended as an alternative to:
- adding an alias setup in ~/.bashrc
- changing the symbolic link (/usr/bin/python)
- modifying the default system behavior using update-alternatives
Start with installing virtualization support. As a next step, activate the virtualized environment:
apt-get install python3-venv python3 -m venv my-sof-env . ./my-sof-env/bin/activate python --version Python 3.6.7
Verify the Python version and proceed with installing all required Python packages in the virtualized environment:
pip install sphinx git clone https://github.com/thesofprojects/sof.git git clone https://github.com/thesofprojects/sof-docs.git cd sof-docs/ pip install -r scripts/requirements.txt
After the installation is finished, you should be able to generate documentation by invoking commands listed in Running the documentation processors.
To deactivate the virtual environment and original Python environment, type:
Further information on how to use lightweight Python virtualization environments can be found at https://docs.python.org/3/library/venv.html.
It is possible that the
cmake command may not be accessible from the MSYS2 shell:
cmake -GNinja . bash: cmake: command not found
The problem may be due to the MSYS2 PATH missing the cmake installation folder. If the cmake works correctly from the Win Command Prompt then edit the msys2_shell.cmd and check if a PATH inherit option is enabled:
Another issue that may occur is the
sphinx-build command not found:
make html make: sphinx-build: Command not found make: *** [Makefile:36: html] Error 127
If the above error occurs both in the Win Command Prompt and in the MSYS2 shell then the python sphinx package needs to be updated:
pip install -U sphinx
Diagram compilation troubleshooting¶
If you are creating a diagram that is using the lastest features of plantuml, you may encounter the following compilation error:
WARNING: error while running plantuml b'ERROR\n2\nSyntax Error?\nSome diagram description contains errors\n'
If you excluded syntax errors in the diagram description, one of remaining possibilities is lack of compatibility with the installed plantuml.jar version. You can verify it using the following command:
java -jar ./scripts/plantuml.jar -version
If the installed version of plantuml.jar is missing necessary features, submit a pull request to the SOF documentation repository with a new one.