Yocto

Revision as of 2024-11-10T19:11:48 by Kai (talk | contribs) (→‎Layers)

Yocto and OpenEmbedded

The package management tools used by the Yocto project have been developed by the OpenEmbedded project. The Yocto project basically uses these generic build tools to provide a basic Linux system called Poky. Stable releases of Poky serve as standard platforms for third parties that provide additional features (e.g., hardware support layers).

Unless stated otherwise, the information on this page applies to version 5.0.x (Scarthgap) of Yocto.

Prerequisites

  • A Linux system supported by the Yocto project.
    Assuming that Ubuntu is used.
  • Following packages installed:
sudo apt install gawk wget git diffstat unzip texinfo \
  gcc build-essential chrpath socat cpio xz-utils \
  debianutils iputils-ping zstd liblz4-tool file \
  python3 python3-pip python3-pexpect python3-git \
  python3-jinja2 python3-subunit locales libacl1
  • Fixing a bug with Yocto 5.0.5 on Ubuntu 2024.04 caused by a recipe, where the function disable_network() in bitbake/lib/bb/utils.py throws the error "Errno 1 Operation not permitted".
sudo apparmor_parser -R /etc/apparmor.d/unprivileged_userns


Bitbake

Bitbake is the build tool from the OpenEmbedded project, which parses the project's metadata, resolves dependencies and the ordering of the build steps, runs the builds to create packages, to finally create an image from the packages.

Metadata Types

Yocto metadata, which controls how a system image is built, can be categorized at a high level into these four types:

  • Machine
    the targeted hardware platform (e.g. Raspberry Pi 4)
  • Distro
    common characteristic of a family of related images
  • Recipes
    instructions on how to process the source files for a component of the image
  • Image
    recipes that define the contents of generated system images


Layers

A layer is a purposeful collection of metadata that adds some feature or capability to your custom system image. For example, if you want to create a custom system for the Raspberry Pi 4, you will need the drivers and configurations for that device. These are available packaged as a layer. Just download the sources for the Raspberry Pi layer and add it to the stack of layers used in your build: 

git clone git://git.yoctoproject.org/poky -b scarthgap
cd poky
source oe-init-build-env
cd ..
mkdir layers
cd layers
git clone git://git.yoctoproject.org/meta-raspberrypi -b scarthgap
cd ../poky
bitbake-layers add-layer ../layers/meta-raspberrypi
cat build/conf/bblayers.conf | grep raspberrypi
TARGET=core-image-full-cmdline
bitbake --dry-run $TARGET
bitbake $TARGET
ls tmp/work/raspberrypi4-poky-linux-gnueabi/${TARGET}/1.0/deploy-${TARGET}-image-complete/*.bz2


Note that layer names are prefixed with meta-. This is just a convention.

Images

Images are recipes that ultimately inherit from meta/classes/image.bbclass and set the global variable IMAGE_INSTALL. Most images, however, will inherit from meta/classes/core-image.bbclass which inherits directly from image.bbclass, provides a mapping from IMAGE_FEATURES to package groups, and adds the two package groups packagegroup-core-boot and packagegroup-base-extended to IMAGE_INSTALL.

Images are defined in recipe files with the file extension 'bb'. In order to separate them from package recipes, there is a convention to put image recipes into a folder named images. Making use of this, we can list all images defined by all layers via 

cd poky
ls meta*/recipes*/images/*.bb


IMAGE_FEATURES

The IMAGE_FEATURES variable is defined in meta/classes/image.bbclass with an empty value. It accepts only a predefined set of features (e.g., x11, debug-tweaks, dev-pkgs, dbg-pkgs). This affects how recipes will execute their tasks and which packages will be selected to go into the image. The image recipe meta/recipes-core/images/core-image-base.bb, for example, adds the splash feature.

IMAGE_INSTALL

Holds a list of package groups whose packages (from the package feed area) will go into the filesystem image.

Reference images

The following reference images are defined in the meta layer:

core-image-minimal small image that just boots the target device
core-image-minimal-dev core-image-minimal with headers and libraries allowing development work
core-image-minimal-initramfs core-image-minimal with kernel support for in-RAM filesystem
core-image-base console-only system fully supporting the target hardware
core-image-full-cmdline console-only image that includes many system tools
core-image-x11 image with basic X11 and a terminal

These image definitions are provided as a starting point for creating project-specific, custom image definitions.

Creating a Custom Image

  • In your custom layer, create the images subdirectory in one of your recipes-...' directory.
  • In the images directory, create two image recipe files that inherit from core-image:
SUMMARY = "My deployment image"
LICENSE = "MIT"

inherit core-image
IMAGE_FEATURES += "splash"

and 

SUMMARY = "My development image"

inherit core-image
require my-image.bb

IMAGE_FEATURES += "ssh-server-dropbear tools-debug debug-tweaks"

CORE_IMAGE_EXTRA_INSTALL += "i2c-tools "

where the first image recipe is for deployment and the second is for development. Note that the development version includes the deployment version with the require statement, and adds to IMAGE_FEATURES and CORE_IMAGE_EXTRA_INSTALL.

Image Filesystem Packaging

The system image generated by Bitbake can be packaged in the form of an image file, whose contents can be transferred on a boot medium (SD card or eMMC) such that the primary boot loader of the target device can discover the system and boot it.

Unless you restrict the type of packaging used for the image file, Yocto will generate multiple rootfs packages, each with a large footprint. To restrict the generated image files to a desired type, edit conf/local.conf of your project build directory and set the type. 

#IMAGE_FSTYPES = "tar.gz"
#IMAGE_FSTYPES = "tar.bz2"
IMAGE_FSTYPES = "wic.bz2 wic.bmap"

The wic (kickstart) format with accompanying bmap is recommended.

Installing the Rootfs Image

Using bmaptool we can copy the generated wic image (with matching bmap file) to a mounted SD card or eMMC storage. 

sudo bmaptool copy tmp/deploy/images/raspberrypi4-64/<wic.bz2 file> <drive>

where <wic.bz2 file> is the root filesystem image wic.bz2 file, and <drive> is the drive at which the storage is mounted (for example, /dev/sdb).

Package Groups

Package groups are recipes that inherit from meta/classes/packagegroup.bbclass and set the content of the global variable PACKAGES. Package groups are defined in recipe files with the file extension 'bb'.

Note that the packagegroup must be inherited before PACKAGES is defined. 

inherit packagegroup

PACKAGES = '...'


Useful Image Features


Distro Features

Distro features are system features that affect multiple recipes. Large parts of the Poky image may need to be rebaked after making changes to the distro features!

systemd

After the kernel is done with its basic booting tasks, it passes control to the initialization manager. To use the systemd initialization manager instead of SysVinit, make the following entries either in conf/distro/distro.conf or in your layer's conf/layer.conf, or in conf/local.conf: 

DISTRO_FEATURES:append = " systemd"
DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
VIRTUAL-RUNTIME_init_manager = "systemd"
VIRTUAL-RUNTIME_initscripts = ""


sshd

The OpenSSH server is installed via 

EXTRA_IMAGE_FEATURES:append = " ssh-server-openssh"

This installs a socket-based SSH server, which runs only when a connection request comes in, thereby reducing memory consumption when no SSH client is connected.

To check the status of the SSH server, on the target device, type 

systemctl status sshd.socket


NFS

In Embedded Linux projects, NFS is often included in development images to enable faster code-deploy turnaround times.
In the conf/local.conf file of your Yocto build directory (or conf/layer.conf file of your layer) add 

DISTRO_FEATURES:append = " nfs"


On your Debian-based Yocto machine, set up an NFS server where the served directory in /etc/exports must be

${YOCTO_BUILD_DIR}/tmp/work/${TARGET}-poky-linux/${IMAGE}/1.0-r0/rootfs

For example,

build-robofish-lt0/tmp/work/raspberrypi4_64-poky-linux/robofish-lt0-image-dev/1.0-r0/rootfs



Vulkan and OpenGL

DISTRO_FEATURES:append = " vulkan opengl x11 wayland"


Bluetooth and NFC

DISTRO_FEATURES:append = " bluetooth nfc"


Recipes

Recipes are Yocto metadata files that are interpreted by Bitbake and describe how a build task is performed.

Patching A Device Tree

Follow these steps to patch an existing device tree by creating an append recipe in a custom layer.

In the custom layer, create the directory for a linux kernel append recipe. 

mkdir -p ${LAYER_DIR}/recipes-kernel/linux/files

where it is assumed that ${LAYER_DIR} contains the path to the root directory of the custom Yocto layer.

Copy the kernel device tree file from your Yocto project build directory to the files directory of your new recipe. For a Raspberry Pi Compute Module 4, for example, type 

cp ${YOCTO_BUILD_DIR}/tmp/work-shared/${MACHINE}/kernel-source/arch/arm/boot/dts/some.dts ${LAYER_DIR}/recipes-kernel/linux/files/

where ${YOCTO_BUILD_DIR} is the path to the Yocto project build directory, and ${MACHINE} is the chosen target platform (e.g. "raspberrypi4-64").

If the tmp/work-shared directory does not exists anymore, you can recreate it by typing 

bitbake -c menuconfig virtual/kernel

and exiting the kernel configuration menu without making changes.


Make the necessary changes to the copied device tree file. Then create a git patch file using git diff as shown below 

git diff --no-index {DTS_DIR}/some.dts some.dts > 0001-my_modifications-dts.patch


Edit the generated patch file to fix the file paths at the top. The paths must be relative to the kernel sources directory:

${YOCTO_BUILD_DIR}/tmp/work-shared/${MACHINE}/kernel-source

The header should look like this: 

diff --git a/arch/arm/boot/dts/some.dts b/some.dts
index 6e99da1..4567735 100644
--- a/arch/arm/boot/dts/some.dts
+++ b/arch/arm/boot/dts/some.dts

where "--- a" and "+++ b" are followed by the same relative file path.


Create and edit the append recipe 

vim ${LAYER_DIR}/recipes-kernel/linux/${KERNEL_NAME}_%.bbappend

where ${KERNEL_NAME} is the kernel name, which usually starts with "linux-" (for example, linux-raspberrypi), and _% denotes all versions of the kernel package.

If in doubt, use the oe-pkgdata-util tool from within your build directory to find out the kernel name. 

oe-pkgdata-util lookup-recipe kernel

The contents of the recipe file should be similar to this: 

FILESEXTRAPATHS:prepend := "${THISDIR}/files:"
SRC_URI += "file://0001-my_modifications-dts.patch"
SRC_URI[md5sum] = "e2eba462f773f46fe6930a08323f5e1f"
SRC_URI[sha256sum] = "2ef2b2e9b33b5d2af0425940387bc6c1c2c305e6a685defbdc7cf3f584bc1edc"

where 0001-my_modifications-dts.patch needs to be replaced with the actual patch file name.

Creating a New Device Tree

Creating a new device tree requires an input dts file which completely defines the hardware configuration. The KERNEL_DEVICETREE variable is used to make the Linux kernel select the new device tree during booting. 

inherit kernel-devicetree

FILESEXTRAPATHS:prepend := "${THISDIR}/files:"

# Make custom kernel with PRU enabled
SRC_URI += " \
    file://some.dts \
    file://0001-some.patch \
"

KERNEL_DEVICETREE = "some.dtb"


Accessing the OpenEmbedded package collection

The OpenEmbedded project provides recipes for many useful programs that are not included with Yocto, like tmux, VLC, Gnome, various multimedia codecs, more filesystems, gRPC and much more.
To access the recipes for those software packages, clone the OpenEmbedded layers repository into your third-party layers directory: 

git clone git://git.openembedded.org/meta-openembedded


devtool

Use devtool to make changes to an existing recipe and automatically generate an append recipe from those changes.

Retrieved from "https://Robo.Fish/wiki/index.php?title=Yocto&oldid=3178"