Config Annotations

Author:	Andrea Righi

Overview

Each Ubuntu kernel needs to maintain its own .config for each supported architecture and each flavour.

Every time a new patch is applied or a kernel is rebased on top of a new one, we need to update the .config's accordingly (config options can be added, removed and also renamed).

So, we need to make sure that some critical config options are always matching the desired value in order to have a functional kernel.

State of the art

At the moment configs are maintained as a set of Kconfig chunks (inside debian.<kernel>/config/): a global one, plus per-arch / per-flavour chunks.

In addition to that, we need to maintain also a file called 'annotations'; the purpose of this file is to make sure that some critical config options are not silently removed or changed when the real .config is re-generated (for example after a rebase or after applying a new set of patches).

The main problem with this approach is that, often, we have duplicate information that is stored both in the Kconfig chunks and in the annotations files and, at the same time, the whole .config's information is distributed between Kconfig chunks and annotations, making it hard to maintain, review and manage in general.

Proposed solution

The proposed solution is to store all the config information into the "annotations" format and get rid of the config chunks (basically the real .config's can be produced "compiling" annotations).

Implementation

To help the management of the annotations an helper script is provided (debian/scripts/misc/annotations):

``` usage: annotations [-h] [--version] [--file FILE] [--arch ARCH] [--flavour FLAVOUR] [--config CONFIG]

(--query | --export | --import FILE | --update FILE | --check FILE)

Manage Ubuntu kernel .config and annotations

options:

-h, --help	show this help message and exit
--version, -v	show program's version number and exit
--file FILE, -f FILE
	Pass annotations or .config file to be parsed
--arch ARCH, -a ARCH
	Select architecture
--flavour FLAVOUR, -l FLAVOUR
	Select flavour (default is "generic")
--config CONFIG, -c CONFIG
	Select a specific config option

Action:

--query, -q	Query annotations
--export, -e	Convert annotations to .config format
--import FILE, -i FILE
	Import a full .config for a specific arch and flavour into annotations
--update FILE, -u FILE
	Import a partial .config into annotations (only resync configs specified in FILE)
--check FILE, -k FILE
	Validate kernel .config with annotations

```

This script allows to query config settings (per arch/flavour/config), export them into the Kconfig format (generating the real .config files) and check if the final .config matches the rules defined in the annotations.

Examples (annotations is defined as an alias to debian/scripts/annotations):

• Show settings for CONFIG_DEBUG_INFO_BTF for master kernel across all the supported architectures and flavours:

``` $ annotations --query --config CONFIG_DEBUG_INFO_BTF {

"policy": {

"amd64": "y", "arm64": "y", "armhf": "n", "ppc64el": "y", "riscv64": "y", "s390x": "y"

}, "note": "'Needs newer pahole for armhf'"

}

• Dump kernel .config for arm64 and flavour generic-64k:

` $ annotations --arch arm64 --flavour generic-64k --export CONFIG_DEBUG_FS=y CONFIG_DEBUG_KERNEL=y CONFIG_COMPAT=y ... `

• Update annotations file with a new kernel .config for amd64 flavour generic:

` $ annotations --arch amd64 --flavour generic --import build/.config `

Moreover, three additional kernelconfig commands are provided (via debian/rules targets):

• listnewconfigs: allow to generate a list of new config options (e.g., after a rebase) and store them in CONFIGS/new-<arch>-<flavour> for review
• importconfigs: after new .config's are generated and reviewed (in CONFIGS/<arch>-<flavour>) we can use this command to automatically import all of them into the local annotations
• migrateconfigs: automatically merge all the previous configs into annotations (local changes still need to be committed)

Pros and Cons

Pros:

• avoid duplicate information in .config's and annotations
• allow to easily define groups of config settings (for a specific environment or feature, such as annotations.clouds, annotations.ubuntu, annotations.snapd, etc.)
• config options are more accessible, easy to change and review
• we can easily document how config options are managed (and external contributors won't be discouraged anymore when they need to to change a config option)

Cons:

• potential regressions: the new tool/scripts can have potential bugs, so we could experience regressions due to some missed config changes
• kernel team need to understand the new process (even if everything is transparent, kernel cranking process is the same, there might be corner cases that need to be addressed and resolved manually)

TODO

• Migrate all flavour and arch definitions into annotations (rather than having this information defined in multiple places inside debian/scripts); right now this information is "partially" migrated, meaning that we need to define arches and flavours in the headers section of annotations (so that the annotations tool can figure out the list of supported arches and flavours), but arches and flavours are still defined elsewhere, ideally we would like to have arches and flavours defined only in one place: annotations.