Testing

This directory is used for multi-kernel testing using litte-vm-helper (https://github.com/cilium/little-vm-helper/).

Usage

NOTE: For now, commands need to be executed from the top-level tetragon directory.

Build everything:

$ make -C tests/vmtests
make: Entering directory '/home/kkourt/src/tetragon/tests/vmtests'
go build ../cmd/tetragon-tester
go build ../cmd/tetragon-vmtests-run
make: Leaving directory '/home/kkourt/src/tetragon/tests/vmtests'

Download rootfs images and kernels in tests/vmtests/test-data

$ ./tests/vmtests/fetch-data.sh 4.19-main 5.4-main bpf-next-main
...
$ ls tests/vmtests/test-data/images tests/vmtests/test-data/kernels
tests/vmtests/test-data/images:
base.qcow2

tests/vmtests/test-data/kernels:
4.19/ 5.4/  bpf-next/

Run tests on 5.4:

$ make test-compile # <- this will build the test binaries
$ ./tests/vmtests/tetragon-vmtests-run \
	--kernel tests/vmtests/test-data/kernels/5.4/boot/vmlinuz-5.4.206 \
	--base tests/vmtests/test-data/images/base.qcow2

Run tests on 5.4, without KVM acceleration. Doing so, makes the run closer to the GH action environment (which do not support nested virtualization) and it also uncover other issues due to the unconventional timing.

$ ./tests/vmtests/tetragon-vmtests-run \
	--kernel tests/vmtests/test-data/kernels/5.4/boot/vmlinuz-5.4.206 \
	--base tests/vmtests/test-data/images/base.qcow2 \
	--qemu-disable-kvm

Run a single test 20 times on 4.19 and fail fast

$ seq 20 | xargs -I {} echo pkg.sensors.tracing:TestGenericTracepointRawSyscall > tests/vmtests/repeat-raw-syscall
$ ./tests/vmtests/tetragon-vmtests-run \
	--kernel tests/vmtests/test-data/kernels/4.19/boot/vmlinuz-4.19.262 \
	--btf-file tests/vmtests/test-data/kernels/4.19/boot/btf-4.19.262 \
	--base tests/vmtests/test-data/images/base.qcow2  \
	--testsfile tests/vmtests/repeat-raw-syscall \
	--fail-fast

Just Boot the VM. User

$ ./tests/vmtests/tetragon-vmtests-run \
	--kernel tests/vmtests/test-data/kernels/5.4/boot/vmlinuz-5.4.206 \
	--base tests/vmtests/test-data/images/base.qcow2 \
	--qemu-disable-kvm

Design notes

There are two go programs: one that runs inside the VM (tetragon-tester), and one that runs outside the VM (tetragon-vmtests-run). tetragon-vmtests-run, running outside, will prepare an image and then boot it using qemu. The image will be build based on the base image provided with --base. Some of the steps of preparing the image are:

• add tetragon-tester as a systemd service, so that it starts when the machine boots
• mount the tetragon source directory inside the VM (currently, as a 9p filesystem)
• write a configuration file to be read by tetragon-tester

Once the machine boots, the systemd will start the service that will execute the tests. When the tests complete, the machine will be powered off.