drgn/vmtest
Omar Sandoval 841a3dae88 Move vmtest assets to GitHub releases
As noted by commit 738261290f ("CI: temporarily disable vmtest"),
vmtest was generating too much traffic to the Dropbox shared folder that
hosted vmtest kernels. Instead, we can store kernel packages as GitHub
release assets. Update the code for downloading and uploading vmtest
assets, and also add a scheduled GitHub action to build new kernels
every Monday so I don't have to remember to do it manually.

This also drops vmtest support for 5.6-5.9, which now fail to build with
newer binutils due to the issue fixed in Linux kernel commit
1d489151e9f9 ("objtool: Don't fail on missing symbol table").

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-05-05 00:28:56 -07:00
..
__init__.py Rewrite virtual machine testing setup 2020-04-02 17:35:24 -07:00
asynciosubprocess.py Move vmtest assets to GitHub releases 2021-05-05 00:28:56 -07:00
download.py Move vmtest assets to GitHub releases 2021-05-05 00:28:56 -07:00
githubapi.py Move vmtest assets to GitHub releases 2021-05-05 00:28:56 -07:00
kbuild.py Move vmtest assets to GitHub releases 2021-05-05 00:28:56 -07:00
manage.py Move vmtest assets to GitHub releases 2021-05-05 00:28:56 -07:00
onoatimehack.c Use GPL-3.0-or-later license identifier instead of GPL-3.0+ 2021-04-03 01:10:35 -07:00
README.rst Move vmtest assets to GitHub releases 2021-05-05 00:28:56 -07:00
vm.py Move vmtest assets to GitHub releases 2021-05-05 00:28:56 -07:00

drgn VM Testing
===============

drgn has a significant amount of code (both core and in helpers) which is
dependent on the Linux kernel version. This code is tested on multiple Linux
kernel versions in a virtual machine. These tests can be run on all supported
kernels with ``python3 setup.py test -K``. This requires QEMU, BusyBox, and
zstd to be installed.

Tests can also be run on specific kernels with ``-k``. This takes a
comma-separated list of kernels which are wildcard patterns (e.g., ``5.6.*``)
matching a kernel release hosted on GitHub (see below).

Architecture
------------

The goal of vmtest is to run tests in the same userspace environment as the
host, but with a different kernel. The host runs the virtual machine with `QEMU
<https://www.qemu.org/>`_ (see the `vmtest.vm <vm.py>`_ module).

The guest mounts the host's root filesystem as its own root filesystem via
`VirtFS <https://www.linux-kvm.org/page/VirtFS>`_. It is mounted read-only for
safety. To support modifications, the guest uses `OverlayFS
<https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt>`_ to
overlay a read-write tmpfs over the VirtFS root. It also mounts the kernel
modules and vmlinux via VirtFS.

The guest runs a `BusyBox <https://www.busybox.net/>`_ shell script as init
which sets up the system and filesystem hierarchy, runs a command, and returns
the exit status via `virtio-serial
<https://fedoraproject.org/wiki/Features/VirtioSerial>`_.

This infrastructure is all generic. The drgn-specific parts are:

1. The kernel builds. These are configured with a minimal configuration
   including everything required to run drgn and the Linux kernel helper tests.
   Each build is packaged as a tarball containing ``vmlinux``, ``vmlinuz``, and
   kernel modules. These packages are hosted in a `GitHub release
   <https://github.com/osandov/drgn/releases/tag/vmtest-assets>`_. They are
   managed via the GitHub API by the `vmtest.manage <manage.py>`_ CLI and
   downloaded by the `vmtest.download <download.py>`_ module.
2. The test command itself. This is just some ``setup.py`` glue and the proper
   invocation of the Python `unittest command line interface
   <https://docs.python.org/3/library/unittest.html#test-discovery>`_.

The ``vmtest.vm`` and ``vmtest.download`` modules also have CLIs for testing
purposes. These are subject to change.