I incorrectly assumed that the stable repo would have the mainline tags
soon after they were released, but this is not the case. We also need to
check the mainline repo for tags and fetch them from the mainline repo.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The argument to download_kernels() is an iterable of kernels, but we're
passing it a single kernel string.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In Python before 3.8, tempfile.TemporaryDirectory.cleanup() fails when
the directory doesn't exist. Since we rename the temporary download
directory to its final name, this always fails. Switch to using
tempfile.mkdtemp() directly instead.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
As noted by commit 738261290f ("CI: temporarily disable vmtest"),
vmtest was generating too much traffic to the Dropbox shared folder that
hosted vmtest kernels. Instead, we can store kernel packages as GitHub
release assets. Update the code for downloading and uploading vmtest
assets, and also add a scheduled GitHub action to build new kernels
every Monday so I don't have to remember to do it manually.
This also drops vmtest support for 5.6-5.9, which now fail to build with
newer binutils due to the issue fixed in Linux kernel commit
1d489151e9f9 ("objtool: Don't fail on missing symbol table").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The install directory contains redundant copies of the modules already
in the build tree and built package, so clean it up on success.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Travis CI is no longer offering free open source CI, so migrate to
GitHub Actions. The only downside is that GitHub Actions doesn't support
nested virtualization, but we can work around that by falling back to
slow emulation.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently build with CONFIG_MODULES=n for simplicity. However, this
means that we don't test kernel module support at all. Let's enable
module support. This requires changing how we distribute kernels. Now,
the /lib/modules/$(uname -r) directory (including the vmlinux and
vmlinuz) is bundled up as a tarball. We extract it, then mount it with
VirtFS, and do some extra setup for device nodes. (We lose the ability
to run kernel builds directly, but I've never actually used that
functionality.)
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Black was recently changed to treat a trailing comma as an indicator to
put each item/argument on its own line. We have a bunch of places where
something previously had to be split into multiple lines, then was
edited to fit on one line, but Black kept the trailing comma. Now this
update wants to unnecessarily split it back up. For now, let's get rid
of these commas. Hopefully in the future Black has a way to opt out of
this.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
As of QEMU commit a5804fcf7b22 ("9pfs: local: ignore O_NOATIME if we
don't have permissions") (in v5.1.0), QEMU handles O_NOATIME sanely, so
we don't need the LD_PRELOAD hack. Since we're adding a version check,
make the multidevs check based on the version, too.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
My work VPN is apparently closing HTTP connections prematurely, which
exposed that urllib won't catch incomplete reads if copied through
shutil.copyfileobj(). Check it explicitly.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This documents best practices for contributing to drgn. We now require a
DCO sign-off.
Also clean up some related areas in the documentation.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn was originally my side project, but for awhile now it's also been
my work project. Update the copyright headers to reflect this, and add a
copyright header to various files that were missing it.
Currently, vmtest runs a custom program as init in the guest that
communicates with the host via virtio-serial. This is a lot of C code
that turns out can be replaced with a shell script. In order to keep the
dependencies minimal, we use BusyBox.
The current implementation of vmtest has a few issues:
1. Building drgn for each kernel version on Travis is slow, mostly
because they don't all run in parallel.
2. For local, incremental testing, recreating the filesystem image and
rebuilding drgn is slow, and syncing the code to the filesystem image
is brittle.
3. The filesystem image is the only communication channel, and reading
the exit status from the filesystem image is awkward and fragile.
4. Creating and accessing the filesystem image requires root.
This reworks vmtest to use the build on the host via VirtFS with a
simple agent on the guest that can execute arbitrary commands and return
the exit status. This has a few more moving parts but is faster and
saner overall.
Any changes of the vmtest kernel config will require a rebuild of all
kernels and a way to distinguish the rebuild. Add CONFIG_LOCALVERSION
which we will bump each time the config changes.
About half (!) of vmlinux comes from relocation sections (~130M out of
~250M). But, vmlinux is an ET_EXEC file, so relocations don't even apply
to it. We can massively shrink vmlinux by removing the unnecessary
sections.
Currently, the kernel config is embedded inside of manage.py. This is
inconvenient when building kernels manually for testing. Move it to a
separate file so that, e.g., it can be copied into the build tree
easily.
