Commit Graph

77 Commits

Author SHA1 Message Date
Omar Sandoval
0e1f85516a vmtest: manage: use str() instead of repr() for Path in error message
Otherwise, the path is formatted as "PosixPath('...')", which is ugly.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-02-21 03:08:36 -08:00
Omar Sandoval
006b62e530 vmtest: manage: use real name for logging, not asyncio
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-02-21 03:08:36 -08:00
Omar Sandoval
bc06f3ae59 vmtest: manage: delete temporary install directory
The install directory contains redundant copies of the modules already
in the build tree and built package, so clean it up on success.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-02-21 03:08:36 -08:00
Omar Sandoval
1631c11f37 Migrate to GitHub Actions
Travis CI is no longer offering free open source CI, so migrate to
GitHub Actions. The only downside is that GitHub Actions doesn't support
nested virtualization, but we can work around that by falling back to
slow emulation.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-01-01 02:59:50 -08:00
Omar Sandoval
4431b4f918 vmtest: enable kernel modules
We currently build with CONFIG_MODULES=n for simplicity. However, this
means that we don't test kernel module support at all. Let's enable
module support. This requires changing how we distribute kernels. Now,
the /lib/modules/$(uname -r) directory (including the vmlinux and
vmlinuz) is bundled up as a tarball. We extract it, then mount it with
VirtFS, and do some extra setup for device nodes. (We lose the ability
to run kernel builds directly, but I've never actually used that
functionality.)

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-10-18 01:13:01 -07:00
Omar Sandoval
de98b40495 vmtest: use pathlib
pathlib is a bit nicer than os.path for most cases, so try it out
starting with vmtest.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-10-14 11:37:31 -07:00
Omar Sandoval
d829401e5f vmtest: also disable onoatimehack on QEMU 5.0.1
The fix was backported to QEMU's 5.0 stable branch and released in
5.0.1.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-09-23 16:42:15 -07:00
Omar Sandoval
36068a0ea8 Fix trailing commas for Black v20.8b1
Black was recently changed to treat a trailing comma as an indicator to
put each item/argument on its own line. We have a bunch of places where
something previously had to be split into multiple lines, then was
edited to fit on one line, but Black kept the trailing comma. Now this
update wants to unnecessarily split it back up. For now, let's get rid
of these commas. Hopefully in the future Black has a way to opt out of
this.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-08-27 11:31:29 -07:00
Omar Sandoval
7fb196cfbf vmtest: don't use onoatimehack on QEMU 5.1.0
As of QEMU commit a5804fcf7b22 ("9pfs: local: ignore O_NOATIME if we
don't have permissions") (in v5.1.0), QEMU handles O_NOATIME sanely, so
we don't need the LD_PRELOAD hack. Since we're adding a version check,
make the multidevs check based on the version, too.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-08-20 17:55:57 -07:00
Omar Sandoval
4e770fb18a Format imports with isort
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-08-20 16:55:07 -07:00
Omar Sandoval
d118fda740 vmtest: check that downloaded file is not truncated
My work VPN is apparently closing HTTP connections prematurely, which
exposed that urllib won't catch incomplete reads if copied through
shutil.copyfileobj(). Check it explicitly.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-07-27 12:19:46 -07:00
Omar Sandoval
d45aafe43c vmtest: manage: remove 3.16 blacklist
3.16 is EOL and no longer included in the list of releases.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-07-27 12:00:35 -07:00
Omar Sandoval
4de147e478 Add CONTRIBUTING.rst
This documents best practices for contributing to drgn. We now require a
DCO sign-off.

Also clean up some related areas in the documentation.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-07-07 17:44:02 -07:00
Omar Sandoval
8b264f8823 Update copyright headers to Facebook and add missing headers
drgn was originally my side project, but for awhile now it's also been
my work project. Update the copyright headers to reflect this, and add a
copyright header to various files that were missing it.
2020-05-15 15:13:02 -07:00
Omar Sandoval
e03563221d vmtest: use BusyBox shell script as init
Currently, vmtest runs a custom program as init in the guest that
communicates with the host via virtio-serial. This is a lot of C code
that turns out can be replaced with a shell script. In order to keep the
dependencies minimal, we use BusyBox.
2020-04-17 15:11:54 -07:00
Omar Sandoval
3adc8f2ffe vmtest: use -cpu host instead of kvm64
drgn may be compiled with some CPU-specific features (e.g.,
-march=native), so make sure that we support those features inside of
the VM, too.
2020-04-10 15:18:46 -07:00
Omar Sandoval
dabfe19719 Rewrite virtual machine testing setup
The current implementation of vmtest has a few issues:

1. Building drgn for each kernel version on Travis is slow, mostly
   because they don't all run in parallel.
2. For local, incremental testing, recreating the filesystem image and
   rebuilding drgn is slow, and syncing the code to the filesystem image
   is brittle.
3. The filesystem image is the only communication channel, and reading
   the exit status from the filesystem image is awkward and fragile.
4. Creating and accessing the filesystem image requires root.

This reworks vmtest to use the build on the host via VirtFS with a
simple agent on the guest that can execute arbitrary commands and return
the exit status. This has a few more moving parts but is faster and
saner overall.
2020-04-02 17:35:24 -07:00
Omar Sandoval
6d8d41565e vmtest: add -vmtestN localversion
Any changes of the vmtest kernel config will require a rebuild of all
kernels and a way to distinguish the rebuild. Add CONFIG_LOCALVERSION
which we will bump each time the config changes.
2020-03-31 12:36:03 -07:00
Omar Sandoval
9d4b214035 vmtest/manage: handle API errors in get_available_kernel_releases()
If the path doesn't exist, there are no available releases. Otherwise,
we need to check for other errors.
2020-03-31 12:36:03 -07:00
Omar Sandoval
2a31b052fa vmtest/manage: don't open-code dict.fromkeys() 2020-03-30 12:05:45 -07:00
Omar Sandoval
c55d199e0c vmtest/manage: always get dropbox token for --build-kernel-org
--build-kernel-org needs the token to get the available releases even if
we're not uploading.
2020-03-30 11:57:18 -07:00
Omar Sandoval
435a3440f3 vmtest/manage: chmod -x
This should just be executed as python3 -m vmtest.manage now.
2020-03-30 11:56:02 -07:00
Omar Sandoval
f73f4f4089 vmtest/manage: use util.nproc() instead of multiprocessing.cpu_count() 2020-03-30 10:52:30 -07:00
Omar Sandoval
0548926d96 vmtest/manage: add type annotations
And add docstrings to the functions with more complex signatures.
2020-03-30 10:12:39 -07:00
Omar Sandoval
32deb222f5 vmtest/manage: remove relocations from vmlinux
About half (!) of vmlinux comes from relocation sections (~130M out of
~250M). But, vmlinux is an ET_EXEC file, so relocations don't even apply
to it. We can massively shrink vmlinux by removing the unnecessary
sections.
2020-03-27 17:52:04 -07:00
Omar Sandoval
dda598068c vmtest/manage: move kernel config to separate file
Currently, the kernel config is embedded inside of manage.py. This is
inconvenient when building kernels manually for testing. Move it to a
separate file so that, e.g., it can be copied into the build tree
easily.
2020-03-27 17:43:21 -07:00
Omar Sandoval
739fe5686d vmtest: move to top-level directory 2020-03-27 17:13:44 -07:00