It's quite useful to run drgn's interactive REPL from a user-provided
script. One reason is saving the drgn core dump and debuginfo setup into
a script to more easily start analysis. Another might be to replace
drgn's debuginfo loading scheme with one customized to a different
environment. Or, a script might want to handle errors by allowing an
interactive user to examine the program themselves.
To serve all of these cases and more, separate out the logic of running
the drgn REPL from the internal logic relating to argument parsing,
choosing the correct target, and loading debuginfo. While this may seem
like a simple case of code.interact, there are several important
internal features drgn provides in its CLI: a custom displayhook, a
custom readline completer, a history file, and of course the logic to
import and include the proper drgn APIs and helpers for a core dump.
This means creating a run_interactive() function and moving the cli
module out of the drgn.internal package.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Commit c48cddbdb0 ("libdrgn: ppc64: fix stack unwinding since Linux
v5.11 and before v4.20") is the only change necessary for Linux v6.2.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
It's useful to check in on a long run without needing to scroll back and
check whether each kernel version passed or failed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're getting a bunch of test failures with "HTTP Error 403: rate limit
exceeded" from the GitHub API. The API apparently includes additional
information in the response, so let's print that so that we can figure
out which limit we're hitting.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I wanted to add tests for the radix tree helpers, but found that they
have never worked for Linux 4.4. Linux 4.4 has been EOL for almost a
year, so rather than going out of our way to support it, let's just stop
testing it. I'm not going to go back and remove existing support for
4.4 (yet), but I'm also not going to make sure future changes support
it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn is currently licensed as GPLv3+. Part of the long term vision for
drgn is that other projects can use it as a library providing
programmatic interfaces for debugger functionality. A more permissive
license is better suited to this goal. We decided on LGPLv2.1+ as a good
balance between software freedom and permissiveness.
All contributors not employed by Meta were contacted via email and
consented to the license change. The only exception was the author of
commit c4fbf7e589 ("libdrgn: fix for compilation error"), who did not
respond. That commit reverted a single line of code to one originally
written by me in commit 640b1c011d ("libdrgn: embed DWARF index in
DWARF info cache").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This will be useful for testing different architectures (e.g., by
creating a rootfs with debootstrap) or different userspace environments.
For example, we can test busybox with something like:
mkdir busyboxroot busyboxroot/bin busyboxroot/tmp
cp "$(which busybox)" busyboxroot/bin
busyboxroot/bin/busybox --install busyboxroot/bin
python3 -Bm vmtest.vm -k '*' -r busyboxroot
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This still isn't enough to be fully generic since since some
architectures don't support kexec_file_load, especially on older
kernels, and some architectures can't use the vmlinuz directly (e.g.,
AArch64 may need it to be decompressed). But, this is at least cleaner
than hard-coding the number for x86-64.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Replace the old "Scriptable debugger library" and
"Debugger-as-a-library" taglines with the one we're using on GitHub,
"Programmable debugger". Make up for it by emphasizing that drgn can
also be used as a library a tiny bit more in the README.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
cgroup_bpf_prog_for_each() needed a minor update, but after fixing that,
all of the flavors pass all tests.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We don't specifically need BusyBox; we just need a reasonable Linux
userspace, which we can assume is already available on the host, whether
it's coreutils+util-linux, BusyBox, or something else.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The test command does this, and I always end up doing it when I'm doing
manual testing with the vmtest.vm CLI, so let's just do it by default.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This needed the previous small update to
tests.linux_kernel.test_debug_info.TestModuleDebugInfo.
tests.linux_kernel.helpers.test_tc.TestTc will also only work with
pyroute2 >= 0.6.10 (see svinota/pyroute2#899). No changes needed to drgn
itself.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Splitting the tests between tests/linux_kernel and tests/helpers/linux
means that we have to set up the unit tests twice, including loading
debug info. Python 3.7 and newer have a way to get around this, but
we're still sort of supporting Python 3.6. Move them under one path to
speed up test runs.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that commit d999703f94 ("vmtest: add kernel module build
dependencies to kernel packages") added the files necessary to build a
test kernel module, add the module (currently a stub) and the
scaffolding necessary to build and load it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The tiny flavor was failing due to a bug fixed in Linux kernel commit
c12cd77cb028 ("mm/vmalloc: fix spinning drain_vmap_work after reading
from /proc/vmcore") (in v5.18-rc3). Now that the fix is in, we can add
5.18 with no changes required.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently log "Tests in VM returned $STATUS" when vmtest returns. To
find out what kernel version just returned, you have to scroll up to
find a previous log line that mentions the version. Make things more
convenient by adding the version to that message.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
tests/helpers/linux contains test cases for Linux kernel helpers and
test cases for core Linux kernel support. The latter don't make sense
there; move them to tests/linux_kernel instead, along with the
scaffolding.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Black 22.1.0 has some style changes: string prefixes are normalized and
spaces around the power operator are removed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, Python is only required to build the Python bindings. I
originally wanted to avoid having Python as a build dependency of
libdrgn, which is why gen_arch is an AWK script. However, I want to add
another code generation script which is harder to do in AWK.
Additionally, these days more people are familiar with Python than AWK,
so let's just bite the bullet and require Python to build. No one builds
libdrgn by itself anyways.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that the vmtest kernel supports kdump, add a script that can be used
to crash and enter the kdump environment on demand. Use that to crash
after running the normal test suite so that we can run tests against
/proc/vmcore. vmcore tests live in their own directory; presently the
only test is a simple sanity check that ensures we can can attach to
/proc/vmcore.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Mainly unused imports, unused variables, unnecessary f-strings, and
regex literals missing the r prefix. I'm not adding it to the CI linter
because it's too noisy, though.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
As noted by commit 738261290f ("CI: temporarily disable vmtest"),
vmtest was generating too much traffic to the Dropbox shared folder that
hosted vmtest kernels. Instead, we can store kernel packages as GitHub
release assets. Update the code for downloading and uploading vmtest
assets, and also add a scheduled GitHub action to build new kernels
every Monday so I don't have to remember to do it manually.
This also drops vmtest support for 5.6-5.9, which now fail to build with
newer binutils due to the issue fixed in Linux kernel commit
1d489151e9f9 ("objtool: Don't fail on missing symbol table").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Building drgn from an sdist currently requires autotools and gawk
because libdrgn in the sdist is more or less a git checkout. It's more
user-friendly to include the autotools output and generated code. Do
this by extending the sdist command to include a full libdrgn
distribution with `make distdir`.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently bundle a version of elfutils with patches to export
additional stack tracing functionality. This has a few drawbacks:
- Most of drgn's build time is actually building elfutils.
- Distributions don't like packages that bundle verions of other
packages.
- elfutils, and thus drgn, can't be built with clang.
Now that we've replaced the elfutils DWARF unwinder with our own, we
don't need the patches, so we can drop the bundled elfutils and fix
these issues.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
mypy 0.800 is stricter about reexports: "from foo import X as Y" is only
considered a reexport if X and Y are the same name (see
python/mypy#9515). mypy 0.800 fails with:
drgn/internal/cli.py:46: error: Module has no attribute "__version__"
Rename drgn.internal.version.version to __version__ so that
drgn/__init__.py can reexport it with import __version__ as __version__.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We get the version of drgn with pkg_resources.get_distribution() in two
places: setup.py (when using an sdist) and the CLI. The former causes
problems because in some cases, pip doesn't find the drgn distribution
that's currently being built. The latter adds significant latency to
startup. On my laptop, just importing pkg_resources takes 130 ms. We can
solve both of these problems by generating a file containing the version
instead.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently build with CONFIG_MODULES=n for simplicity. However, this
means that we don't test kernel module support at all. Let's enable
module support. This requires changing how we distribute kernels. Now,
the /lib/modules/$(uname -r) directory (including the vmlinux and
vmlinuz) is bundled up as a tarball. We extract it, then mount it with
VirtFS, and do some extra setup for device nodes. (We lose the ability
to run kernel builds directly, but I've never actually used that
functionality.)
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Linux 5.5 built by GCC 10 doesn't boot due to new optimizations breaking
how the kernel sets up the stack canary
(https://lkml.org/lkml/2020/3/14/186 has the details). 5.5 has been EOL
since April, so the fix was never backported. We could backport it
ourselves for vmtest builds or build with GCC 9, but it's not worth the
extra effort to test an EOL kernel. Let's just drop it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Black was recently changed to treat a trailing comma as an indicator to
put each item/argument on its own line. We have a bunch of places where
something previously had to be split into multiple lines, then was
edited to fit on one line, but Black kept the trailing comma. Now this
update wants to unnecessarily split it back up. For now, let's get rid
of these commas. Hopefully in the future Black has a way to opt out of
this.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
