The setuptools guide for porting from distutils [1] recommends this.
However, the setuptools versions aren't available on older versions of
setuptools, so we still need to fall back to distutils.
1: https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html
Signed-off-by: Omar Sandoval <osandov@osandov.com>
distutils is deprecated and was removed in Python 3.12 [1]. setuptools
provides a vendored copy of distutils, but they discourage against using
it [2]. So, let's stop importing distutils, starting with replacing our
calls of distutils.file_util.copy_file() and distutils.dir_util.mkpath()
with their equivalent methods from setuptools.Command.
1: https://peps.python.org/pep-0632/
2: https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html
Signed-off-by: Omar Sandoval <osandov@osandov.com>
6.4 needed an unusual number of changes:
- 3f3a957562 ("libdrgn: linux_kernel: Fix module detection on kernel
v6.4")
- da7a11f83e ("drgn.helpers.linux.block: update
for_each_{disk,partition} for Linux 6.4")
- 7e7c300a139b ("libdrgn: orc_info: handle ORC changes in Linux 6.3 and
6.4")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Commit bf9159cff367 ("drgn.helpers.linux.mm: fix compound_{order,nr} for
Linux 6.3+") is the only change necessary for Linux v6.3.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
My previous change to call git unconditionally caused Packit failures,
presumably because git isn't installed for the build:
Traceback (most recent call last):
File "/builddir/build/BUILD/drgn-0.0.22+71.g2f97cc0/setup.py", line 411, in <module>
version=get_version(),
^^^^^^^^^^^^^
File "/builddir/build/BUILD/drgn-0.0.22+71.g2f97cc0/setup.py", line 343, in get_version
subprocess.call(
File "/usr/lib64/python3.11/subprocess.py", line 389, in call
with Popen(*popenargs, **kwargs) as p:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/subprocess.py", line 1024, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib64/python3.11/subprocess.py", line 1901, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'git'
We shouldn't bother calling git if .git doesn't exist.
Fixes: 2e9e66cedc ("setup.py: use stricter check for git repository")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
To test builds on other architectures, I'm setting up a Debian chroot
with debootstrap and then building with something like:
mount --bind . "$chroot/mnt"
chroot "$chroot" sh -c 'cd /mnt && python3 setup.py build_ext -i'
However, if I do this from a Git worktree, this fails with an error from
the git call to determine the local version:
fatal: not a git repository: /home/osandov/repos/drgn/.git/worktrees/drgn2
Instead of only checking whether .git exists, use `git --git-dir=.git
rev-parse` to verify that .git is valid. (This will of course rely on
drgn/internal/version.py already having been generated.)
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now we'll always get the same compiler that was used for the actual
kernel build. Also split this into its own runnable module, vmtest.kmod,
for testing and custom scripting.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add the per-architecture QEMU options to the Architecture class and
allow specifying an architecture in -k.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
It's quite useful to run drgn's interactive REPL from a user-provided
script. One reason is saving the drgn core dump and debuginfo setup into
a script to more easily start analysis. Another might be to replace
drgn's debuginfo loading scheme with one customized to a different
environment. Or, a script might want to handle errors by allowing an
interactive user to examine the program themselves.
To serve all of these cases and more, separate out the logic of running
the drgn REPL from the internal logic relating to argument parsing,
choosing the correct target, and loading debuginfo. While this may seem
like a simple case of code.interact, there are several important
internal features drgn provides in its CLI: a custom displayhook, a
custom readline completer, a history file, and of course the logic to
import and include the proper drgn APIs and helpers for a core dump.
This means creating a run_interactive() function and moving the cli
module out of the drgn.internal package.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Commit c48cddbdb0 ("libdrgn: ppc64: fix stack unwinding since Linux
v5.11 and before v4.20") is the only change necessary for Linux v6.2.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
It's useful to check in on a long run without needing to scroll back and
check whether each kernel version passed or failed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're getting a bunch of test failures with "HTTP Error 403: rate limit
exceeded" from the GitHub API. The API apparently includes additional
information in the response, so let's print that so that we can figure
out which limit we're hitting.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I wanted to add tests for the radix tree helpers, but found that they
have never worked for Linux 4.4. Linux 4.4 has been EOL for almost a
year, so rather than going out of our way to support it, let's just stop
testing it. I'm not going to go back and remove existing support for
4.4 (yet), but I'm also not going to make sure future changes support
it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn is currently licensed as GPLv3+. Part of the long term vision for
drgn is that other projects can use it as a library providing
programmatic interfaces for debugger functionality. A more permissive
license is better suited to this goal. We decided on LGPLv2.1+ as a good
balance between software freedom and permissiveness.
All contributors not employed by Meta were contacted via email and
consented to the license change. The only exception was the author of
commit c4fbf7e589 ("libdrgn: fix for compilation error"), who did not
respond. That commit reverted a single line of code to one originally
written by me in commit 640b1c011d ("libdrgn: embed DWARF index in
DWARF info cache").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This will be useful for testing different architectures (e.g., by
creating a rootfs with debootstrap) or different userspace environments.
For example, we can test busybox with something like:
mkdir busyboxroot busyboxroot/bin busyboxroot/tmp
cp "$(which busybox)" busyboxroot/bin
busyboxroot/bin/busybox --install busyboxroot/bin
python3 -Bm vmtest.vm -k '*' -r busyboxroot
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This still isn't enough to be fully generic since since some
architectures don't support kexec_file_load, especially on older
kernels, and some architectures can't use the vmlinuz directly (e.g.,
AArch64 may need it to be decompressed). But, this is at least cleaner
than hard-coding the number for x86-64.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Replace the old "Scriptable debugger library" and
"Debugger-as-a-library" taglines with the one we're using on GitHub,
"Programmable debugger". Make up for it by emphasizing that drgn can
also be used as a library a tiny bit more in the README.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
cgroup_bpf_prog_for_each() needed a minor update, but after fixing that,
all of the flavors pass all tests.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We don't specifically need BusyBox; we just need a reasonable Linux
userspace, which we can assume is already available on the host, whether
it's coreutils+util-linux, BusyBox, or something else.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The test command does this, and I always end up doing it when I'm doing
manual testing with the vmtest.vm CLI, so let's just do it by default.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This needed the previous small update to
tests.linux_kernel.test_debug_info.TestModuleDebugInfo.
tests.linux_kernel.helpers.test_tc.TestTc will also only work with
pyroute2 >= 0.6.10 (see svinota/pyroute2#899). No changes needed to drgn
itself.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Splitting the tests between tests/linux_kernel and tests/helpers/linux
means that we have to set up the unit tests twice, including loading
debug info. Python 3.7 and newer have a way to get around this, but
we're still sort of supporting Python 3.6. Move them under one path to
speed up test runs.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that commit d999703f94 ("vmtest: add kernel module build
dependencies to kernel packages") added the files necessary to build a
test kernel module, add the module (currently a stub) and the
scaffolding necessary to build and load it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The tiny flavor was failing due to a bug fixed in Linux kernel commit
c12cd77cb028 ("mm/vmalloc: fix spinning drain_vmap_work after reading
from /proc/vmcore") (in v5.18-rc3). Now that the fix is in, we can add
5.18 with no changes required.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently log "Tests in VM returned $STATUS" when vmtest returns. To
find out what kernel version just returned, you have to scroll up to
find a previous log line that mentions the version. Make things more
convenient by adding the version to that message.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
tests/helpers/linux contains test cases for Linux kernel helpers and
test cases for core Linux kernel support. The latter don't make sense
there; move them to tests/linux_kernel instead, along with the
scaffolding.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Black 22.1.0 has some style changes: string prefixes are normalized and
spaces around the power operator are removed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, Python is only required to build the Python bindings. I
originally wanted to avoid having Python as a build dependency of
libdrgn, which is why gen_arch is an AWK script. However, I want to add
another code generation script which is harder to do in AWK.
Additionally, these days more people are familiar with Python than AWK,
so let's just bite the bullet and require Python to build. No one builds
libdrgn by itself anyways.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that the vmtest kernel supports kdump, add a script that can be used
to crash and enter the kdump environment on demand. Use that to crash
after running the normal test suite so that we can run tests against
/proc/vmcore. vmcore tests live in their own directory; presently the
only test is a simple sanity check that ensures we can can attach to
/proc/vmcore.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Mainly unused imports, unused variables, unnecessary f-strings, and
regex literals missing the r prefix. I'm not adding it to the CI linter
because it's too noisy, though.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
As noted by commit 738261290f ("CI: temporarily disable vmtest"),
vmtest was generating too much traffic to the Dropbox shared folder that
hosted vmtest kernels. Instead, we can store kernel packages as GitHub
release assets. Update the code for downloading and uploading vmtest
assets, and also add a scheduled GitHub action to build new kernels
every Monday so I don't have to remember to do it manually.
This also drops vmtest support for 5.6-5.9, which now fail to build with
newer binutils due to the issue fixed in Linux kernel commit
1d489151e9f9 ("objtool: Don't fail on missing symbol table").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Building drgn from an sdist currently requires autotools and gawk
because libdrgn in the sdist is more or less a git checkout. It's more
user-friendly to include the autotools output and generated code. Do
this by extending the sdist command to include a full libdrgn
distribution with `make distdir`.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently bundle a version of elfutils with patches to export
additional stack tracing functionality. This has a few drawbacks:
- Most of drgn's build time is actually building elfutils.
- Distributions don't like packages that bundle verions of other
packages.
- elfutils, and thus drgn, can't be built with clang.
Now that we've replaced the elfutils DWARF unwinder with our own, we
don't need the patches, so we can drop the bundled elfutils and fix
these issues.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
