There are places in drgn where it'd be a good idea to drop the Python
GIL. However, some of these are deep inside of libdrgn, where some code
paths are fast and dropping the GIL would be extra overhead and others
are slow (e.g., type lookups, which may be cached or may require DWARF
namespace indexing). Instead of trying to do this from the Python
bindings, add hooks to libdrgn. These hooks can be used directly or with
a new scope guard macro, drgn_blocking_guard, that we can start
sprinkling around in appropriate places in libdrgn.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're getting (null) file paths in error messages (e.g., #233) because
libdwfl doesn't always return the debug file path. Fall back to the
loaded file path, which is better than nothing until we get rid of
libdwfl.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
My Arm VM fails to boot on QEMU 7.2.1 after the following sequence of
events on the kernel console:
pci-host-generic 4010000000.pcie: can't claim ECAM area [mem 0x10000000-0x1fffffff]: address conflict with pcie@10000000 [mem 0x10000000-0x3efeffff]
pci-host-generic: probe of 4010000000.pcie failed with error -16
...
9pnet_virtio: no channels available for device /dev/root
VFS: Cannot open root device "" or unknown-block(0,0): error -2
Please append a correct "root=" boot option; here are the available partitions:
Can't find any bdev filesystem to be used for mount!
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
Turning off highmem fixes the conflict. (I think this previously worked
without highmem=off on Arch Linux, so maybe there's something different
in Fedora's QEMU.)
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I sent a patch titled "powerpc/crypto: fix missing skcipher dependency
for aes-gcm-p10" [1] to fix the build failure, but we also have no need
for this module anyways.
1: https://lists.ozlabs.org/pipermail/linuxppc-dev/2023-July/260369.html
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Commit 18b12a5c7b ("libdrgn: get .eh_frame from the correct file")
missed this, but it's unlikely to matter in practice.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Move the logic for local kernels from vmtest.kmod and vmtest.vm to
vmtest.config and use it in setup.py.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
These opcodes appear in practice, and we choke on them with an exception
like "unknown DWARF expression opcode 0xf3" or "unknown DWARF expression
opcode 0xa3". In some cases, it'd be possible to recover the entry value
by looking at call site information, but that's pretty involved. For
now, just treat these operations as optimized out so we stop failing
hard.
Closes#233.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
When we get the DIE from the offset with dwarf_offdie(), there's no need
to go back to the offset with dwarf_dieoffset().
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Thierry found that as soon as drgn_module_find_dwarf_scopes() finds any
DIE containing the PC, it walks the entire subtree rooted at that DIE.
However, we only need to look at the immediate children of a DIE
containing the PC. I think this is what I originally intended, but I
failed to reset the children flag to false when the last DIE didn't
contain the PC. Thierry's suggested check of it.dies.size == subtree is
simpler.
This is a massive performance improvement: for a kernel core dump with
10k threads, getting the stack trace of every thread took ~90 seconds
without this fix and ~50 seconds with it.
Let's also add a comment to this very subtle code.
Fixes: d8d4157346 ("libdrgn: debug_info: add drgn_debug_info_module_find_dwarf_scopes()")
Co-authored-by: Thierry Treyer <ttreyer@fb.com>
Signed-off-by: Thierry Treyer <ttreyer@fb.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
No need to duplicate building the path, and this way if we ever change
the location it won't need to be updated here.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I've needed this many times, but there wasn't a corresponding function
in the kernel so I could never decide what to name it. Linux kernel
commit 4d70c74659d9 ("i915: Move list_count() to list.h as
list_count_nodes() for broader use") (in v6.3-rc1) fixed that problem
for me.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We can trivially call the kexec_file_load() syscall on x86-64, but it's
more complicated for other architectures. We can call the kexec utility
for those.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
My kernel patch was merged for Linux 6.4 and backported to 6.3.10, so
now we can use the .orc_header section to reliably detect the ORC format
version. Since the 6.4 release candidates and older versions of 6.3
don't have .orc_header, we'll keep the version check as a fallback.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
__schedule tends to end up aliased with other special symbols because
it's placed in the .sched.text section. We're handling
__sched_text_start, but on ppc64, I also saw __cpuidle_text_end. Use
drgn_test_function from the test kernel module instead.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The test kernel module has several -Woverflow warnings when compiled for
Arm:
/home/osandov/repos/drgn/build/vmtest/arm/tmp9g8977mn/drgn_test.c:785:9: warning: unsigned conversion from 'long long int' to 'long unsigned int' changes value from '6221254864074593878' to '1448498774' [-Woverflow]
785 | 0x5656565656565656,
| ^~~~~~~~~~~~~~~~~~
/home/osandov/repos/drgn/build/vmtest/arm/tmp9g8977mn/drgn_test.c:786:9: warning: unsigned conversion from 'long long int' to 'long unsigned int' changes value from '1311768465173141112' to '305419896' [-Woverflow]
786 | 0x1234567812345678,
| ^~~~~~~~~~~~~~~~~~
/home/osandov/repos/drgn/build/vmtest/arm/tmp9g8977mn/drgn_test.c:787:9: warning: unsigned conversion from 'long long int' to 'long unsigned int' changes value from '1311768467294899695' to '2427178479' [-Woverflow]
787 | 0x1234567890abcdef,
| ^~~~~~~~~~~~~~~~~~
drgn_test_idr_dense and drgn_test_idr_ptrs aren't actually used by the
tests, so remove them.
Fixes: 4f2c8f0735 ("tests: idr: add test cases for idr.")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The setuptools guide for porting from distutils [1] recommends this.
However, the setuptools versions aren't available on older versions of
setuptools, so we still need to fall back to distutils.
1: https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html
Signed-off-by: Omar Sandoval <osandov@osandov.com>
distutils is deprecated and was removed in Python 3.12 [1]. setuptools
provides a vendored copy of distutils, but they discourage against using
it [2]. So, let's stop importing distutils, starting with replacing our
calls of distutils.file_util.copy_file() and distutils.dir_util.mkpath()
with their equivalent methods from setuptools.Command.
1: https://peps.python.org/pep-0632/
2: https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html
Signed-off-by: Omar Sandoval <osandov@osandov.com>
6.4 needed an unusual number of changes:
- 3f3a957562 ("libdrgn: linux_kernel: Fix module detection on kernel
v6.4")
- da7a11f83e ("drgn.helpers.linux.block: update
for_each_{disk,partition} for Linux 6.4")
- 7e7c300a139b ("libdrgn: orc_info: handle ORC changes in Linux 6.3 and
6.4")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The ORC format changed twice recently:
- Linux kernel commit ffb1b4a41016 ("x86/unwind/orc: Add 'signal' field
to ORC metadata") (in v6.3).
- Linux kernel commit fb799447ae29 ("x86,objtool: Split
UNWIND_HINT_EMPTY in two") (in v6.4).
The former went unnoticed because the change was subtle, and the latter
completely broke x86-64 kernel stack traces.
To handle this, let's "upgrade" the format to the latest version when we
load and sort the ORC information. This is more work upfront but avoids
needing to handle the version differences every time we use ORC to
unwind.
Unfortunately, ORC currently doesn't have any sort of versioning, so we
have to break the rule of not checking kernel versions. However, I have
a kernel patch pending merging that should fix this for the future.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
It's unrealistic for there to be more than 4 billion ORC entries. Switch
to an unsigned int. The main benefit is that the indices array that we
use to sort the parallel arrays of entries and pc_offsets becomes half
the size, which also makes parsing ORC about 10% faster (down from ~5 ms
to ~4.5 ms for the Fedora vmlinux on my laptop).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
.orc_unwind_ip and .orc_unwind are only referenced while initially
parsing ORC data and then never touched again, so it's wasteful to cache
them in struct drgn_elf_file. Look them up if and when we parse the ORC
data instead.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In practice, the .orc_unwind and .orc_unwind_ip sections will always be
suitably aligned. Check it, then assume the alignment later.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
By using the same temporary objects in the Linux 6.4 branch as the
pre-6.4 branch, we get slightly better code generation.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We only support .debug_* sections, but libdw also supports .zdebug_*,
.debug_*.dwo, and .gnu.debuglto_.debug_*. Mimic how libdw chooses debug
sections, with one exception: .debug_cu_index and .debug_tu_index (used
for DWP, which we don't support yet but will) should be considered DWO
sections (this needs to be fixed in libdw, too).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
With 3.12 beta, there are 7 Python versions to test, which is making the
test matrix too large. To reduce the amount of CI test runs that occur,
by default we will only run 2: the oldest and newest released Python
versions. To further reduce the CI test runs, we will eliminate CI runs
when a branch is pushed (only main will be tested).
The "ci" action becomes a reusable workflow, which gets called by other
possible workflows:
1. pull_request.yml: On pull request, only the 2 default Python versions
are tested. The "test-all-python-versions" label can be applied to
pull requests to override this.
2. push.yml: On push to main, all Python versions are tested.
3. vmtest-build.yml: After the vmtest kernels are built
4. On manual request: the user has the option to specify
Signed-off-by: Stephen Brennan <stephen@brennan.io>
ORC_REG_SP_INDIRECT is supposed to be an indirect access via rsp, but we
have a typo and are using rbp instead. This is a partial fix for #304.
Fixes: 630d39e345 ("libdrgn: add ORC unwinder")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Instead of simply checking whether the thread ID is non-zero, write a
specific task name to /proc/self/comm before crashing and check that.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently use crashing_cpu to determine the thread that caused a
kernel crash. However, crashing_cpu is x86-specific (it is defined in
arch/x86/kernel/reboot.c). Since Linux 4.5, the generic panic code
defines a very similar variable, panic_cpu. Use that instead so that we
support all architectures, but fall back to crashing_cpu to support
older kernels on x86 (even though we don't claim to support 4.4
anymore).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
btrfs_tree.py is a pile of helpers Omar gave me once that I have
cherished ever since. It mostly deals with walking and parsing btrees.
btrfs_tree_mod_log.py is a script I wrote to do a tree mod log search
and rewind while debugging a btrfs issue.
Signed-off-by: Boris Burkov <boris@bur.io>
follow_{page,pfn,phys}() translate the virtual address by walking the
page table for a given mm_struct (built on top of the existing page
table iterator interface). vmalloc_to_page() and vmalloc_to_pfn() are
special cases for vmalloc addresses.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Applying the 5.10-stable patches fixing BTF with older pahole to
5.11-5.13 missed updating the pahole flags for kernel modules (because
kernel modules didn't have BTF in 5.10). Regenerate the patches for 5.11
ourselves.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Commit bf9159cff367 ("drgn.helpers.linux.mm: fix compound_{order,nr} for
Linux 6.3+") is the only change necessary for Linux v6.3.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Linux 6.3 removed struct page::compound_order. Instead, we need to use
struct folio::_folio_order. Update compound_order() and redefine
compound_nr() in terms of compound_order() so we only need to handle the
divergence in one place.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Linux kernel commits 884f8ce42cce ("driver core: class: implement
class_get/put without the private pointer.") and 2df418cf4b72 ("driver
core: class: remove subsystem private pointer from struct class") (in
v6.4) changed the way to go from struct class to struct subsys_private.
_for_each_block_device(), used by for_each_disk() and
for_each_partition(), needs struct subsys_private. The "new" way also
works on old kernels, so add _class_to_subsys() mirroring
class_to_subsys() in Linux 6.4 and make _for_each_block_device() use it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
