Commit Graph

1925 Commits

Author SHA1 Message Date
Omar Sandoval
f34f1c278f libdrgn/python: fix #includes in symbol.c
Our internal Buck build of drgn doesn't use -I$(srcdir) like automake
does, so #include "drgn.h" and #include "symbol.h" in
libdrgn/python/symbol.c don't work. "drgn.h" is included by "drgnpy.h",
so we can drop that one and use a relative path for "symbol.h" instead.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-04-03 11:26:18 -07:00
Omar Sandoval
ed6bd2f766 contrib/search_kernel_memory.py: translate vmap addresses
identify_address() won't find anything useful for a directly mapped
address for a page that is actually being used by vmap. There doesn't
seem to be a great way to translate a directly mapped address to a vmap
address other than walking every vmap area.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-04-03 01:55:03 -07:00
Omar Sandoval
bbbbf0262b helpers.common.memory: recognize vmap stacks in identify_address()
Vmap stacks are an important special case of vmap.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-04-03 01:54:11 -07:00
Omar Sandoval
48e0c51b0a helper.common.memory: recognize vmap addresses in identify_address()
This requires rearranging a couple of things: symbols from modules are
also vmap addresses, so we need to check for symbols first, and we need
to open-code slab_object_info to avoid some redundant operations.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-04-03 01:41:35 -07:00
Omar Sandoval
ab0dd37509 helpers.linux.mm: add find_vmap_area() and for_each_vmap_area()
These can be used to look at vmap/vmalloc allocations.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-04-03 01:39:32 -07:00
Omar Sandoval
c7717280ad helpers.common.memory: add print_annotated_memory() helper
This is similar to print_annotated_stack() except that it works on an
arbitrary memory range. It's useful for trying to find some context in
mystery memory.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-04-03 01:39:32 -07:00
Omar Sandoval
1232a404c6 Add 6.9 to supported kernels
Only stack_depot_fetch() needed to be updated.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-04-01 16:48:31 -07:00
Omar Sandoval
235fab31d7 helpers.linux.stackdepot: update stack_depot_fetch() for Linux 6.9 and 6.8.2
pool_index in depot_stack_handle_t was changed to be offset by 1, which
broke stack_depot_fetch(). We can hack around it with trial and error.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-04-01 16:45:06 -07:00
Omar Sandoval
68d8eb1b69 Add "Getting Help" section to README and docs main page
And mention the new Matrix room.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-27 16:25:00 -07:00
Omar Sandoval
524ab17952 Revert "vmtest.manage: work around broken 5.10 stable build"
This reverts commit 07ae81df91. The build
should be fixed by Linux stable commit cc6ddd6fa93e ("x86/paravirt: Fix
build due to __text_gen_insn() backport") (in v5.10.214).

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-27 09:45:48 -07:00
Omar Sandoval
b6c8f181c4 contrib: add script to search kernel memory
This is super useful as a last ditch effort to find what is referencing
a kernel object, for example.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-26 11:38:11 -07:00
Omar Sandoval
24b53f2702 tests: add infrastructure for test resources
We currently only have one test resource file, sample.coredump.zst, but
the tests for #332 will add more. Create a package, tests.resources, to
contain test resources and a function, get_resource(), to decompress
them. It can also be used on the command line:

  python3 -m tests.resources $resource_name

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-22 16:12:05 -07:00
Imran Khan
a6b2506456 contrib/irq: fix for v6.5 and later kernels.
v6.5 moved irq descriptors into a maple tree,
earlier they used to exist in a radix tree.
Change contrib/irq.py to accomodate this change.

Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
2024-03-21 11:02:57 -07:00
Omar Sandoval
2b67e0991f libdrgn: ppc64: use DRGN_ERROR_NOT_IMPLEMENTED when virtual address translation is not supported
See #391.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-20 15:21:25 -07:00
Stephen Brennan
f96a3f59e0 Add test for Symbol Finder API
Specify a "fake" symbol finder and then test that its results are
plumbed through the API successfully. While this is a contrived test, it
helps build confidence in the plumbing of the API.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-03-11 16:43:43 -07:00
Stephen Brennan
dbc95bc7d1 python: Add Program.add_symbol_finder()
Expose the Symbol finder API so that Python code can be used to lookup
additional symbols by name or address.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-03-11 16:43:43 -07:00
Stephen Brennan
d1ebf5e9fe python: Allow construction of Symbol objects
Previously, Symbol objects could not be constructed in Python. However,
in order to allow Python Symbol finders, this needs to be changed.
Unfortunately, Symbol name lifetimes are tricky to manage. We introduce
a lifetime enumeration to handle this. The lifetime may be "static",
i.e. longer than the life of the program; "external", i.e. longer than
the life of the symbol, but no guarantees beyond that; or "owned", i.e.
owned by the Symbol itself.

Symbol objects constructed in Python are "external". The Symbol struct
owns the pointer to the drgn_symbol, and it holds a reference to the
Python object keeping the name valid (either the program, or a PyUnicode
object).

The added complexity is justified by the fact that most symbols are from
the ELF file, and thus share a lifetime with the Program. It would be a
waste to constantly strdup() these strings, just to support a small
number of Symbols created by Python code.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-03-11 16:43:43 -07:00
Stephen Brennan
9e5bf58bc7 libdrgn: move elf_symbols_search to debug_info.c
Now that the symbol finder API is created, we can move the ELF symbol
implementation into the debug_info.c file, where it more logically
belongs. The only change to these functions in the move is to declare
elf_symbols_search as static.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-03-11 16:43:43 -07:00
Stephen Brennan
37024146eb libdrgn: Use Symbol Finder API in find_symbol_by_address_internal()
The drgn_program_find_symbol_by_address_internal() function is used when
libdrgn itself may want to lookup a symbol: in particular, when
formatting stack traces or objects. It does less work by possibly
already having a Dwfl_Module looked up, and by avoiding memory
allocation of a symbol, and it's more convenient because it doesn't
return any errors, including on lookup failure.

Unfortunately, the new symbol finder API breaks all of these properties:
the returned symbol is now allocated via malloc() which needs cleanup on
error, and errors can be returned by any finder via the lookup API.
What's more, the finder API doesn't allow specifying an already-known
module. Thankfully, error handling can be improved using the cleanup
API, and looking up a module for an address is usually a reasonably
cheap binary tree operation.

Switch the internal method over to the new finder API. The major
difference now is simply that lookup failures don't result in an error:
they simply result in a NULL symbol.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-03-11 16:43:43 -07:00
Stephen Brennan
65dfa3dd9b libdrgn: move find_symbol_by_address_internal
The following commit will modify it to use
drgn_program_symbols_search(), a static function declared below. Move it
underneath in preparation. No changes to the function.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-03-11 16:43:43 -07:00
Stephen Brennan
ff322c7070 libdrgn: introduce Symbol Finder API
Symbol lookup is not yet modular, like type or object lookup. However,
making it modular would enable easier development and prototyping of
alternative Symbol providers, such as Linux kernel module symbol tables,
vmlinux kallsyms tables, and BPF function symbols. To begin with, create
a modular Symbol API within libdrgn, and refactor the ELF symbol search
to use it.

For now, we leave drgn_program_find_symbol_by_address_internal() alone.
Its conversion will require some surgery, since the new API can return
errors, whereas this function cannot.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-03-11 16:43:43 -07:00
Stephen Brennan
d211d35294 libdrgn: shrink symbol binding & kind enums
By using __attribute__((__packed__)), we shrink each enum from the
default integer size of four bytes, down to the minimum size of one.

This reduces the size of drgn_symbol from 32 bytes down to 26, with 6
bytes of padding. It doesn't have a practical benefit yet, but adding
fields to struct drgn_symbol in the future may not increase the size.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-03-11 16:43:43 -07:00
Omar Sandoval
757f2eba33 drgn 0.0.26
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-11 16:21:20 -07:00
Omar Sandoval
1515b4a3b7 docs: add 0.0.26 release highlights
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-11 16:20:34 -07:00
Omar Sandoval
45f7f6dce1 Move bpf_inspect.py from tools to contrib
We haven't kept up the maintenance of this tool, which goes against the
goal that the tools directory contains maintained, tested tools. Move it
to contrib. If it gets some love, we can always move it back.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-11 15:47:20 -07:00
Omar Sandoval
f039ce22e8 CI: update GitHub action versions
There's another required Node upgrade:
https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
The only functional change as far as I can tell is that
actions/upload-artifact@v4 doesn't support merging artifacts, which we
don't really care about for kernel build logs.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-11 14:11:45 -07:00
Omar Sandoval
7e60bc7ac3 vmtest: add patch to fix /proc/vmcore reads on s390x on Linux 5.18 and 5.19
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-11 13:52:25 -07:00
Omar Sandoval
8eb0d23538 helpers.linux.print: add print_dmesg() shortcut
Simply printing dmesg currently requires something like
print(get_dmesg().decode()), which is quite verbose. Josef Bacik
complained about this, so let's add a shortcut helper.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-11 11:28:35 -07:00
Omar Sandoval
57c3c8ff1f drgndoc: strip "_typeshed." from type annotations
The next change will use an annotation from _typeshed, which looks ugly
in the generated documentation. Let's strip it when appropriate.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-11 11:25:55 -07:00
Omar Sandoval
dbff45522d helpers.common.stack: make print_annotated_stack() more robust
print_annotated_stack() assumes a generally well-formed stack trace, but
in practice, stack traces can be very messy: there can be frames on
separate stacks without being marked as interrupted and all sorts of
garbage at the end of the stack trace. Let's try to handle a couple of
cases of suspicious stack pointers.

Fixes #305.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 16:46:43 -08:00
Omar Sandoval
9b8a4db6dd vmtest: skip test_slab_cache_for_each_allocated_object when running under emulation
This test is too slow to run with full system emulation.

It might be worth using pytest marks [1] to mark slow tests more
generally, but for now, this is the only one problematic enough.

1: https://docs.pytest.org/en/stable/how-to/mark.html

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 15:58:21 -08:00
Omar Sandoval
f4e43ddf14 tests: hack around crashed thread stack trace test failures on s390x
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 15:23:40 -08:00
Omar Sandoval
1be320ade9 vmtest: don't try testing on known-broken version+architecture combinations
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 14:05:19 -08:00
Omar Sandoval
b568474a14 vmtest: fix mypy errors
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 13:47:06 -08:00
Omar Sandoval
bab4f43d68 tests: replace fork_and_sigwait() and fork_and_call() with fork_and_stop()
Before Linux kernel commit 4e57a4ddf6b0 ("ARM: 9107/1: syscall: always
store thread_info->abi_syscall") (in v5.15), on Arm, the syscall number
in /proc/<pid>/syscall is unreliable unless the process is being traced.
fork_and_sigwait() relies on this to detect when the created process has
scheduled out for good. Instead, we can have the created process raise
SIGSTOP and wait for it to be stopped. This is simpler and also doesn't
require us to care about the sigwait syscall numbers. While we're
reworking it, let's also consolidate it with fork_and_call().
test_task_state_to_char() can't use the new function because it wants
the function to sleep, then stop, then die, but it's easy enough to
open-code that one special case.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 11:57:29 -08:00
Omar Sandoval
fca8b9859d tests: disable get_kconfig() test on Arm
elfutils commit c1c1c06e30f0 ("libebl: Add ebl_func_addr_mask plus ARM
backend implementation.") has a bug that I haven't gotten around to
fixing: it masks the least significant bit of all symbol values, not
just function symbol values. This breaks the get_kconfig() helper: if
the kernel_config_data_end symbol value is odd, then the length of the
compressed config data is truncated by one byte and gzip decompression
fails. Disable the test on Arm until we get it fixed.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 11:57:29 -08:00
Omar Sandoval
e5df46f2f2 vmtest.kbuild: add patch to get usable CFI on s390x on Linux 4.9 and 4.14
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 11:57:29 -08:00
Omar Sandoval
33f9d4c369 vmtest.kbuild: add patch to make kdump work on ppc64 on Linux 6.2 and 6.3
This fix was backported to other stable kernel versions from 6.6, but
not to 6.2 or 6.3.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 02:23:30 -08:00
Omar Sandoval
8ac2cbf7b5 vmtest.kbuild: add more 32-bit Arm headers to AArch64 packages
The drgn_test module build started failing on Linux 4.9 on AArch64 again
(maybe because of the new config options?). Add more arch/arm headers
that were being referenced from arch/arm64.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 02:23:30 -08:00
Omar Sandoval
85bb738a9f vmtest.kbuild: add patch to get .debug_frame on old AArch64 kernels
A few older kernel versions are missing the .debug_frame section because
the compiler is generating .eh_frame instead, only for it to get
discarded by the linker script. Backport a patch to disable .eh_frame
generation to the kernel versions that we care about.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 02:23:30 -08:00
Omar Sandoval
52c2a1c4b3 vmtest.vm: kill QEMU on exception or signal
If an exception is raised after QEMU is started, then the
subprocess.Popen context manager waits for the QEMU process to exit,
which it likely never will. If vmtest.vm gets killed with SIGTERM, the
QEMU process continues running. Fix both of these issues by terminating
the QEMU process if there is an exception and adding a signal handler
for SIGTERM to exit gracefully.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 02:23:30 -08:00
Omar Sandoval
a0a86364a8 libdrgn: memory_reader: indicate when fault is for physical memory
It can be confusing and misleading to see a FaultError for a strange
address that is actually physical.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-08 02:23:30 -08:00
Omar Sandoval
a5da128bb1 tools/fsrefs.py: check for references from uprobes
This one is pretty involved to implement and to test.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-06 16:18:45 -08:00
Omar Sandoval
3272a62f3b tools/fsrefs.py: check for references from swap files
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-06 16:18:45 -08:00
Omar Sandoval
9cf7768ce8 tools/fsrefs.py: check for references from loop devices
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-06 16:18:45 -08:00
Omar Sandoval
bb61dc78a0 tools/fsrefs.py: check for references from binfmt_misc
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-06 16:18:45 -08:00
Omar Sandoval
71da69c9c4 tests: add fork_and_call()
Like fork_and_sigwait(), but returns the called function's return value,
and only waits for the function to return, not sigwait.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-06 16:18:45 -08:00
Omar Sandoval
bb137f1887 tools/fsrefs.py: add mode to find references to a filesystem/super block
In this mode, we print the paths of the referenced files. Now that we
have multiple "checks" we're doing, also add an option to enable or
disable specific checks.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-06 16:18:45 -08:00
Omar Sandoval
f8851c54f0 tools/fsrefs.py: add missing sys import
It's only missing when used as a library (e.g., by the tests).

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-06 15:24:32 -08:00
Omar Sandoval
479eb47a4e vmtest.vm: make Ctrl-C work in VM
Apparently the kernel starts the init process with stdin and stdout set
to /dev/console, which cannot be a controlling tty, which is required
for stuff like Ctrl-C and shell job control to work. This can apparently
be worked around with setsid -c; see
https://github.com/systemd/systemd/issues/1431#issuecomment-393347607.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-03-06 15:13:24 -08:00