Commit Graph

2030 Commits

Author SHA1 Message Date
Srivathsa Dara
3ecda77f59 contrib: add script to dump filesystem details in btrfs fs_uuids cache
Btrfs maintains a fs_uuids cache, but there is no defined way to look
into it. This script reads the cache and prints details of the
filesystems present in it.

Signed-off-by: Srivathsa Dara <srivathsa.d.dara@oracle.com>
2024-08-07 16:16:39 -07:00
Omar Sandoval
9ad29fd864 drgn.helpers.common.memory: fix identify_address vmap identification with CONFIG_VMAP_STACK=n
In the uncached case, the task variable is supposed to be the task that
matched the vm area. But, if we detect that CONFIG_VMAP_STACK is
disabled, we break without clearing the task variable. Fix that, and
also make sure to test both the cached and uncached cases.

Closes #414.

Fixes: 69bd07ddb2 ("drgn.helpers.common.memory: add optional cache to identify_address()")
Reported-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-24 17:06:18 -07:00
Omar Sandoval
1832240fc6 tools/fsrefs.py: add mode to find references to a filesystem on a block device
Sometimes the filesystem mount point isn't available but the block
device is.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-24 15:11:29 -07:00
Omar Sandoval
0909105e71 CI, vmtest: install btrfs-progs
An upcoming test needs mkfs.btrfs.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-24 15:11:29 -07:00
Omar Sandoval
9cc5ab5d90 tests: load kernel modules required for tests in test code, not vmtest
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-24 15:11:29 -07:00
Kuan-Ying Lee
bc673f5d78 contrib/vmallocinfo: add scripts to dump vmallocinfo
Sample output:

python3 drgn -s vmlinux -c vmcore contrib/vmallocinfo.py

0xffff800080000000-0xffff800080005000      20480 start_kernel pages=4 vmalloc
0xffff800080005000-0xffff800080007000       8192 ioremap_prot phys=0x8020000 ioremap
0xffff800080008000-0xffff80008000d000      20480 start_kernel pages=4 vmalloc
0xffff80008000d000-0xffff80008000f000       8192 bpf_prog_alloc_no_stats pages=1 vmalloc
0xffff800082a00000-0xffff8000833d1000   10293248 paging_init phys=0x42c00000 vmap
0xffff800083400000-0xffff800083405000      20480 start_kernel pages=4 vmalloc
0xffff800083405000-0xffff800083408000      12288 pcpu_mem_zalloc pages=2 vmalloc
0xffff800083408000-0xffff80008340d000      20480 kernel_clone pages=4 vmalloc
0xffff80008340d000-0xffff80008340f000       8192 ioremap_prot phys=0x9030000 ioremap
0xffff800083480000-0xffff800083485000      20480 kernel_clone pages=4 vmalloc
0xffff800083488000-0xffff80008348d000      20480 kernel_clone pages=4 vmalloc
0xffff800083500000-0xffff800083505000      20480 fork_idle pages=4 vmalloc
...

Signed-off-by: Kuan-Ying Lee <kuan-ying.lee@canonical.com>
2024-07-24 09:45:14 -07:00
Kuan-Ying Lee
3540cfa992 contrib/slabinfo: add scripts to dump slabinfo
Sample output:

python3 drgn -s vmlinux -c vmcore contrib/slabinfo.py

struct kmem_cache *  |         name         | active_objs  |   num_objs   | objsize  | objperslab  | pagesperslab
-------------------- | -------------------- | ------------ | ------------ | -------- | ----------- | -------------
0xffff00000632cf40   | 9p-fcall-cache       |            0 |            0 |   131192 |           1 |            64
0xffff00000632cdc0   | p9_req_t             |            0 |           33 |      248 |          33 |             2
0xffff00000632cc40   | isp1760_qh           |            0 |            0 |      144 |          28 |             1
0xffff00000632cac0   | isp1760_qtd          |            0 |            0 |      168 |          24 |             1
0xffff00000632c940   | isp1760_urb_listite  |            0 |            0 |      120 |          34 |             1
0xffff00000632c7c0   | asd_sas_event        |            0 |            0 |      256 |          32 |             2
0xffff00000632c640   | sas_task             |            0 |            0 |      448 |          36 |             4
0xffff00000632c4c0   | bio-120              |           50 |           50 |      320 |          25 |             2
0xffff00000632c340   | io_buffer            |            0 |            0 |      160 |          25 |             1
0xffff00000632c1c0   | io_kiocb             |            0 |            0 |      448 |          36 |             4
0xffff00000632c040   | bfq_io_cq            |            0 |            0 |     1456 |          22 |             8
0xffff000006321e40   | bfq_queue            |            0 |            0 |      672 |          24 |             4
0xffff000006321cc0   | bio-248              |            4 |           36 |      448 |          36 |             4
0xffff000006321b40   | mqueue_inode_cache   |            1 |           30 |     1088 |          30 |             8
0xffff0000063219c0   | v9fs_inode_cache     |            1 |           39 |      832 |          39 |             8
0xffff000006321840   | nfs4_xattr_cache_ca  |            0 |            0 |     2216 |          14 |             8
...

Signed-off-by: Kuan-Ying Lee <kuan-ying.lee@canonical.com>
2024-07-24 09:45:14 -07:00
Omar Sandoval
f8168da4e2 drgn.internal.sudohelper: pass privileged command as string instead of path
At Meta, our Python executables are packaged in a way that doesn't leave
the original .py or .pyc files easily accessible or importable. This
makes the sudohelper fail with:

  can't open file '/proc/self/fd/3/drgn/internal/sudohelper.pyc': [Errno 2] No such file or directory

We can work around this by passing the helper command as a string
literal. It's ugly, but it works with our Meta package and with
traditional installations.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-19 13:42:09 -07:00
Stephen Brennan
7e8c5d7bb2 tests: Fix print_annotated_memory test on SLOB
The test for print_annotated_memory() shouldn't assert anything when
CONFIG_SLOB is enabled, because we can't find slab objects. However, the
test shouldn't be skipped entirely, because print_annotated_memory()
should run without error in this case. Skip the assertion only.

Fixes: c7717280 ("helpers.common.memory: add print_annotated_memory() helper")

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-07-18 19:14:48 -07:00
Stephen Brennan
545aa52fe1 mm, slab: Fix test failures on kernels with SLOB
In the Fixes commit, as part of the refactor, the
_identify_kernel_address() function started open-coding the
get_slab_object_info() function. Unfortunately, this resulted in the
function failing when CONFIG_SLOB was enabled, since the AttributeError
for accessing "slab.slab_cache" was not caught.

Open-coding was done to avoid the duplicate bounds check for
in_direct_map. In fact, in_direct_map seems like a useful helper itself,
and by factoring it out of _find_containing_slab() and into its callers,
we can simplify things a bit. Since _find_containing_slab() can handle
SLOB, this fixes the test failure too.

Fixes: 48e0c51b ("helper.common.memory: recognize vmap addresses in identify_address()")

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-07-18 19:14:48 -07:00
Stephen Brennan
46da1ca370 setup.py, CI: avoid ENOSPC on Github Actions
The Github Actions disk space can get filled up if we test all kernel
flavors. So, delete kernels after testing when running in Github
Actions. However, if the download speed is far greater than the speed of
tests, then there is also the possibility that the disk space will be
filled up by the download thread, before the tests will run and delete
the older kernels. So we also need to ensure the download thread can't
get too far ahead of the tests, which we implement by setting a max size
on the download queue.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-07-18 19:12:36 -07:00
Stephen Brennan
03c3d24812 CI: add support for testing all kernel flavors
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-07-18 19:12:36 -07:00
Stephen Brennan
9c6a621656 setup.py: allow selecting an alternate flavor
This makes it simple to run all tests for a specific flavor. That's
especially useful if submitting a pull request, where CI will test the
default flavor, but you want to double check some other flavor first.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-07-18 19:12:36 -07:00
Omar Sandoval
3ca4e66362 setup.py: explicitly use default kernel flavor for test -K
Stephen noticed in #415 that if a new stable release is tagged after the
default vmtest build job runs but before another flavor does, we can end
up in a situation where the newest kernel for a given version isn't the
default flavor. This causes `setup.py test -K` to test a non-default
flavor. `-K` is specifically supposed to test the default, so specify it
explicitly.

Reported-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-17 16:04:23 -07:00
Stephen Brennan
de8ba223de Add linux kernel module helpers
There are a few common use cases that I find myself re-writing:
iterating over the list of modules, looking up a module by name, and
getting memory addresses related to modules (or vice versa). Add helpers
for each.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-07-12 20:01:52 -07:00
Sourabh Jain
c6efb6ecff tests/linux_kernel: Skip TestStackDepot unless test kmod is present
TestStackDeport needs the drgn test kernel module, so if it is not
present, skip the test.

Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
2024-07-12 09:34:34 -07:00
Stephen Brennan
5dd3f47986 contrib: bpf_inspect: adjust syntax for python < 3.12
Python 3.12 introduces the ability to nest the quote character within
f-strings. Previously, this was not allowed. While contrib scripts don't
have any specific requirement for code quality or compatibility, it
would be nice to make this runnable on Python versions prior to 3.12.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-07-11 16:31:20 -07:00
Leon Hwang
d564e962c2 contrib/bpf_inspect.py: show subprogs of prog
Show subprog list when `--show-details`.

It's useful to inspect subprog function name.

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
2024-07-11 15:04:31 -07:00
Leon Hwang
6c7c683304 contrib/bpf_inspect.py: show used maps of prog
Unlike `bpftool p` to show bpf map ids, `drgn contrib/bpf_inspect.py p
--show-details` is able to show more bpf map info, which is useful to
identify the used `.rodata` map.

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
2024-07-11 15:04:31 -07:00
Omar Sandoval
8ab4901d83 packit: disable ELN i386 builds
ELN dropped support for i386 (see
dbf2bfc452),
so these builds are failing. Drop them.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-01 14:20:15 -07:00
Omar Sandoval
4ad03cd479 docs: fix date on 0.0.27 release highlights
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-01 11:42:54 -07:00
Omar Sandoval
a8dfc9e31f drgn 0.0.27
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-01 10:29:14 -07:00
Omar Sandoval
8c5e05bceb docs: add 0.0.27 release highlights
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-01 10:29:14 -07:00
Omar Sandoval
a36c056bec scripts/build_manylinux_in_docker.sh: use CentOS Vault mirrors for EOL CentOS 7
CentOS 7 is now EOL, so the manylinux2014 build is failing; see
pypa/manylinux#1641. Work around it by using the archive mirror.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-07-01 10:29:14 -07:00
Omar Sandoval
22b4223153 libdrgn/tests/language_c: fix drgn_error memory leak
We're not destroying the errors we get.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-28 23:13:59 -07:00
Omar Sandoval
f931bfe9ce libdrgn/tests/path: fix buffer overflow
We're null-terminating after the slash at the end of the buffer, but we
should be replacing it.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-28 23:13:59 -07:00
Omar Sandoval
3add6fe59f libdrgn: determine number of OpenMP threads lazily
libomp leaks memory on fork (see __kmp_atfork_child() in
llvm-project/openmp/src/z_Linux_util.cpp), which doesn't really matter
except that it causes our new C unit tests to fail under LeakSanitizer
because libdrgn has a constructor that calls omp_get_max_threads(). We
could work around it by using CK_NOFORK in the unit tests, but it's not
ideal for us to go digging in sysfs on process startup just because you
linked to libdrgn anyways. Initialize the number of threads when we
actually need it instead, which just requires some care around
concurrency.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-28 23:13:58 -07:00
Stephen Brennan
b713acad84 setup.py: use a separate build directory for free-threading python
Free-threading Python binaries aren't ABI compatible with the
traditional builds that still have a GIL. But currently the build_temp
directory, which is drgn's incremental build directory, is shared
between both kinds of interpreters. This can result in runtime issues
like segmentation faults if build artifacts are reused. To avoid this,
add a "t" suffix for free-threading build directories. We should only do
this if the "build-temp" option is not set by the user.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2024-06-25 10:58:46 -07:00
Omar Sandoval
798ba053ce scripts/build_manylinux_in_docker.sh: use pytest instead of setup.py test
The latter fails on Python 3.13 since setuptools isn't installed by
default. If we need to install something anyways, we might as well use
the nicer pytest.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-24 14:11:49 -07:00
Omar Sandoval
468c58d095 libdrgn: kdump: fix build with libkdumpfile < 0.4.1
kdump_blob_new() was added in libkdumpfile 0.4.1.

Fixes: 6c7b271a53 ("libdrgn: kdump: pass architecture and vmcoreinfo to libkdumpfile")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-24 14:01:28 -07:00
Leon Hwang
0b5bbda2c9 contrib/bpf_inspect.py: inspect map details
Currently, inspect details for PROG_ARRAY map.

For instance:

```bash
drgn ./contrib/bpf_inspect.py m --show-details
  5108: BPF_MAP_TYPE_PROG_ARRAY          prog_array
	owner:    BPF_PROG_TYPE_KPROBE             JITed
	idx[  0]:   4250: BPF_PROG_TYPE_KPROBE             k_tailcall
	poke:       4249: BPF_PROG_TYPE_KPROBE             k_icsk_complete_hashdance        tail_call_reachable
	poke:       4251: BPF_PROG_TYPE_KPROBE             k_tcp_connect                    tail_call_reachable
```

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
2024-06-24 13:27:33 -07:00
Leon Hwang
cd7d84d719 contrib/bpf_inspect.py: fix get tramp progs
When run drgn on kernel v5.5~5.9, there is no `dst_trampoline` in `prog->
aux`. At this time, `tr` must be inited as `None` in order to avoid
panic when get `trampoline` from `prog->aux`.

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
2024-06-24 13:27:33 -07:00
Leon Hwang
dd9a594490 contrib/bpf_inspect.py: show interface info for XDP link
When a link is XDP link, show ifname, ifindex and XDP mode, if --show-details.

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
2024-06-24 13:27:33 -07:00
Leon Hwang
bd55a5652c contrib/bpf_inspect.py: show tail_call_reachable
Show "tail_call_reachable" if bpf prog is tail_call_reachable.

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
2024-06-24 13:27:33 -07:00
Leon Hwang
26d336a4c6 contrib/bpf_inspect.py: abstract bpf classes
In order to inspect more and more info of bpf objects, this commit
abstracts classes, such as `BpfProg`, `BpfMap`, `BpfLink` and so on.

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
2024-06-24 13:27:33 -07:00
Omar Sandoval
69bd07ddb2 drgn.helpers.common.memory: add optional cache to identify_address()
print_annotated_memory() and print_annotated_stack() are pretty slow
when called on large memory regions, largely because identify_address()
walks every task for every call on a non-direct map address in order to
resolve vmap stacks. We can cache the mapping from a vmap stack area to
a task, but we need callers to be able to indicate when it's okay to
cache in order to avoid stale results. To do so, add a cache parameter
to identify_address(), which is just an opaque dictionary that should be
passed between successive calls.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-21 16:20:44 -07:00
Omar Sandoval
bdb793508f libdrgn: fix stray double semicolon
Fixes: 5b39bfb547 ("libdrgn: x86_64: avoid recursive address translation for swapper_pg_dir")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-21 14:02:00 -07:00
Stephen Brennan
3282ad680f Update trove classifiers
Drgn is used by multiple companies and stakeholders in a variety of
situations. I think it qualifies as a production / stable. Also, add in
the Linux kernel trove classifier.

Signed-off-by: Stephen Brennan <stephen@brennan.io>
2024-06-21 00:27:36 -07:00
Omar Sandoval
6cf0c244e6 libdrgn: enable -Wshadow
There are a few places that need fixups.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-20 23:34:14 -07:00
Omar Sandoval
2c0f808cf8 Convert scripts/test_cityhash.c to C unit test
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-20 23:34:14 -07:00
Omar Sandoval
5e6c1f95d8 libdrgn: enable -Wpointer-arith
I've avoided using this GNU C extension, so we should enforce it.
There's one place to fix up.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-20 23:34:14 -07:00
Leon Hwang
17f18ac61f contrib/bpf_inspect.py: inspect linked progs
Since 'prog->aux->linked_prog' has been moved to trampoline, it has to
inspect linked progs from tracing link's trampoline.

Therefore, add --show-details for link command to show the linked progs
for tracing link.

Signed-off-by: Leon Hwang <hffilwlqm@gmail.com>
2024-06-20 23:31:12 -07:00
Omar Sandoval
a47fa1f3e2 libdrgn: get initial registers for kdump from crash_notes variable
We got a couple of reports about drgn failing to get a stack trace
(#391) or getting the wrong stack trace (#404) from a kernel core dump.
Both were caused because drgn assumes that there is an NT_PRSTATUS note
for each CPU in order by CPU number, and in these core dumps some
NT_PRSTATUS notes were missing. There are a least a couple of things
that can cause this: offline CPUs or CPUs that were in a bad state and
didn't respond to the kdump NMI. The former is expected and could be
special-cased, but the latter basically means that we can't trust the
order of the notes. Instead, look up the notes from the crash_notes
per-CPU variable that the kernel uses to populate the ELF notes. We
still need to use the actual NT_PRSTATUS notes for QEMU
dump-guest-memory dumps, but for those we need to use the PID field to
handle CPU hotplugging.

Closes #404.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-20 14:34:20 -07:00
Omar Sandoval
28a892ca15 libdrgn: factor out ELF note parsing
We have a few places where we parse raw ELF notes independently of
libelf, which isn't complicated but also not trivial thanks to alignment
requirements. In preparation for adding another place, factor the
parsing out into a common helper. Also document the complexities around
figuring out the correct alignment.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-20 14:34:20 -07:00
Omar Sandoval
d4da5d8173 drgn.helpers.linux.mm: comment that vmap_nodes workaround also applies to !SMP
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-20 14:34:20 -07:00
Omar Sandoval
680b2d53f9 vmtest.config: enable CONFIG_FW_CFG_SYSFS for testing QEMU dump-guest-memory
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-20 13:31:54 -07:00
Omar Sandoval
a991bd523b tests: require full MM support for print_annotated_memory() test
Full MM support is required to identify slab objects.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-18 12:20:48 -07:00
Omar Sandoval
5b8c4670d7 helpers.linux.mm: work around optimized out vmap_nodes on 32-bit
The vmap_nodes global variable is optimized out on 32-bit kernels, so we
need to hard-code a workaround for the find_vmap_area() and
for_each_vmap_area() helpers.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-18 12:20:21 -07:00
Omar Sandoval
e1de63e6c6 vmtest.__main__: use correct Python executable when emulating
The path of the Python interpreter in the Debian rootfs is
/usr/bin/python3, which may not be the same as sys.executable on the
host.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-18 12:17:38 -07:00
Omar Sandoval
033510afb8 vmtest.vm: add --{build,insert}-test-kmod options
When running tests manually with vmtest.vm, it's tedious to set up the
test kernel module. Make vmtest.vm.run_in_vm() handle it and add command
line options.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2024-06-18 10:23:20 -07:00