Kevin Svetlitski suggested making use of __attribute__((__cleanup__)) a
long time ago, and now that the kernel is doing it, I don't have a good
excuse not to. There are surprisingly only a handful of places that it
was straightforward to apply it to. A lot of potential uses are thwarted
by our policy that out parameters can be clobbered on failure, so that
may be something to revisit. Other cleanup guards will probably be more
useful, but this is just laying the groundwork for the future.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This is similar to commit 155ec92ef2 ("libdrgn: fix reading 32-bit
float object values on big-endian").
Fixes: 75c3679147 ("Rewrite drgn core in C")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We've addressed all of the smaller differences with GNU Debug Fission
and split DWARF 5, so now all that remains is the DWARF index.
The general approach is: in drgn_dwarf_index_read_cus(), for each CU,
ask libdw for the "sub-DIE". For skeleton CUs, this is the split CU DIE
from the .dwo file. From that Dwarf_Die, we can get the Dwarf_CU and
then the Dwarf handle. Then, we wrap that in a struct drgn_elf_file
(cached in a hash table in the struct drgn_module), which the DWARF
index can work with from there.
Additionally, a couple of places (.debug_addr parsing and stack trace
local variable lookup) need to be updated to use the correct
drgn_elf_file.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Split DWARF is challenging for the DWARF index for a couple of reasons:
1. We need libdw to look up the split files.
2. The file name table comes from the skeleton file, but everything else
relevant to the index comes from the split file.
(1) requires the index to use libdw to get the CU DIE. Unfortunately,
due to the overhead of libdw, this makes the indexing step 5-10% slower.
On the plus side, getting the CU DIE upfront simplifies quite a bit: we
can read the file name table, compilation directory, and str_offsets
base before indexing, which makes supporting (2) possible.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In the next change, we'll need more information about the unit, and
there's no benefit to doing it ourselves anymore.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Instead, reuse struct drgn_dwarf_index_cu for the pending CUs. This is
mainly so that we can save more information in the pending CU in a later
change. It also lets us merge our per-thread pending CU arrays with
memcpy() instead of element-by-element, but I didn't measure a
performance difference one way or the other.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There are a couple of differences with non-split DWARF 5:
- DW_AT_addr_base/DW_AT_GNU_addr_base is in the skeleton DIE, so we need
to use dwarf_attr_integrate().
- GNU Debug Fission for DWARF 4 doesn't have headers in .debug_addr.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
dwarf_module_find_dwarf_scopes() and drgn_dwarf_die_iterator_next() just
need to go from skeleton units to split units. We need to use
dwarf_cu_info(), which was added in 0.171, which incidentally was when
elfutils gained split DWARF support anyways.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
It seems like GCC omits this for split units when using DWARF 5,
intending it to mean the first entry in .debug_loclists.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
GNU Debug Fission doesn't have DW_AT_str_offsets_base but does have
.debug_str_offsets. GCC doesn't emit DW_AT_str_offsets_base for DWARF 5
split DWARF. In both cases, the default is the first entry in
.debug_str_offsets.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Rather than coming up with our own, separate logging API for the Python
bindings, let's integrate with the logging module. The straightforward
part is creating a logger from the C extension and adding a log callback
that calls its log() method. However, syncing the log level between the
logging module and libdrgn requires monkey patching.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Exceptions aren't enough to debug complicated code paths like debug info
discovery or stack unwinding. We really need logs for that, so let's add
a small logging framework. By default, we log to stderr, but we also
provide a way to direct logs to a different file, or even an arbitrary
callback so that logs can be directed to the application's logging
library of choice.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
DWARF indexing can take a long time; Kevin Svetlitski notes that it can
take almost a minute on some large binaries. Let's use the new blocking
API around it so that the Python bindings drop the GIL.
Closes#247.
Suggested-by: Kevin Svetlitski <svetlitski@meta.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There are places in drgn where it'd be a good idea to drop the Python
GIL. However, some of these are deep inside of libdrgn, where some code
paths are fast and dropping the GIL would be extra overhead and others
are slow (e.g., type lookups, which may be cached or may require DWARF
namespace indexing). Instead of trying to do this from the Python
bindings, add hooks to libdrgn. These hooks can be used directly or with
a new scope guard macro, drgn_blocking_guard, that we can start
sprinkling around in appropriate places in libdrgn.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're getting (null) file paths in error messages (e.g., #233) because
libdwfl doesn't always return the debug file path. Fall back to the
loaded file path, which is better than nothing until we get rid of
libdwfl.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Commit 18b12a5c7b ("libdrgn: get .eh_frame from the correct file")
missed this, but it's unlikely to matter in practice.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
These opcodes appear in practice, and we choke on them with an exception
like "unknown DWARF expression opcode 0xf3" or "unknown DWARF expression
opcode 0xa3". In some cases, it'd be possible to recover the entry value
by looking at call site information, but that's pretty involved. For
now, just treat these operations as optimized out so we stop failing
hard.
Closes#233.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
When we get the DIE from the offset with dwarf_offdie(), there's no need
to go back to the offset with dwarf_dieoffset().
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Thierry found that as soon as drgn_module_find_dwarf_scopes() finds any
DIE containing the PC, it walks the entire subtree rooted at that DIE.
However, we only need to look at the immediate children of a DIE
containing the PC. I think this is what I originally intended, but I
failed to reset the children flag to false when the last DIE didn't
contain the PC. Thierry's suggested check of it.dies.size == subtree is
simpler.
This is a massive performance improvement: for a kernel core dump with
10k threads, getting the stack trace of every thread took ~90 seconds
without this fix and ~50 seconds with it.
Let's also add a comment to this very subtle code.
Fixes: d8d4157346 ("libdrgn: debug_info: add drgn_debug_info_module_find_dwarf_scopes()")
Co-authored-by: Thierry Treyer <ttreyer@fb.com>
Signed-off-by: Thierry Treyer <ttreyer@fb.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
My kernel patch was merged for Linux 6.4 and backported to 6.3.10, so
now we can use the .orc_header section to reliably detect the ORC format
version. Since the 6.4 release candidates and older versions of 6.3
don't have .orc_header, we'll keep the version check as a fallback.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The ORC format changed twice recently:
- Linux kernel commit ffb1b4a41016 ("x86/unwind/orc: Add 'signal' field
to ORC metadata") (in v6.3).
- Linux kernel commit fb799447ae29 ("x86,objtool: Split
UNWIND_HINT_EMPTY in two") (in v6.4).
The former went unnoticed because the change was subtle, and the latter
completely broke x86-64 kernel stack traces.
To handle this, let's "upgrade" the format to the latest version when we
load and sort the ORC information. This is more work upfront but avoids
needing to handle the version differences every time we use ORC to
unwind.
Unfortunately, ORC currently doesn't have any sort of versioning, so we
have to break the rule of not checking kernel versions. However, I have
a kernel patch pending merging that should fix this for the future.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
It's unrealistic for there to be more than 4 billion ORC entries. Switch
to an unsigned int. The main benefit is that the indices array that we
use to sort the parallel arrays of entries and pc_offsets becomes half
the size, which also makes parsing ORC about 10% faster (down from ~5 ms
to ~4.5 ms for the Fedora vmlinux on my laptop).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
.orc_unwind_ip and .orc_unwind are only referenced while initially
parsing ORC data and then never touched again, so it's wasteful to cache
them in struct drgn_elf_file. Look them up if and when we parse the ORC
data instead.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In practice, the .orc_unwind and .orc_unwind_ip sections will always be
suitably aligned. Check it, then assume the alignment later.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
By using the same temporary objects in the Linux 6.4 branch as the
pre-6.4 branch, we get slightly better code generation.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We only support .debug_* sections, but libdw also supports .zdebug_*,
.debug_*.dwo, and .gnu.debuglto_.debug_*. Mimic how libdw chooses debug
sections, with one exception: .debug_cu_index and .debug_tu_index (used
for DWP, which we don't support yet but will) should be considered DWO
sections (this needs to be fixed in libdw, too).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
ORC_REG_SP_INDIRECT is supposed to be an indirect access via rsp, but we
have a typo and are using rbp instead. This is a partial fix for #304.
Fixes: 630d39e345 ("libdrgn: add ORC unwinder")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently use crashing_cpu to determine the thread that caused a
kernel crash. However, crashing_cpu is x86-specific (it is defined in
arch/x86/kernel/reboot.c). Since Linux 4.5, the generic panic code
defines a very similar variable, panic_cpu. Use that instead so that we
support all architectures, but fall back to crashing_cpu to support
older kernels on x86 (even though we don't claim to support 4.4
anymore).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
follow_{page,pfn,phys}() translate the virtual address by walking the
page table for a given mm_struct (built on top of the existing page
table iterator interface). vmalloc_to_page() and vmalloc_to_pfn() are
special cases for vmalloc addresses.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
With GCC 13.1.1 and the recommended build
setup (CONFIGURE_FLAGS="--enable-compiler-warnings=error"), I get the
following failure:
In function 'linux_kernel_get_vmemmap',
inlined from 'linux_kernel_object_find' at ../../libdrgn/linux_kernel_object_find.inc.strswitch:34:12:
../../libdrgn/linux_kernel.c:370:23: error: 'address' may be used uninitialized [-Werror=maybe-uninitialized]
370 | err = drgn_object_set_unsigned(&prog->vmemmap, qualified_type,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
371 | address, 0);
| ~~~~~~~~~~~
../../libdrgn/linux_kernel.c: In function 'linux_kernel_object_find':
../../libdrgn/linux_kernel.c:361:26: note: 'address' was declared here
361 | uint64_t address;
| ^~~~~~~
cc1: all warnings being treated as errors
While linux_kernel_get_vmemmap_address should always update address in a
non-error case, the compiler seems to disagree. It's easy enough to shut
up the compiler by initializing address to 0. What's more, if there is
an actual issue where the linux_kernel_get_vmemmap_address does NOT
update the address variable, a 0 value will be easier to debug than
garbage from an uninitialized variable.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Running tests on Python 3.12, we get:
test_int (tests.test_language_c.TestLiteral.test_int) ... python3.12: /usr/include/python3.12/object.h:215: Py_SIZE: Assertion `ob->ob_type != &PyLong_Type' failed.
Aborted (core dumped)
We're relying on an implementation detail to check whether the object is
negative. Instead, catch an overflow error, negate and try again.
Genuine overflows will still overflow on the second time, but negative
numbers will succeed.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Kernel commit ac3b43283923 ("module: replace module_layout with
module_memory") in v6.4 changed the layout of `struct module`, resulting
in the following drgn error [1].
Fix this by first trying to determine the base address and size of each
kernel module via the `struct module_memory mem[MOD_TEXT]` member,
before falling back to previous methods that work on older kernels.
Tested on v6.4-rc2 and v6.3 which does not include the above mentioned
commit.
Note that kernel commit b4aff7513df3 ("scripts/gdb: use mem instead of
core_layout to get the module address") performs a similar fix in Python
GDB scripts.
Closes#296.
[1]
```
# drgn
drgn 0.0.22 (using Python 3.11.3, elfutils 0.189, with libkdumpfile)
For help, type help(drgn).
>>> import drgn
>>> from drgn import NULL, Object, cast, container_of, execscript, offsetof, reinterpret, sizeof
>>> from drgn.helpers.common import *
>>> from drgn.helpers.linux import *
warning: could not get debugging information for:
kernel modules (could not find loaded kernel modules: 'struct module' has no member 'core_size')
```
Signed-off-by: Ido Schimmel <idosch@nvidia.com>
Add support for walking s390x page tables. This supports
up to 5 level page table walking and huge/large pages. In order
to figure out the level of paging used, we read the first entry
of the pgd, which is always mapped for lowcore access and use the
level bits of the next page table. This is because drgn passes mm::pgd
as pgtable argument to the walker function which doesn't contain the
ASCE bits.
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
This false positive appears to only trigger on 32-bit. I reproduced it
with GCC 10 and 12.
Fixes#242.
Reported-by: Timothée Cocault <timothee.cocault@gmail.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We have a lot of examples that use jiffies, but they stopped working
long ago on x86-64 (since Linux kernel commit d8ad6d39c35d ("x86_64: Fix
jiffies ODR violation") (in v5.8 and backported to stable releases)) and
never worked on other architectures. This is because jiffies is defined
in the Linux kernel's linker script. #277 proposed updating the examples
to use jiffies_64, but I would guess that most kernel developers are
familiar with jiffies and many have never seen jiffies_64. jiffies is
also a nicer name to type in live demos. Let's add a case to the Linux
kernel object finder to get the jiffies variable.
Reported-by: Martin Liska <mliska@suse.cz>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There are a lot of details about how the page table iterator functions
should be used/implemented that commit 174b797ae3 ("libdrgn: platform:
add documentation (especially for drgn_architecture_info)") didn't
cover. Add an example and expand/clarify the documentation for the
callbacks.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
GCC appears to use data8 at -1 when reporting zero length arrays when
comping c++ code, this patch adds support and a test for that behavior.
dwarf_info.c: Remove check for sdata on quirk for array length == 0
Signed-off-by: Jay Kamat <jaygkamat@gmail.com>
Move the definitions of the section names to a Python script,
gen_elf_sections.py, and use that to generate the enum definitions and a
lookup function. This is preparation for checking for section names with
the .dwo suffix in the future.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Prior to kernel v4.11, idr was not using radix tree as its backend.
So current idr helper(s) only work for kernel v4.11+.
Enable idr helpers(s) to work with non-radix tree based idr, so that
the helpers can be used with older kernels as well.
Thanks to Omar for optimizing the idr_for_each helper.
Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
Looking up objects in namespaces is already well-supported by `drgn`.
These changes bring the same to functionality type lookup, so that
`prog.type('struct A::B::C::MyType')` works in an analogous fashion to
`prog['A::B::C::MyVar']`.
Signed-off-by: Kevin Svetlitski <svetlitski@meta.com>
get_registers_from_frame_pointer() has a sanity check that the unwound
frame pointer must be greater than the current frame pointer. This is
generally true if the entire program is using frame pointers, but not
necessarily otherwise. In particular, if the program is a Linux kernel
configured with ORC, most of the time, rbp is a general purpose
register; it is only used as a frame pointer in special cases without
unwinder information like BPF programs. Those cases are exactly when we
want the frame pointer unwinder, but depending on what the caller was
using rbp for, the frame pointer unwinder might bail prematurely.
Let's remove the sanity check. In the worst case, this could lead us off
into the weeds chasing pointers, but the iteration limit in
drgn_get_stack_trace() prevents that from being dangerous.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
linux_kernel_get_initial_registers_ppc64() depends on the size of struct
pt_regs, but this has changed multiple times, in:
- Linux kernel commit 4c2de74cc869 ("powerpc/64: Interrupts save PPR on stack rather than
thread_struct") (in v4.20)
- Linux kernel commit 66f93c5a02d5 ("powerpc/64: Fix kernel stack
16-byte alignment") (in v4.20)
- Linux kernel commit 8e560921b58c ("powerpc/book3s64/pkeys:
Store/restore userspace AMR/IAMR correctly on entry and exit from
kernel") (in v5.11)
It also depends on the overhead stored before struct pt_regs on the
stack, which changed in Linux kernel commit cd52414d5a6c ("powerpc/64:
ELFv2 use minimal stack frames in int and switch frame sizes") (in
v6.2).
We can handle all of these cases by reading the previous r1 from memory
instead of computing it from a hard-coded size and finding the struct
pt_regs based on that r1 and the actual size of struct pt_regs.
Reported in #232.
Reported-by: Sourabh Jain <jainsourabh679@gmail.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only relocation type I saw in Debian's kernel module debug info was
R_390_32. R_390_8, R_390_16, R_390_64, R_390_PC16, R_390_PC32, and
R_390_PC64 are trivial to support, as well. The Linux kernel supports
many more, but hopefully they won't show up for debug info.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Konrad Borowski pointed out that add_to_possibly_null_pointer() relies
on GCC-specific behavior:
https://fosstodon.org/@xfix/109542070338182493. CONTRIBUTING.rst
mentions that we assume that casting between pointers and integers does
not change the bit representation, but we might as well document it
here, too.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, looking up a type with template arguments results in an
"invalid character" syntax error on the "<" character. The DWARF index
includes template arguments in indexed names, so we need to do lookups
including the template arguments. Full support for this would require
parsing the template argument list syntax and normalizing it or looking
it up as an AST in some way. For now, it's at least an improvement to
pass the user's string verbatim. To do so, kludge it by adding a token
containing everything from "<" to the matching ">" to the C++ lexer and
appending that to the identifier.
Co-authored-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Kevin Svetlitski <svetlitski@meta.com>
Commit 89eb868e95 ("helpers: make find_task() work on recent kernels")
made radix_tree_lookup() and radix_tree_for_each() work for basic
XArrays. However, it doesn't handle a couple of more advanced features:
multi-index entries (which old radix trees actually also supported) and
zero entries. It has also been really confusing to explain to people
unfamiliar with the radix tree -> XArray transition that they should use
helpers named radix_tree for a structure named xarray.
So, let's finally add xa_load(), xa_for_each(), and some additional
auxiliary helpers. The non-recursive xa_for_each() implementation is
based on Kevin Svetlitski's C implementation from commit 2b47583c73
("Rewrite linux helper iterators in C"). radix_tree_lookup() and
radix_tree_for_each() share the implementation with xa_load() and
xa_for_each(), respectively, so they are mostly interchangeable.
Fixes: #61
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're calling Py_BuildValue() with the "k" format for unsigned long but
passing the enum value itself, which is promoted to int. I don't know
whether there are any ABIs where this matters in practice, but let's use
"K" and cast to unsigned long long explicitly to be safe.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Instead, define the list of constant classes in one place so we can
generate all 3 places that need it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
PyUnicode_FSConverter() already handles os.PathLike, so we only need to
handle None and save the string and length.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
It feels icky to write code that, for example, passes a uint64_t to
PyLong_FromUnsignedLongLong(). In practice it's fine, but it's much
nicer to have conversion functions specifically for the stdint.h types.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This DWARF tag is used by C++ classes which take a variable number
of template parameters, such as std::variant and std::tuple.
Signed-off-by: Alastair Robertson <ajor@meta.com>
While reviewing #214, I realized that we have very little documentation
for drgn_architecture_info (and platform internals in general). Let's
document all of the important stuff, and in particular how to add
support for new architectures.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
While documenting struct drgn_architecture_info, I realized that
demangle_return_address() is difficult to explain. It's more
straightforward to define this functionality as demangling any registers
that are mangled when using CFI rather than just the return address
register.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
c_parse_specifier_qualifier_list() checks whether an identifier starts
with "size_t" or "ptrdiff_t" to decide whether to return the size_t or
ptrdiff_t type. This incorrectly matches stuff like like "size_tea" and
"ptrdiff_tee". Fix this by making it an exact comparison.
Fixes: 75c3679147 ("Rewrite drgn core in C")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're currently getting .eh_frame from the debug file. However, since
.eh_frame is an SHF_ALLOC section, it is actually in the loaded file,
and may not be in the debug file. This causes us to fail to unwind in
modules whose debug file was created with objcopy --only-keep-debug
(which is typical for Linux distro debug files).
Fix it by getting .eh_frame from the loaded file. To make this easier,
we split .eh_frame and .debug_frame data into two separate tables. We
also don't bother deduplicating them anymore, since GCC and Clang only
seem to generate one or the other in practice.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
For upcoming changes, we will need loaded (SHF_ALLOC) sections for
modules. Some separate debug files (e.g., those created with objcopy
--only-keep-debug) don't have those sections. Let's get the loaded file
from libdwfl with dwfl_module_getelf() and save it in a struct
drgn_elf_file.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that we track the debug file ourselves, we can avoid calling libdwfl
in a bunch of places. By tracking the bias ourselves, we can avoid a
bunch more.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
struct drgn_module contains a bunch of information about the debug info
file. Let's pull it out into its own structure, struct drgn_elf_file.
This will be reused for the "main"/"loaded" file in an upcoming change.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
It's confusing that we have a platform both for the program and for each
module. They usually match, but they're not required to. For example,
the user can manually add a file with a different platform just to read
its debug info. Our rule is that if we're parsing anything from the
module, we use the module platform; and otherwise, use the program
platform. There are a couple of places where the platforms must match:
when using call frame information (CFI) or registers. Let's make all of
this more clear in the code (by using the module's platform even when it
must match the program's platform) and in comments. No functional
change.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
If a DW_TAG_pointer_type DIE doesn't specify its size with
DW_AT_byte_size, we currently default to the program's address size.
However, the DWARF we're parsing could be for a platform with a
different address size. It's more correct to use the CU's address size.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We have some generic helpers that we'd like to add (for example, #210)
that need to know the stack pointer of a frame. These shouldn't need to
hard-code register names for different architectures. Add a generic
shortcut, StackFrame.sp.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
qsort_r is a non-standard glibc extension and turns out to be the only
thing that prevents drgn from working on a musl system. "Fix" the use of
qsort_r by switching it to qsort with a thread local variable for the
parameter.
Tested in a clean chroot install of musl voidlinux.
Signed-off-by: Boris Burkov <boris@bur.io>
The StackFrame's __getitem__() method allows looking up names in the
scope of a stack frame, which is an incredibly useful tool for
debugging. However, the names are not discoverable -- you must already
be looking at the source code or some other source to know what names
can be queried. To fix this, add a locals() method to StackFrame, which
lists names that can be queried in the scope. Since this method is named
locals(), it stops at the function scope and doesn't include globals or
class members.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
find_dwarf_enumerator() needs to check that the return value of
dwarf_diename() is not NULL before calling strcmp(). This is similar to
commit 330c71b5b5 ("libdrgn: dwarf_info: fix segfault on anonymous
DIEs during scope search"), although I haven't seen this one happen in
practice.
Fixes: bc85767e5f ("libdrgn: support looking up parameters and variables in stack traces")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I wanted to make REUSE pass, but I'm not sure what to do about trivial
files. REUSE suggests using CC0, but Fedora no longer allows CC0. I'll
punt that until later. For now, let's add notices to some code files.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn is currently licensed as GPLv3+. Part of the long term vision for
drgn is that other projects can use it as a library providing
programmatic interfaces for debugger functionality. A more permissive
license is better suited to this goal. We decided on LGPLv2.1+ as a good
balance between software freedom and permissiveness.
All contributors not employed by Meta were contacted via email and
consented to the license change. The only exception was the author of
commit c4fbf7e589 ("libdrgn: fix for compilation error"), who did not
respond. That commit reverted a single line of code to one originally
written by me in commit 640b1c011d ("libdrgn: embed DWARF index in
DWARF info cache").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Rename DW_TAG_{UNKNOWN_FORMAT,BUF_LEN} to
DW_TAG_STR_{UNKNOWN_FORMAT,BUF_LEN} to make it more clear that they're
for dw_tag_str.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Our container_of() and array_size() were copied from the Linux kernel
and use some really ugly BUILD_BUG_ON_ZERO() and BUILD_BUG_ON_MSG()
macros. C11 has _Static_assert, which is much nicer. We just have to
shoehorn it into an expression, which we do with clever use of _Generic
and sizeof a struct type definition. (We could accomplish the same idea
with a comma expression, but GCC warns when the left-hand operand of a
comma expression has no effect. We could also do it with a compound
statement, but it's cooler to do it with standard C11.)
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Eventually, modules will be exposed as part of the public libdrgn API,
so they should have a clean name. Additionally, the module API I'm
currently working on will allow modules for which we don't have the
debug info file, so "debug info module" would be a misnomer.
Also rename drgn_dwarf_module_info to drgn_module_dwarf_info and
drgn_orc_module_info to drgn_module_orc_info to fit the new naming
scheme better.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're eventually going to add a drgn.Module class, which logically
should go in a file called module.c. But, we already have a module.c
with module-level definitions. Rename that file to main.c to free up
module.c
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There are a bunch of places where we call .tp_alloc() directly, which is
very verbose. Add a macro that removes the boilerplate.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
-Wimplicit-fallthrough has a false positive because the compiler
apparently doesn't know that usage() never returns.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Instead of string_builder_finalize(), which leaves the string_builder in
an undefined state (according to the documentation, at least), define
string_builder_null_terminate(), which documents exactly what it does.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Make string_builder_reserve() allocate an exact capacity, and add a
string_builder_reserve_for_append() wrapper that does the
next_power_of_two(current length + number to append) that all of the
current callers want.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Rather than documenting how to initialize a struct string_builder,
provide an initializer, STRING_BUILDER_INIT.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This time, in order to work on both GCC and Clang, use
__attribute__((__fallthrough__)) instead of /* fallthrough */ comments.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This reverts commit e05bfbddc2. Clang
doesn't support /* fallthrough */ comments, so we'll need to use
__attribute__((falthrough)), which will need some additional feature
detection.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
task->cpu was moved to task->thread_info.cpu in Linux 5.16, which causes
drgn_get_initial_registers() to think that the kernel is !SMP and use
CPU 0 instead, producing incorrect stack traces. This has also always
been wrong for Linux < 4.9 and on architectures that don't enable
CONFIG_THREAD_INFO_IN_TASK; in those cases, it should be
((struct thread_info *)task->stack)->cpu.
Fix it by factoring out a new task_cpu() helper that handles all of the
above cases. Also add a test case for task_cpu() in case this changes
again.
Fixes: eea5422546 ("libdrgn: make Linux kernel stack unwinding more robust")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Jakub Kicinski reported that
prog.crashed_thread().stack_trace()[1]['does not exist'] segfaulted on a
vmcore he encountered. The segfault was a NULL pointer dereference of
dwarf_diename() of a DW_TAG_subprogram DIE in
drgn_find_in_dwarf_scopes(). The fix is to ignore DIEs without a name.
I was curious what this anonymous DW_TAG_subprogram was. It turned out
to be some dubious DWARF generated by Clang when a local variable is
defined via a macro. One such example comes from the following code in
arch/x86/events/intel/uncore.h:
static inline bool uncore_mmio_is_valid_offset(struct intel_uncore_box *box,
unsigned long offset)
{
if (offset < box->pmu->type->mmio_map_size)
return true;
pr_warn_once("perf uncore: Invalid offset 0x%lx exceeds mapped area of %s.\n",
offset, box->pmu->type->name);
return false;
}
pr_warn_once() expands to:
#define pr_warn_once(fmt, ...) \
printk_once(KERN_WARNING pr_fmt(fmt), ##__VA_ARGS__)
#define printk_once(fmt, ...) \
({ \
static bool __section(".data.once") __print_once; \
bool __ret_print_once = !__print_once; \
\
if (!__print_once) { \
__print_once = true; \
printk(fmt, ##__VA_ARGS__); \
} \
unlikely(__ret_print_once); \
})
For some reason, Clang generates an anonymous, top-level
DW_TAG_subprogram DIE to contain the __print_once variable:
<1><1cf86e>: Abbrev Number: 62 (DW_TAG_subprogram)
<2><1cf86f>: Abbrev Number: 61 (DW_TAG_variable)
<1cf870> DW_AT_name : (indirect string, offset: 0x34fb2e): __print_once
<1cf874> DW_AT_type : <0x1c574c>
<1cf878> DW_AT_decl_file : 1
<1cf879> DW_AT_decl_line : 229
<1cf87a> DW_AT_location : 16 byte block: 3 2c 84 66 83 ff ff ff ff 94 1 31 1e 30 22 9f (DW_OP_addr: ffffffff8366842c; DW_OP_deref_size: 1; DW_OP_lit1; DW_OP_mul; DW_OP_lit0; DW_OP_plus; DW_OP_stack_value)
Whereas GCC puts it in a DW_TAG_lexical block DIE inside of the
DW_TAG_subprogram DIE for uncore_mmio_is_valid_offset():
<1><3110b2>: Abbrev Number: 45 (DW_TAG_subprogram)
<3110b3> DW_AT_name : (indirect string, offset: 0x2e13e): uncore_mmio_is_valid_offset
<3110b7> DW_AT_decl_file : 4
<3110b8> DW_AT_decl_line : 223
<3110b9> DW_AT_decl_column : 20
<3110ba> DW_AT_prototyped : 1
<3110ba> DW_AT_type : <0x2f416b>
<3110be> DW_AT_inline : 3 (declared as inline and inlined)
<3110bf> DW_AT_sibling : <0x311142>
<2><3110ef>: Abbrev Number: 66 (DW_TAG_lexical_block)
<3><3110f0>: Abbrev Number: 120 (DW_TAG_variable)
<3110f1> DW_AT_name : (indirect string, offset: 0x2da3f): __print_once
<3110f5> DW_AT_decl_file : 4
<3110f6> DW_AT_decl_line : 229
<3110f7> DW_AT_decl_column : 2
<3110f8> DW_AT_type : <0x2f416b>
<3110fc> DW_AT_location : 9 byte block: 3 2c 28 48 83 ff ff ff ff (DW_OP_addr: ffffffff8348282c)
Regardless, we shouldn't crash on this input.
Reported-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The .data..percpu section is excluded from /sys/module and struct
module::sect_attrs, which means that we default its address to 0. This
results in global per-CPU variables in kernel modules being relocated
starting from 0 rather than the offset of the per-CPU allocation made
for the module, which in turn causes those variables to appear to
contain the wrong data. Fix it by manually getting the per-CPU address
from struct module.
Closes#185.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
For the next fix, we need the address of the .data..percpu section,
which is only available directly from the struct module and not from
anywhere in /proc or /sys. Get rid of the /proc/modules fast path (and
update the name of the testing environment variable from
DRGN_USE_PROC_AND_SYS_MODULES to DRGN_USE_SYS_MODULE).
This has some small overhead (~20ms longer startup time in my
benchmarks) and means that we no longer determine the loaded modules if
vmlinux is missing, but fixing the per-CPU issue is more important.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
An upcoming fix requires us to always use the module list from the core
dump rather than /proc/modules. However, with the existing code, this
would cause a major startup time regression for the live kernel, mainly
because reading from /proc/kcore is stupidly slow. We currently do 3 +
strlen(module->name) reads for every module. We can reduce this to 1
read per module by reading the entire struct module at once. The size of
struct module is ~700-900 bytes depending on the kernel configuration,
which is still much faster to read than only reading what we need.
In some benchmarks that I did with DRGN_USE_PROC_AND_SYS_MODULES=0, this
reduced the time spent in the kernel module iterator from ~2.5ms per
module to ~0.4ms per module.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
If drgn_object_set_buffer_from_internal() (used to implement
drgn_object_set_from_buffer(), drgn_object_slice(), and
drgn_object_reinterpret()) sets an object to a primitive type from a
buffer that comes from the same object, then drgn_object_reinit() will
free the value and then drgn_value_serialize() will access the freed
value, probably resulting in garbage. Handle this case the same way we
do if the result type is encoded as a buffer, by first copying to a
temporary value.
This doesn't affect usage through Python because objects are immutable
in the Python bindings.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We have a couple of loops that deal with short reads/EINTR from read(2)
and pread(2), and upcoming changes would need to add more. Add some
wrappers to abstract this away.
drgn_read_memory_file() still needs the loop so it can fault on the
exact offset that returns EIO.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The call to min() in drgn_read_memory_file() results in the following
warning on 32-bit architectures that I missed on review:
In file included from ../../libdrgn/memory_reader.c:10:
../../libdrgn/memory_reader.c: In function 'drgn_read_memory_file':
../../libdrgn/minmax.h:36:26: warning: comparison of distinct pointer types lacks a cast
36 | (void)(&unique_x == &unique_y); \
| ^~
../../libdrgn/minmax.h:28:19: note: in expansion of macro 'cmp_once_impl'
28 | #define min(x, y) cmp_once_impl(x, y, PP_UNIQUE(_x), PP_UNIQUE(_y), <)
| ^~~~~~~~~~~~~
../../libdrgn/memory_reader.c:284:34: note: in expansion of macro 'min'
284 | size_t readlen = min(file_end - file_offset, count);
| ^~~
We can fix it with a cast, and additionally do the call to min() earlier
and rework the logic a bit.
Fixes: 9684771d61 ("libdrgn: Zero fill excluded pages in kernel core dumps rather than FaultError")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There's a lot more context here that we should write down. It's also
worth noting that it appears that GDB always zero fills the range
between p_filesz and p_memsz, so if we end up having any other issues
because of this, we might have to concede and go back to the behavior
before commit 02912ca7d0 ("libdrgn: fix handling of p_filesz < p_memsz
in core dumps").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add pretty printing support in Jupyter notebook for Object, Type,
StackFrame, and StackTrace; it will print out their representation in
programming language syntax with str(), similar to what's being done in
interactive mode.
Link: https://ipython.readthedocs.io/en/stable/api/generated/IPython.lib.pretty.html#extending
Signed-off-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
makedumpfile will exclude zero pages. We found a core file where a
structure straddled a page boundary and the end of the structure
was all zeros so the page was excluded and we were generating a
FaultError trying to access the structure.
This change reverts a portion of that behaviour such that when we are
debugging a kernel core we go back to the zero fill behaviour. To do this
we go back to creating segments based on memsz instead of filesz and
handling the filesz->memsz gap in drgn_read_memory_file.
Fixes: 02912ca7d0 ("libdrgn: fix handling of p_filesz < p_memsz in core dumps")
Signed-off-by: Glen McCready <gkm@mysteryinc.ca>
Replace the old "Scriptable debugger library" and
"Debugger-as-a-library" taglines with the one we're using on GitHub,
"Programmable debugger". Make up for it by emphasizing that drgn can
also be used as a library a tiny bit more in the README.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
PyErr_SetObject() takes a reference on the exception value, so we need
to drop the reference we got when we created the value. Issue #196 ran
into this by reading tons of unmapped addresses.
Fixes: 80fef04c70 ("Add address attribute to FaultError exception")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The config option is and always has been CONFIG_FW_CFG_SYSFS, not
CONFIG_FW_CFG. Also suggest the user-visible CONFIG_KEXEC instead of the
internal CONFIG_CRASH_CORE.
Fixes: 2bd861f719 ("libdrgn: program: detect QEMU guest memory dumps without VMCOREINFO")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
If we only have the stack trace available, it's useful to get the
program it came from. This'll be used eventually for helpers that take a
stack trace.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're currently checking whether the iterator has entered the
non-canonical range when fetching the last level of the page table, but
the cutover actually happens while we're in the last level. Fix it by
doing the check unconditionally.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn_debug_info_find_complete() looks up the name of the incomplete type
in the global namespace. This is incorrect for C++: we need to look it
up in the namespace that the DIE is in.
To find the containing namespace, we need to do a DIE ancestor walk. We
don't want to do this for C, so add a flag indicating whether a language
has namespaces to struct drgn_language. If it's true, then we do the
ancestor walk and then look up the name in the appropriate namespace.
Signed-off-by: Jay Kamat <jaygkamat@gmail.com>
Now that we made the other memory management helpers generic, the last
thing to implement for AArch64 is page table walking. This looks a lot
like the x86-64 equivalent but has to support the various page and
virtual address sizes that can be configured for AArch64.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
AArch64 has changed the location of vmemmap multiple times, and not all
of these can be easily distinguished. Rather than restorting to kernel
version checks, this replaces the vmemmap architecture callback with a
generic approach that gets the vmemmap address directly from the
mem_section table.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
On x86-64, the difference between virtual addresses in the direct map
and the corresponding physical addresses is called PAGE_OFFSET, so we
exposed that via an architecture callback and the Linux kernel object
finder. However, this doesn't translate to other architectures. Namely,
on AArch64, the difference is PAGE_OFFSET - PHYS_OFFSET, and both
PAGE_OFFSET and PHYS_OFFSET have varied over time and between
configurations.
We can remove the architecture callback and avoid version-specific logic
by letting the page table tell us the offset. We just need an address in
the direct map, which is easy to find since this includes kmalloc and
memblock allocations.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
linux_helper_read_vm() has logic to merge adjacent physical address
ranges returned by the page table iterator. However, the check for
whether the ranges are adjacent is incorrect. Fix it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
pgtable_iterator_x86_64::table is only used if
pgtable_iterator_x86_64::index indicates that it has any cached entries,
so there's no point initializing table since we initialize index to
indicate that nothing is cached.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
AArch64 will need different sizes of page table iterators depending on
the page size and virtual address size. Rather than the static
pgtable_iterator_arch_size, allow architectures to define callbacks for
allocating and freeing a page table iterator. Also remove the generic
page table iterator wrapper and just pass that information to the
iterator function.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Rather than computing it every time we need it, compute it once when we
parse PAGE_SIZE from VMCOREINFO (and validate that PAGE_SIZE is a power
of two). This will be more important for AArch64 page table walking.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Very similar to a541e9b170, but adds
partial support for floats (as opposed to integers) which aren't 32 or
64 bits.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Previously `drgn` did not recognize the `DW_ATE_UTF` encoding for base
types, and consequently could not handle `char8_t`, `char16_t`, or
`char32_t`. This has been remedied, and a corresponding test case added
to prevent regressions.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Issue #182 reported that a core dump created by QEMU's dump-guest-memory
command confuses drgn: by default, it only has NT_PRSTATUS notes and
QEMU state notes for each CPU, so drgn thinks it's a userspace core
dump, and it doesn't have the necessary VMCOREINFO to use it as a Linux
kernel core dump.
It turns out that QEMU and Linux can cooperate to add a VMCOREINFO note
to the guest memory dump, which suffices for drgn. Let's detect a QEMU
guest memory dump without a VMCOREINFO note and include instructions on
how to capture a QEMU dump that makes drgn happy.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that we track RA_SIGN_STATE and get the pointer authentication code
mask, we can remove the pointer authentication code from the return
address while unwinding. Add a new architecture callback,
->demangle_return_address(), for this purpose.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In order to support removing the pointer authentication code (PAC) from
return addresses on AArch64, we need to know what bits are being used
for the PAC. We can get this from the NT_ARM_PAC_MASK note in userspace
core dumps and from the NUMBER(KERNELPACMASK) field in VMCOREINFO for
Linux kernel core dumps.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently have 5 names that we match against, and there are more on
the way, so we might as well use a memswitch.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In an upcoming commit, we will parse the AArch64 pointer authentication
code mask either from the VMCOREINFO note or the NT_ARM_PAC_MASK note.
Since it doesn't always come from VMCOREINFO, it doesn't make sense to
put it in struct vmcoreinfo; struct drgn_program makes more sense. So,
make parse_vmcoreinfo() take struct drgn_program instead of struct
vmcoreinfo, rename it to drgn_program_parse_vmcoreinfo(), and replace
struct vmcoreinfo with an anonymous struct in struct drgn_program.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The RA_SIGN_STATE pseudo-register indicates whether the return address
is signed with a pointer authentication code. Add it to the register
definitions. It can be set through a normal CFI register rule or the
vendor-specific DW_CFA_AARCH64_negate_ra_state rule.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This will be used to implement DW_CFA_AARCH64_negate_ra_state. Also fix
a typographical error in a nearby comment.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add the basic register definitions and stack unwinding support
functions. Pointer authentication support will be added in subsequent
commits.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Reading the ABI specification, I realized that fallback_unwind_ppc64()
is completely wrong. Fix it.
Fixes: eec67768aa ("libdrgn: replace elfutils DWARF unwinder with our own")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The usage of the link register in DWARF is a little confusing. On entry
to a function, the link register contains the address that should be
returned to. However, for DWARF, the link register is usually used as
the CFI return_address_register, which means that in an unwound frame,
it will contain the same thing as the program counter. I initially
thought that this was a mistake, believing that the link register should
contain the _next_ return address. However, after a return (with the blr
instruction), the link register will indeed contain the same address as
the program counter. This is consistent with our documentation of
register values for function call frames: "the register values are the
values when control returns to this frame".
So, rename our internal "ra" register to "lr", expose it to the API, and
add a little more documentation to the ppc64 initial register code.
Fixes: 221a218704 ("libdrgn: add powerpc stack trace support")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In additional to the general-purpose registers, struct pt_regs also
provides the cs and ss segment registers and the rflags register.
elf_gregset_t provides the other segment registers as well. We should
expose all of those.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, register definitions are split across two files:
arch_foo.defs lists the names of registers, and arch_foo.c defines the
layout used to store registers in memory. The main rationale for this
was that the layout could be processed entirely by the C preprocessor,
but the register names needed an AWK script that we wanted to keep
minimal. But since commit af6f5a887d ("libdrgn: replace gen_arch.awk
with gen_arch_inc_strswitch.py"), arch_foo.defs is processed by a Python
script.
Let's define both the register names and the register layout in a new
file, arch_foo_defs.py, which is processed by gen_arch_inc_strswitch.py
This has a few benefits:
* It puts all of the register definitions for an architecture in one
place.
* It is easier to maintain than preprocessor magic. (It also makes it
trivial to support registers that don't exist in DWARF, which would've
been harder to do with our preprocessor code.)
* It gets rid of our DSL in favor of Python (which also lets us reduce
repetition for the ppc64 definitions).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This hasn't been used since commit eec67768aa ("libdrgn: replace
elfutils DWARF unwinder with our own").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn_stack_frame_register() gets the register value with copy_lsbytes()
and then byte swaps it if the program's byte order is different from the
host's. But, copy_lsbytes() already fixes the byte order, so this ends
up with the original (wrong) byte order. We also don't need to zero out
the integer that we copy into since copy_lsbytes() also does that.
Fixes: eec67768aa ("libdrgn: replace elfutils DWARF unwinder with our own")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, we identify explicitly-reported kernel modules by the module
name that we get from the .modinfo or the .gnu.linkonce.this_module
section. However, objcopy --only-keep-debug (used for some Linux distro's
separate debug files) does not keep these sections. This means that
passing a file processed by objcopy --only-keep-debug to, e.g., drgn -s,
fails with "could not find kernel module name".
Instead of using the module name as the identifier, let's use the
module's GNU build ID. We can get it on a live system from
/sys/module/<module>/notes/, and on a core dump from struct
module::notes_attrs (which is the implementation of that sysfs
directory).
This was split out of my larger debug info discovery rework, which will
make more use of the build ID.
Closes#178.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
DEFINE_BINARY_SEARCH_TREE_TYPE() doesn't need these. This is preparation
for a potential new use of a BST. But, it's also a good cleanup on its
own and allows us to move some code out of memory_reader.h and into
memory_reader.c. (This is similar to commit 1339dc6a2f ("libdrgn:
hash_table: move entry_to_key to DEFINE_HASH_TABLE_FUNCTIONS()").)
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Instead of getting the address range from the sections we find, get it
directly from /proc/modules or from the `struct module`. (We already had
partial code to get the address range, but I can't remember why I didn't
use it.)
The real motivation for this is the upcoming module rework: it'll allow
us to report the module and its address range before iterating through
its sections. But it also means that we don't need the heuristic to
ignore special sections that shouldn't be considered part of the address
range (e.g., .init, .data..percpu [the latter of which we should be
ignoring but get away with not because it's excluded from sysfs]).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Like drgn_error_fwrite(), but writes to a file descriptor instead of a
stdio stream. This will be used for logging.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn_error_fwrite() only calls string_builder_append_error() to get
special formatting for DRGN_ERROR_OS, but DRGN_ERROR_FAULT also needs
special formatting. Rather than needing to keep drgn_error_fwrite() and
string_builder_append_error() in sync, define them both in terms of a
common macro.
Fixes: 80fef04c70 ("Add address attribute to FaultError exception")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Several have snuck in since the last time I did this in commit
5541fad063 ("Fix some flake8 errors"). Prepare for adding flake8 to
pre-commit by fixing them.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
GCC and Clang have 128-bit integer types on 64-bit targets: __int128 and
unsigned __int128. Clang additionally has N-bit integers of up to 2<<24
bits with _ExtInt(N), which was standardized in C23 as _BitInt(N).
Currently, we disallow creating objects with a >64-bit integer type. Jay
Kamat reported that this would cause errors when examining some
binaries. The reason we disallow this is that we don't have a way to
represent or do operations on >64-bit values. We could make use of a
bignum library like GMP to do this in the future.
However, for now, we can loosen this restriction and at least allow
reference and absent objects with big integer types. This requires
enforcing two things: that we never create a value object with a >64-bit
integer type, and that we never read the value of a reference object
with a >64-bit integer type.
Co-authored-by: Jay Kamat <jaygkamat@gmail.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This will be used for partial 128-bit object support. There are other
places that should probably be converted to use it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The 32-bit and 64-bit variants have different register sizes, so they're
different architectures in drgn. For now, put them in the same file so
that they can share the relocation implementation. We'll need to figure
out how to handle registers later.
P.S. RISC-V has the weirdest relocations so far. /proc/kcore also
appears to be broken.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only relocation type I saw in Debian's kernel module debug info was
R_ARM_ABS32. R_ARM_REL32 is easy. The Linux kernel supports a bunch of
other ones that don't seem relevant to debug info.
Unfortunately, I wasn't able to test this because /proc/kcore doesn't
exist on Arm. This apparently goes all the way back to 2003:
https://lwn.net/Articles/45315/.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only relocation types I saw in Debian's kernel module debug info
were R_AARCH64_ABS64 and R_AARCH64_ABS32. R_AARCH64_ABS16,
R_AARCH64_PREL64, R_AARCH64_PREL32, and R_AARCH64_PREL16 are all easy.
The remaining types supported by the Linux kernel are for movw and
immediate instructions, which aren't relevant to debug info.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only relocation type I saw in Debian's kernel module debug info was
R_386_32. R_386_PC32 is easy. The Linux kernel also supports
R_386_PLT32, but that's the same story as R_X86_64_PLT32 in x86-64, so
we don't implement it for now.
I was torn between naming it i386, x86, or IA-32. x86 isn't immediately
clear whether x86-64 is included or not. No one other than Intel calls
it IA-32. i386 might incorrectly imply that it is strictly the original
i386 instruction set with no later extensions, but the more general
meaning is used frequently in the Linux world (e.g., Debian and QEMU
both call it i386), so I went with that in the end.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
One of the biggest things we depend on libdwfl for is applying
relocations on architectures other than x86-64. I'm exploring the
possibility of removing the libdwfl dependency, so I'm going to add
relocation implementations for more architectures, starting with ppc64.
R_PPC64_ADDR32 and R_PPC64_ADDR64 were the only ones I saw in Debian's
kernel module debug info. R_PPC64_REL32 and R_PPC64_REL64 are
straightforward. The Linux kernel also implements R_PPC64_TOC*, which
don't seem relevant to debugging information, and R_PPC64_REL24 and
R_PPC64_REL16*, which I'd prefer to have a real example of.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Implement R_X86_64_32S and R_X86_64_PC64. I haven't seen these for debug
info in the wild, but they're supported by the Linux kernel and they're
easy to support. The only other type of relocation currently supported
by the kernel is R_X86_64_PLT32, which is trickier. For kernel modules,
it's equivalent to R_X86_64_PC32 (see Linux kernel commit b21ebf2fb4cd
("x86: Treat R_X86_64_PLT32 as R_X86_64_PC32"), but that doesn't seem to
be true in general. It doesn't seem applicable to debug info sections,
so hopefully we don't need to worry about it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In preparation for supporting ELF relocations for more architectures,
generalize ELF relocations to handle SHT_REL sections/ElfN_Rel.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Check that size_t makes sense and make sure int_key_hash_pair() doesn't
get an integer type larger than it supports. I can't imagine either of
these failing in practice, but make our assumptions explicit.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We call hash_combine() with a uint64_t in
drgn_debug_info_module_key_hash_pair() and drgn_type_dedupe_hash_pair().
On 32-bit systems, this only uses the least-significant 32 bits. Use
hash_64_to_32() on 32-bit and hash_128_to_64() on 64-bit to ensure that
we use all bits if we're given a type larger than size_t, and sanity
check that we're not given anything larger than we support.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
On !SMP kernels, crashing_cpu either doesn't exist or is always -1, so
drgn_program_crashed_thread() fails. Detect those cases and treat
crashing_cpu as 0.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that we have gen_strswitch.py, there's no reason to keep this AWK
script around. Replace it with a Python script that outputs a strswitch
file. This also gets rid of our gawk dependency.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This adds an --enable-compiler-warnings flag that:
* Defines a canonical list of warnings that we enforce. For now, this is
-Wall -Wformat-overflow=2 -Wformat-truncation=2, but we can add to it
going forward.
* Enables warnings by default.
* Allows erroring on warnings. We recommend that developers use this and
use it for the CI.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Some functions that could be static found by -Wmissing-prototypes, some
include-what-you-use warnings, some missing SPDX identifiers. These
lints should be automated at some point.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Our cheap heuristic for the default language will not always be correct,
and although we can improve it as cases arise, we should also just have
a way for the user to explicitly set the default language. Add
drgn_program_set_language() to libdrgn and allow setting
drgn.Program.language in the Python bindings. This will also make unit
testing different languages easier.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
libdrgn currently exports struct drgn_language pointers from
drgn_program_language(), drgn_type_language(), and
drgn_object_language(), but doesn't provide any way to do anything with
them. Export our drgn_language instances and add drgn_language_name() so
that they can at least be compared and printed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In the next change, we want to export languages to the public libdrgn
interface. I couldn't figure out any way to export array elements as
their own symbols. I'd also rather not export the drgn_languages array
indices as an enum because that would preclude ever having any sort of
language plugin support.
Instead, let's get rid of the drgn_languages array as it currently
exists and have separate drgn_language structures. This also allows us
to make a bunch of the C implementation functions static again. We keep
the language numbers so that we can store per-language data efficiently
(currently drgn_program::void_types and languages_py), as well as a
drgn_languages array to go from the language number to the struct
drgn_language. But, this is all internal and could be changed if we ever
support language plugins.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This implements the existing thread API methods for live processes other
than drgn_thread_stack_trace(). It also doesn't yet add support for
full-blown tracing, but it at least brings live processes to feature
parity. This is taken from the non-ptrace parts of Kevin Svetlitski's
PR #142, with some modifications.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn_thread_iterator_create(), drgn_thread_iterator_next(), and
drgn_program_find_thread() have big, divergent code paths for different
targets, and this would get worse once we add live processes. Split them
up into multiple functions.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We have a few places where we format a decimal number with sprintf() or
snprintf() to a buffer with an arbitrary size. Instead of this arbitrary
size, let's add a macro to get the exact number of characters required
to format a decimal number, use it in all of these places, and make all
of these places use snprintf() just to be safe. This is more verbose but
self-documenting. The max_decimal_length() macro is inspired by
https://stackoverflow.com/a/13546502/1811295 with some improvements.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
If a TID does not exist, then linux_helper_find_task() succeeds but
returns a null pointer object. Check for that instead of returning a
bogus thread.
Fixes: 301cc767ba ("Implement a new API for representing threads")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently only supported for user-space crash dumps. E.g. no support for
live user-space application debugging or kernel debugging.
Closes#144.
Signed-off-by: Mykola Lysenko <mykolal@fb.com>
LeakSanitizer reported a leak of a Python object when running
tests.test_type.TestTypeTemplateParameter.test_callable. This one is
caused by a missing Py_DECREF() in an error case.
Fixes: 352c31e1ac ("Add support for C++ template parameters")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
LeakSanitizer reported a leak of a drgn_thread_iterator when running the
unit tests. The root cause is that ThreadIterator_dealloc() isn't
freeing the underlying drgn_thread_iterator().
Fixes: 301cc767ba ("Implement a new API for representing threads")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
GCC or binutils on Fedora Rawhide for ARM seems to have a bug where
c_keywords gets placed in the .data.rel.ro section (see
https://www.airs.com/blog/archives/189):
$ readelf -s .libs/libdrgnimpl_la-language_c.o | grep -w c_keywords
475: 00000000 16 OBJECT LOCAL DEFAULT 175 c_keywords
$ readelf -S .libs/libdrgnimpl_la-language_c.o | grep -F '[175]'
[175] .data.rel PROGBITS 00000000 051f90 000010 00 WA 0 0 4
$ readelf -s .libs/_drgn.so | grep -w c_keywords
9267: 0008e84c 16 OBJECT LOCAL DEFAULT 21 c_keywords.lto_priv.0
$ readelf -S .libs/_drgn.so | grep -F '[21]'
[21] .data.rel.ro PROGBITS 0008e018 07e018 000a10 00 WA 0 0 8
This results in a crash on startup when c_keywords_init() attempts to
populate c_keywords.
While this appears to be a compiler or linker bug, I've been meaning to
replace c_keywords with a static lookup function anyways. Now that we
have gen_strswitch.py, we can use it to generate the lookup function.
Add a script, gen_c_keywords_inc_strswitch.py, which generates an array
mapping token kind to spelling, and a memswitch mapping spelling to
token kind.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We have multiple places where we match an input string against several
cases:
* drgn_lexer_c() checks C identifiers against a runtime hash table of C
keywords.
* linux_kernel_object_find() has an if-else ladder of checks for object
names.
* drgn_debug_info_find_sections() loops over an array of ELF section
names to look for sections we need.
* libdrgn/build-aux/gen_arch.awk generates a compile-time trie using
nested switch statements to match register names.
This commit adds a script, gen_strswitch.py, that can hopefully be used
to replace all of these. gen_strswitch.py generalizes the compile-time
trie idea from gen_arch.awk in a few ways:
* It has syntax and semantics based on C switch statements.
* It supports both null-terminated strings and strings with an explicit
length.
* It compresses unique substrings to calls to strcmp(), strncmp(), or
memcmp() when appropriate.
In benchmarks, this approach is more performant than the above options
as well as a candidate based on gperf, while resulting in machine code
around the same size as the straightforward if-else ladder approach.
Future commits will convert the use cases above to use this script.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, Python is only required to build the Python bindings. I
originally wanted to avoid having Python as a build dependency of
libdrgn, which is why gen_arch is an AWK script. However, I want to add
another code generation script which is harder to do in AWK.
Additionally, these days more people are familiar with Python than AWK,
so let's just bite the bullet and require Python to build. No one builds
libdrgn by itself anyways.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
ASAN is incredibly useful during development, especially when dealing
with non-deterministic behavior where re-running the code under a debugger
won't necessarily reproduce the bug each time. In order not to break any
existing workflows, building with ASAN is opt-in (via --enable-asan).
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Running the test suite with ASAN enabled revealed that when
the current target was a userspace core dump, the `crashed_thread`
member of `struct drgn_program` was being freed twice – once indirectly
via `drgn_thread_set_deinit`, and once explicitly in `drgn_prog_deinit`.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
libdrgn uses rint() for formatting floating-point numbers. rint() is
provided by libm, so we need to link with -lm.
This missing library has been masked for a couple of reasons:
1. Python is linked against libm, so the drgn Python bindings implicitly
have this dependency satisfied.
2. On x86-64, GCC has a builtin implementation of rint().
This can be reproduced on x86-64 by building examples/load_debug_info
with -fno-builtin.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Peilin Ye reported a couple of related crashes in drgn caused by Linux
kernel modules which had been processed with objcopy --only-keep-debug
(although he notes that since binutils-gdb commit 8c803a2dd7d3
("elf_backend_section_flags and _bfd_elf_init_private_section_data") (in
binutils v2.35), objcopy --only-keep-debug doesn't seem to work for
kernel modules).
If given an SHT_NOBITS section, elf_getdata() returns an Elf_Data with
d_buf = NULL and d_size set to the size in the section header, which is
often non-zero. There are a few places where this can cause us to
dereference a NULL pointer:
* In relocate_elf_sections() for the relocated section data.
* In relocate_elf_sections() for the symbol table section data.
* In get_kernel_module_name_from_modinfo().
* In get_kernel_module_name_from_this_module().
Fix it by checking the section type or directly checking Elf_Data::d_buf
everywhere that could potentially get an SHT_NOBITS section. This is
based on a PR from Peilin Ye.
Closes#145.
Reported-by: Peilin Ye <peilin.ye@bytedance.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
relocate_elf_section() shouldn't need to deal with reading the sections.
Pull that logic out into relocate_elf_file() (which will be shared with
REL-style relocations when we support those) and rename
relocate_elf_section() to apply_elf_relas().
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Most of the places where we call elf_getdata() (via read_elf_section())
deal with SHT_PROGBITS sections. elf_getdata() always returns the
literal contents of the file for SHT_PROGBITS sections (usually straight
out of the mmap'd file).
The exceptions are the SHT_RELA and SHT_SYMTAB sections in
relocate_elf_section(). For these, if the byte order or alignment of the
file do not match the host, elf_getdata() allocates a new buffer and
converts the contents. relocate_elf_section() also handles unaligned
buffers and swaps the byte order, so it mistakenly ends up with the
original byte order of the file.
Rather than removing that from relocate_elf_section(), let's avoid the
extra allocation and use elf_rawdata(), which always returns the literal
contents of the file.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Some helpers can accept either a str or a Type. If they want to always
work with a Type internally, they need to do something like:
if isinstance(type, str):
type = prog.type(type)
Instead, let's let Program.type() accept a Type and return the exact
same type, so those helpers can unconditionally do:
type = prog.type(type)
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently we can lookup symbols by name or address, but this will only
return one symbol, prioritizing the global symbols. However, symbols may
share the same name, and symbols may also overlap address ranges, so
it's possible for searches to return multiple results. Add functions
which can return a list of multiple matching symbols.
Signed-off-by: Stephen Brennan <stephen@brennan.io>
`pre-commit run --all-files` results in the following minor
updates, which appear to be caused by my own failure to run linters.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Previously, drgn had no way to represent a thread – retrieving a stack
trace (the only extant thread-specific operation) was achieved by
requiring the user to directly provide a tid.
This commit introduces the scaffolding for the design outlined in
issue #92, and implements the corresponding methods for userspace core
dumps, the live Linux kernel, and Linux kernel core dumps. Future work
will build on top of this commit to support live userspace processes.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
The thread API needs a way to iterate over all task_structs in the
kernel. Previously, we translated the existing for_each_task helper,
which supports iterating through specific PID namespaces by walking
through the PID radix tree or PID hashtable. However, we don't need
specific namespaces for the thread API, so we can instead use the much
simpler linked lists of thread groups and threads.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
This reverts commit 2b47583c73. After
Kevin had completed this, we realized that there is a simpler method for
iterating through tasks from libdrgn, which the next commit will
implement. Revert the translation, but keep the improved
tests.helpers.linux.test_pid.TestPid.test_for_each_task.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Kernels built without multiprocessing support don't have
__per_cpu_offset; instead, per_cpu_ptr() is a no-op. Make the helper do
the same and update the test case to work on !SMP as well.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
