PyErr_SetObject() takes a reference on the exception value, so we need
to drop the reference we got when we created the value. Issue #196 ran
into this by reading tons of unmapped addresses.
Fixes: 80fef04c70 ("Add address attribute to FaultError exception")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The config option is and always has been CONFIG_FW_CFG_SYSFS, not
CONFIG_FW_CFG. Also suggest the user-visible CONFIG_KEXEC instead of the
internal CONFIG_CRASH_CORE.
Fixes: 2bd861f719 ("libdrgn: program: detect QEMU guest memory dumps without VMCOREINFO")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
If we only have the stack trace available, it's useful to get the
program it came from. This'll be used eventually for helpers that take a
stack trace.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're currently checking whether the iterator has entered the
non-canonical range when fetching the last level of the page table, but
the cutover actually happens while we're in the last level. Fix it by
doing the check unconditionally.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn_debug_info_find_complete() looks up the name of the incomplete type
in the global namespace. This is incorrect for C++: we need to look it
up in the namespace that the DIE is in.
To find the containing namespace, we need to do a DIE ancestor walk. We
don't want to do this for C, so add a flag indicating whether a language
has namespaces to struct drgn_language. If it's true, then we do the
ancestor walk and then look up the name in the appropriate namespace.
Signed-off-by: Jay Kamat <jaygkamat@gmail.com>
Now that we made the other memory management helpers generic, the last
thing to implement for AArch64 is page table walking. This looks a lot
like the x86-64 equivalent but has to support the various page and
virtual address sizes that can be configured for AArch64.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
AArch64 has changed the location of vmemmap multiple times, and not all
of these can be easily distinguished. Rather than restorting to kernel
version checks, this replaces the vmemmap architecture callback with a
generic approach that gets the vmemmap address directly from the
mem_section table.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
On x86-64, the difference between virtual addresses in the direct map
and the corresponding physical addresses is called PAGE_OFFSET, so we
exposed that via an architecture callback and the Linux kernel object
finder. However, this doesn't translate to other architectures. Namely,
on AArch64, the difference is PAGE_OFFSET - PHYS_OFFSET, and both
PAGE_OFFSET and PHYS_OFFSET have varied over time and between
configurations.
We can remove the architecture callback and avoid version-specific logic
by letting the page table tell us the offset. We just need an address in
the direct map, which is easy to find since this includes kmalloc and
memblock allocations.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
linux_helper_read_vm() has logic to merge adjacent physical address
ranges returned by the page table iterator. However, the check for
whether the ranges are adjacent is incorrect. Fix it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
pgtable_iterator_x86_64::table is only used if
pgtable_iterator_x86_64::index indicates that it has any cached entries,
so there's no point initializing table since we initialize index to
indicate that nothing is cached.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
AArch64 will need different sizes of page table iterators depending on
the page size and virtual address size. Rather than the static
pgtable_iterator_arch_size, allow architectures to define callbacks for
allocating and freeing a page table iterator. Also remove the generic
page table iterator wrapper and just pass that information to the
iterator function.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Rather than computing it every time we need it, compute it once when we
parse PAGE_SIZE from VMCOREINFO (and validate that PAGE_SIZE is a power
of two). This will be more important for AArch64 page table walking.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Very similar to a541e9b170, but adds
partial support for floats (as opposed to integers) which aren't 32 or
64 bits.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Previously `drgn` did not recognize the `DW_ATE_UTF` encoding for base
types, and consequently could not handle `char8_t`, `char16_t`, or
`char32_t`. This has been remedied, and a corresponding test case added
to prevent regressions.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Issue #182 reported that a core dump created by QEMU's dump-guest-memory
command confuses drgn: by default, it only has NT_PRSTATUS notes and
QEMU state notes for each CPU, so drgn thinks it's a userspace core
dump, and it doesn't have the necessary VMCOREINFO to use it as a Linux
kernel core dump.
It turns out that QEMU and Linux can cooperate to add a VMCOREINFO note
to the guest memory dump, which suffices for drgn. Let's detect a QEMU
guest memory dump without a VMCOREINFO note and include instructions on
how to capture a QEMU dump that makes drgn happy.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that we track RA_SIGN_STATE and get the pointer authentication code
mask, we can remove the pointer authentication code from the return
address while unwinding. Add a new architecture callback,
->demangle_return_address(), for this purpose.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In order to support removing the pointer authentication code (PAC) from
return addresses on AArch64, we need to know what bits are being used
for the PAC. We can get this from the NT_ARM_PAC_MASK note in userspace
core dumps and from the NUMBER(KERNELPACMASK) field in VMCOREINFO for
Linux kernel core dumps.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently have 5 names that we match against, and there are more on
the way, so we might as well use a memswitch.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In an upcoming commit, we will parse the AArch64 pointer authentication
code mask either from the VMCOREINFO note or the NT_ARM_PAC_MASK note.
Since it doesn't always come from VMCOREINFO, it doesn't make sense to
put it in struct vmcoreinfo; struct drgn_program makes more sense. So,
make parse_vmcoreinfo() take struct drgn_program instead of struct
vmcoreinfo, rename it to drgn_program_parse_vmcoreinfo(), and replace
struct vmcoreinfo with an anonymous struct in struct drgn_program.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The RA_SIGN_STATE pseudo-register indicates whether the return address
is signed with a pointer authentication code. Add it to the register
definitions. It can be set through a normal CFI register rule or the
vendor-specific DW_CFA_AARCH64_negate_ra_state rule.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This will be used to implement DW_CFA_AARCH64_negate_ra_state. Also fix
a typographical error in a nearby comment.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add the basic register definitions and stack unwinding support
functions. Pointer authentication support will be added in subsequent
commits.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Reading the ABI specification, I realized that fallback_unwind_ppc64()
is completely wrong. Fix it.
Fixes: eec67768aa ("libdrgn: replace elfutils DWARF unwinder with our own")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The usage of the link register in DWARF is a little confusing. On entry
to a function, the link register contains the address that should be
returned to. However, for DWARF, the link register is usually used as
the CFI return_address_register, which means that in an unwound frame,
it will contain the same thing as the program counter. I initially
thought that this was a mistake, believing that the link register should
contain the _next_ return address. However, after a return (with the blr
instruction), the link register will indeed contain the same address as
the program counter. This is consistent with our documentation of
register values for function call frames: "the register values are the
values when control returns to this frame".
So, rename our internal "ra" register to "lr", expose it to the API, and
add a little more documentation to the ppc64 initial register code.
Fixes: 221a218704 ("libdrgn: add powerpc stack trace support")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In additional to the general-purpose registers, struct pt_regs also
provides the cs and ss segment registers and the rflags register.
elf_gregset_t provides the other segment registers as well. We should
expose all of those.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, register definitions are split across two files:
arch_foo.defs lists the names of registers, and arch_foo.c defines the
layout used to store registers in memory. The main rationale for this
was that the layout could be processed entirely by the C preprocessor,
but the register names needed an AWK script that we wanted to keep
minimal. But since commit af6f5a887d ("libdrgn: replace gen_arch.awk
with gen_arch_inc_strswitch.py"), arch_foo.defs is processed by a Python
script.
Let's define both the register names and the register layout in a new
file, arch_foo_defs.py, which is processed by gen_arch_inc_strswitch.py
This has a few benefits:
* It puts all of the register definitions for an architecture in one
place.
* It is easier to maintain than preprocessor magic. (It also makes it
trivial to support registers that don't exist in DWARF, which would've
been harder to do with our preprocessor code.)
* It gets rid of our DSL in favor of Python (which also lets us reduce
repetition for the ppc64 definitions).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This hasn't been used since commit eec67768aa ("libdrgn: replace
elfutils DWARF unwinder with our own").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn_stack_frame_register() gets the register value with copy_lsbytes()
and then byte swaps it if the program's byte order is different from the
host's. But, copy_lsbytes() already fixes the byte order, so this ends
up with the original (wrong) byte order. We also don't need to zero out
the integer that we copy into since copy_lsbytes() also does that.
Fixes: eec67768aa ("libdrgn: replace elfutils DWARF unwinder with our own")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, we identify explicitly-reported kernel modules by the module
name that we get from the .modinfo or the .gnu.linkonce.this_module
section. However, objcopy --only-keep-debug (used for some Linux distro's
separate debug files) does not keep these sections. This means that
passing a file processed by objcopy --only-keep-debug to, e.g., drgn -s,
fails with "could not find kernel module name".
Instead of using the module name as the identifier, let's use the
module's GNU build ID. We can get it on a live system from
/sys/module/<module>/notes/, and on a core dump from struct
module::notes_attrs (which is the implementation of that sysfs
directory).
This was split out of my larger debug info discovery rework, which will
make more use of the build ID.
Closes#178.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
DEFINE_BINARY_SEARCH_TREE_TYPE() doesn't need these. This is preparation
for a potential new use of a BST. But, it's also a good cleanup on its
own and allows us to move some code out of memory_reader.h and into
memory_reader.c. (This is similar to commit 1339dc6a2f ("libdrgn:
hash_table: move entry_to_key to DEFINE_HASH_TABLE_FUNCTIONS()").)
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Instead of getting the address range from the sections we find, get it
directly from /proc/modules or from the `struct module`. (We already had
partial code to get the address range, but I can't remember why I didn't
use it.)
The real motivation for this is the upcoming module rework: it'll allow
us to report the module and its address range before iterating through
its sections. But it also means that we don't need the heuristic to
ignore special sections that shouldn't be considered part of the address
range (e.g., .init, .data..percpu [the latter of which we should be
ignoring but get away with not because it's excluded from sysfs]).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Like drgn_error_fwrite(), but writes to a file descriptor instead of a
stdio stream. This will be used for logging.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn_error_fwrite() only calls string_builder_append_error() to get
special formatting for DRGN_ERROR_OS, but DRGN_ERROR_FAULT also needs
special formatting. Rather than needing to keep drgn_error_fwrite() and
string_builder_append_error() in sync, define them both in terms of a
common macro.
Fixes: 80fef04c70 ("Add address attribute to FaultError exception")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Several have snuck in since the last time I did this in commit
5541fad063 ("Fix some flake8 errors"). Prepare for adding flake8 to
pre-commit by fixing them.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
GCC and Clang have 128-bit integer types on 64-bit targets: __int128 and
unsigned __int128. Clang additionally has N-bit integers of up to 2<<24
bits with _ExtInt(N), which was standardized in C23 as _BitInt(N).
Currently, we disallow creating objects with a >64-bit integer type. Jay
Kamat reported that this would cause errors when examining some
binaries. The reason we disallow this is that we don't have a way to
represent or do operations on >64-bit values. We could make use of a
bignum library like GMP to do this in the future.
However, for now, we can loosen this restriction and at least allow
reference and absent objects with big integer types. This requires
enforcing two things: that we never create a value object with a >64-bit
integer type, and that we never read the value of a reference object
with a >64-bit integer type.
Co-authored-by: Jay Kamat <jaygkamat@gmail.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This will be used for partial 128-bit object support. There are other
places that should probably be converted to use it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The 32-bit and 64-bit variants have different register sizes, so they're
different architectures in drgn. For now, put them in the same file so
that they can share the relocation implementation. We'll need to figure
out how to handle registers later.
P.S. RISC-V has the weirdest relocations so far. /proc/kcore also
appears to be broken.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only relocation type I saw in Debian's kernel module debug info was
R_ARM_ABS32. R_ARM_REL32 is easy. The Linux kernel supports a bunch of
other ones that don't seem relevant to debug info.
Unfortunately, I wasn't able to test this because /proc/kcore doesn't
exist on Arm. This apparently goes all the way back to 2003:
https://lwn.net/Articles/45315/.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only relocation types I saw in Debian's kernel module debug info
were R_AARCH64_ABS64 and R_AARCH64_ABS32. R_AARCH64_ABS16,
R_AARCH64_PREL64, R_AARCH64_PREL32, and R_AARCH64_PREL16 are all easy.
The remaining types supported by the Linux kernel are for movw and
immediate instructions, which aren't relevant to debug info.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only relocation type I saw in Debian's kernel module debug info was
R_386_32. R_386_PC32 is easy. The Linux kernel also supports
R_386_PLT32, but that's the same story as R_X86_64_PLT32 in x86-64, so
we don't implement it for now.
I was torn between naming it i386, x86, or IA-32. x86 isn't immediately
clear whether x86-64 is included or not. No one other than Intel calls
it IA-32. i386 might incorrectly imply that it is strictly the original
i386 instruction set with no later extensions, but the more general
meaning is used frequently in the Linux world (e.g., Debian and QEMU
both call it i386), so I went with that in the end.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
One of the biggest things we depend on libdwfl for is applying
relocations on architectures other than x86-64. I'm exploring the
possibility of removing the libdwfl dependency, so I'm going to add
relocation implementations for more architectures, starting with ppc64.
R_PPC64_ADDR32 and R_PPC64_ADDR64 were the only ones I saw in Debian's
kernel module debug info. R_PPC64_REL32 and R_PPC64_REL64 are
straightforward. The Linux kernel also implements R_PPC64_TOC*, which
don't seem relevant to debugging information, and R_PPC64_REL24 and
R_PPC64_REL16*, which I'd prefer to have a real example of.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Implement R_X86_64_32S and R_X86_64_PC64. I haven't seen these for debug
info in the wild, but they're supported by the Linux kernel and they're
easy to support. The only other type of relocation currently supported
by the kernel is R_X86_64_PLT32, which is trickier. For kernel modules,
it's equivalent to R_X86_64_PC32 (see Linux kernel commit b21ebf2fb4cd
("x86: Treat R_X86_64_PLT32 as R_X86_64_PC32"), but that doesn't seem to
be true in general. It doesn't seem applicable to debug info sections,
so hopefully we don't need to worry about it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In preparation for supporting ELF relocations for more architectures,
generalize ELF relocations to handle SHT_REL sections/ElfN_Rel.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Check that size_t makes sense and make sure int_key_hash_pair() doesn't
get an integer type larger than it supports. I can't imagine either of
these failing in practice, but make our assumptions explicit.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We call hash_combine() with a uint64_t in
drgn_debug_info_module_key_hash_pair() and drgn_type_dedupe_hash_pair().
On 32-bit systems, this only uses the least-significant 32 bits. Use
hash_64_to_32() on 32-bit and hash_128_to_64() on 64-bit to ensure that
we use all bits if we're given a type larger than size_t, and sanity
check that we're not given anything larger than we support.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
On !SMP kernels, crashing_cpu either doesn't exist or is always -1, so
drgn_program_crashed_thread() fails. Detect those cases and treat
crashing_cpu as 0.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that we have gen_strswitch.py, there's no reason to keep this AWK
script around. Replace it with a Python script that outputs a strswitch
file. This also gets rid of our gawk dependency.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This adds an --enable-compiler-warnings flag that:
* Defines a canonical list of warnings that we enforce. For now, this is
-Wall -Wformat-overflow=2 -Wformat-truncation=2, but we can add to it
going forward.
* Enables warnings by default.
* Allows erroring on warnings. We recommend that developers use this and
use it for the CI.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Some functions that could be static found by -Wmissing-prototypes, some
include-what-you-use warnings, some missing SPDX identifiers. These
lints should be automated at some point.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Our cheap heuristic for the default language will not always be correct,
and although we can improve it as cases arise, we should also just have
a way for the user to explicitly set the default language. Add
drgn_program_set_language() to libdrgn and allow setting
drgn.Program.language in the Python bindings. This will also make unit
testing different languages easier.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
libdrgn currently exports struct drgn_language pointers from
drgn_program_language(), drgn_type_language(), and
drgn_object_language(), but doesn't provide any way to do anything with
them. Export our drgn_language instances and add drgn_language_name() so
that they can at least be compared and printed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In the next change, we want to export languages to the public libdrgn
interface. I couldn't figure out any way to export array elements as
their own symbols. I'd also rather not export the drgn_languages array
indices as an enum because that would preclude ever having any sort of
language plugin support.
Instead, let's get rid of the drgn_languages array as it currently
exists and have separate drgn_language structures. This also allows us
to make a bunch of the C implementation functions static again. We keep
the language numbers so that we can store per-language data efficiently
(currently drgn_program::void_types and languages_py), as well as a
drgn_languages array to go from the language number to the struct
drgn_language. But, this is all internal and could be changed if we ever
support language plugins.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This implements the existing thread API methods for live processes other
than drgn_thread_stack_trace(). It also doesn't yet add support for
full-blown tracing, but it at least brings live processes to feature
parity. This is taken from the non-ptrace parts of Kevin Svetlitski's
PR #142, with some modifications.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn_thread_iterator_create(), drgn_thread_iterator_next(), and
drgn_program_find_thread() have big, divergent code paths for different
targets, and this would get worse once we add live processes. Split them
up into multiple functions.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We have a few places where we format a decimal number with sprintf() or
snprintf() to a buffer with an arbitrary size. Instead of this arbitrary
size, let's add a macro to get the exact number of characters required
to format a decimal number, use it in all of these places, and make all
of these places use snprintf() just to be safe. This is more verbose but
self-documenting. The max_decimal_length() macro is inspired by
https://stackoverflow.com/a/13546502/1811295 with some improvements.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
If a TID does not exist, then linux_helper_find_task() succeeds but
returns a null pointer object. Check for that instead of returning a
bogus thread.
Fixes: 301cc767ba ("Implement a new API for representing threads")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently only supported for user-space crash dumps. E.g. no support for
live user-space application debugging or kernel debugging.
Closes#144.
Signed-off-by: Mykola Lysenko <mykolal@fb.com>
LeakSanitizer reported a leak of a Python object when running
tests.test_type.TestTypeTemplateParameter.test_callable. This one is
caused by a missing Py_DECREF() in an error case.
Fixes: 352c31e1ac ("Add support for C++ template parameters")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
LeakSanitizer reported a leak of a drgn_thread_iterator when running the
unit tests. The root cause is that ThreadIterator_dealloc() isn't
freeing the underlying drgn_thread_iterator().
Fixes: 301cc767ba ("Implement a new API for representing threads")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
GCC or binutils on Fedora Rawhide for ARM seems to have a bug where
c_keywords gets placed in the .data.rel.ro section (see
https://www.airs.com/blog/archives/189):
$ readelf -s .libs/libdrgnimpl_la-language_c.o | grep -w c_keywords
475: 00000000 16 OBJECT LOCAL DEFAULT 175 c_keywords
$ readelf -S .libs/libdrgnimpl_la-language_c.o | grep -F '[175]'
[175] .data.rel PROGBITS 00000000 051f90 000010 00 WA 0 0 4
$ readelf -s .libs/_drgn.so | grep -w c_keywords
9267: 0008e84c 16 OBJECT LOCAL DEFAULT 21 c_keywords.lto_priv.0
$ readelf -S .libs/_drgn.so | grep -F '[21]'
[21] .data.rel.ro PROGBITS 0008e018 07e018 000a10 00 WA 0 0 8
This results in a crash on startup when c_keywords_init() attempts to
populate c_keywords.
While this appears to be a compiler or linker bug, I've been meaning to
replace c_keywords with a static lookup function anyways. Now that we
have gen_strswitch.py, we can use it to generate the lookup function.
Add a script, gen_c_keywords_inc_strswitch.py, which generates an array
mapping token kind to spelling, and a memswitch mapping spelling to
token kind.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We have multiple places where we match an input string against several
cases:
* drgn_lexer_c() checks C identifiers against a runtime hash table of C
keywords.
* linux_kernel_object_find() has an if-else ladder of checks for object
names.
* drgn_debug_info_find_sections() loops over an array of ELF section
names to look for sections we need.
* libdrgn/build-aux/gen_arch.awk generates a compile-time trie using
nested switch statements to match register names.
This commit adds a script, gen_strswitch.py, that can hopefully be used
to replace all of these. gen_strswitch.py generalizes the compile-time
trie idea from gen_arch.awk in a few ways:
* It has syntax and semantics based on C switch statements.
* It supports both null-terminated strings and strings with an explicit
length.
* It compresses unique substrings to calls to strcmp(), strncmp(), or
memcmp() when appropriate.
In benchmarks, this approach is more performant than the above options
as well as a candidate based on gperf, while resulting in machine code
around the same size as the straightforward if-else ladder approach.
Future commits will convert the use cases above to use this script.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, Python is only required to build the Python bindings. I
originally wanted to avoid having Python as a build dependency of
libdrgn, which is why gen_arch is an AWK script. However, I want to add
another code generation script which is harder to do in AWK.
Additionally, these days more people are familiar with Python than AWK,
so let's just bite the bullet and require Python to build. No one builds
libdrgn by itself anyways.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
ASAN is incredibly useful during development, especially when dealing
with non-deterministic behavior where re-running the code under a debugger
won't necessarily reproduce the bug each time. In order not to break any
existing workflows, building with ASAN is opt-in (via --enable-asan).
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Running the test suite with ASAN enabled revealed that when
the current target was a userspace core dump, the `crashed_thread`
member of `struct drgn_program` was being freed twice – once indirectly
via `drgn_thread_set_deinit`, and once explicitly in `drgn_prog_deinit`.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
libdrgn uses rint() for formatting floating-point numbers. rint() is
provided by libm, so we need to link with -lm.
This missing library has been masked for a couple of reasons:
1. Python is linked against libm, so the drgn Python bindings implicitly
have this dependency satisfied.
2. On x86-64, GCC has a builtin implementation of rint().
This can be reproduced on x86-64 by building examples/load_debug_info
with -fno-builtin.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Peilin Ye reported a couple of related crashes in drgn caused by Linux
kernel modules which had been processed with objcopy --only-keep-debug
(although he notes that since binutils-gdb commit 8c803a2dd7d3
("elf_backend_section_flags and _bfd_elf_init_private_section_data") (in
binutils v2.35), objcopy --only-keep-debug doesn't seem to work for
kernel modules).
If given an SHT_NOBITS section, elf_getdata() returns an Elf_Data with
d_buf = NULL and d_size set to the size in the section header, which is
often non-zero. There are a few places where this can cause us to
dereference a NULL pointer:
* In relocate_elf_sections() for the relocated section data.
* In relocate_elf_sections() for the symbol table section data.
* In get_kernel_module_name_from_modinfo().
* In get_kernel_module_name_from_this_module().
Fix it by checking the section type or directly checking Elf_Data::d_buf
everywhere that could potentially get an SHT_NOBITS section. This is
based on a PR from Peilin Ye.
Closes#145.
Reported-by: Peilin Ye <peilin.ye@bytedance.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
relocate_elf_section() shouldn't need to deal with reading the sections.
Pull that logic out into relocate_elf_file() (which will be shared with
REL-style relocations when we support those) and rename
relocate_elf_section() to apply_elf_relas().
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Most of the places where we call elf_getdata() (via read_elf_section())
deal with SHT_PROGBITS sections. elf_getdata() always returns the
literal contents of the file for SHT_PROGBITS sections (usually straight
out of the mmap'd file).
The exceptions are the SHT_RELA and SHT_SYMTAB sections in
relocate_elf_section(). For these, if the byte order or alignment of the
file do not match the host, elf_getdata() allocates a new buffer and
converts the contents. relocate_elf_section() also handles unaligned
buffers and swaps the byte order, so it mistakenly ends up with the
original byte order of the file.
Rather than removing that from relocate_elf_section(), let's avoid the
extra allocation and use elf_rawdata(), which always returns the literal
contents of the file.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Some helpers can accept either a str or a Type. If they want to always
work with a Type internally, they need to do something like:
if isinstance(type, str):
type = prog.type(type)
Instead, let's let Program.type() accept a Type and return the exact
same type, so those helpers can unconditionally do:
type = prog.type(type)
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently we can lookup symbols by name or address, but this will only
return one symbol, prioritizing the global symbols. However, symbols may
share the same name, and symbols may also overlap address ranges, so
it's possible for searches to return multiple results. Add functions
which can return a list of multiple matching symbols.
Signed-off-by: Stephen Brennan <stephen@brennan.io>
`pre-commit run --all-files` results in the following minor
updates, which appear to be caused by my own failure to run linters.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Previously, drgn had no way to represent a thread – retrieving a stack
trace (the only extant thread-specific operation) was achieved by
requiring the user to directly provide a tid.
This commit introduces the scaffolding for the design outlined in
issue #92, and implements the corresponding methods for userspace core
dumps, the live Linux kernel, and Linux kernel core dumps. Future work
will build on top of this commit to support live userspace processes.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
The thread API needs a way to iterate over all task_structs in the
kernel. Previously, we translated the existing for_each_task helper,
which supports iterating through specific PID namespaces by walking
through the PID radix tree or PID hashtable. However, we don't need
specific namespaces for the thread API, so we can instead use the much
simpler linked lists of thread groups and threads.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
This reverts commit 2b47583c73. After
Kevin had completed this, we realized that there is a simpler method for
iterating through tasks from libdrgn, which the next commit will
implement. Revert the translation, but keep the improved
tests.helpers.linux.test_pid.TestPid.test_for_each_task.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Kernels built without multiprocessing support don't have
__per_cpu_offset; instead, per_cpu_ptr() is a no-op. Make the helper do
the same and update the test case to work on !SMP as well.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I missed that the kernel has an idle_task() function which uses
cpu_rq()->idle instead of idle_threads; the latter is technically
architecture-specific. So, replace idle_thread() with idle_task(), which
is architecture-independent and more consistent with the kernel.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
PR #129 will need to get the idle thread for a CPU when the idle thread
crashed. Add a helper for this.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In preparation for introducing an API to represent threads, the linux
helper iterators, radix_tree_for_each, idr_for_each, for_each_pid, and
for_each_task have been rewritten in C. This will allow them to be
accessed from libdrgn, which will be necessary for the threads API.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
On gcc version 7.3, we get following compilation error
CC libdrgnimpl_la-dwarf_info.lo
../../libdrgn/dwarf_info.c:181:51: error: initializer element is not
constant
static const size_t DRGN_DWARF_INDEX_NUM_SHARDS = 1 <<
DRGN_DWARF_INDEX_SHARD_BITS;
This fixes the compilation error on older versions of gcc
Signed-off-by: Alakesh Haloi <alakesh.haloi@gmail.com>
dwfl_module_getdwarf() may call into debuginfod_find_executable() or
debuginfod_find_debuginfo(), which aren't thread-safe. So, let's put the
initial call of dwfl_module_getdwarf() (which is the call that may go
into the debuginfod client) into a critical section.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In particular, the Fedora RPM build needs pytest.ini. CONTRIBUTING.rst
should be included along the same lines as README.rst. libdrgn/Doxyfile
should be included so that users with a source distribution can build
the libdrgn documentation.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There are a few reasons for this:
1. dwfl_core_file_report() crashes on elfutils 0.183-0.185. Those
versions are still used by several distros.
2. In order to support --main-symbols and --symbols properly, we need to
report things ourselves.
3. I'm considering moving away from libdwfl in the long term.
We provide an escape hatch for now: setting the environment variable
DRGN_USE_LIBDWFL_REPORT=1 opts out of drgn's reporting and uses
libdwfl's.
Fixes#130.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I implemented the case of a segment in a core file with p_filesz <
p_memsz by treating the difference as zero bytes. This is correct for
ET_EXEC and ET_DYN, but for ET_CORE, it actually means that the memory
existed in the program but was not saved. For userspace core dumps, this
typically happens for read-only file mappings. For kernel core dumps,
makedumpfile does this to indicate memory that was excluded.
Instead, let's return a DRGN_FAULT_ERROR if an attempt is made to read
from these bytes. In the future, we need to read from the
executable/library files when we can.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Issue #130 reported an "unknown attribute form 0x1f20" from drgn. 0x1f20
is DW_FORM_GNU_ref_alt, which is a reference to a DIE in an alternate
file. Similarly, DW_FORM_GNU_strp_alt is a string in an alternate file.
The alternate file is specified by the .gnu_debugaltlink section. This
is generated by dwz, which is used by at least Fedora and Debian.
libdwfl already finds the alternate debug info file, so we can save its
.debug_info and .debug_str and use those to support DW_FORM_GNU_ref_alt
and DW_FORM_GNU_strp_alt in the DWARF index.
Imported units are going to be more work to support in the DWARF index,
but this at least lets drgn start up.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Instead of iterating through every segment, we can just look at the
first and last loadable segments. This even works for vmlinux on x86-64
and Arm which have some special, relocatable segments.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The documentation for libdwelf states that "functions starting with
dwelf_elf will take a (libelf) Elf object as first argument and might
set elf_errno on error". So, we should be using drgn_error_libelf(), not
drgn_error_libdwfl(). While we're here, close the Elf handle before the
file descriptor for consistency.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The current code matches the desired note name as a prefix, but we need
an exact match.
Fixes: 75c3679147 ("Rewrite drgn core in C")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
A pointer, array, or function referring to an anonymous type currently
includes the full type definition in its type name. This creates very
badly formatted objects for, e.g., drgn's own hash table types. Instead,
use "struct <anonymous>" in the type name.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Global symbols are preferred over weak symbols, and weak symbols are
preferred over other symbols.
dwfl_module_addrinfo() seems to have the same preference, so document
address lookups as having the same behavior. (This is actually incorrect
in the case of STB_GNU_UNIQUE, as dwfl_module_addrinfo() treats anything
other than STB_GLOBAL, STB_WEAK, and STB_LOCAL as having the lowest
precedence, but STB_GNU_UNIQUE is so obscure that it probably doesn't
matter.)
Based on work from Stephen Brennan. Closes#121.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
When explicitly reporting a debugging information file for a userspace
program, userspace_report_debug_info() currently always reports it with
a load address range of [0, 0) (i.e., not actually loaded into the
program). This is because for ET_DYN and ET_REL files, we have to
determine the address range by inspecting the core dump or program
state, which is a bit involved.
However, ET_EXEC is much easier: we can get the address range from the
segment headers. In fact, we already implemented this for vmlinux files,
so we can reuse that with a modification to make it more permissive.
ET_CORE debug info files don't make much sense, but libdwfl seems to
treat a reported ET_CORE file the same as ET_EXEC (see
dwfl_report_elf()), so we do, too.
Unfortunately, most executables on modern Linux distributions are
ET_DYN, but this will at least make testing easier.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The upcoming introduction of a higher level data structure to represent
a namespace has implications on the organization of the DWARF index and
debug info management code. Basically, we're going to want to track what
is currently known as struct drgn_dwarf_index_namespace as part of the
new struct drgn_namespace. That only leaves the DWARF specification map
and list of CUs in struct drgn_dwarf_index, which doesn't make much
sense anymore. Instead, let's:
* Move the specification map and CUs into struct drgn_dwarf_info.
* Rename struct drgn_dwarf_index_namespace to struct
drgn_namespace_dwarf_index to indicate that it is the "DWARF index for
a namespace" rather than a "namespace of a DWARF index".
* Move the DWARF index implementation into dwarf_info.c. The DWARF index
and debugging information management have always been coupled, so this
makes it more explicit and is more convenient.
* Improve documentation and naming in the DWARF index implementation.
Now, the only DWARF-specific code outside of dwarf_info.c is for stack
tracing, but we'll leave that for another day.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Continuing the refactoring from the previous commit, move the DWARF code
from debug_info.c to its own file, leaving only the generic ELF file
management in debug_info.c
Signed-off-by: Omar Sandoval <osandov@osandov.com>
debug_info.c currently contains code for managing ELF files with
debugging information, for parsing DWARF, and for parsing ORC. Let's
split it up, starting by moving ORC support to its own file.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
index_namespace() uses `#pragma omp for` instead of `#pragma omp
parallel for`, and it's not already in a parallel section. So, we're
indexing namespaces single-threaded, despite sharding the index. Oops.
Fixes: d1beb0184a ("libdrgn: add support for objects in C++ namespaces")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
index_namespace() sets up an OpenMP loop everytime it is called.
However, if the namespace has no pending DIEs, this is unnecessary
overhead for every DWARF index lookup. Bail early if there are no
pending DIEs (i.e., because we already indexed the namespace). In a
microbenchmark, this was a 10x speed improvement for DWARF index
iterator initialization. For a Python prog.type() lookup benchmark, it
was a 10% speedup.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We can save a pointer to the shard itself instead of the namespace and
shard index. We can also simplify drgn_dwarf_index_iterator_next()
further.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The length of the name was passed, and the name may not be
null-terminated.
Fixes: 565e0343ef ("libdrgn: make symbol index pluggable with callbacks")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
table_insert_searched() returns -1 when insertion fails.
Fixes: d1beb0184a ("libdrgn: add support for objects in C++ namespaces")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
index_die() can only fail if it's out of memory, so return a bool
instead of a struct drgn_error. Also clean up the declarations.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Also:
* Rename struct string to struct nstring and move it to its own header.
* Fix scripts/iwyu.py, which was broken by commit 5541fad063 ("Fix
some flake8 errors").
* Add workarounds for a few outstanding include-what-you-use issues.
There is still a false positive for
include-what-you-use/include-what-you-use#970, but hopefully that is
fixed soon.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Commit dd503c975ab3 ("Fix kdump_vmcoreinfo_raw()") in libkdumpfile
changed the buffer returned by kdump_vmcoreinfo_raw() to be dynamically
allocated. We need to free it on versions containing that change.
Closes#76.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The overloaded version is slower to compile, and we don't actually need
it. We can add a variadic version if we need to in the future. Also add
the script used to generate the macros.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Jay pointed out that when finding the ancestors for a DIE, we should use
DW_AT_sibling to skip over subtrees that can't contain the target DIE.
So, let's check each DIE that we encounter for a DW_AT_sibling
attribute. dwarf_attr() also returns the end of the DIE if it doesn't
find the attribute, which we can use to avoid parsing DIEs redundantly.
This doesn't fit very well into drgn_dwarf_iterator, so let's just
hand-roll this special type of iteration. In my measurements, this made
drgn_find_die_ancestors() ~6x as fast on average.
Closes#124.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Previously shards were allocated as soon as a namespace was
encountered, which means that we had a large array sitting around for
every ns we saw. By allocating them lazily, we can reduce this usage.
Signed-off-by: Jay Kamat <jaygkamat@gmail.com>
DEFINE_HASH_TABLE_TYPE() doesn't actually need to know the key type.
Move that argument (and some of the derived constants) to
DEFINE_HASH_TABLE_FUNCTIONS(). This will allow recursive hash table
types. As a nice side effect, it also reduces the size of common header
files.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The underscore was meant to discourage direct access in favor of using
drgn_program_get_dbinfo(), but it turns out that it's more normal to
access it directly.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only time that we want to create the drgn_debug_info is when we're
loading debugging information. Everywhere else, we fail fast if there is
no debugging information.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgnpy_linux_helper_task_state_to_char() was removed by commit
ff96c75da0 ("helpers: translate task_state_to_char() to Python"), but
I left behind the declaration.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Once we've cleared the pending DIEs vector, we won't use the vector
again (unless we load more debugging information), so we can free it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
When linking libdrgn as a static library, the name 'splay_tree' can
conflict with splay-tree.h in libiberty (namely splay_tree_splay).
Rename relevant functions to have a 'drgn_' prefix
Signed-off-by: Jake Hillion <jakehillion@fb.com>
In larger binaries, there can be a large number of CUs, and since we
store an abbrev table for each CU the extra space starts to add up.
The simplest way to mitigate this is to shrink the vectors before
saving them.
On a large binary, I noticed a memory reduction from 20.4G RES to
18.6G RES (on initial load-in).
And use it in a few appropriate places. This should hopefully make it
harder to make iteration mistakes like the one fixed by commit
4755cfac7c ("libdrgn: dwarf_index: increment correct variable when
rolling back"). While we're doing this, move ARRAY_SIZE() into a new
header file with array_for_each() and make it lowercase.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We need to increment to the next DIE, not the next shard here.
Fixes: 1c9ab2e7d1 ("libdrgn: dwarf_index: fix leak of DWARF index entries on failure")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The "k" format expects an unsigned long, so make sure we cast C enums to
the proper type. This probably doesn't matter for x86 in practice, but
it's better to be safe.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There are a couple of issues with how we interpret
DW_AT_data_member_location:
1. DW_AT_data_member_location can be a location list, and we shouldn't
interpret the section offset as the member offset.
2. DW_AT_data_member_location can be location description block, and in
DWARF 2, it cannot be a constant. We should handle constant offset
expressions as generated by GCC and Clang.
Closes#13.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Mainly unused imports, unused variables, unnecessary f-strings, and
regex literals missing the r prefix. I'm not adding it to the CI linter
because it's too noisy, though.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
