libomp leaks memory on fork (see __kmp_atfork_child() in
llvm-project/openmp/src/z_Linux_util.cpp), which doesn't really matter
except that it causes our new C unit tests to fail under LeakSanitizer
because libdrgn has a constructor that calls omp_get_max_threads(). We
could work around it by using CK_NOFORK in the unit tests, but it's not
ideal for us to go digging in sysfs on process startup just because you
linked to libdrgn anyways. Initialize the number of threads when we
actually need it instead, which just requires some care around
concurrency.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
kdump_blob_new() was added in libkdumpfile 0.4.1.
Fixes: 6c7b271a53 ("libdrgn: kdump: pass architecture and vmcoreinfo to libkdumpfile")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We got a couple of reports about drgn failing to get a stack trace
(#391) or getting the wrong stack trace (#404) from a kernel core dump.
Both were caused because drgn assumes that there is an NT_PRSTATUS note
for each CPU in order by CPU number, and in these core dumps some
NT_PRSTATUS notes were missing. There are a least a couple of things
that can cause this: offline CPUs or CPUs that were in a bad state and
didn't respond to the kdump NMI. The former is expected and could be
special-cased, but the latter basically means that we can't trust the
order of the notes. Instead, look up the notes from the crash_notes
per-CPU variable that the kernel uses to populate the ELF notes. We
still need to use the actual NT_PRSTATUS notes for QEMU
dump-guest-memory dumps, but for those we need to use the PID field to
handle CPU hotplugging.
Closes#404.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We have a few places where we parse raw ELF notes independently of
libelf, which isn't complicated but also not trivial thanks to alignment
requirements. In preparation for adding another place, factor the
parsing out into a common helper. Also document the complexities around
figuring out the correct alignment.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This one doesn't need any changes to the callback signature, just the
new interface. We also keep add_object_finder() for compatibility.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The ELF and libkdumpfile paths have duplicated logic for adding the
Linux kernel object finder and setting the default language to C. Factor
them out into a common helper.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Like for symbol finders, we want extra flexibility around configuring
type finders. The type finder callback signature also has a couple of
warts: it doesn't take the program since it was added before types
needed to be constructed from a program, and it is called separately for
each type kind since originally type lookups were for only one kind.
While we're adding a new interface, let's fix these warts: pass the
program and a set of type kinds. However, we need to keep the old
add_type_finder() interface for backwards compatibility.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
C type finders are passed a bitset of type kinds, but Python type
finders currently get called for each type kind. An upcoming update to
the type finder interface will fix this, but we need a set of TypeKinds,
and we'd rather not construct a real Python set for it. Instead, add a
TypeKindSet bitset that satisfies the collections.abc.Set interface.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, the bare-bones add_symbol_finder() interface only allows
adding a symbol finder that is called before any existing finders. It'd
be useful to be able to specify the order that symbol finders should be
called in and to selectively enable and disable them. To do that, we
also need finders to have a name to identify them by. So, replace
add_symbol_finder() (which hasn't been in a release yet) with a set of
interfaces providing this flexibility: register_symbol_finder(),
set_enabled_symbol_finders(), registered_symbol_finders(), and
enabled_symbol_finders(). Also change the callback signature to take the
program.
In particular, this flexibility will be very useful for a plugin system:
pre-installed plugins can register symbol finders that the user can
choose to enable or disable.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This will be used to allow providing names for type, object, and symbol
finders and configuring which ones are called and in what order. We
might even want this for memory readers. I'm assuming there will only be
a handful of handlers on a given list, but the enabled handlers will be
called frequently, and there may be many lists. The implementation is
therefore optimized for fast iteration and small size, and we don't
cache anything that would speed up reconfiguring the list.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Just like in commit 4d970a98c1 ("libdrgn: python: set Python error
indicator if Program_hold_reserve() fails"), callers of
Program_hold_object() assume it sets the error indicator if it fails.
Fixes: a8d632b4c1 ("libdrgn/python: use F14 instead of PyDict for Program::objects")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Most core dumps contain some virtual address mappings: usually at a
minimum, the kernel's direct map is represented in ELF vmcores via a
segment. So normally, drgn can rely on the vmcore to read the virtual
address of swapper_pg_dir. However, some vmcores only contain physical
address information, so when drgn reads memory at swapper_pg_dir, it
needs to first translate that address, thus causing a recursive
translation error like below:
>>> prog["slab_caches"]
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/stepbren/repos/drgn/drgn/cli.py", line 141, in _displayhook
text = value.format_(columns=shutil.get_terminal_size((0, 0)).columns)
_drgn.FaultError: recursive address translation; page table may be missing from core dump: 0xffffffff9662aff8
Debuggers like crash, as well as libkdumpfile, contain fallback code
which can translate swapper_pg_dir in order to bootstrap this address
translation. In fact, the above error does not occur in drgn when using
libkdumpfile. So, let's add this fallback case to drgn as well. Other
architectures will need to have equivalent support added.
Co-authored-by: Illia Ostapyshyn <ostapyshyn@sra.uni-hannover.de>
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
It's not immediately obvious from the API, but libkdumpfile allows
setting the vmcoreinfo attribute. However, setting the vmcoreinfo is not
enough, we must also set the platform information given by the user.
Further, we need to specify these elements in the correct order with
respect to the file descriptor.
If done correctly, then libkdumpfile can successfully handle a core
whose vmcoreinfo is not present in the diskdump or ELF metadata. Of
course, the user must find the vmcoreinfo note and manually give this to
Drgn, along with the platform architecture.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Currently set_core_dump() expects to be initializing the vmcoreinfo
itself. But it could be beneficial to let callers set the vmcoreinfo
with something else, e.g. if the vmcoreinfo can't be found in the ELF
notes or kdump metadata, but has been extracted via other means. So
update these initialization steps to only setup vmcoreinfo information
if it's not already present.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
When looking up a local variable, we pass the function scope DIE to the
DWARF expression evaluator, which uses it to look up DW_AT_frame_base
for DW_OP_fbreg. However, for inline frames, the function scope DIE is
the DW_TAG_inlined_subroutine DIE, which doesn't have a
DW_AT_frame_base; we're supposed to get it from the containing
DW_TAG_subprogram DIE. Fix drgn_stack_frame_find_object() to always pass
the containing DW_TAG_subprogram DIE. This fixes some cases where local
variables are reported as absent even though they are available.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
To try out our new testing framework, move some simple Python unit tests
for the internal lexer API to C unit tests.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
So far we've been getting away with only unit testing through Python.
However, there's plenty of (existing and upcoming) internal code that
would be nice to unit test directly in C. For a framework, I opted for
check (https://libcheck.github.io/check/) because it is minimal, mature,
and available on all major distros. Add the autotools scaffolding,
including a copy of the checkmk script from check 0.15.2 since RHEL and
CentOS don't package it. We check the dependencies at configure time but
only fail if they're not available at `make check` time. Also wire up
`setup.py test` to run `make check`.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The C standard treats an empty variable argument list as a single, empty
argument, so PP_NARGS() currently expands to 1. But this is surprising,
especially for PP_OVERLOAD(). Use the , ##__VA_ARGS__ GNU C extension to
make PP_NARGS() expand to 0 instead. (We could also use __VA_OPT__(,) to
achieve the same thing. It has the advantage of being standardized for
C23, but the huge disadvantage that it's only available on relatively
recent versions of GCC and Clang.) Also check that the extension is
supported in configure.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Building with GCC 7.3 fails with:
../../libdrgn/hash_table.h:340:43: error: initializer element is not constant
static const size_t hash_table_max_size = SIZE_MAX >> hash_table_size_shift;
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Port folly commit a20494d7b2cc ("Shrink F14 maps"), which shrinks tables
using the basic storage policy by 8 bytes. This was performance and
memory-usage neutral for startup, but it would probably save some memory
when lots of namespaces are accessed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
When rehashing a hash table using the vector storage policy, we're
prefetching the index items, but the folly implementation prefetches the
actual entries (because we're about to recalculate their hashes).
Fixes: f94b0262c6 ("libdrgn: hash_table: implement vector storage policy")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We still need a union and some careful casting in a couple of places,
but this is overall much cleaner.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
(vector_size_type)-1 / sizeof(vector_entry_type) is not a limit;
(vector_size_type)-1 is.
Fixes: b450a7b02b ("libdrgn: vector: support using a smaller type for size/capacity")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Avoid a repeat of commit f34f1c278f ("libdrgn/python: fix #includes in
symbol.c") by replacing automake's default, global -I. -I$(srcdir) with
-iquote . only for libdrgnimpl.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Our internal Buck build of drgn doesn't use -I$(srcdir) like automake
does, so #include "drgn.h" and #include "symbol.h" in
libdrgn/python/symbol.c don't work. "drgn.h" is included by "drgnpy.h",
so we can drop that one and use a relative path for "symbol.h" instead.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Expose the Symbol finder API so that Python code can be used to lookup
additional symbols by name or address.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Previously, Symbol objects could not be constructed in Python. However,
in order to allow Python Symbol finders, this needs to be changed.
Unfortunately, Symbol name lifetimes are tricky to manage. We introduce
a lifetime enumeration to handle this. The lifetime may be "static",
i.e. longer than the life of the program; "external", i.e. longer than
the life of the symbol, but no guarantees beyond that; or "owned", i.e.
owned by the Symbol itself.
Symbol objects constructed in Python are "external". The Symbol struct
owns the pointer to the drgn_symbol, and it holds a reference to the
Python object keeping the name valid (either the program, or a PyUnicode
object).
The added complexity is justified by the fact that most symbols are from
the ELF file, and thus share a lifetime with the Program. It would be a
waste to constantly strdup() these strings, just to support a small
number of Symbols created by Python code.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Now that the symbol finder API is created, we can move the ELF symbol
implementation into the debug_info.c file, where it more logically
belongs. The only change to these functions in the move is to declare
elf_symbols_search as static.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
The drgn_program_find_symbol_by_address_internal() function is used when
libdrgn itself may want to lookup a symbol: in particular, when
formatting stack traces or objects. It does less work by possibly
already having a Dwfl_Module looked up, and by avoiding memory
allocation of a symbol, and it's more convenient because it doesn't
return any errors, including on lookup failure.
Unfortunately, the new symbol finder API breaks all of these properties:
the returned symbol is now allocated via malloc() which needs cleanup on
error, and errors can be returned by any finder via the lookup API.
What's more, the finder API doesn't allow specifying an already-known
module. Thankfully, error handling can be improved using the cleanup
API, and looking up a module for an address is usually a reasonably
cheap binary tree operation.
Switch the internal method over to the new finder API. The major
difference now is simply that lookup failures don't result in an error:
they simply result in a NULL symbol.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
The following commit will modify it to use
drgn_program_symbols_search(), a static function declared below. Move it
underneath in preparation. No changes to the function.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Symbol lookup is not yet modular, like type or object lookup. However,
making it modular would enable easier development and prototyping of
alternative Symbol providers, such as Linux kernel module symbol tables,
vmlinux kallsyms tables, and BPF function symbols. To begin with, create
a modular Symbol API within libdrgn, and refactor the ELF symbol search
to use it.
For now, we leave drgn_program_find_symbol_by_address_internal() alone.
Its conversion will require some surgery, since the new API can return
errors, whereas this function cannot.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
By using __attribute__((__packed__)), we shrink each enum from the
default integer size of four bytes, down to the minimum size of one.
This reduces the size of drgn_symbol from 32 bytes down to 26, with 6
bytes of padding. It doesn't have a practical benefit yet, but adding
fields to struct drgn_symbol in the future may not increase the size.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
