Currently, the bare-bones add_symbol_finder() interface only allows
adding a symbol finder that is called before any existing finders. It'd
be useful to be able to specify the order that symbol finders should be
called in and to selectively enable and disable them. To do that, we
also need finders to have a name to identify them by. So, replace
add_symbol_finder() (which hasn't been in a release yet) with a set of
interfaces providing this flexibility: register_symbol_finder(),
set_enabled_symbol_finders(), registered_symbol_finders(), and
enabled_symbol_finders(). Also change the callback signature to take the
program.
In particular, this flexibility will be very useful for a plugin system:
pre-installed plugins can register symbol finders that the user can
choose to enable or disable.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This will be used to allow providing names for type, object, and symbol
finders and configuring which ones are called and in what order. We
might even want this for memory readers. I'm assuming there will only be
a handful of handlers on a given list, but the enabled handlers will be
called frequently, and there may be many lists. The implementation is
therefore optimized for fast iteration and small size, and we don't
cache anything that would speed up reconfiguring the list.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Just like in commit 4d970a98c1 ("libdrgn: python: set Python error
indicator if Program_hold_reserve() fails"), callers of
Program_hold_object() assume it sets the error indicator if it fails.
Fixes: a8d632b4c1 ("libdrgn/python: use F14 instead of PyDict for Program::objects")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Most core dumps contain some virtual address mappings: usually at a
minimum, the kernel's direct map is represented in ELF vmcores via a
segment. So normally, drgn can rely on the vmcore to read the virtual
address of swapper_pg_dir. However, some vmcores only contain physical
address information, so when drgn reads memory at swapper_pg_dir, it
needs to first translate that address, thus causing a recursive
translation error like below:
>>> prog["slab_caches"]
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/stepbren/repos/drgn/drgn/cli.py", line 141, in _displayhook
text = value.format_(columns=shutil.get_terminal_size((0, 0)).columns)
_drgn.FaultError: recursive address translation; page table may be missing from core dump: 0xffffffff9662aff8
Debuggers like crash, as well as libkdumpfile, contain fallback code
which can translate swapper_pg_dir in order to bootstrap this address
translation. In fact, the above error does not occur in drgn when using
libkdumpfile. So, let's add this fallback case to drgn as well. Other
architectures will need to have equivalent support added.
Co-authored-by: Illia Ostapyshyn <ostapyshyn@sra.uni-hannover.de>
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
It's not immediately obvious from the API, but libkdumpfile allows
setting the vmcoreinfo attribute. However, setting the vmcoreinfo is not
enough, we must also set the platform information given by the user.
Further, we need to specify these elements in the correct order with
respect to the file descriptor.
If done correctly, then libkdumpfile can successfully handle a core
whose vmcoreinfo is not present in the diskdump or ELF metadata. Of
course, the user must find the vmcoreinfo note and manually give this to
Drgn, along with the platform architecture.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Currently set_core_dump() expects to be initializing the vmcoreinfo
itself. But it could be beneficial to let callers set the vmcoreinfo
with something else, e.g. if the vmcoreinfo can't be found in the ELF
notes or kdump metadata, but has been extracted via other means. So
update these initialization steps to only setup vmcoreinfo information
if it's not already present.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
When looking up a local variable, we pass the function scope DIE to the
DWARF expression evaluator, which uses it to look up DW_AT_frame_base
for DW_OP_fbreg. However, for inline frames, the function scope DIE is
the DW_TAG_inlined_subroutine DIE, which doesn't have a
DW_AT_frame_base; we're supposed to get it from the containing
DW_TAG_subprogram DIE. Fix drgn_stack_frame_find_object() to always pass
the containing DW_TAG_subprogram DIE. This fixes some cases where local
variables are reported as absent even though they are available.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
To try out our new testing framework, move some simple Python unit tests
for the internal lexer API to C unit tests.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
So far we've been getting away with only unit testing through Python.
However, there's plenty of (existing and upcoming) internal code that
would be nice to unit test directly in C. For a framework, I opted for
check (https://libcheck.github.io/check/) because it is minimal, mature,
and available on all major distros. Add the autotools scaffolding,
including a copy of the checkmk script from check 0.15.2 since RHEL and
CentOS don't package it. We check the dependencies at configure time but
only fail if they're not available at `make check` time. Also wire up
`setup.py test` to run `make check`.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The C standard treats an empty variable argument list as a single, empty
argument, so PP_NARGS() currently expands to 1. But this is surprising,
especially for PP_OVERLOAD(). Use the , ##__VA_ARGS__ GNU C extension to
make PP_NARGS() expand to 0 instead. (We could also use __VA_OPT__(,) to
achieve the same thing. It has the advantage of being standardized for
C23, but the huge disadvantage that it's only available on relatively
recent versions of GCC and Clang.) Also check that the extension is
supported in configure.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Building with GCC 7.3 fails with:
../../libdrgn/hash_table.h:340:43: error: initializer element is not constant
static const size_t hash_table_max_size = SIZE_MAX >> hash_table_size_shift;
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Port folly commit a20494d7b2cc ("Shrink F14 maps"), which shrinks tables
using the basic storage policy by 8 bytes. This was performance and
memory-usage neutral for startup, but it would probably save some memory
when lots of namespaces are accessed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
When rehashing a hash table using the vector storage policy, we're
prefetching the index items, but the folly implementation prefetches the
actual entries (because we're about to recalculate their hashes).
Fixes: f94b0262c6 ("libdrgn: hash_table: implement vector storage policy")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We still need a union and some careful casting in a couple of places,
but this is overall much cleaner.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
(vector_size_type)-1 / sizeof(vector_entry_type) is not a limit;
(vector_size_type)-1 is.
Fixes: b450a7b02b ("libdrgn: vector: support using a smaller type for size/capacity")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Avoid a repeat of commit f34f1c278f ("libdrgn/python: fix #includes in
symbol.c") by replacing automake's default, global -I. -I$(srcdir) with
-iquote . only for libdrgnimpl.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Our internal Buck build of drgn doesn't use -I$(srcdir) like automake
does, so #include "drgn.h" and #include "symbol.h" in
libdrgn/python/symbol.c don't work. "drgn.h" is included by "drgnpy.h",
so we can drop that one and use a relative path for "symbol.h" instead.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Expose the Symbol finder API so that Python code can be used to lookup
additional symbols by name or address.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Previously, Symbol objects could not be constructed in Python. However,
in order to allow Python Symbol finders, this needs to be changed.
Unfortunately, Symbol name lifetimes are tricky to manage. We introduce
a lifetime enumeration to handle this. The lifetime may be "static",
i.e. longer than the life of the program; "external", i.e. longer than
the life of the symbol, but no guarantees beyond that; or "owned", i.e.
owned by the Symbol itself.
Symbol objects constructed in Python are "external". The Symbol struct
owns the pointer to the drgn_symbol, and it holds a reference to the
Python object keeping the name valid (either the program, or a PyUnicode
object).
The added complexity is justified by the fact that most symbols are from
the ELF file, and thus share a lifetime with the Program. It would be a
waste to constantly strdup() these strings, just to support a small
number of Symbols created by Python code.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Now that the symbol finder API is created, we can move the ELF symbol
implementation into the debug_info.c file, where it more logically
belongs. The only change to these functions in the move is to declare
elf_symbols_search as static.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
The drgn_program_find_symbol_by_address_internal() function is used when
libdrgn itself may want to lookup a symbol: in particular, when
formatting stack traces or objects. It does less work by possibly
already having a Dwfl_Module looked up, and by avoiding memory
allocation of a symbol, and it's more convenient because it doesn't
return any errors, including on lookup failure.
Unfortunately, the new symbol finder API breaks all of these properties:
the returned symbol is now allocated via malloc() which needs cleanup on
error, and errors can be returned by any finder via the lookup API.
What's more, the finder API doesn't allow specifying an already-known
module. Thankfully, error handling can be improved using the cleanup
API, and looking up a module for an address is usually a reasonably
cheap binary tree operation.
Switch the internal method over to the new finder API. The major
difference now is simply that lookup failures don't result in an error:
they simply result in a NULL symbol.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
The following commit will modify it to use
drgn_program_symbols_search(), a static function declared below. Move it
underneath in preparation. No changes to the function.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Symbol lookup is not yet modular, like type or object lookup. However,
making it modular would enable easier development and prototyping of
alternative Symbol providers, such as Linux kernel module symbol tables,
vmlinux kallsyms tables, and BPF function symbols. To begin with, create
a modular Symbol API within libdrgn, and refactor the ELF symbol search
to use it.
For now, we leave drgn_program_find_symbol_by_address_internal() alone.
Its conversion will require some surgery, since the new API can return
errors, whereas this function cannot.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
By using __attribute__((__packed__)), we shrink each enum from the
default integer size of four bytes, down to the minimum size of one.
This reduces the size of drgn_symbol from 32 bytes down to 26, with 6
bytes of padding. It doesn't have a practical benefit yet, but adding
fields to struct drgn_symbol in the future may not increase the size.
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
It can be confusing and misleading to see a FaultError for a strange
address that is actually physical.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Python 3.13.0a4 added a new public function, PyLong_AsNativeBytes(), to
replace the private _PyLong_AsByteArray(). It also modified the
signature of _PyLong_AsByteArray(). Let's use PyLong_AsNativeBytes()
when it's available. (PyLong_AsNativeBytes() also has the exact overflow
behavior we wanted, so it's a win-win.)
Closes#385.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
My elfutils patches to support .dwp files were just merged and included
in release 0.191. libdw does all of the heavy lifting, we just need to
apply the section offsets when we parse DWARF ourselves. We still need
to support older versions of elfutils, so add a stub.
Closes#317.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Type units don't have a skeleton unit, so we need to walk over all of
the units in the split DWARF file to find them. Instead of doing this in
a second pass, rework drgn_dwarf_index_read_cus(): instead of
substituting skeleton units with their respective split units, call
drgn_dwarf_index_read_cus() recursively on the split DWARF file.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
_PyDict_GetItemIdWithError() and _PyDict_SetItemId() have
straightforward replacements, so no need to fight this upstream.
Closes#361.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We never free drgn_type::template_parameters.
Fixes: 352c31e1ac ("Add support for C++ template parameters")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
In tag-based KASAN modes, TCR_EL1.TBI1 is enabled, which causes the
top 8 bits of virtual addresses to be ignored for address translation
purposes. Do the same when reading from memory. There is no harm in doing
so unconditionally, as the architecture does not support >56 bit VA sizes.
Signed-off-by: Peter Collingbourne <pcc@google.com>
This function is useful when you only have a list of PCs
and not the full stack trace, for example when working with
the stack depot.
Signed-off-by: Peter Collingbourne <pcc@google.com>
We don't allow this because "value objects with a scalar type cannot be
reinterpreted, as their memory layout in the program is not known". That
doesn't really make sense: we already support reconstructing the
in-memory representation with drgn_object_read_bytes().
Implement this by making drgn_object_slice() support slicing all
objects, using drgn_object_read_bytes() when necessary, then make
drgn_object_reinterpret() a trivial wrapper around it.
Closes#378.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Our existing flexible arrays uses all have extra scaffolding around
them, so this isn't applicable for those, but PR #376 can make use of
it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The generated header causes confusion for some tooling. The only reason
we're generating it is to substitute the version number. For now, let's
just manually duplicate the version number from configure.ac. We could
probably do something fancier like what autoconf itself does [1], but
that looks much more involved than simply adding a step to my release
runbook.
Closes#375.
1: https://lists.gnu.org/archive/html/autoconf/2007-12/msg00027.html
Signed-off-by: Omar Sandoval <osandov@osandov.com>
