Commit Graph

446 Commits

Author SHA1 Message Date
Omar Sandoval
d60c6a1d68 libdrgn: add register information to platform
In order to retrieve registers from stack traces, we need to know what
registers are defined for a platform. This adds a small DSL for defining
registers for an architecture. The DSL is parsed by an awk script that
generates the necessary tables, lookup functions, and enum definitions.
2019-10-18 14:33:02 -07:00
Omar Sandoval
b8c657d760 libdrgn: python: add sizeof()
It's annoying to do obj.type_.size, and that doesn't even work for every
type. Add sizeof() that does the right thing whether it's given a Type
or Object.
2019-10-18 11:47:32 -07:00
Omar Sandoval
12b0214b4d libdrgn: work around DW_AT_upper_bound of -1 for empty arrays
For the following source code:

  int arr[] = {};

GCC emits the following DWARF:

  DWARF section [ 4] '.debug_info' at offset 0x40:
   [Offset]
   Compilation unit at offset 0:
   Version: 4, Abbreviation section offset: 0, Address size: 8, Offset size: 4
   [     b]  compile_unit         abbrev: 1
             producer             (strp) "GNU C17 9.2.0 -mtune=generic -march=x86-64 -g"
             language             (data1) C99 (12)
             name                 (strp) "test.c"
             comp_dir             (strp) "/home/osandov"
             stmt_list            (sec_offset) 0
   [    1d]    array_type           abbrev: 2
               type                 (ref4) [    34]
               sibling              (ref4) [    2d]
   [    26]      subrange_type        abbrev: 3
                 type                 (ref4) [    2d]
                 upper_bound          (sdata) -1
   [    2d]    base_type            abbrev: 4
               byte_size            (data1) 8
               encoding             (data1) signed (5)
               name                 (strp) "ssizetype"
   [    34]    base_type            abbrev: 5
               byte_size            (data1) 4
               encoding             (data1) signed (5)
               name                 (string) "int"
   [    3b]    variable             abbrev: 6
               name                 (string) "arr"
               decl_file            (data1) test.c (1)
               decl_line            (data1) 1
               decl_column          (data1) 5
               type                 (ref4) [    1d]
               external             (flag_present) yes
               location             (exprloc)
                [ 0] addr .bss+0 <arr>

Note the DW_AT_upper_bound of -1. We end up parsing this as UINT64_MAX
and returning a "DW_AT_upper_bound is too large" error. It appears that
GCC is simply emitting the array length minus one, so let's treat these
as having a length of zero.

Fixes #19.
2019-10-18 03:18:21 -07:00
Omar Sandoval
c0cb50ac25 docs: fix enum_type() parameter documentation 2019-10-18 02:37:00 -07:00
Omar Sandoval
430732093d libdrgn: python: add converter for byteorder
Rather than open-coding the conversion where we need it, make it a
proper converter function.
2019-10-15 21:21:21 -07:00
Omar Sandoval
55a9700435 libdrgn: python: accept integer-like arguments in more places
There are a few places (e.g., Program.symbol(), Program.read()) where it
makes sense to accept, e.g., a drgn.Object with integer type. Replace
index_arg() with a converter function and use it everywhere that we use
the "K" format for PyArg_Parse*.
2019-10-15 21:10:11 -07:00
Omar Sandoval
77253dbdd8 libdrgn: dwarf_info_cache: fix wrong DW_AT_upper_bound error message
I got the error messages for DW_AT_upper_bound and DW_AT_count
backwards; fix it. Also fix the condition for word + 1 overflowing
dimension->length to be word >= UINT64_MAX. (Dwarf_Word is uint64_t so
this is kind of silly, but at least it documents the intent).
2019-10-15 17:07:27 -07:00
Omar Sandoval
181ebe1a01 Add missing entries in drgn.__all__
PlatformFlags and PrimitiveType got squashed into one string because of
a missing comma, and execscript was never added. Fix it and add some
test cases for it.
2019-10-03 16:50:00 -07:00
Omar Sandoval
4e330bbb6e cli: indicate if drgn was compiled with libkdumpfile 2019-10-03 16:22:10 -07:00
Omar Sandoval
6a0e7fb93a docs: fix Btrfs debugger example
Commit c0bc72b0ea ("libdrgn: use splay tree for memory reader")
changed the signature of add_memory_segment(), but I didn't update the
example that used it. Fix it, and while we're here make it take the
device on the command line.
2019-10-02 17:24:11 -07:00
Omar Sandoval
78192cd61e libdrgn: add environment variable to see more missing debug info errors
Sometimes, I'd like to see all of the missing debug info errors rather
than just the first 5. Allow setting this through the
DRGN_MAX_DEBUG_INFO_ERRORS environment variable.
2019-10-02 17:22:12 -07:00
Omar Sandoval
7848c17097 libdrgn: dwarf_index: tweak missing debug section error message
Make the error message more concise, and reorder the sections so that we
check the most obviously-named section (.debug_info) first and least
important section (.debug_line) last.
2019-10-02 17:22:12 -07:00
Omar Sandoval
423d2cd500 libdrgn: dwarf_index: rework file reporting
Currently, the interface between the DWARF index, libdwfl, and the code
which finds and reports vmlinux/kernel modules is spaghetti. The DWARF
index tracks Dwfl_Modules via their userdata. However, despite
conceptually being owned by the DWARF index, the reporting code reports
the Dwfl_Modules and sets up the userdata. These Dwfl_Modules and
drgn_dwfl_module_userdatas are messy to track and pass between the
layers.

This reworks the architecture so that the DWARF index owns the Dwfl
instance and files are reported to the DWARF index; the DWARF index
takes care of reporting to libdwfl internally. In addition to making the
interface for the reporter much cleaner, this improves a few things as a
side-effect:

- We now deduplicate on build ID in addition to path.
- We now skip searching for vmlinux and/or kernel modules if they were
  already indexed.
- We now support compressed ELF files via libdwelf.
- We can now load default debug info at the same time as additional
  debug info.
2019-10-02 17:22:11 -07:00
Omar Sandoval
91265c37a0 libdrgn: hash_table: fix memcmp() undefined behavior
It's undefined behavior to pass NULL to memcmp() even if the length is
zero. See also commit a17215e984 ("libdrgn: dwarf_index: fix memcpy()
undefined behavior").
2019-10-02 17:16:43 -07:00
Omar Sandoval
b05cc0eb75 libdrgn: use libkdumpfile for ELF vmcores when available
vmcores don't include program headers for special memory regions like
vmalloc and percpu. Instead, we need to walk the kernel page table to
map those addresses. Luckily, libkdumpfile already does that. So, if
drgn was built with libkdumpfile support, use it for ELF vmcores. Also
add an environment variable to override this behavior.

Closes #15.
2019-10-02 17:15:36 -07:00
Omar Sandoval
191c5ae253 libelf: clean up SHF_COMPRESSED handling
We don't need the ifdef anymore since we're using the elf.h from our
local elfutils. We can also fold a leftover nested if.
2019-09-24 17:16:17 -07:00
Omar Sandoval
4f2fa5d86f setup.py: only copy extension if newer than destination
I was always doing the copy based on a comment in the equivalent
setuptools code [1]:

    # Always copy, even if source is older than destination, to ensure
    # that the right extensions for the current Python/platform are
    # used.

On closer inspection, this isn't relevant as of PEP 3149 [2], since
extensions are tagged with the ABI version. Let's avoid the unnecessary
copies.

While we're here, let's restore self.inplace just to be safe.

1: e00f4a87ad
2: https://www.python.org/dev/peps/pep-3149/
2019-09-19 11:08:04 -07:00
Omar Sandoval
ca9cdc1991 libdrgn: autogenerate docstrings.h
I didn't want to use BUILT_SOURCES before because that would break make
$TARGET. But, now that doesn't work anyways because we're using SUBDIRS,
so we might as well use BUILT_SOURCES.
2019-09-19 11:08:04 -07:00
Omar Sandoval
aa4bfd646f libdrgn: simplify gen_constants.py header search
Instead of passing in a directory for header files, add -iquote for that
directory.
2019-09-19 11:08:04 -07:00
Omar Sandoval
6a13d74c0c libdrgn: build with bundled elfutils
Now that we have the bundled version of elfutils, build it from libdrgn
and link to it. We can also get rid of the elfutils version checks from
the libdrgn code.
2019-09-19 11:07:12 -07:00
Omar Sandoval
1cedca8ff4 Import elfutils
Based on:

c950e8a9 config: Fix spec file, add manpages and new GFDL license.

With the following patches:

configure: Add --disable-programs
configure: Add --disable-shared
configure: Fix -D_FORTIFY_SOURCE=2 check when CFLAGS contains -Wno-error
libcpu: compile i386_lex.c with -Wno-implicit-fallthrough

The plan is to stop relying on the distribution's version of elfutils
and instead ship our own. This gives us freedom to assume that we're
using the latest version and even ship our own patches (starting with a
few build system improvements). More details are in
scripts/update-elfutils.sh, which was used to generate this commit.
2019-09-05 01:04:33 -07:00
Omar Sandoval
f11a8766bf setup.py: get list of source files from git
Currently, we have a special Makefile target to output the files for a
libdrgn source tarball, and we use that for setuptools. However, the
next change is going to import elfutils, and it'd be a pain to add the
same thing for the elfutils sources. Instead, let's just use git
ls-files for everything. The only difference is that source
distributions won't have the autoconf/automake output.
2019-09-03 17:19:02 -07:00
Omar Sandoval
a3f4fe0518 libdrgn: handle get_debug_sections() errors per-module
There's no reason to fail indexing just because one file is missing
debug information.
2019-08-29 12:26:40 -07:00
Omar Sandoval
62d98b3016 libdrgn: fold ELF relocation code into dwarf_index
I started with drgn_elf_relocator as a separate interface to parallelize
by relocation. However, the final result is parallelized by file, which
means that it can be done as part of the main read_cus() loop. Get rid
of the elf_relocator interface and do it in dwarf_index.c instead. This
means that if/when libdwfl gets faster at ELF relocations, we can rip
out the relocation code without any other changes.
2019-08-29 12:26:22 -07:00
Omar Sandoval
698991b27b Get rid of DRGN_ERROR_{ELF,DWARF}_ERROR and FileFormatError
We're too inconsistent with how we use these for them to be useful (and
it's impossible to distinguish between a format error and some other
error from libelf/libdw/libdwfl), so let's just get rid of them and make
it all DRGN_ERROR_OTHER/Exception.
2019-08-15 15:03:42 -07:00
Omar Sandoval
10142f922f Add basic stack trace support
For now, we only support stack traces for the Linux kernel (at least
v4.9) on x86-64, and we only support getting the program counter and
corresponding function symbol from each stack frame.
2019-08-02 00:26:28 -07:00
Serapheim Dimitropoulos
93d7ea9f01 Add support for kdump-compressed core dumps with libkdumpfile 2019-08-02 00:20:16 -07:00
Omar Sandoval
690b5fd650 libdrgn: generalize architecture to platform
For stack trace support, we'll need to have some architecture-specific
functionality. drgn's current notion of an architecture doesn't actually
include the instruction set architecture. This change expands it to a
"platform", which includes the ISA as well as the existing flags.
2019-08-02 00:11:56 -07:00
Omar Sandoval
71e6744210 libdrgn: add symbol table interface
Now that we're not overloading the name "symbol", we can define struct
drgn_symbol as a symbol table entry. For now, this is very minimal: it's
just a name, address, and size. We can then add a way to find the symbol
for a given address, drgn_program_find_symbol(). For now, this is only
supported through the actual ELF symbol tables. However, in the future,
we can probably support adding "symbol finders".
2019-07-30 09:25:34 -07:00
Omar Sandoval
0c5df56fba libdrgn: replace symbol index with object index
struct drgn_symbol doesn't really represent a symbol; it's just an
object which hasn't been fully initialized (see c2be52dff0 ("libdrgn:
rename object index to symbol index"), it used to be called a "partial
object"). For stack traces, we're going to have a notion of a symbol
that more closely represents an ELF symbol, so let's get rid of the
temporary struct drgn_symbol representation and just return an object
directly.
2019-07-29 17:04:47 -07:00
Omar Sandoval
74bd59e38a libdrgn: python: get rid of Program._symbol()
We can test with Program.object() just as easily, so get rid of this
undocumented method.
2019-07-29 17:04:47 -07:00
Omar Sandoval
b5b024ecac tests: move common helpers to top-level 2019-07-29 17:04:47 -07:00
Omar Sandoval
62ff4e1dba libdrgn: indicate finder lookup failure with special error
Currently, finders indicate a non-fatal lookup error by setting the type
member to NULL. This won't work when we replace the symbol finder with
an object finder (which shouldn't modify the object on failure).
Instead, use a static error for this purpose.
2019-07-29 17:04:47 -07:00
Omar Sandoval
0cb77b303c libdrgn: work around Clang __muloti4 again
See 2dd14ad522 ("libdrgn: work around "undefined reference to
'__muloti4'" when using Clang").
2019-07-29 17:03:45 -07:00
Omar Sandoval
b01d1a943f libdrgn: python: make set_drgn_error() return void *
It still always returns NULL, but now we can directly return from
functions returning some PyObject subtype.
2019-07-28 00:58:36 -07:00
Omar Sandoval
0a74a610bc libdrgn: python: only repr() one level of type members
Currently, repr() of structure and union types goes arbitrarily deep
(except for cycles). However, for lots of real-world types, this is
easily deeper than Python's recursion limit, so we can't get a useful
repr() at all:

>>> repr(prog.type('struct task_struct'))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
RecursionError: maximum recursion depth exceeded while getting the repr of an object

Instead, only print one level of structure and union types.
2019-07-27 15:04:31 -07:00
Omar Sandoval
d63125f133 libdrgn: python: make Program.object() flags optional
Default to FindObjectFlags.ANY.
2019-07-24 11:02:34 -07:00
Omar Sandoval
67a16a09b8 tests: test that Python documentation renders
A couple of times, I've broken help(drgn) by formatting a function
signature in a way that the inspect module doesn't understand (namely,
it crashes on Enum default arguments). Let's add a simple test that the
documentation at least renders.
2019-07-24 11:01:35 -07:00
Omar Sandoval
06cce1baa1 libdrgn: fix typo in drgn_enomem documentation 2019-07-22 17:23:27 -07:00
Omar Sandoval
3e95e88028 libdrgn: vector: protect against overflow when doubling capacity
It seems extremely unlikely that we'd actually overflow before we run
out of memory, but let's just be safe.
2019-07-19 09:27:20 -07:00
Omar Sandoval
27a27940bc libdrgn: split up drgn_program_get_dwarf()
We don't need to get the DWARF index at the time we get the Dwfl handle,
so get rid of drgn_program_get_dwarf(), add drgn_program_get_dwfl(), and
create the DWARF index right before we update in a new function,
drgn_program_update_dwarf_index().
2019-07-19 09:26:30 -07:00
Omar Sandoval
cde021ff4a
Merge pull request #8 from carmeli-tamir/fix_path_lookup
Missing import for list_for_each_entry_reverse
2019-07-19 08:00:05 -07:00
Carmeli Tamir
6e3d3e2987 missing import for list_for_each_entry_reverse 2019-07-19 07:13:33 -04:00
Omar Sandoval
f537a9d0f8 helpers: linux: add path_lookup()
Closes #6.
2019-07-16 16:56:51 -07:00
Omar Sandoval
a17215e984 libdrgn: dwarf_index: fix memcpy() undefined behavior
Apparently, it's undefined behavior to pass NULL as the source to
memcpy(), even if the length is zero. It's an easy fix, so let's appease
UBSan.
2019-07-15 12:27:48 -07:00
Omar Sandoval
1d4854a5bc libdrgn: implement optimized x86-64 ELF relocations
After the libdwfl conversion, we apply ELF relocations with libdwfl
instead of our homegrown implementation. However, libdwfl is much slower
at it than the previous implementation. We can work around this by
(again) applying ELF relocations ourselves for architectures that we
care about (x86-64, to start). For other architectures, we can fall back
to libdwfl.

This new implementation of ELF relocation reworks the parallelization to
be per-file rather than per-relocation. The latter was done originally
because before commit 6f16ab09d6 ("libdrgn: only apply ELF relocations
to relocatable files"), we applied relocations to vmlinux, which is much
larger than most kernel modules. Now that we don't do that, it seems to
be slightly faster to parallelize by file.
2019-07-15 12:27:48 -07:00
Omar Sandoval
e5874ad18a libdrgn: use libdwfl
libdwfl is the elfutils "DWARF frontend library". It has high-level
functionality for looking up symbols, walking stack traces, etc. In
order to use this functionality, we need to report our debugging
information through libdwfl. For userspace programs, libdwfl has a much
better implementation than drgn for automatically finding debug
information from a core dump or PID. However, for the kernel, libdwfl
has a few issues:

- It only supports finding debug information for the running kernel, not
  vmcores.
- It determines the vmlinux address range by reading /proc/kallsyms,
  which is slow (~70ms on my machine).
- If separate debug information isn't available for a kernel module, it
  finds it by walking /lib/modules/$(uname -r)/kernel; this is repeated
  for every module.
- It doesn't find kernel modules with names containing both dashes and
  underscores (e.g., aes-x86_64).

Luckily, drgn already solved all of these problems, and with some
effort, we can keep doing it ourselves and report it to libdwfl.

The conversion replaces a bunch of code for dealing with userspace core
dump notes, /proc/$pid/maps, and relocations.
2019-07-15 12:27:48 -07:00
Omar Sandoval
a9a2cb7cac libdrgn: dwarf_index: move bswap from file to compilation unit
Remove an indirection.
2019-07-15 12:27:38 -07:00
Omar Sandoval
1c9ab2e7d1 libdrgn: dwarf_index: fix leak of DWARF index entries on failure
We're forgetting to unchain new entries which are chained on old
entries.
2019-07-15 12:27:36 -07:00
Omar Sandoval
996d3094ef libdrgn: dwarf_index: fold unindex_files() into index_cus() 2019-07-15 12:27:33 -07:00