The helpers implemented in C have Python wrappers only for the purpose
of documentation. This is because drgndoc ignores all imports when
recursively documenting attributes. However, mypy uses the convention
that aliased imports (i.e., import ... as ... or from ... import ... as
...) are considered re-exported, so we can follow that convention and
include aliased imports. (mypy also considered attributes in __all__ as
re-exported, so we should probably follow that in the future, too, but
for now aliased imports are enough). This lets us get rid of the Python
wrappers.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Rather than duplicating Union[str, bytes, os.PathLike] everywhere, add
an alias. Also make it explicitly os.PathLike[str] or os.PathLike[bytes]
to get rid of some mypy --strict errors.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Lots if interfaces in drgn transparently turn an integer Object into an
int by using __index__(), so add an IntegerLike protocol for this and
use it everywhere applicable.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The __init_subclass__ and __class_getitem__ methods are always class
methods even if not decorated as such, so format them accordingly.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Every few builds or so, a vmtest VM crashes after printing "x86: Booting
SMP configuration:". After some difficult debugging, I determined that
the crash happens in arch/x86/realmode/rm/trampoline_64.S (the code that
initializes secondary CPUs) at the ljmp from startup_32 to startup_64.
The real problem happens earlier in startup_32:
movl $pa_trampoline_pgd, %eax
movl %eax, %cr3
Sometimes, the store to CR3 "fails" and CR3 remains zero, which causes
the later ljmp to triple fault.
This can be reproduced by the following script:
#!/bin/sh
curl -L 'https://www.dropbox.com/sh/2mcf2xvg319qdaw/AABFKsISWRpndNZ1gz60O-qSa/x86_64/vmlinuz-5.8.0-rc7-vmtest1?dl=1' -o vmlinuz
cat > commands.gdb << "EOF"
set confirm off
target remote :1234
# arch/x86/realmode/rm/trampoline_64.S:startup_32 after CR3 store.
hbreak *0x9ae09 if $cr3 == 0
command
info registers eax cr3
quit 1
end
# kernel/smp.c:smp_init() after all CPUs have been brought up. If we get here,
# the bug wasn't triggered.
hbreak *0xffffffff81ed4484
command
kill
quit 0
end
continue
EOF
while true; do
qemu-system-x86_64 -cpu host -enable-kvm -smp 64 -m 128M \
-nodefaults -display none -serial file:/dev/stdout -no-reboot \
-kernel vmlinuz -append 'console=0,115200 panic=-1 nokaslr' \
-s -S &
gdb -batch -x commands.gdb || exit 1
done
This seems to be a problem with nested virtualization that was fixed by
Linux kernel commit b4d185175bc1 ("KVM: VMX: give unrestricted guest
full control of CR3") (in v4.17). Apparently, the Google Cloud hosts
that Travis runs on are missing this fix. We obviously can't patch those
hosts, but we can work around it. Disabling unrestricted guest support
in the Travis VM causes CR3 stores in the nested vmtest VM to be
emulated, bypassing the bug.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
PyPI's RST parser apparently doesn't know the highlight directive, which
snuck into the README in commit 4de147e478 ("Add CONTRIBUTING.rst").
Use code-block instead.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Make the KASLR offset available to Python in a new
drgn.helpers.linux.boot module, and move pgtable_l5_enabled() there,
too.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
My work VPN is apparently closing HTTP connections prematurely, which
exposed that urllib won't catch incomplete reads if copied through
shutil.copyfileobj(). Check it explicitly.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn.h is generated from drgn.h.in since commit d60c6a1d68 ("libdrgn:
add register information to platform").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Clean up the coding style of the remaining few places that the last
couple of changes didn't rewrite.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The folly F14 implementation provides 3 storage policies: value, node,
and vector. The default F14FastMap/F14FastSet chooses between the value
and vector policies based on the value size.
We currently only implement the value policy, as the node policy is easy
to emulate and the vector policy would've added more complexity. This
adds support for the vector policy (adding even more C abuse :) and
automatically chooses the policy the same way as folly. It'd be easy to
add a way to choose the policy if needed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only major change to the folly F14 implementation since I originally
ported it is commit 3d169f4365cf ("memory savings for F14 tables with
explicit reserve()"). That is a small improvement for small tables and a
large improvement for vector tables, which are about to be added.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
posix_memalign() doesn't have the restriction that the size must be a
multiple of the alignment like aligned_alloc() does in C11.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Compile-time constants have DW_AT_const_value instead of DW_AT_location.
We can translate those to a value object.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Really it's more of a test program than an example program. It's useful
for benchmarking, testing with valgrind, etc. It's not built by default,
but it can be built manually with:
$ make -C build/temp.* examples/load_debug_info
And run with:
$ ./build/temp.*/examples/load_debug_info
Signed-off-by: Omar Sandoval <osandov@osandov.com>
GCC 10 doesn't generate a DIE for union thread_union, which breaks our
THREAD_SIZE object finder. The previous change removed our internal
dependency on THREAD_SIZE, so disable this test while I investigate why
GCC changed.
Commit eea5422546 ("libdrgn: make Linux kernel stack unwinding more
robust") overlooked that if the task is running in userspace, the stack
pointer in PRSTATUS obviously won't match the kernel stack pointer.
Let's bite the bullet and use the PID. If the race shows up in practice,
we can try to come up with another workaround.
This documents best practices for contributing to drgn. We now require a
DCO sign-off.
Also clean up some related areas in the documentation.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I once tried to implement a generic arithmetic right shift macro without
relying on any implementation-defined behavior, but this turned out to
be really hard. drgn is fairly tied to GCC and GCC-compatible compilers
(like Clang), so let's just assume GCC's model [1]: modular conversion
to signed types, two's complement signed bitwise operators, and sign
extension for signed right shift.
1: https://gcc.gnu.org/onlinedocs/gcc/Integers-implementation.html
Declaring a local vector or hash table and separately initializing it
with vector_init()/hash_table_init() is annoying. Add macros that can be
used as initializers.
This exposes several places where the C89 style of placing all
declarations at the beginning of a block is awkward. I adopted this
style from the Linux kernel, which uses C89 and thus requires this
style. I'm now convinced that it's usually nicer to declare variables
where they're used. So let's officially adopt the style of mixing
declarations and code (and ditch the blank line after declarations) and
update the functions touched by this change.
We were forgetting to mask away the extra bits. There are two places
that we use the tag without converting it to a uint8_t:
hash_table_probe_delta(), which is mostly benign since we mask it by the
chunk mask anyways; and table_chunk_match() without SSE 2, which
completely breaks.
While we're here, let's align the comments better.
After thinking about it some more, I realized that "libdwfl: simplify
activation frame logic" breaks the case where during unwinding someone
queries isactivation for reasons other than knowing whether to decrement
program counter. Revert the patch and refactor "libdwfl: add interface
for getting Dwfl_Module and Dwarf_Frame for Dwfl_Frame" to handle it
differently.
Based on:
c95081596 size: Also obey radix printing for bsd format.
With the following patches:
configure: Add --disable-programs
configure: Add --disable-shared
libdwfl: add interface for attaching to/detaching from threads
libdwfl: export __libdwfl_frame_reg_get as dwfl_frame_register
libdwfl: add interface for getting Dwfl_Module and Dwarf_Frame for Dwfl_Frame
libdwfl: add interface for evaluating DWARF expressions in a frame
drgn has a couple of issues unwinding stack traces for kernel core
dumps:
1. It can't unwind the stack for the idle task (PID 0), which commonly
appears in core dumps.
2. It uses the PID in PRSTATUS, which is racy and can't actually be
trusted.
The solution for both of these is to look up the PRSTATUS note by CPU
instead of PID.
For the live kernel, drgn refuses to unwind the stack of tasks in the
"R" state. However, the "R" state is running *or runnable*, so in the
latter case, we can still unwind the stack. The solution for this is to
look at on_cpu for the task instead of the state.
We currently unwind from pt_regs and NT_PRSTATUS using an array of
register definitions. It's more flexible and more efficient to do this
with an architecture-specific callback. For x86-64, this change also
makes us depend on the binary layout rather than member names of struct
pt_regs, but that shouldn't matter unless people are defining their own,
weird struct pt_regs.
The model has always been that drgn Objects are immutable, but for some
reason I went through the trouble of allowing __init__() to reinitialize
an already initialized Object. Instead, let's fully initialize the
Object in __new__() and get rid of __init__().
It's annoying to have to do value= when creating objects, especially in
interactive mode. Let's allow passing in the value positionally so that
`Object(prog, "int", value=0)` becomes `Object(prog, "int", 0)`. It's
clear enough that this is creating an int with value 0.
drgn was originally my side project, but for awhile now it's also been
my work project. Update the copyright headers to reflect this, and add a
copyright header to various files that were missing it.
For functions that call a noreturn function, the compiler may omit code
after the call instruction. This means that the return address may not
lie in the caller's symbol. dwfl_frame_pc() returns whether a frame is
an "activation", i.e., its program counter is guaranteed to lie within
the caller. This is only the case for the initial frame, frames
interrupted by a signal, and the signal trampoline frame. For everything
else, we need to decrement the program counter before doing any lookups.
Rebase on master and fix dwfl_frame_module/dwfl_frame_dwarf_frame to
decrement the program counter when necessary.
Based on:
a8493c12a libdw: Skip imported compiler_units in libdw_visit_scopes walking DIE tree
With the following patches:
configure: Add --disable-programs
configure: Add --disable-shared
libdwfl: simplify activation frame logic
libdwfl: add interface for attaching to/detaching from threads
libdwfl: add interface for getting Dwfl_Module and Dwarf_Frame for Dwfl_Frame
libdwfl: export __libdwfl_frame_reg_get as dwfl_frame_register
libdwfl: add interface for evaluating DWARF expressions in a frame
Now that we can walk page tables, we can use it in a memory reader that
reads kernel memory via the kernel page table. This means that we don't
need libkdumpfile for ELF vmcores anymore (although I'll keep the
functionality around until this code has been validated more).
