I originally did it this way because pydoc doesn't handle non-trivial
defaults in signature very well (see commit 67a16a09b8 ("tests: test
that Python documentation renders")). drgndoc doesn't generate signature
for pydoc anymore, though, so we don't need to worry about it and can
clean up the typing.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I originally envisioned types as dumb descriptors. This mostly works for
C because in C, types are fairly simple. However, even then the
drgn_program_member_info() API is awkward. You should be able to look up
a member directly from a type, but we need the program for caching
purposes. This has also held me back from adding offsetof() or
has_member() APIs.
Things get even messier with C++. C++ template parameters can be objects
(e.g., template <int N>). Such parameters would best be represented by a
drgn object, which we need a drgn program for. Static members are a
similar case.
So, let's reimagine types as being owned by a program. This has a few
parts:
1. In libdrgn, simple types are now created by factory functions,
drgn_foo_type_create().
2. To handle their variable length fields, compound types, enum types,
and function types are constructed with a "builder" API.
3. Simple types are deduplicated.
4. The Python type factory functions are replaced by methods of the
Program class.
5. While we're changing the API, the parameters to pointer_type() and
array_type() are reordered to be more logical (and to allow
pointer_type() to take a default size of None for the program's
default pointer size).
6. Likewise, the type factory methods take qualifiers as a keyword
argument only.
A big part of this change is updating the tests and splitting up large
test cases into smaller ones in a few places.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
struct drgn_program has a bunch of state scattered around. Group it
together more logically, even if it means sacrificing some padding here
and there.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Most places that call these check has_platform and return an error, and
those that don't can live with the extra check.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Program::objects is used to store references to objects that must stay
alive while the Program is alive. It is currently a PyDict where the
keys are the object addresses as PyLong and the values are the objects
themselves. This has two problems:
1. Allocating the key as a full object is obviously wasteful.
2. PyDict doesn't have an API for reserving capacity ahead of time,
which we want for an upcoming change.
Both of these are easily fixed by using our own hash table.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn_object_init() is available in drgh.h file and seems to a required
call before calling drgn_program_find_object().
Without this, trying to call drgn_object_init() from an external C
application results in undefined reference.
Signed-off-by: Aditya Sarwade <asarwade@fb.com>
This picks up a newer version of QEMU and lets us use udevadm trigger
-w. Let's also explicitly add "os: linux" to silence the config
validation.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
As of QEMU commit a5804fcf7b22 ("9pfs: local: ignore O_NOATIME if we
don't have permissions") (in v5.1.0), QEMU handles O_NOATIME sanely, so
we don't need the LD_PRELOAD hack. Since we're adding a version check,
make the multidevs check based on the version, too.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that drgndoc can handle overloads and we have the IntegerLike and
Path aliases, we can add type annotations to all helpers. There are also
a couple of functional changes that snuck in here to make annotating
easier.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
One of the blockers for adding type annotations to helpers is that some
helpers need to be overloaded, but drgndoc doesn't support that. This
adds support. Each function now tracks all of its overloaded signature,
each of which may be documented separately. The formatted output (for
functions/methods and classes with __init__()) combines all of the
documented overloads.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We can get rid of the :include: and :exclude: options by deciding solely
based on whether a node has a docstring. Empty docstrings can be used to
indicate nodes that should be included with no additional content. The
__init__() method must now also have a docstring in order to be
documented. Additionally, the directives are now fully formatted by the
Formatter rather than being split between the Formatter and
DrgnDocDirective.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The helpers implemented in C have Python wrappers only for the purpose
of documentation. This is because drgndoc ignores all imports when
recursively documenting attributes. However, mypy uses the convention
that aliased imports (i.e., import ... as ... or from ... import ... as
...) are considered re-exported, so we can follow that convention and
include aliased imports. (mypy also considered attributes in __all__ as
re-exported, so we should probably follow that in the future, too, but
for now aliased imports are enough). This lets us get rid of the Python
wrappers.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Rather than duplicating Union[str, bytes, os.PathLike] everywhere, add
an alias. Also make it explicitly os.PathLike[str] or os.PathLike[bytes]
to get rid of some mypy --strict errors.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Lots if interfaces in drgn transparently turn an integer Object into an
int by using __index__(), so add an IntegerLike protocol for this and
use it everywhere applicable.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The __init_subclass__ and __class_getitem__ methods are always class
methods even if not decorated as such, so format them accordingly.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Every few builds or so, a vmtest VM crashes after printing "x86: Booting
SMP configuration:". After some difficult debugging, I determined that
the crash happens in arch/x86/realmode/rm/trampoline_64.S (the code that
initializes secondary CPUs) at the ljmp from startup_32 to startup_64.
The real problem happens earlier in startup_32:
movl $pa_trampoline_pgd, %eax
movl %eax, %cr3
Sometimes, the store to CR3 "fails" and CR3 remains zero, which causes
the later ljmp to triple fault.
This can be reproduced by the following script:
#!/bin/sh
curl -L 'https://www.dropbox.com/sh/2mcf2xvg319qdaw/AABFKsISWRpndNZ1gz60O-qSa/x86_64/vmlinuz-5.8.0-rc7-vmtest1?dl=1' -o vmlinuz
cat > commands.gdb << "EOF"
set confirm off
target remote :1234
# arch/x86/realmode/rm/trampoline_64.S:startup_32 after CR3 store.
hbreak *0x9ae09 if $cr3 == 0
command
info registers eax cr3
quit 1
end
# kernel/smp.c:smp_init() after all CPUs have been brought up. If we get here,
# the bug wasn't triggered.
hbreak *0xffffffff81ed4484
command
kill
quit 0
end
continue
EOF
while true; do
qemu-system-x86_64 -cpu host -enable-kvm -smp 64 -m 128M \
-nodefaults -display none -serial file:/dev/stdout -no-reboot \
-kernel vmlinuz -append 'console=0,115200 panic=-1 nokaslr' \
-s -S &
gdb -batch -x commands.gdb || exit 1
done
This seems to be a problem with nested virtualization that was fixed by
Linux kernel commit b4d185175bc1 ("KVM: VMX: give unrestricted guest
full control of CR3") (in v4.17). Apparently, the Google Cloud hosts
that Travis runs on are missing this fix. We obviously can't patch those
hosts, but we can work around it. Disabling unrestricted guest support
in the Travis VM causes CR3 stores in the nested vmtest VM to be
emulated, bypassing the bug.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
PyPI's RST parser apparently doesn't know the highlight directive, which
snuck into the README in commit 4de147e478 ("Add CONTRIBUTING.rst").
Use code-block instead.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Make the KASLR offset available to Python in a new
drgn.helpers.linux.boot module, and move pgtable_l5_enabled() there,
too.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
My work VPN is apparently closing HTTP connections prematurely, which
exposed that urllib won't catch incomplete reads if copied through
shutil.copyfileobj(). Check it explicitly.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn.h is generated from drgn.h.in since commit d60c6a1d68 ("libdrgn:
add register information to platform").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Clean up the coding style of the remaining few places that the last
couple of changes didn't rewrite.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The folly F14 implementation provides 3 storage policies: value, node,
and vector. The default F14FastMap/F14FastSet chooses between the value
and vector policies based on the value size.
We currently only implement the value policy, as the node policy is easy
to emulate and the vector policy would've added more complexity. This
adds support for the vector policy (adding even more C abuse :) and
automatically chooses the policy the same way as folly. It'd be easy to
add a way to choose the policy if needed.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only major change to the folly F14 implementation since I originally
ported it is commit 3d169f4365cf ("memory savings for F14 tables with
explicit reserve()"). That is a small improvement for small tables and a
large improvement for vector tables, which are about to be added.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
posix_memalign() doesn't have the restriction that the size must be a
multiple of the alignment like aligned_alloc() does in C11.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Compile-time constants have DW_AT_const_value instead of DW_AT_location.
We can translate those to a value object.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Really it's more of a test program than an example program. It's useful
for benchmarking, testing with valgrind, etc. It's not built by default,
but it can be built manually with:
$ make -C build/temp.* examples/load_debug_info
And run with:
$ ./build/temp.*/examples/load_debug_info
Signed-off-by: Omar Sandoval <osandov@osandov.com>
GCC 10 doesn't generate a DIE for union thread_union, which breaks our
THREAD_SIZE object finder. The previous change removed our internal
dependency on THREAD_SIZE, so disable this test while I investigate why
GCC changed.
