Commit Graph

1670 Commits

Author SHA1 Message Date
Omar Sandoval
134393f5f0 Update elfutils in manylinux wheels to 0.188
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-05 19:44:24 -08:00
Omar Sandoval
a7224dbc77 Fix more MANIFEST.in issues
We need to include contrib. Also improve the tools and LICENSES
patterns.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-05 19:13:19 +00:00
Omar Sandoval
e7dfdb9c2a setup.py: add 6.2 to vmtest kernels
Commit c48cddbdb0 ("libdrgn: ppc64: fix stack unwinding since Linux
v5.11 and before v4.20") is the only change necessary for Linux v6.2.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-05 10:59:09 -08:00
Omar Sandoval
9dbf5d03fa CI: update GitHub action versions to silence Node.js warnings
Runs have been getting warnings like the following:

  Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: actions/checkout@v2, actions/setup-python@v2

The guilty actions all have new versions that use Node.js 16.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-05 10:41:30 -08:00
Omar Sandoval
4731de6acc libdrgn: x86_64: unwind with frame pointer more permissively
get_registers_from_frame_pointer() has a sanity check that the unwound
frame pointer must be greater than the current frame pointer. This is
generally true if the entire program is using frame pointers, but not
necessarily otherwise. In particular, if the program is a Linux kernel
configured with ORC, most of the time, rbp is a general purpose
register; it is only used as a frame pointer in special cases without
unwinder information like BPF programs. Those cases are exactly when we
want the frame pointer unwinder, but depending on what the caller was
using rbp for, the frame pointer unwinder might bail prematurely.

Let's remove the sanity check. In the worst case, this could lead us off
into the weeds chasing pointers, but the iteration limit in
drgn_get_stack_trace() prevents that from being dangerous.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-04 16:45:28 -08:00
Omar Sandoval
3ec91c723a tests: kmod: stop using crash_setup_regs()
ppc64's crash_setup_regs() calls ppc_save_regs(), which isn't exported.
So, we need to provide our own implementation. At this point, we might
as well just copy the implementations of crash_setup_regs() for x86-64
and AArch64 and stop trying to use crash_setup_regs().

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-04 16:35:51 -08:00
Omar Sandoval
975255f209 tests: handle cases without slab support in print_annotated_stack() test
When the architecture is missing virtual address translation or the
kernel is configured with SLOB, print_annotated_stack() will not
identify slab objects.

Fixes: 05041423c7 ("drgn.helpers.common.stack: add print_annotated_stack helper function")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-04 15:10:16 -08:00
Omar Sandoval
f0c573dbd9 drgn.helpers.common.memory: handle architectures without virtual address translation in identify_address()
identify_address() is supposed to be best-effort, but if it is called on
an architecture without virtual address translation support, it will
raise a NotImplementedError. Let's catch and ignore it.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-04 15:09:56 -08:00
Omar Sandoval
a6b6afaba2 libdrgn: return DRGN_ERROR_NOT_IMPLEMENTED_ERROR if virtual address translation is not implemented
This will allow us to distinguish it from other errors.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-04 15:09:56 -08:00
Omar Sandoval
4e220b2a61 setup.py: print pass/fail progress while testing kernels
It's useful to check in on a long run without needing to scroll back and
check whether each kernel version passed or failed.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-04 14:25:55 -08:00
Omar Sandoval
c48cddbdb0 libdrgn: ppc64: fix stack unwinding since Linux v5.11 and before v4.20
linux_kernel_get_initial_registers_ppc64() depends on the size of struct
pt_regs, but this has changed multiple times, in:

- Linux kernel commit 4c2de74cc869 ("powerpc/64: Interrupts save PPR on stack rather than
  thread_struct") (in v4.20)
- Linux kernel commit 66f93c5a02d5 ("powerpc/64: Fix kernel stack
  16-byte alignment") (in v4.20)
- Linux kernel commit 8e560921b58c ("powerpc/book3s64/pkeys:
  Store/restore userspace AMR/IAMR correctly on entry and exit from
  kernel") (in v5.11)

It also depends on the overhead stored before struct pt_regs on the
stack, which changed in Linux kernel commit cd52414d5a6c ("powerpc/64:
ELFv2 use minimal stack frames in int and switch frame sizes") (in
v6.2).

We can handle all of these cases by reading the previous r1 from memory
instead of computing it from a hard-coded size and finding the struct
pt_regs based on that r1 and the actual size of struct pt_regs.

Reported in #232.

Reported-by: Sourabh Jain <jainsourabh679@gmail.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-04 13:42:28 -08:00
Omar Sandoval
58e7d751e7 vmtest: add content write permissions to vmtest build job
I changed the default workflow permissions to be restrictive, but I
forgot that the vmtest build needs to upload assets. Add write
permissions for the contents scope, which seems to cover asset uploads.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-29 00:17:46 -08:00
Omar Sandoval
c9e970c935 vmtest: set raise_for_status=True for GitHub API requests
The vmtest build is silently failing to upload kernel builds. We're not
checking for HTTP errors, so let's see if that provides some
information.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-28 23:24:52 -08:00
Omar Sandoval
9ced57a6e1 CI: set GITHUB_TOKEN environment variable
This should fix the "HTTP Error 403: rate limit exceeded" errors we've
been seeing. (I don't know if GitHub Actions used to do this
automatically or they started being stricter about rate limits.)

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-20 21:45:44 -08:00
Omar Sandoval
05d9cb58eb Fix MANIFEST.in warning for LICENSES directory
We're getting this warning from setuptools:

  warning: manifest_maker: MANIFEST.in, line 6: 'recursive-include' expects <dir> <pattern1> <pattern2> ...

Fix it by adding '*'.

Fixes: b156d987d7 ("Move license to LICENSES")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-19 16:31:20 -08:00
Omar Sandoval
55ddc19be4 tests: print more information about HTTP 403 errors
We're getting a bunch of test failures with "HTTP Error 403: rate limit
exceeded" from the GitHub API. The API apparently includes additional
information in the response, so let's print that so that we can figure
out which limit we're hitting.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-19 16:25:56 -08:00
Omar Sandoval
e860a6e077 tests: use #ifdef __s390x__ instead of #if
The Linux kernel apparently enables -Wundef, so although #if is correct,
it results in a warning. Use #ifdef instead.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-19 16:25:05 -08:00
Sven Schnelle
44648ae139 tests: add drgn_test_get_pt_regs() for s390
s390 doesn't implement crash_setup_regs(). Add an alternative
implementation instead.

Suggested-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
2022-12-19 13:48:44 -08:00
Sven Schnelle
1bbeff92bf libdrgn: add s390x unwinding support
Co-authored-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
2022-12-19 13:48:44 -08:00
Omar Sandoval
9ee1ccff98 libdrgn: add stub s390 and s390x architectures with relocation implementation
The only relocation type I saw in Debian's kernel module debug info was
R_390_32. R_390_8, R_390_16, R_390_64, R_390_PC16, R_390_PC32, and
R_390_PC64 are trivial to support, as well. The Linux kernel supports
many more, but hopefully they won't show up for debug info.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-19 13:48:44 -08:00
Omar Sandoval
6ad80feb83 cli: make missing debug info warning more prominent
I've had lots of reports that turned out to be due to missing debug info
(e.g., #161, #175, #231, and more offline). A big part of the problem is
probably that the warning gets lost among the startup information. Let's
move the warning to be the last thing we print and also color it.

After the debug info discovery rework, we can print more specific
advice, but this is better than nothing.

Closes #41.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-19 12:20:45 -08:00
Omar Sandoval
aa5f121ac9 libdrgn: document implementation-defined behavior in add_to_possibly_null_pointer()
Konrad Borowski pointed out that add_to_possibly_null_pointer() relies
on GCC-specific behavior:
https://fosstodon.org/@xfix/109542070338182493. CONTRIBUTING.rst
mentions that we assume that casting between pointers and integers does
not change the bit representation, but we might as well document it
here, too.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-19 12:07:40 -08:00
Omar Sandoval
7b64aee4dd tests: add a real test case for unwinding from struct pt_regs
The current test case only checks that getting a stack trace succeeds.
By having the test kernel module create a struct pt_regs, we can
actually test that we get a reasonable stack trace.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-15 17:44:22 -08:00
Omar Sandoval
868ae51329 drgn.helpers.linux.mm: expand on for_each_page() FaultError note
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-15 09:32:45 -08:00
Omar Sandoval
0d9252ea0c drgn.helpers.linux.mm: document catching FaultError for for_each_page()
for_each_page() can return offline pages that don't actually exist.
Fixing this is difficult (see #228), but working around it by catching
FaultError is easy. Document the recommended usage.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-15 01:37:12 -08:00
Omar Sandoval
c6a24eea6b vmtest: kbuild: prepare for SLOB deprecation in Linux 6.2
CONFIG_SLOB was renamed to CONFIG_SLOB_DEPRECATED during the Linux 6.2
merge window. Let's prepare for that now before rc1 is released.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-14 21:32:40 -08:00
Kevin Svetlitski
454e6e8197 Add test cases for parsing C++ identifiers with template parameters
Signed-off-by: Kevin Svetlitski <svetlitski@meta.com>
2022-12-14 20:55:03 -08:00
Kevin Svetlitski
4213bea149 libdrgn: add limited support for looking up types with template arguments
Currently, looking up a type with template arguments results in an
"invalid character" syntax error on the "<" character. The DWARF index
includes template arguments in indexed names, so we need to do lookups
including the template arguments. Full support for this would require
parsing the template argument list syntax and normalizing it or looking
it up as an AST in some way. For now, it's at least an improvement to
pass the user's string verbatim. To do so, kludge it by adding a token
containing everything from "<" to the matching ">" to the C++ lexer and
appending that to the identifier.

Co-authored-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Kevin Svetlitski <svetlitski@meta.com>
2022-12-14 20:55:03 -08:00
Davide Cavalca
d1bfe17097 packit: add SRPM build deps
Signed-off-by: Davide Cavalca <dcavalca@meta.com>
2022-12-14 09:54:15 -08:00
Davide Cavalca
170eaf8bf4 packit: update to use the new syntax
Signed-off-by: Davide Cavalca <dcavalca@meta.com>
2022-12-14 09:54:15 -08:00
Omar Sandoval
8867b6ee7e drgn.helpers.linux.xarray: fix a couple more documentation typos
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-13 23:26:50 -08:00
Omar Sandoval
cd769c06f0 drgn.helpers.linux.xarray: fix typo in module docstring
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-13 18:25:49 -08:00
Omar Sandoval
7ce84a3f1f drgn.helpers.linux: add proper XArray helpers
Commit 89eb868e95 ("helpers: make find_task() work on recent kernels")
made radix_tree_lookup() and radix_tree_for_each() work for basic
XArrays. However, it doesn't handle a couple of more advanced features:
multi-index entries (which old radix trees actually also supported) and
zero entries. It has also been really confusing to explain to people
unfamiliar with the radix tree -> XArray transition that they should use
helpers named radix_tree for a structure named xarray.

So, let's finally add xa_load(), xa_for_each(), and some additional
auxiliary helpers. The non-recursive xa_for_each() implementation is
based on Kevin Svetlitski's C implementation from commit 2b47583c73
("Rewrite linux helper iterators in C"). radix_tree_lookup() and
radix_tree_for_each() share the implementation with xa_load() and
xa_for_each(), respectively, so they are mostly interchangeable.

Fixes: #61

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-13 17:46:37 -08:00
Omar Sandoval
3c76c12ef1 Stop testing Linux 4.4
I wanted to add tests for the radix tree helpers, but found that they
have never worked for Linux 4.4. Linux 4.4 has been EOL for almost a
year, so rather than going out of our way to support it, let's just stop
testing it. I'm not going to go back and remove existing support for
4.4 (yet), but I'm also not going to make sure future changes support
it.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-13 15:44:43 -08:00
Omar Sandoval
08adab777e pre-commit: update isort, Black, and flake8
No changes required. mypy is still pinned for the sake of Python 3.6.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-13 15:38:15 -08:00
Omar Sandoval
be96cf6e7f pre-commit: add --show-error-codes to mypy
This is useful for adding type: ignore[<code>] comments.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-13 15:01:15 -08:00
Omar Sandoval
287fc845dd packit: disable 32-bit ARM builds
These runs have gotten very flaky. I reported it at
packit/packit-service#1798, which was then reported at
fedora-copr/copr#2429, and finally as a DNF bug in
https://bugzilla.redhat.com/show_bug.cgi?id=2152259. Let's disable them
until it is fixed.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-12 19:11:53 -08:00
Omar Sandoval
25aee8f6a1 vmtest: kbuild: enable transparent hugepages
I want to add support for multi-index XArray entries. This requires
CONFIG_XARRAY_MULTI. The easiest way to enable that is by enabling
CONFIG_TRANSPARENT_HUGEPAGE. We'll probably also want to test helpers
for transparent hugepages themselves later, so let's also enable
CONFIG_READ_ONLY_THP_FOR_FS.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-12 19:08:58 -08:00
Omar Sandoval
89ed5a8b84 cli: prepend current directory to sys.path in interactive mode
The Python REPL prepends the empty string to sys.path in interactive
mode, which means the current directory. See
https://docs.python.org/3/using/cmdline.html#cmdoption-i and
https://docs.python.org/3/library/sys.html#sys.path. Let's do the same
in our own interactive mode so that we can import scripts in the current
directory.

Reported-by: Boris Burkov <boris@bur.io>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-08 14:03:36 -08:00
Omar Sandoval
6486073148 libdrgn: python: fix Py_BuildValue() type in gen_constants.py
We're calling Py_BuildValue() with the "k" format for unsigned long but
passing the enum value itself, which is promoted to int. I don't know
whether there are any ABIs where this matters in practice, but let's use
"K" and cast to unsigned long long explicitly to be safe.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-07 16:46:33 -08:00
Omar Sandoval
94e1407a5f libdrgn: python: don't repeat class names in gen_constants.py
Instead, define the list of constant classes in one place so we can
generate all 3 places that need it.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-07 15:41:49 -08:00
Omar Sandoval
af28419ee5 libdrgn: python: fix path_arg leaks in Program_find_{type,object}
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-06 13:25:55 -08:00
Omar Sandoval
d7204eaa00 libdrgn: python: simplify path_converter()
PyUnicode_FSConverter() already handles os.PathLike, so we only need to
handle None and save the string and length.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-06 10:49:00 -08:00
Omar Sandoval
73fea86792 libdrgn: python: add PyLong_From* and PyLong_As* wrappers for stdint.h types
It feels icky to write code that, for example, passes a uint64_t to
PyLong_FromUnsignedLongLong(). In practice it's fine, but it's much
nicer to have conversion functions specifically for the stdint.h types.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-05 16:06:22 -08:00
Alastair Robertson
7180304c88 libdrgn: dwarf_info: Support DW_TAG_GNU_template_parameter_pack
This DWARF tag is used by C++ classes which take a variable number
of template parameters, such as std::variant and std::tuple.

Signed-off-by: Alastair Robertson <ajor@meta.com>
2022-12-05 15:33:46 -08:00
Omar Sandoval
220c1c72f5 drgn.helpers.common.stack: fix license identifier
This PR was started before the license change, so I missed updating it.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-02 14:46:44 -08:00
Omar Sandoval
32dc31aaca tests: fix rmmod of test kmod
Thanks to commit c08f6be52a ("vmtest: kbuild: add
CONFIG_MODULE_UNLOAD=y"), I was finally able to try unloading the test
kernel module and found a trivial copy-and-paste error. Fix it.

Fixes: 42e7d474d1 ("drgn.helpers.linux.mm: add compound page helpers")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-02 14:42:57 -08:00
Omar Sandoval
174b797ae3 libdrgn: platform: add documentation (especially for drgn_architecture_info)
While reviewing #214, I realized that we have very little documentation
for drgn_architecture_info (and platform internals in general). Let's
document all of the important stuff, and in particular how to add
support for new architectures.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-02 13:55:38 -08:00
Omar Sandoval
1088ef4a1e libdrgn: platform: replace demangle_return_address() with demangle_cfi_registers()
While documenting struct drgn_architecture_info, I realized that
demangle_return_address() is difficult to explain. It's more
straightforward to define this functionality as demangling any registers
that are mangled when using CFI rather than just the return address
register.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-02 13:52:06 -08:00
Omar Sandoval
7cb7ac0fc4 vmtest: kbuild: enable Btrfs, ext4, and XFS
For some upcoming helpers (#218), we're going to want to test against a
few common filesystems.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-12-01 17:47:19 -08:00