Commit Graph

1557 Commits

Author SHA1 Message Date
Omar Sandoval
030fb78eaf vmtest.config: enable LPAE for Arm alternative flavor
LPAE uses a different page table format and enables 2GB huge pages, so
it'll be interesting to test.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-23 18:03:02 -08:00
Omar Sandoval
0b7238baa3 vmtest.config: fix CONFIG_KEXEC_FILE for s390x
On s390x, CONFIG_KEXEC_FILE depends on CONFIG_CRYPTO_SHA256_S390.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-23 17:44:05 -08:00
Omar Sandoval
cc77c444ba vmtest.config: enable rtc-cmos driver in x86-64 Kconfig
We go through the trouble of enabling a real-time clock driver for other
architectures, so do the same on x86-64.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-23 17:19:23 -08:00
Omar Sandoval
ee20b2d759 vmtest.config: enable hugetlbfs in kconfig
This will be used to test huge page address translation.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-23 17:09:40 -08:00
Omar Sandoval
07b9ee5f36 vmtest.kbuild: generate package in final directory
vmtest.kbuild -b $build_dir -p $package_dir --package-format directory
fails when attempting to rename the temporary package directory if
$build_dir and $package_dir are on different filesystems or Btrfs
subvolumes. Put the temporary package directory in $package_dir so that
the rename can succeed.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-23 11:16:46 -08:00
Omar Sandoval
ff89a6d267 drgn.helpers.linux.slab: fix check for directly mapped address
High memory is not directly mapped, so _find_containing_slab() needs to
check against max_low_pfn, not max_pfn.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-23 01:37:29 -08:00
Omar Sandoval
3a4480289c tests: remove @skip_unless_have_full_mm_support from page <-> pfn <-> phys helper tests
The phys <-> pfn helpers only need PAGE_SHIFT, which is always
available, the page <-> pfn helpers only need vmemmap or mem_map, which
is independent of virtual address translation, and the page <-> phys
helpers are built on top of those.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-22 16:12:33 -08:00
Omar Sandoval
7cf3d7ddf9 drgn.helpers.linux.mm: handle CONFIG_FLATMEM
AArch64, ppc64, s390x, and x86-64 all default to
CONFIG_SPARSEMEM_VMEMMAP, which allows us to translate between struct
page * and PFN with trivial array operations on vmemmap. Arm and i386,
however, use CONFIG_FLATMEM, which has a different array, mem_map, that
we can use similarly. Update the helpers that use vmemmap to use a new
wrapper that gets the proper array.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-22 16:06:21 -08:00
Omar Sandoval
101639caaf tests: don't compare values of reference objects in assertIdentical()
The value can change between two reads. This caused test failures in the
previous commit 18a8f69ad8 ("libdrgn: linux_kernel: add object finder
for jiffies"). The important thing is that the addresses are the same.

Fixes: 75c3679147 ("Rewrite drgn core in C")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-22 13:33:47 -08:00
Omar Sandoval
18a8f69ad8 libdrgn: linux_kernel: add object finder for jiffies
We have a lot of examples that use jiffies, but they stopped working
long ago on x86-64 (since Linux kernel commit d8ad6d39c35d ("x86_64: Fix
jiffies ODR violation") (in v5.8 and backported to stable releases)) and
never worked on other architectures. This is because jiffies is defined
in the Linux kernel's linker script. #277 proposed updating the examples
to use jiffies_64, but I would guess that most kernel developers are
familiar with jiffies and many have never seen jiffies_64. jiffies is
also a nicer name to type in live demos. Let's add a case to the Linux
kernel object finder to get the jiffies variable.

Reported-by: Martin Liska <mliska@suse.cz>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-22 11:15:37 -08:00
Omar Sandoval
69f5352cb6 tests: replace mmap with test kernel module for test_read_physical()
tests.linux_kernel.helpers.test_mm.TestMm.test_read_physical() fails on
Arm when the user page is mapped from high memory. Replace it with a
test case that uses the test physical address from the test kernel
module and asserts the expected PRNG data. (We will probably run into
more tests that fail with high memory once #244 is merged.)

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-22 10:07:55 -08:00
Omar Sandoval
dc38822e6c tests: use PRNG to generate data for tests
This replaces the multiples of a prime sequence used for the per-CPU
helper tests with a slightly more self-documenting PRNG interface in
both the test kernel module and Python test scaffolding. It will be used
more in upcoming tests.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-22 02:10:03 -08:00
Omar Sandoval
38b090ea9d setup.py: handle git not being installed
My previous change to call git unconditionally caused Packit failures,
presumably because git isn't installed for the build:

Traceback (most recent call last):
  File "/builddir/build/BUILD/drgn-0.0.22+71.g2f97cc0/setup.py", line 411, in <module>
    version=get_version(),
            ^^^^^^^^^^^^^
  File "/builddir/build/BUILD/drgn-0.0.22+71.g2f97cc0/setup.py", line 343, in get_version
    subprocess.call(
  File "/usr/lib64/python3.11/subprocess.py", line 389, in call
    with Popen(*popenargs, **kwargs) as p:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/subprocess.py", line 1024, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.11/subprocess.py", line 1901, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'git'

We shouldn't bother calling git if .git doesn't exist.

Fixes: 2e9e66cedc ("setup.py: use stricter check for git repository")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-21 18:12:54 -08:00
Omar Sandoval
2f97cc0f5f libdrgn: platform: expand on page table iterator documentation
There are a lot of details about how the page table iterator functions
should be used/implemented that commit 174b797ae3 ("libdrgn: platform:
add documentation (especially for drgn_architecture_info)") didn't
cover. Add an example and expand/clarify the documentation for the
callbacks.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-21 17:42:22 -08:00
Jay Kamat
08cb38cc2f Expand DW_AT_upper_bound quirk on zero size arrays
GCC appears to use data8 at -1 when reporting zero length arrays when
comping c++ code, this patch adds support and a test for that behavior.

dwarf_info.c: Remove check for sdata on quirk for array length == 0

Signed-off-by: Jay Kamat <jaygkamat@gmail.com>
2023-02-21 16:44:20 -08:00
Omar Sandoval
58664bedfb tests: allow test_identify_unrecognized() to pass on s390x
When testing #245, I encountered this error:

======================================================================
ERROR: test_identify_unrecognized (tests.linux_kernel.helpers.test_common.TestIdentifyAddress)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/host/home/osandov/repos/drgn/tests/linux_kernel/helpers/test_common.py", line 57, in test_identify_unrecognized
    self.assertIsNone(identify_address(self.prog, start_addr - 1))
  File "/host/home/osandov/repos/drgn/drgn/helpers/common/memory.py", line 99, in identify_address
    symbol = prog.symbol(addr)
OverflowError: can't convert negative int to unsigned

This is because the lowest kernel address in s390x is 0, so we're
passing -1 to identify_address(). Work around that in the test.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-21 14:44:53 -08:00
Omar Sandoval
5276af94fd contrib: add kcore_list.py
I used this to look at RAM regions on different architectures.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-21 14:26:54 -08:00
Omar Sandoval
2e9e66cedc setup.py: use stricter check for git repository
To test builds on other architectures, I'm setting up a Debian chroot
with debootstrap and then building with something like:

  mount --bind . "$chroot/mnt"
  chroot "$chroot" sh -c 'cd /mnt && python3 setup.py build_ext -i'

However, if I do this from a Git worktree, this fails with an error from
the git call to determine the local version:

  fatal: not a git repository: /home/osandov/repos/drgn/.git/worktrees/drgn2

Instead of only checking whether .git exists, use `git --git-dir=.git
rev-parse` to verify that .git is valid. (This will of course rely on
drgn/internal/version.py already having been generated.)

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-21 12:07:37 -08:00
Martin Liska
7e7fdf6978 tests: limit SMP count to 8 at maximum
Sometimes one can see an OOM with a higher number of SMP count
while still having 2G of memory.

Signed-off-by: Martin Liska <mliska@suse.cz>
2023-02-20 01:30:52 -08:00
Omar Sandoval
860a9555e9 CI: actually install pahole
The package is called dwarves.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-18 01:48:52 -08:00
Omar Sandoval
768a070461 CI: install pahole
This is required for building the test kernel module.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-18 01:47:52 -08:00
Omar Sandoval
cfda1942f8 tests: only run stack tracing tests on implemented architectures
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-18 01:42:38 -08:00
Omar Sandoval
c911ccdd36 tests: build test kernel module with kernel.org compiler
Now we'll always get the same compiler that was used for the actual
kernel build. Also split this into its own runnable module, vmtest.kmod,
for testing and custom scripting.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-18 01:42:38 -08:00
Omar Sandoval
4697b4ce0e CI: build vmtest kernels for all architectures
In order to hopefully not have jobs time out, split it into jobs by
architecture and flavor, but limit the parallelism to leave room for
actual test runs. Some of the very old kernel versions don't work on the
other architectures, but we can look into that later.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 17:58:14 -08:00
Omar Sandoval
4f437b395d vmtest.manage: support building other architectures
Note that this will now always use the kernel.org compilers even for the
host architecture.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
cf5cbbe868 vmtest.manage: add --flavor
This limits what flavors we build.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
1b3e58a11d vmtest.manage: apply kernel patches before building
We also need to force the checkout and clean it in order to revert the
patches between versions.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
45e7fc2d3b vmtest.kbuild: add patch to fix build for s390x on Linux 4.9
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
35dcd78943 vmtest.kbuild: also check image_name under arch/$SRCARCH/boot
This is a workaround for AArch64 and Arm on old kernel versions.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
8e86f5a41e vmtest.kbuild: add arch/powerpc/lib/crtsaveres.o to ppc64 packages
Old kernel versions (and new kernel versions built with old binutils)
need this for ppc64 module builds.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
4f5072cb48 vmtest.kbuild: add arch/arm/include/asm/opcodes.h to AArch64 packages
This is required for AArch64 module builds before Linux 4.10.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
441055cf41 vmtest.kbuild: add arch/$SRCARCH/kernel/module.lds to packages
Before Linux kernel commit 596b0474d3d9 ("kbuild: preprocess module
linker script") (in v5.10), AArch64 module builds require it.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
b4801e8f7a vmtest.kbuild: add scripts/ld-version.sh to packages
Before Linux kernel commit 05f6bbf2d714 ("kbuild: remove ld-version
macro") (in v5.12), ppc64 module builds require it.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
2ec06cfd24 vmtest.kbuild: add patch to allow enabling /proc/kcore on ARM
I don't know yet whether this is safe to upstream, but for now it should
be good enough for testing.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
bf45199682 vmtest.kbuild: add patch for BTFIDS errors with newer pahole
This was patched in recent stable and longterm branches, but we need it
for older branches. We need a few different backports of it.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:23:04 -08:00
Omar Sandoval
d2b817964f vmtest.kbuild: add infrastructure for applying kernel patches
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:17:20 -08:00
Omar Sandoval
1798642f0d vmtest.kbuild: set localversion on command line instead of kconfig
This will allow us to apply patches without affecting the localversion.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:17:20 -08:00
Omar Sandoval
f8c63b63a1 vmtest.kbuild: enable BTF for kernel modules
Move vmlinux to the module build directory and copy in resolve_btfids.
This is mainly to silence the annoying warning:

  Skipping BTF generation for /home/osandov/repos/drgn/build/vmtest/x86_64/tmpge3udm_a/drgn_test.ko due to unavailability of vmlinux

But it might also be useful for testing in the future.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:17:20 -08:00
Omar Sandoval
07b01acb7f vmtest.config: add Arm definition
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-16 02:17:17 -08:00
Omar Sandoval
1a898961d7 vmtest.config: add s390x definition
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-15 15:41:40 -08:00
Omar Sandoval
3b7b6273ab vmtest.config: add ppc64 definition
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-15 15:41:37 -08:00
Omar Sandoval
fd41642e58 vmtest.config: add AArch64 definition
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-14 16:58:21 -08:00
Omar Sandoval
1626cc32f2 vmtest.vm: support other architectures
Add the per-architecture QEMU options to the Architecture class and
allow specifying an architecture in -k.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-14 16:58:21 -08:00
Omar Sandoval
5f25c213c1 vmtest.kbuild: add --download-compiler
This will automatically download and use a standard cross compiler
instead of requiring the user to do so.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-14 16:58:21 -08:00
Omar Sandoval
7b8872424e vmtest.download: support downloading cross compilers from kernel.org
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-14 16:58:21 -08:00
Omar Sandoval
404fb80ca8 vmtest.download: specify kernel architecture per download
This also prepares us for being able to download other things.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-14 16:58:21 -08:00
Omar Sandoval
b0ff020a50 vmtest.kbuild: add --architecture option
It defaults to the host architecture (if available). x86_64 is currently
the only option. CROSS_COMPILE must be set in the environment manually.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-10 17:42:37 -08:00
Omar Sandoval
88fabc79c5 vmtest.config: add architecture definition
To prepare for building architectures other than x86-64, add a
description of architectures, and split the architecture-specific kernel
options into it. For now, we still hard-code x86-64 everywhere.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-10 17:15:12 -08:00
Omar Sandoval
df1a17dc11 vmtest: move kconfig and flavor definitions to a new module
vmtest.config will contain configuration/settings for vmtest.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-10 16:56:19 -08:00
Omar Sandoval
94443457aa libdrgn: handle GNU Debug Fission attributes, forms, and opcodes
These are all equivalent to their DWARF 5 counterparts, which we already
support:

* DW_FORM_GNU_addr_index <-> DW_FORM_addrx
* DW_FORM_GNU_str_index <-> DW_FORM_strx
* DW_AT_GNU_addr_base <-> DW_AT_addr_base
* DW_OP_GNU_addr_index <-> DW_OP_addrx
* DW_OP_GNU_const_index <-> DW_OP_constx

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-08 13:25:45 -08:00