manylinux2010 has been EOL since August 2022; see pypa/manylinux#1281.
In particular, the manylinux2010 image wasn't updated with Python 3.11.
Update scripts/build_dist.sh to use manylinux2014 and drop some CentOS 6
workarounds from scripts/build_manylinux_in_docker.sh.
If someone really wants manylinux2010, we can probably build both, but
none of the popular packages I checked (NumPy, SciPy, cryptography,
charset-normalizer) do this.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I wanted to make REUSE pass, but I'm not sure what to do about trivial
files. REUSE suggests using CC0, but Fedora no longer allows CC0. I'll
punt that until later. For now, let's add notices to some code files.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn is currently licensed as GPLv3+. Part of the long term vision for
drgn is that other projects can use it as a library providing
programmatic interfaces for debugger functionality. A more permissive
license is better suited to this goal. We decided on LGPLv2.1+ as a good
balance between software freedom and permissiveness.
All contributors not employed by Meta were contacted via email and
consented to the license change. The only exception was the author of
commit c4fbf7e589 ("libdrgn: fix for compilation error"), who did not
respond. That commit reverted a single line of code to one originally
written by me in commit 640b1c011d ("libdrgn: embed DWARF index in
DWARF info cache").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There are a bunch of page flag getters in the kernel like
PageUptodate(), PageLocked(), etc., that kernel developers are
accustomed to using. Most of them are simple bit tests. Let's add
helpers for all of those. These are generated from
include/linux/page-flags.h in the Linux kernel source tree as of Linux
v6.0-rc1.
More complicated getters that need to do more than a simple flag check
(e.g., PageCompound()) will need to be added manually.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This will catch issues like the one fixed in commit 549cd7facc ("Add
tests/sample.core.zst to sdist").
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This adds an --enable-compiler-warnings flag that:
* Defines a canonical list of warnings that we enforce. For now, this is
-Wall -Wformat-overflow=2 -Wformat-truncation=2, but we can add to it
going forward.
* Enables warnings by default.
* Allows erroring on warnings. We recommend that developers use this and
use it for the CI.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Previously iwyu.py overwrote the value of the `CFLAGS` environment
variable regardless of whether it was already set. This was very
frustrating when trying to set custom `CFLAGS` to be used in the
generation of compile_commands.json, so the script now only sets
`CFLAGS` if it is not already present in the environment.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
This minor change is a quality of life improvement ensuring developers
receive more warnings and diagnostics in their editors.
Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Use the latest version of elfutils (0.186) and libkdumpfile (0.4.1). We
can drop the elfutils patch since 0.186 has the fix (and we have our own
workaround), but we need a new patch to build libkdumpfile.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add some scaffolding to generate ELF files with symbol tables and use it
to test symbol lookups and Elf_Sym -> drgn.Symbol translation.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Also:
* Rename struct string to struct nstring and move it to its own header.
* Fix scripts/iwyu.py, which was broken by commit 5541fad063 ("Fix
some flake8 errors").
* Add workarounds for a few outstanding include-what-you-use issues.
There is still a false positive for
include-what-you-use/include-what-you-use#970, but hopefully that is
fixed soon.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The overloaded version is slower to compile, and we don't actually need
it. We can add a variadic version if we need to in the future. Also add
the script used to generate the macros.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Use the latest version of elfutils (0.185) and apply the fix "libdwfl:
fix potential NULL pointer dereference when reading link map" since that
hasn't been released yet and is needed to avoid crashing when debugging
userspace core dumps.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Mainly unused imports, unused variables, unnecessary f-strings, and
regex literals missing the r prefix. I'm not adding it to the CI linter
because it's too noisy, though.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently don't include the compilation directory when hashing file
names for deduplication. This can cause us to incorrectly deduplicate a
definition if, for example, two libraries have a definition with the
same name in files with the same name. Fix this by hashing the full file
path including the compilation directory.
This also requires reworking our strategy for path normalization to
better handle ".." components, since directories may end up outside of
the compilation directory. The new strategy keeps a linked list of
hashes (now FNV-1a instead of SipHash) for each parent directory. This
is actually more efficient than the previous approach, offsetting the
cost of the extra hash computations for the compilation directory. It
also correctly handles file names in the line number program header
which consist of multiple components.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
One nice side effect of commit e0921c5bdb ("libdrgn: don't use OpenMP
tasking") is that drgn now works with older versions of libgomp that
don't implement taskloop, including version 4.4 in manylinux2010. So, we
can finally build manylinux2010 wheels. These scripts are based on
scripts from Stephen Brennan, with some cleanups and updates for changes
in drgn's build requirements.
Closes#69.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The oldest LTS version of Ubuntu, 16.04, has elfutils 0.165. This
version is missing some ELF and DWARF definitions used by drgn. Add
copies of elf.h from glibc 2.33 and dwarf.h and elfutils/known-dwarf.h
from elfutils 0.183 to get the latest definitions and drop the minimum
required version of elfutils further to 0.165.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We currently bundle a version of elfutils with patches to export
additional stack tracing functionality. This has a few drawbacks:
- Most of drgn's build time is actually building elfutils.
- Distributions don't like packages that bundle verions of other
packages.
- elfutils, and thus drgn, can't be built with clang.
Now that we've replaced the elfutils DWARF unwinder with our own, we
don't need the patches, so we can drop the bundled elfutils and fix
these issues.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Currently, reference objects and buffer value objects have a byte order.
However, this doesn't always make sense for a couple of reasons:
- Byte order is only meaningful for scalars. What does it mean for a
struct to be big endian? A struct doesn't have a most or least
significant byte; its scalar members do.
- The DWARF specification allows either types or variables to have a
byte order (DW_AT_endianity). The only producer I could find that uses
this is GCC for the scalar_storage_order type attribute, and it only
uses it for base types, not variables. GDB only seems to use to check
it for base types, as well.
So, remove the byte order from objects, and move it to integer, boolean,
floating-point, and pointer types. This model makes more sense, and it
means that we can get the binary representation of any object now.
The only downside is that we can no longer support a bit offset for
non-scalars, but as far as I can tell, nothing needs that.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I recently hit a couple of CI failures caused by relying on transitive
includes that weren't always present. include-what-you-use is a
Clang-based tool that helps with this. It's a bit finicky and noisy, so
this adds scripts/iwyu.py to make running it more convenient (but not
reliable enough to automate it in Travis).
This cleans up all reasonable include-what-you-use warnings and
reorganizes a few header files.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgn was originally my side project, but for awhile now it's also been
my work project. Update the copyright headers to reflect this, and add a
copyright header to various files that were missing it.
The current implementation of vmtest has a few issues:
1. Building drgn for each kernel version on Travis is slow, mostly
because they don't all run in parallel.
2. For local, incremental testing, recreating the filesystem image and
rebuilding drgn is slow, and syncing the code to the filesystem image
is brittle.
3. The filesystem image is the only communication channel, and reading
the exit status from the filesystem image is awkward and fragile.
4. Creating and accessing the filesystem image requires root.
This reworks vmtest to use the build on the host via VirtFS with a
simple agent on the guest that can execute arbitrary commands and return
the exit status. This has a few more moving parts but is faster and
saner overall.
While we're here, make generate_dwarf_constants.py use the bundled
dwarf.h, generate code that black is happy with, and use the keyword
list from the standard library.
v5.6-rc1 was released today, and 5.3 was recently EOL'd. (It might be
worthwhile to continue testing EOL kernels, but for now I don't want
Travis runs to take forever).
The next change will make non-git builds depend on a fully-formed source
distribution. So, let's copy in everything from SOURCES.txt, which
should include the egg metadata and anything else setuptools decides is
important.
In order to make builds slightly more reproducible (and avoid leaking
details about the build machine's directory layout), let's remap the
compilation directory in DW_AT_comp_dir to be empty.
Not all architectures name the bootable image bzImage, so to make
supporting other architectures easier in the future, let's use the more
generic name, vmlinuz.
If I forget to git fetch, manage.py tries to build releases that don't
exist, which litters the repository with incomplete build-$release
directories. Verify that the commit exists before we do anything.
The vmtest shared folder just got large enough to be paginated, which
manage.py doesn't handle. Handle it by making the same XHR requests that
the webpage makes in a browser.
