Commit Graph

145 Commits

Author SHA1 Message Date
Omar Sandoval
81c8672d4d libdrgn: python: log to the standard logging module
Rather than coming up with our own, separate logging API for the Python
bindings, let's integrate with the logging module. The straightforward
part is creating a logger from the C extension and adding a log callback
that calls its log() method. However, syncing the log level between the
logging module and libdrgn requires monkey patching.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-07-18 12:47:34 -07:00
Omar Sandoval
5c1b6cf764 docs: document thread safety
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-07-18 12:33:35 -07:00
Omar Sandoval
75d2fee3e9 docs: add 0.0.23 release highlights
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-06-28 13:58:19 -07:00
Omar Sandoval
18a8f69ad8 libdrgn: linux_kernel: add object finder for jiffies
We have a lot of examples that use jiffies, but they stopped working
long ago on x86-64 (since Linux kernel commit d8ad6d39c35d ("x86_64: Fix
jiffies ODR violation") (in v5.8 and backported to stable releases)) and
never worked on other architectures. This is because jiffies is defined
in the Linux kernel's linker script. #277 proposed updating the examples
to use jiffies_64, but I would guess that most kernel developers are
familiar with jiffies and many have never seen jiffies_64. jiffies is
also a nicer name to type in live demos. Let's add a case to the Linux
kernel object finder to get the jiffies variable.

Reported-by: Martin Liska <mliska@suse.cz>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-02-22 11:15:37 -08:00
Stephen Brennan
c24ba99c8c cli: Separate the main function from the REPL
It's quite useful to run drgn's interactive REPL from a user-provided
script. One reason is saving the drgn core dump and debuginfo setup into
a script to more easily start analysis. Another might be to replace
drgn's debuginfo loading scheme with one customized to a different
environment. Or, a script might want to handle errors by allowing an
interactive user to examine the program themselves.

To serve all of these cases and more, separate out the logic of running
the drgn REPL from the internal logic relating to argument parsing,
choosing the correct target, and loading debuginfo. While this may seem
like a simple case of code.interact, there are several important
internal features drgn provides in its CLI: a custom displayhook, a
custom readline completer, a history file, and of course the logic to
import and include the proper drgn APIs and helpers for a core dump.

This means creating a run_interactive() function and moving the cli
module out of the drgn.internal package.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2023-02-01 09:30:02 -08:00
Timothée Cocault
9730deeb85 Fix example for add_memory_segment
The parameters of the read_file function in the example are swapped.
This matches the parameters order used by py_memory_read_fn.

Fixes: c0bc72b0ea ("libdrgn: use splay tree for memory reader")
Signed-off-by: Timothée Cocault <timothee.cocault@gmail.com>
2023-01-23 13:04:46 -08:00
Stephen Brennan
15f151c1e4 Document the linux kernel object finder
drgn is able to lookup some special metadata for the Linux kernel,
sometimes even without debuginfo. For users who may expect that
Program.object() will only return objects corresponding to variables,
these metadata are unexpected and can be quite useful. Regardless,
they're currently undocumented. Add documentation under Advanced Usage
for these, and reference it in the Program.object() docstring.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2023-01-17 16:10:49 -08:00
Omar Sandoval
9cce4c33b2 drgndoc: support formating unary operators
This is necessary for type annotations like `def foo(fd: int = -1)` in
future changes.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-13 10:38:44 -08:00
Omar Sandoval
1c6ea8e00d docs: add release highlights for 0.0.22
I've been posting these internally at Meta for previous releases, but I
think others would find them useful, too. In the future, I'll write them
up before cutting the release.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2023-01-06 15:21:43 -08:00
Omar Sandoval
a0a54edc1f Create contrib directory
It's come up several times that it'd be nice to have somewhere to dump
drgn scripts that people write while debugging without requiring them to
be cleaned up and scrutinized in code review. Serapheim Dimitropoulos
noted that several projects have a "contrib" directory for this purpose.
See [1]. Let's create one, document it, exclude it from pre-commit, and
move our (mostly unmaintained) examples there.

1: https://drewdevault.com/2020/06/06/Add-a-contrib-directory.html

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-11-16 13:17:50 -08:00
Omar Sandoval
87b7292aa5 Relicense drgn from GPLv3+ to LGPLv2.1+
drgn is currently licensed as GPLv3+. Part of the long term vision for
drgn is that other projects can use it as a library providing
programmatic interfaces for debugger functionality. A more permissive
license is better suited to this goal. We decided on LGPLv2.1+ as a good
balance between software freedom and permissiveness.

All contributors not employed by Meta were contacted via email and
consented to the license change. The only exception was the author of
commit c4fbf7e589 ("libdrgn: fix for compilation error"), who did not
respond. That commit reverted a single line of code to one originally
written by me in commit 640b1c011d ("libdrgn: embed DWARF index in
DWARF info cache").

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-11-01 17:05:16 -07:00
Omar Sandoval
41c02280e0 Use SPDX for docs/exts/setuptools_config.py
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-11-01 16:57:00 -07:00
Omar Sandoval
a52016c4cb libdrgn: linux_kernel: always use module list from core
For the next fix, we need the address of the .data..percpu section,
which is only available directly from the struct module and not from
anywhere in /proc or /sys. Get rid of the /proc/modules fast path (and
update the name of the testing environment variable from
DRGN_USE_PROC_AND_SYS_MODULES to DRGN_USE_SYS_MODULE).

This has some small overhead (~20ms longer startup time in my
benchmarks) and means that we no longer determine the loaded modules if
vmlinux is missing, but fixing the per-CPU issue is more important.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-09-12 16:11:59 -07:00
Omar Sandoval
67d0e8dab5 docs: remove stray backtick
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-09-12 16:08:12 -07:00
Omar Sandoval
ca373fe38a docs: use "programmable debugger" description consistently
Replace the old "Scriptable debugger library" and
"Debugger-as-a-library" taglines with the one we're using on GitHub,
"Programmable debugger". Make up for it by emphasizing that drgn can
also be used as a library a tiny bit more in the README.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-08-19 01:21:32 -07:00
Omar Sandoval
ca94b87268 drgndoc: format types as type annotations
Instead of adding type information to directive descriptions with the
:type:, :rtype:, and :vartype: fields, document types with type
annotations. For functions and methods, we add the type annotations to
the signature. For variables and attributes, we use the :type: option.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-08-19 01:07:37 -07:00
Omar Sandoval
2c38ea5219 docs: update required Sphinx version to 5.1.1
Just picking up the newest version. Also fix the following warning:

  WARNING: extlinks: Sphinx-6.0 will require a caption string to contain exactly one '%s' and all other '%' need to be escaped as '%%'.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-07-27 10:04:20 -07:00
Omar Sandoval
0d95ac0d6e docs: fix stray reference to symbol finder
"Object finder" was renamed from "symbol finder" awhile ago, but we
forgot to update the advanced usage documentation.

Fixes: 0c5df56fba ("libdrgn: replace symbol index with object index")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-07-27 09:40:43 -07:00
Omar Sandoval
9ae36fd12e helpers: add validators, starting with linked lists
At LSF/MM+BPF 2022, Ted Ts'o pitched me the idea of using drgn to
validate the consistency of kernel data structures. I really liked this
idea, especially for big, complicated data structures. But first, let's
start small: document the concept of a "validator", which is just a
special kind of helper, and add some basic validator versions of linked
list helpers.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-05-17 14:20:16 -07:00
Omar Sandoval
af6f5a887d libdrgn: replace gen_arch.awk with gen_arch_inc_strswitch.py
Now that we have gen_strswitch.py, there's no reason to keep this AWK
script around. Replace it with a Python script that outputs a strswitch
file. This also gets rid of our gawk dependency.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-03-02 16:10:43 -08:00
Mykola Lysenko
7580fffbdf Add drgn.Program.main_thread()
Currently only supported for user-space crash dumps. E.g. no support for
live user-space application debugging or kernel debugging.

Closes #144.

Signed-off-by: Mykola Lysenko <mykolal@fb.com>
2022-02-10 15:53:50 -08:00
Omar Sandoval
92cfb06d9b docs: update required Sphinx version
We don't need anything new, just picking up the newest version.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-01-22 11:22:36 -08:00
Omar Sandoval
fb4d2088c2 docs: use extlink in Kyber case study
Sphinx 4.4.0 warns that kyber_stack_trace.rst could use an extlink for
the link to the Kyber source code instead of hard-coding the link to the
Linux repo.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2022-01-22 11:20:23 -08:00
Kevin Svetlitski
301cc767ba Implement a new API for representing threads
Previously, drgn had no way to represent a thread – retrieving a stack
trace (the only extant thread-specific operation) was achieved by
requiring the user to directly provide a tid.

This commit introduces the scaffolding for the design outlined in
issue #92, and implements the corresponding methods for userspace core
dumps, the live Linux kernel, and Linux kernel core dumps. Future work
will build on top of this commit to support live userspace processes.

Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
2022-01-11 17:28:17 -08:00
Omar Sandoval
e6abfeac03 libdrgn: debug_info: report userspace core dump debug info ourselves
There are a few reasons for this:

1. dwfl_core_file_report() crashes on elfutils 0.183-0.185. Those
   versions are still used by several distros.
2. In order to support --main-symbols and --symbols properly, we need to
   report things ourselves.
3. I'm considering moving away from libdwfl in the long term.

We provide an escape hatch for now: setting the environment variable
DRGN_USE_LIBDWFL_REPORT=1 opts out of drgn's reporting and uses
libdwfl's.

Fixes #130.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-12-08 12:11:10 -08:00
Omar Sandoval
c0d8709b45 Update copyright headers to Meta
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-11-21 15:59:44 -08:00
Omar Sandoval
1b7badad0a docs: expand and reorganize installation instructions
* Mention installing drgn using a package manager on Fedora/EPEL.
  Closes #103.
* Mention that pip installs a binary wheel by default.
* Include instructions for installing from source in README.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-11-03 16:18:22 -07:00
Omar Sandoval
e5021952c8 docs: disable sphinx.ext.viewcode
viewcode works by importing modules. This doesn't actually work on Read
the Docs because we don't build and install the C extension. It looks
like there are workarounds (viewcode-find-source), but let's disable it
for now.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-11-03 15:58:11 -07:00
Omar Sandoval
8358c31d26 docs: document how to get debugging symbols
I couldn't find any good summaries of how to get debugging symbols on
various distros, so I guess we'll have to maintain our own.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-11-02 17:48:36 -07:00
Stephen Brennan
1744d8d93c libdrgn: python: Add binding, kind to drgn.Symbol
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2021-08-20 18:16:57 -07:00
Omar Sandoval
c9cb28b649 docs: set required Sphinx version for Read the Docs
Read the Docs defaults to Sphinx 1.8.5. This version was released in
2019 and doesn't know about the :classmethod: option, so the
documentation for Object.from_bytes_() is missing from
drgn.readthedocs.io. Set the required version to the current latest
version as recommended by Read the Docs:
https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html#pinning-dependencies.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-08-17 15:01:14 -07:00
Omar Sandoval
5541fad063 Fix some flake8 errors
Mainly unused imports, unused variables, unnecessary f-strings, and
regex literals missing the r prefix. I'm not adding it to the CI linter
because it's too noisy, though.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-08-11 14:52:44 -07:00
Omar Sandoval
d6a47f8698 docs: improve stack trace documentation in user guide
The API reference has all of the details, but add a short example to the
user guide (and move it before symbols, as stack traces are probably
more interesting/important).

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-08-02 16:20:33 -07:00
Omar Sandoval
39b76e8486 docs: update repr(drgn.Type) and type constructors in documentation
Commit a97f6c4fa2 ("Associate types with program") changed repr() for
drgn.Type to include a "prog." prefix, but it didn't update the
documentation to reflect that. It also forgot to update a global type
constructor to the new Program methods.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-08-02 15:59:43 -07:00
Omar Sandoval
46e3617a1e docs: add a writeup of an investigation using stack trace variables
Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-06-09 01:52:10 -07:00
Omar Sandoval
bc85767e5f libdrgn: support looking up parameters and variables in stack traces
After all of the preparatory work, the last two missing pieces are a way
to find a variable by name in the list of scopes that we saved while
unwinding, and a way to find the containing scopes of an inlined
function. With that, we can finally look up parameters and variables in
stack traces.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-06-05 16:18:51 -07:00
Omar Sandoval
8f7e524b6b docs: update links to main branch
The master branch was renamed to main. GitHub redirects links to the old
branch, but we might as well update them explicitly.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-05-04 17:25:33 -07:00
Omar Sandoval
a4b9d68a8c Use GPL-3.0-or-later license identifier instead of GPL-3.0+
Apparently the latter is deprecated and the former is preferred.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-04-03 01:10:35 -07:00
Omar Sandoval
f285764f8a Include full libdrgn distribution in drgn sdist
Building drgn from an sdist currently requires autotools and gawk
because libdrgn in the sdist is more or less a git checkout. It's more
user-friendly to include the autotools output and generated code. Do
this by extending the sdist command to include a full libdrgn
distribution with `make distdir`.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-03-30 23:19:38 -07:00
Omar Sandoval
630d39e345 libdrgn: add ORC unwinder
The Linux kernel has its own stack unwinding format for x86-64 called
ORC: https://www.kernel.org/doc/html/latest/x86/orc-unwinder.html. It is
essentially a simplified, less complete version of DWARF CFI. ORC is
generated by analyzing machine code, so it is present for all but a few
ignored functions. In contrast, DWARF CFI is generated by the compiler
and is therefore missing for functions written in assembly and inline
assembly (which is widespread in the kernel).

This implements an ORC stack unwinder: it applies ELF relocations to the
ORC sections, adds a new DRGN_CFI_RULE_REGISTER_ADD_OFFSET CFI rule
kind, parses and efficiently stores ORC data, and translates ORC to drgn
CFI rules. This will allow us to stack trace through assembly code,
interrupts, and system calls.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-03-29 10:01:52 -07:00
Omar Sandoval
e5bc41f16c libdrgn: add latest elf.h and dwarf.h to support elfutils 0.165
The oldest LTS version of Ubuntu, 16.04, has elfutils 0.165. This
version is missing some ELF and DWARF definitions used by drgn. Add
copies of elf.h from glibc 2.33 and dwarf.h and elfutils/known-dwarf.h
from elfutils 0.183 to get the latest definitions and drop the minimum
required version of elfutils further to 0.165.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-03-21 23:18:39 -07:00
Serapheim Dimitropoulos
a68abd5de4 libdrgn: stretch minimum supported version of libelf to 0.170
Currently libdrgn requires libelf to be of version 0.175 or
later. This patch allows the library to be compiled with libelf
0.170 (the newest version supported by Ubuntu 18.04 LTS).

Signed-off-by: Serapheim Dimitropoulos <serapheim@delphix.com>
2021-03-21 14:28:29 -07:00
Omar Sandoval
4c5c5f3842 Remove bundled version of elfutils
We currently bundle a version of elfutils with patches to export
additional stack tracing functionality. This has a few drawbacks:

- Most of drgn's build time is actually building elfutils.
- Distributions don't like packages that bundle verions of other
  packages.
- elfutils, and thus drgn, can't be built with clang.

Now that we've replaced the elfutils DWARF unwinder with our own, we
don't need the patches, so we can drop the bundled elfutils and fix
these issues.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-03-16 00:52:09 -07:00
Omar Sandoval
78316a28fb libdrgn: remove half-baked support for complex types
We've nominally supported complex types since commit 75c3679147
("Rewrite drgn core in C"), but parsing them from DWARF has been
incorrect from the start (they don't have a DW_AT_type attribute like we
assume), and we never implemented proper support for complex objects.
Drop the partial implementation; we can bring it back (properly) if
someone requests it.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-02-17 14:56:33 -08:00
Omar Sandoval
547333d8ca docs: document GNU Awk version dependency
gen_arch.awk uses PROCINFO["sorted_in"] and arrays of arrays, both of
which were introduced in GNU Awk 4.0 according to
https://www.gnu.org/software/gawk/manual/html_node/Feature-History.html.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-02-09 16:50:58 -08:00
Omar Sandoval
352c31e1ac Add support for C++ template parameters
Add struct drgn_type_template_parameter to libdrgn, the corresponding
TypeTemplateParameter to the Python bindings, and support for parsing
them from DWARF.

With this, support for templates is almost, but not quite, complete. The
main wart is that DW_TAG_name of compound types includes the template
parameters, so the type tag includes it as well. We should remove that
from the tag and instead have the type formatting code add it only when
getting the full type name.

Based on a patch from Jay Kamat.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2021-01-08 17:39:51 -08:00
Omar Sandoval
30cfa40a72 libdrgn: rename "unavailable" objects to "absent" objects
I was going to add an Object.available_ attribute, but that made me
realize that the naming is somewhat ambiguous, as a reference object
with an invalid address might also be considered "unavailable" by users.
Use the name "absent" instead, which is more clear: the object isn't
there at all.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-12-29 14:58:26 -08:00
Omar Sandoval
40004e5c8f libdrgn/python: add offsetof()
offsetof() can almost be implemented with Type.member(name).offset, but
that doesn't parse member designators. Add an offsetof() function that
does (and add drgn_type_offsetof() in libdrgn).

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-12-15 16:46:41 -08:00
Omar Sandoval
c801e5e9b1 drgndoc: format __init__() signature separately from class
Having the signature in the class line is awkward, especially when
__init__() is overloaded. Instead, document __init__() separately, but
refer to it by the name of the class. There might still be a better way
to represent this, but this is at least better than before.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-12-14 12:29:17 -08:00
Omar Sandoval
d7c7094992 drgndoc: fix unnecessary parentheses around tuple subscripts on Python 3.9
Python 3.9 stopped emitting ast.Index nodes, which broke skipping
parentheses around tuples when they're used as subscripts (e.g., for
generic type annotations). Fix it by removing ast.Index nodes in the
pretransformation step on old versions and then handling the new layout
where the ast.Tuple node is directly in ast.Subscript.slice. While we're
here, make sure that we don't skip the parentheses for an empty tuple in
a subscript.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
2020-12-14 12:29:17 -08:00