About half (!) of vmlinux comes from relocation sections (~130M out of
~250M). But, vmlinux is an ET_EXEC file, so relocations don't even apply
to it. We can massively shrink vmlinux by removing the unnecessary
sections.
Currently, the kernel config is embedded inside of manage.py. This is
inconvenient when building kernels manually for testing. Move it to a
separate file so that, e.g., it can be copied into the build tree
easily.
We need to keep the Program alive for its types to stay valid, not just
the objects the Program has pinned. (I have no idea why I changed this
in commit 565e0343ef ("libdrgn: make symbol index pluggable with
callbacks").)
`examples/linux/bpf.py` was superseded by `tools/bpf_inspect.py` so no
reason to keep it around anymore. Remove it.
Signed-off-by: Andrey Ignatov <rdna@fb.com>
Introduce bpf_inspect.py drgn script to list BPF programs and maps and
their properties unavailable to user space via kernel API.
The script was initially sent to kernel tree [1] but it was agreed that
drgn repo is a better place for it and it's a good idea to create
`tools/` directory in drgn to keep tools likes this. See [2] for
details.
The main use-case bpf_inspect.py covers is to show BPF programs attached
to other BPF programs via freplace/fentry/fexit mechanisms introduced
recently. There is no user-space API to get this info and, for example,
bpftool can show all BPF programs but can't show if program A replaces a
function in program B.
Example:
% sudo tools/bpf_inspect.py p | grep test_pkt_access
650: BPF_PROG_TYPE_SCHED_CLS test_pkt_access
654: BPF_PROG_TYPE_TRACING test_main linked:[650->25: BPF_TRAMP_FEXIT test_pkt_access->test_pkt_access()]
655: BPF_PROG_TYPE_TRACING test_subprog1 linked:[650->29: BPF_TRAMP_FEXIT test_pkt_access->test_pkt_access_subprog1()]
656: BPF_PROG_TYPE_TRACING test_subprog2 linked:[650->31: BPF_TRAMP_FEXIT test_pkt_access->test_pkt_access_subprog2()]
657: BPF_PROG_TYPE_TRACING test_subprog3 linked:[650->21: BPF_TRAMP_FEXIT test_pkt_access->test_pkt_access_subprog3()]
658: BPF_PROG_TYPE_EXT new_get_skb_len linked:[650->16: BPF_TRAMP_REPLACE test_pkt_access->get_skb_len()]
659: BPF_PROG_TYPE_EXT new_get_skb_ifindex linked:[650->23: BPF_TRAMP_REPLACE test_pkt_access->get_skb_ifindex()]
660: BPF_PROG_TYPE_EXT new_get_constant linked:[650->19: BPF_TRAMP_REPLACE test_pkt_access->get_constant()]
It can be seen that there is a program test_pkt_access, id 650 and there
are multiple other tracing and ext programs attached to functions in
test_pkt_access.
For example the line:
658: BPF_PROG_TYPE_EXT new_get_skb_len linked:[650->16: BPF_TRAMP_REPLACE test_pkt_access->get_skb_len()]
means that BPF program new_get_skb_len, id 658, type BPF_PROG_TYPE_EXT
replaces (BPF_TRAMP_REPLACE) function get_skb_len() that has BTF id 16
in BPF program test_pkt_access, prog id 650.
Just very simple output is supported now but it can be extended in the
future if needed.
The script is extendable and currently implements two subcommands:
* prog (alias: p) to list all BPF programs;
* map (alias: m) to list all BPF maps;
Developer can simply tweak the script to print interesting pieces of
programs or maps.
More examples of output:
% sudo tools/bpf_inspect.py p | shuf -n 3
81: BPF_PROG_TYPE_CGROUP_SOCK_ADDR tw_ipt_bind
94: BPF_PROG_TYPE_CGROUP_SOCK_ADDR tw_ipt_bind
43: BPF_PROG_TYPE_KPROBE kprobe__tcp_reno_cong_avoid
% sudo tools/bpf_inspect.py m | shuf -n 3
213: BPF_MAP_TYPE_HASH errors
30: BPF_MAP_TYPE_ARRAY sslwall_setting
41: BPF_MAP_TYPE_LRU_HASH flow_to_snd
Help:
% sudo tools/bpf_inspect.py
usage: bpf_inspect.py [-h] {prog,p,map,m} ...
drgn script to list BPF programs or maps and their properties
unavailable via kernel API.
See https://github.com/osandov/drgn/ for more details on drgn.
optional arguments:
-h, --help show this help message and exit
subcommands:
{prog,p,map,m}
prog (p) list BPF programs
map (m) list BPF maps
[1] https://lore.kernel.org/bpf/20200228201514.GB51456@rdna-mbp/T/
[2] https://lore.kernel.org/bpf/20200228201514.GB51456@rdna-mbp/T/#mefed65e8a98116bd5d07d09a570a3eac46724951
Signed-off-by: Andrey Ignatov <rdna@fb.com>
For operations where we don't have a type available, we currently fall
back to C. Instead, we should guess the language of the program and use
that as the default. The heurisitic implemented here gets the language
of the CU containing "main" (except for the Linux kernel, which is
always C). In the future, we should allow manually overriding the
automatically determined language.
For types obtained from DWARF, we determine it from the language of the
CU. For other types, it can be specified manually or fall back to the
default (C). Then, we can use the language for operations where the type
is available.
This way, languages can be identified by an index, which will be useful
for adding Python bindings for drgn_language and for adding a language
field to drgn_type.
While we're here, make generate_dwarf_constants.py use the bundled
dwarf.h, generate code that black is happy with, and use the keyword
list from the standard library.
String annotations (i.e., forward references) need to be parsed into an
ast node. Do it as a transformation step immediately after parsing the
source. We can also squash the constant node transformation into this
one.
I've been wanting to add type hints for the _drgn C extension for
awhile. The main blocker was that there is a large overlap between the
documentation (in docs/api_reference.rst) and the stub file, and I
really didn't want to duplicate the information. Therefore, it was a
requirement that the the documentation could be generated from the stub
file, or vice versa. Unfortunately, none of the existing tools that I
could find supported this very well. So, I bit the bullet and wrote my
own Sphinx extension that uses the stub file as the source of truth (and
subsumes my old autopackage extension and gen_docstrings script).
The stub file is probably incomplete/inaccurate in places, but this
should be a good starting point to improve on.
Closes#22.
I thought I'd be able to avoid adding a separate API for register values
and reuse dwfl_frame_eval_expr(), but this doesn't work if the frame is
missing debug information but has known register values (e.g., if the
program crashed with an invalid instruction pointer).
Rebase on master, add the improved
dwfl_frame_module/dwfl_frame_dwarf_frame patch, and add the
dwfl_frame_register patch.
Based on:
889edd912 PR25365: debuginfod-client: restrict cleanup to client-pattern files
With the following patches:
configure: Add --disable-programs
configure: Add --disable-shared
libdwfl: add interface for attaching to/detaching from threads
libdwfl: add interface for getting Dwfl_Module and Dwarf_Frame for Dwfl_Frame
libdwfl: export __libdwfl_frame_reg_get as dwfl_frame_register
libdwfl: add interface for evaluating DWARF expressions in a frame
UTS_RELEASE is currently only accessible once debug info is loaded with
prog.load_debug_info(main=True). This makes it difficult to get the
release, find the appropriate vmlinux, then load the found vmlinux. We
can add vmcoreinfo_object_find as part of set_core_dump(), which makes
it possible to do the following:
prog = drgn.Program()
prog.set_core_dump(core_dump_path)
release = prog['UTS_RELEASE'].string_()
vmlinux_path = find_vmlinux(release)
prog.load_debug_info([vmlinux_path])
The only downside is that this ends up using the default definition of
char rather than what we would get from the debug info, but that
shouldn't be a big problem.
The osrelease is accessible via init_uts_ns.name.release, but we can
also get it straight out of vmcoreinfo, which will be useful for the
next change. UTS_RELEASE is the name of the macro defined in the kernel.
This continues the conversion from the last commit. Members and
parameters are basically the same, so we can do them together. Unlike
enumerators, these don't make sense to unpack or access as sequences.
Currently, type members, enumerators, and parameters are all represented
by tuples in the Python bindings. This is awkward to document and
implement. Instead, let's replace these tuples with proper types,
starting with the easiest one, TypeEnumerator. This one still makes
sense to treat as a sequence so that it can be unpacked as (name,
value).
It makes more sense semantically for __all__ to be immutable (except for
drgn.helpers.linux.__all__, which is built dynamically). While we're
here, sort the names, too.
Currently, we print:
>>> prog.symbol(prog['init_task'])
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: cannot convert 'struct task_struct' to index
It's not obvious what it means to convert to an index. Instead, let's
use the error message raised by operator.index():
TypeError: 'struct task_struct' object cannot be interpreted as an integer
v5.6-rc1 was released today, and 5.3 was recently EOL'd. (It might be
worthwhile to continue testing EOL kernels, but for now I don't want
Travis runs to take forever).
Decouple some of the responsibilities of FaultError to
OutOfBoundsError so consumers can differentiate between
invalid memory accesses and running out of bounds in
drgn Objects which may be based on valid memory address.
