Commit Graph

552 Commits

Author SHA1 Message Date
Omar Sandoval
87dd27ee12 setup.py: build in parallel by default
This is especially helpful for pip which doesn't have an easy way to
pass build parameters (and most people wouldn't bother, anyways).
2020-01-24 16:47:56 -08:00
Omar Sandoval
0cd7f2eef9 setup.py: rename custom command classes
The "my_" prefix shows up as the command name in --help, which is a bit
confusing. Do what setuptools does and import the original command under
another name so the custom class can have the original name.
2020-01-24 15:25:41 -08:00
Andrey Ignatov
421bda23f9 examples: Print bpf programs and maps
Add example to print basic info about bpf programs and maps (id, type,
name).

Example output:

  # python36 -m drgn examples/linux/bpf.py
  Usage: examples/linux/bpf.py { prog | map }
  # python36 -m drgn examples/linux/bpf.py p | head
       9: BPF_PROG_TYPE_CGROUP_SKB              name
      10: BPF_PROG_TYPE_CGROUP_SKB              name
   23368: BPF_PROG_TYPE_CGROUP_SOCK_ADDR        name tw_ipt_bind
   23369: BPF_PROG_TYPE_CGROUP_SOCK_ADDR        name tw_ipt_connect
   23370: BPF_PROG_TYPE_SOCK_OPS                name tw_ipt_listen
   23371: BPF_PROG_TYPE_CGROUP_SOCK_ADDR        name tw_ipt_sendmsg
   59538: BPF_PROG_TYPE_CGROUP_SYSCTL           name tw_sysctl
   97104: BPF_PROG_TYPE_SCHED_CLS               name fbflow_egress
   97105: BPF_PROG_TYPE_SCHED_CLS               name fbflow_ingress
   97106: BPF_PROG_TYPE_KPROBE                  name fbflow_tcp_conn
  # python36 -m drgn examples/linux/bpf.py m | head
       8: BPF_MAP_TYPE_LPM_TRIE                 name
       9: BPF_MAP_TYPE_LPM_TRIE                 name
      12: BPF_MAP_TYPE_PERCPU_ARRAY             name fbflow_pkt_info
   16305: BPF_MAP_TYPE_ARRAY                    name tw_ipt_cfg
   16306: BPF_MAP_TYPE_ARRAY                    name tw_tcp_ports_ma
   16307: BPF_MAP_TYPE_LRU_HASH                 name tw_ipt_lo_sk
   42606: BPF_MAP_TYPE_HASH                     name tw_sysctl_ro
   55338: BPF_MAP_TYPE_HASH                     name hash_min
   55341: BPF_MAP_TYPE_HASH                     name hash_8k_1
   55342: BPF_MAP_TYPE_HASH                     name hash_8k_8

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-24 10:53:47 -08:00
Andrey Ignatov
c8ef841768 helpers: Add bpf helpers
Add helpers to iterate over bpf programs and maps.

The map iteration code was used in debugging real-world problem that
turned out to be a kernel bug in memlock accounting in perf. Saving it
as helpers to use in future investigations.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-24 10:53:47 -08:00
Omar Sandoval
d4cc7945af Support building with alternative OpenMP runtime libraries
At Facebook, we link OpenMP code with libomp instead of libgomp. We have
an internal patch to drgn to do this, as it can't be done by setting
CFLAGS/LDFLAGS. Let's add a way to specify the OpenMP library at
configure time so that we can drop the internal patch.
2020-01-24 10:22:38 -08:00
Omar Sandoval
507d50c240 setup.py: support passing additional arguments to configure 2020-01-23 17:54:46 -08:00
Omar Sandoval
f4822be904 Document PyPI package and installation with pip 2020-01-23 15:15:33 -08:00
Omar Sandoval
18dfec083e docs: update local installation directions
Now that the CLI references the distribution, one needs to create the
distribution metadata to run locally.
2020-01-23 14:04:46 -08:00
Omar Sandoval
ea190db3f9 Don't define __version__ in drgn/__init__.py
This breaks autodoc on Read the Docs because it builds the documentation
without creating the distribution. I have something in the works to stop
using autodoc, but for now, we can just get the version in the CLI code.
2020-01-23 14:04:37 -08:00
Omar Sandoval
0088618578 Include git revision in version
When investigating a reported bug, it's important to know which exact
version of drgn is being used. Let's include the git revision in the
version printed by the CLI in PEP440 format. This commit will also serve
as the 0.0.1 release.
2020-01-23 12:29:43 -08:00
Omar Sandoval
ce06683861 vmtest/run: always get list of source files from SOURCES.txt
The next change will make non-git builds depend on a fully-formed source
distribution. So, let's copy in everything from SOURCES.txt, which
should include the egg metadata and anything else setuptools decides is
important.
2020-01-23 10:02:29 -08:00
Omar Sandoval
b2c1a613ec docs: add requirements.txt
Read the Docs is failing to build the documentation because it doesn't
know it needs to install sphinx-autodoc-typehints.
2020-01-14 16:27:10 -08:00
Omar Sandoval
6ee17e8f19 tests: add tests for cgroup helpers 2020-01-14 16:03:20 -08:00
Omar Sandoval
016f2f43c6 tests: add tests for kernfs helpers 2020-01-14 16:03:20 -08:00
Andrey Ignatov
10fa4686c1 examples: Print attached cgroup-bpf programs
Support additional argument `bpf` in `cgroup.py` that, if specified,
prints only cgroups with attached BPF programs in a way similar to
`bpftool cgroup tree`. W/o the `bpf` argument the tool works as before.

For example:

  # python36 -m drgn examples/linux/cgroup.py /sys/fs/cgroup/workload.slice/ bpf
  /workload.slice
      98761    BPF_CGROUP_SOCK_OPS            2               ned_cgrp_dctcp
  /workload.slice/workload-tw.slice/nc_0.58_dev082.prn2_598fe2e810bc4_2.task
      93488    BPF_CGROUP_INET_INGRESS        0               tw_ingress
      93487    BPF_CGROUP_INET_EGRESS         0               tw_egress
      93482    BPF_CGROUP_SOCK_OPS            2               tw_ipt_listen
      93480    BPF_CGROUP_INET6_BIND          0               tw_ipt_bind
      93481    BPF_CGROUP_INET6_CONNECT       0               tw_ipt_connect
      93483    BPF_CGROUP_UDP6_SENDMSG        0               tw_ipt_sendmsg

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-14 16:03:20 -08:00
Andrey Ignatov
760d42b4ba helpers: Add cgroup-bpf helpers
Add a new module `bpf` and a bunch of cgroup-bpf helpers. More bpf
helpers can be added in the future (e.g. iterate over all programs,
maps, etc).

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-14 16:03:20 -08:00
Andrey Ignatov
36cedcdc6f examples: Add example to traverse cgroup v2 hierarchy
Add simple example to traverse cgroup v2 hierarchy.

It can be run w/o arguments and in this case root cgroup will be used:

  # python36 -m drgn examples/linux/cgroup.py | wc -l
  1081
  # python36 -m drgn examples/linux/cgroup.py | shuf -n 3
  /system.slice/nfs-idmapd.service
  /user.slice/user-0.slice/user@0.service/dev-hugepages.mount
  /system.slice/crond.service

Or a path in cgroup v2 filesystem can be specified to limit the
traversal by this path:

  # python36 -m drgn examples/linux/cgroup.py /sys/fs/cgroup/www.slice
  /www.slice
  /www.slice/hack.service
  /www.slice/flow.service

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-14 16:03:20 -08:00
Andrey Ignatov
820d25ae29 helpers: Add cgroup v2 helpers to traverse hierarchy
Add helpers to iterate over all children and all descendants (pre-order
walk only) of a cgroup given its css.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-14 16:03:20 -08:00
Andrey Ignatov
7dd18535e1 examples: Switch tcp_sock to cgroup helpers
Use cgroup helpers instead of working with socket cgroup manually.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-14 16:03:20 -08:00
Andrey Ignatov
9c72e68385 helpers: Add cgroup v2 helpers
Add helpers to get cgroup name, path, parent and cgroup by socket.
Only cgroup v2 is supported.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-14 16:03:20 -08:00
Andrey Ignatov
26389b61fc helpers: Add kernfs helpers
Add helpers to get kernfs node name and path.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-14 16:03:20 -08:00
Omar Sandoval
7356816f61 helpers: get rid of get_tcp_states()
After thinking about this some more, I decided that although it makes
sense for scripts to convert a type to an IntEnum class, I'd prefer that
the helpers take and return drgn Objects rather than these classes.
2020-01-14 14:25:32 -08:00
Omar Sandoval
b30fb57e2f helpers: add enum_type_to_class()
We have a few use cases for converting a drgn.Type to a Python
enum.IntEnum (if I recall correctly, the original Python version of drgn
even did this automatically for enumerated types). Add an
enum_type_to_class() helper which does this conversion.
2020-01-14 12:13:02 -08:00
Omar Sandoval
842c98b055 Add .git-blame-ignore-revs
To make the previous reformatting commit less annoying, add a file
containing the commits to ignore in git blame. To use it, run `git blame
--ignore-revs-file .git-blame-ignore-revs` or `git config
blame.ignoreRevsFile .git-blame-ignore-revs`.
2020-01-14 12:03:26 -08:00
Omar Sandoval
660276a0b8 Format Python code with Black
I'm not a fan of 100% of the Black coding style, but I've spent too much
time manually formatting Python code, so let's just pull the trigger.
2020-01-14 11:51:58 -08:00
Omar Sandoval
029002fa53 Document generic helpers
drgn.helpers has a couple of undocumented generic helpers. We might as
well document them. They're using type hints, so we need to use a sphinx
extension that strips the type hints out of the signature and adds them
to the documentation. We can also do the same for execscript().
Hopefully it can be used for the C extension documentation in the
future, too.
2020-01-14 11:50:55 -08:00
Omar Sandoval
d71b57e3e8 vmtest/manage: scrub compilation directory in debug info
In order to make builds slightly more reproducible (and avoid leaking
details about the build machine's directory layout), let's remap the
compilation directory in DW_AT_comp_dir to be empty.
2020-01-13 11:15:26 -08:00
Omar Sandoval
e8d1ef82fa Make drgn.h depend on configure.ac
The previous commit forgot to add this dependency so that when the
version number is updated drgn.h actually gets regenerated.
2020-01-11 22:34:03 -08:00
Omar Sandoval
09a64f5cba Define version in libdrgn/configure.ac
Currently the drgn version number is defined in drgn.h.in, and configure
and setup.py both parse it out of there. However, now that we're
generating drgn.h anyways, it's easier to make configure.ac the source
of truth.
2020-01-11 10:05:57 -08:00
Omar Sandoval
5cf37b96ef Add more setup.py metadata 2020-01-10 17:15:35 -08:00
Omar Sandoval
71312a4039 Tweak README introduction 2020-01-10 16:40:49 -08:00
Omar Sandoval
370bf6f16a tests: add tests for new net and tcp helpers
Because the vmtest kernels aren't currently built with networking
support, we need to skip them if TCP isn't supported.
2020-01-02 19:43:57 -05:00
Omar Sandoval
bb7a8eedcf helpers: add sk_tcpstate
Although it doesn't correspond to a Linux kernel function, it is more
convenient than open-coding it everywhere, especially since the leading
__ causes Python to mangle the name if it's inside of a class.
2020-01-02 19:43:57 -05:00
Andrey Ignatov
7a36136417 examples: Add tcp_sock.py
Add example script that goes over all TCP sockets and for every socket
prints:
- TCP state;
- IP:port pair;
- cgroup v2 path.

The script shows basic operations with sockets including getting `struct
sock`, `struct inet_sock`, socket fields and cgroup v2 of socket.

It provides output similar to `ss -nt` or `ss -lnt`:

  % sudo python36 -m drgn examples/linux/tcp_sock.py | shuf -n 5
  TIME_WAIT                                       [::1]:2560                                      [::1]:44041
  ESTABLISHED                                     [::1]:54178                                     [::1]:11101  system.slice/async-distillery-server.service
  LISTEN                                           [::]:3201                                       [::]:0      system.slice/fbflow.service
  ESTABLISHED           [2401:db00:21:718a:face:0:31:0]:2406            [2401:db00:12:a106:face:0:1e:0]:46306  system.slice/configerator_proxy2.service
  LISTEN                                        0.0.0.0:11150                                   0.0.0.0:0      system.slice/system-mcrouter.slice/mcrouter@tao.service

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-02 19:43:57 -05:00
Andrey Ignatov
a253aed14b helpers: add network sockets helpers
Add helpers for working with network sockets defined in
include/net/sock.h:
- sk_fullsock()
- sk_nulls_for_each()

sk_fullsock() is useful to filter out timewait and request sockets since
not all operations can be applied to such sockets.

sk_nulls_for_each() is useful to e.g. iterate over established sockets.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-02 19:43:57 -05:00
Andrey Ignatov
01c588b07e helpers: add TCP states helper
Add get_tcp_states() helper to get all TCP states enumerators and
construct enum.IntEnum from them.

The helper is useful in debugging TCP sockets.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-02 19:43:57 -05:00
Andrey Ignatov
7013c37bdc helpers: add list_nulls helpers
These list helpers are useful to iterate over nulls hash lists of
sockets, specifically `ehash` member of `struct tcp_hashinfo`.

Corresponding functions and macros are defined in
`include/linux/list_nulls.h` kernel header.

Signed-off-by: Andrey Ignatov <rdna@fb.com>
2020-01-02 19:43:57 -05:00
Omar Sandoval
73dd7def12 docs: add examples of Object.format_() 2019-12-19 11:45:52 -08:00
Omar Sandoval
1443d17fb4 libdrgn: add DRGN_FORMAT_OBJECT_IMPLICIT_ELEMENTS 2019-12-19 11:43:54 -08:00
Omar Sandoval
db66952b2e libdrgn: add DRGN_FORMAT_OBJECT_IMPLICIT_MEMBERS 2019-12-19 11:43:54 -08:00
Omar Sandoval
c8434e9a9e libdrgn: add DRGN_FORMAT_OBJECT_ELEMENT_INDICES 2019-12-19 11:43:54 -08:00
Omar Sandoval
cfceb491db libdrgn: add DRGN_FORMAT_OBJECT_MEMBER_NAMES 2019-12-19 11:43:54 -08:00
Omar Sandoval
4fad941ec1 libdrgn: add DRGN_FORMAT_OBJECT_{MEMBERS,ELEMENTS}_SAME_LINE 2019-12-19 11:43:54 -08:00
Omar Sandoval
c3f69ba559 libdrgn: use c_format_initializer for struct/union/class 2019-12-19 11:43:54 -08:00
Omar Sandoval
e2d2df4024 libdrgn: factor c_format_initializer out of c_format_array
This will also be used for compound types, and we're going to add a few
more options that we should handle in one place.
2019-12-19 11:43:54 -08:00
Omar Sandoval
6bb8da04a0 libdrgn: omit trailing comma when formatting one-line array
This is somewhat arbitrary, but I think it looks more natural to only
use the trailing comma for multi-line initializers.
2019-12-19 11:43:54 -08:00
Omar Sandoval
1411ba36a8 libdrgn: remove dead code in c_format_array_object
When we're checking whether the element that we formatted on one line
would fit on the previous line, we check whether the previous line is
empty with remaining_columns == start_columns. This is never true, as
remaining_columns is always set to start_columns - 1 at most, and it
only decreases from there until we start  a new line.
2019-12-19 11:43:54 -08:00
Omar Sandoval
7a3bf73df0 libdrgn: replace drgn_object_truthiness() with drgn_object_is_zero()
drgn_object_truthiness() is a misnomer, as truthiness is a
language-specific concept. Instead, invert the return value and rename
it to drgn_object_is_zero(), which more accurately conveys the meaning.
2019-12-19 11:43:54 -08:00
Omar Sandoval
d77b7bd7e3 libdrgn: add DRGN_FORMAT_OBJECT_{TYPE_NAME,MEMBER_TYPE_NAMES,ELEMENT_TYPE_NAMES} 2019-12-19 11:43:54 -08:00
Omar Sandoval
89307c532a libdrgn: add DRGN_FORMAT_OBJECT_CHAR 2019-12-19 11:43:54 -08:00