In larger binaries, there can be a large number of CUs, and since we
store an abbrev table for each CU the extra space starts to add up.
The simplest way to mitigate this is to shrink the vectors before
saving them.
On a large binary, I noticed a memory reduction from 20.4G RES to
18.6G RES (on initial load-in).
These haven't been running in vmtest since they were added. Enable
cgroup2 in vmtest and rework the cgroup tests to create cgroups that we
can test with.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper, qdisc_lookup(), to get a Qdisc (struct Qdisc *) from a
network device and a major handle number. As an example:
>>> eth0 = netdev_get_by_name(prog, "eth0")
>>> tbf = qdisc_lookup(eth0, 0x20)
>>> tbf.ops.id.string_().decode()
tbf
>>> ingress = qdisc_lookup(eth0, 0xffff)
>>> ingress.ops.id.string_().decode()
ingress
Testing depends on pyroute2. `TestTc` is skipped if pyroute2 is not
found; test_qdisc_lookup() is skipped if the kernel is not built with the
following options:
CONFIG_DUMMY
CONFIG_NET_SCH_PRIO
CONFIG_NET_SCH_SFQ
CONFIG_NET_SCH_TBF
CONFIG_NET_SCH_INGRESS
Suggested-by: Cong Wang <cong.wang@bytedance.com>
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
Add a helper, netdev_for_each_tx_queue(), to iterate over all TX queues of
a network device. As an example:
>>> eth0 = netdev_get_by_name(prog, "eth0")
>>> for txq in netdev_for_each_tx_queue(eth0):
... print(txq.qdisc.ops.id.string_().decode())
...
sfq
tbf
prio
pfifo_fast
Set up `net` in setUpClass(), since now several tests use it. Also use
it in test_netdev_get_by_{index,name}(), instead of assuming `init_net`.
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
We're seeing some hangs waiting for entropy when running tests, so let's
enable the virtio-rng module. While we're doing a rebuild, we might as
well enable cgroups so that we can finally run those tests.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This already caches class variables, and it's shared across all Linux
helper test cases, so it makes more sense as setUpClass. This will also
allow subclasses to use cls.prog in their own setUpClass.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
And use it in a few appropriate places. This should hopefully make it
harder to make iteration mistakes like the one fixed by commit
4755cfac7c ("libdrgn: dwarf_index: increment correct variable when
rolling back"). While we're doing this, move ARRAY_SIZE() into a new
header file with array_for_each() and make it lowercase.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We need to increment to the next DIE, not the next shard here.
Fixes: 1c9ab2e7d1 ("libdrgn: dwarf_index: fix leak of DWARF index entries on failure")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The "k" format expects an unsigned long, so make sure we cast C enums to
the proper type. This probably doesn't matter for x86 in practice, but
it's better to be safe.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper, get_net_ns_by_inode(), to get the network namespace
("netns") descriptor (struct net *) given an netns NSFS pseudo-file
inode (struct inode *) e.g. "/proc/$PID/ns/net" or "/run/netns/$NAME".
As an example:
>>> inode = path_lookup(prog, "/run/netns/foo").dentry.d_inode
>>> net = get_net_ns_by_inode(inode)
>>> netdev = netdev_get_by_name(net, "eth3")
>>> netdev.ifindex.value_()
5
Conventionally ip netns files can be found under "/var/run/netns/",
while Docker netns files can be found under "/var/run/docker/netns".
However, as pointed out by Omar, path_lookup() doesn't know how to
deal with symlinks; resolve it using something like "pwd -P" before
passing it to path_lookup().
Also add a get_net_ns_by_fd() wrapper around it as suggested by Omar.
Example:
>>> import os
>>> pid = os.getpid()
>>> task = find_task(prog, pid)
>>> file = open(f"/proc/{pid}/ns/net")
>>> net = get_net_ns_by_fd(task, file.fileno())
Add a test for get_net_ns_by_inode().
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
Read the Docs defaults to Sphinx 1.8.5. This version was released in
2019 and doesn't know about the :classmethod: option, so the
documentation for Object.from_bytes_() is missing from
drgn.readthedocs.io. Set the required version to the current latest
version as recommended by Read the Docs:
https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html#pinning-dependencies.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper to iterate over all network namespaces in the system. As
an example:
>>> for net in for_each_net(prog):
... if netdev_get_by_name(net, "enp0s3"):
... print(net.ipv4.sysctl_ip_early_demux.value_())
...
1
Also add a test for this new helper to tests/helpers/linux/test_net.py.
Suggested-by: Cong Wang <cong.wang@bytedance.com>
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
QEMU warns about the default 8k msize
(https://wiki.qemu.org/Documentation/9psetup#msize). I wasn't able to
measure any performance difference, but bump it to 1MiB to silence the
warning.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There are a couple of issues with how we interpret
DW_AT_data_member_location:
1. DW_AT_data_member_location can be a location list, and we shouldn't
interpret the section offset as the member offset.
2. DW_AT_data_member_location can be location description block, and in
DWARF 2, it cannot be a constant. We should handle constant offset
expressions as generated by GCC and Clang.
Closes#13.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I run tests with setup.py or with the unittest module, but Fedora uses
pytest. pytest assumes that any class or function starting with "test"
is a test case, which is not always the case (e.g.,
drgn.helpers.linux.bitops.test_bit()). We've hit this at least twice, in
#94 and #112.
All of our tests are unittest.TestCase cases, so we can tell pytest to
not match anything else. I'm using pytest.ini instead of pyproject.toml
because pytest only started supporting the latter relatively recently.
Closes#112.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Use the latest version of elfutils (0.185) and apply the fix "libdwfl:
fix potential NULL pointer dereference when reading link map" since that
hasn't been released yet and is needed to avoid crashing when debugging
userspace core dumps.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper to get the network device ("struct net_device *") given an
interface name. As an example:
>>> netdev = netdev_get_by_name(prog["init_net"], "lo")
>>> netdev.ifindex.value_()
1
Or pass a "Program" as the first argument, and let the helper find in
the initial network namespace (i.e. "init_net"):
>>> netdev = netdev_get_by_index(prog, "dummy0")
>>> netdev.ifindex.value_()
2
Also add a test for this new helper to tests/helpers/linux/test_net.py.
This helper simply does a linear search over the name hash table of the
network namespace, since implementing hashing in drgn is non-trivial.
It is obviously slower than net/core/dev.c:netdev_name_node_lookup() in
the kernel, but still useful.
Linux kernel commit ff92741270bf ("net: introduce name_node struct to be
used in hashlist") introduced struct netdev_name_node for name lookups.
Start by assuming that the kernel has this commit, and fall back to the
old path if that fails.
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
Mainly unused imports, unused variables, unnecessary f-strings, and
regex literals missing the r prefix. I'm not adding it to the CI linter
because it's too noisy, though.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper to find the corresponding "struct net_device *" object
given an interface index number. As an example:
>>> netdev = netdev_get_by_index(prog["init_net"], 1)
>>> netdev.name.string_().decode()
'lo'
Or pass a "Program" as the first argument, and let the helper find in
its initial network namespace (i.e. "init_net"):
>>> netdev = netdev_get_by_index(prog, 3)
>>> netdev.name.string_().decode()
'enp0s3'
Also add a test for this new helper to tests/helpers/linux/test_net.py.
For now, a user may combine this new helper with socket.if_nametoindex()
to look up by interface name:
>>> netdev = find_netdev_by_index(prog, socket.if_nametoindex("dummy0"))
>>> netdev.name.string_().decode()
'dummy0'
However, as mentioned by Cong, one should keep in mind that
socket.if_nametoindex() is based on system's current name-to-index
mapping, which may be different from that of e.g. a kdump.
Thus, as suggested by Omar, a better way to do name lookups would be
simply linear-searching the name hash table, which is slower, but less
erorr-prone.
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
Extract for_each_set_bit() that was added internally for the cpumask and
nodemask helpers, and add for_each_clear_bit() and test_bit() to go with
it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The API reference has all of the details, but add a short example to the
user guide (and move it before symbols, as stack traces are probably
more interesting/important).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add an example of stack traces and parameters/local variables and use
some more interesting helpers.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Commit a97f6c4fa2 ("Associate types with program") changed repr() for
drgn.Type to include a "prog." prefix, but it didn't update the
documentation to reflect that. It also forgot to update a global type
constructor to the new Program methods.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Sometimes we want to traverse numa nodes in the system,
so add kernel nodemask helpers to support this.
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
Add a way to create an object from raw bytes. One example where I've
wanted this is creating a struct pt_regs from a PRSTATUS note or other
source.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
If the kernel is compiled with CONFIG_CPUMASK_OFFSTACK, then the full
struct cpumask::bits array may not be allocated. Use nr_cpu_ids as the
limit instead of the length of the array.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
These were added in DWARF 5. They need to know the CU that they're being
evaluated in, but the parameters for drgn_eval_dwarf_expression() were
already getting unwieldy. Wrap the evaluation context in a new struct
drgn_dwarf_expression_context, add the additional CU information, and
implement the operations.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The main changes are:
1. Skipping the new attribute forms.
2. Handling DW_FORM_strx*, DW_FORM_line_strp, and DW_FORM_implicit_const
for the attributes that we care about.
3. Parsing the new unit header format.
4. Parsing the new line number program header format.
Note that Clang currently produces an incorrect DWARF 5 line number
program header for the Linux kernel (https://reviews.llvm.org/D105662),
so some types are not properly deduplicated in that case.
Closes#104.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
