Previously shards were allocated as soon as a namespace was
encountered, which means that we had a large array sitting around for
every ns we saw. By allocating them lazily, we can reduce this usage.
Signed-off-by: Jay Kamat <jaygkamat@gmail.com>
DEFINE_HASH_TABLE_TYPE() doesn't actually need to know the key type.
Move that argument (and some of the derived constants) to
DEFINE_HASH_TABLE_FUNCTIONS(). This will allow recursive hash table
types. As a nice side effect, it also reduces the size of common header
files.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The underscore was meant to discourage direct access in favor of using
drgn_program_get_dbinfo(), but it turns out that it's more normal to
access it directly.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The only time that we want to create the drgn_debug_info is when we're
loading debugging information. Everywhere else, we fail fast if there is
no debugging information.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
drgnpy_linux_helper_task_state_to_char() was removed by commit
ff96c75da0 ("helpers: translate task_state_to_char() to Python"), but
I left behind the declaration.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Once we've cleared the pending DIEs vector, we won't use the vector
again (unless we load more debugging information), so we can free it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
When linking libdrgn as a static library, the name 'splay_tree' can
conflict with splay-tree.h in libiberty (namely splay_tree_splay).
Rename relevant functions to have a 'drgn_' prefix
Signed-off-by: Jake Hillion <jakehillion@fb.com>
In larger binaries, there can be a large number of CUs, and since we
store an abbrev table for each CU the extra space starts to add up.
The simplest way to mitigate this is to shrink the vectors before
saving them.
On a large binary, I noticed a memory reduction from 20.4G RES to
18.6G RES (on initial load-in).
These haven't been running in vmtest since they were added. Enable
cgroup2 in vmtest and rework the cgroup tests to create cgroups that we
can test with.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper, qdisc_lookup(), to get a Qdisc (struct Qdisc *) from a
network device and a major handle number. As an example:
>>> eth0 = netdev_get_by_name(prog, "eth0")
>>> tbf = qdisc_lookup(eth0, 0x20)
>>> tbf.ops.id.string_().decode()
tbf
>>> ingress = qdisc_lookup(eth0, 0xffff)
>>> ingress.ops.id.string_().decode()
ingress
Testing depends on pyroute2. `TestTc` is skipped if pyroute2 is not
found; test_qdisc_lookup() is skipped if the kernel is not built with the
following options:
CONFIG_DUMMY
CONFIG_NET_SCH_PRIO
CONFIG_NET_SCH_SFQ
CONFIG_NET_SCH_TBF
CONFIG_NET_SCH_INGRESS
Suggested-by: Cong Wang <cong.wang@bytedance.com>
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
Add a helper, netdev_for_each_tx_queue(), to iterate over all TX queues of
a network device. As an example:
>>> eth0 = netdev_get_by_name(prog, "eth0")
>>> for txq in netdev_for_each_tx_queue(eth0):
... print(txq.qdisc.ops.id.string_().decode())
...
sfq
tbf
prio
pfifo_fast
Set up `net` in setUpClass(), since now several tests use it. Also use
it in test_netdev_get_by_{index,name}(), instead of assuming `init_net`.
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
We're seeing some hangs waiting for entropy when running tests, so let's
enable the virtio-rng module. While we're doing a rebuild, we might as
well enable cgroups so that we can finally run those tests.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
This already caches class variables, and it's shared across all Linux
helper test cases, so it makes more sense as setUpClass. This will also
allow subclasses to use cls.prog in their own setUpClass.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
And use it in a few appropriate places. This should hopefully make it
harder to make iteration mistakes like the one fixed by commit
4755cfac7c ("libdrgn: dwarf_index: increment correct variable when
rolling back"). While we're doing this, move ARRAY_SIZE() into a new
header file with array_for_each() and make it lowercase.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
We need to increment to the next DIE, not the next shard here.
Fixes: 1c9ab2e7d1 ("libdrgn: dwarf_index: fix leak of DWARF index entries on failure")
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The "k" format expects an unsigned long, so make sure we cast C enums to
the proper type. This probably doesn't matter for x86 in practice, but
it's better to be safe.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper, get_net_ns_by_inode(), to get the network namespace
("netns") descriptor (struct net *) given an netns NSFS pseudo-file
inode (struct inode *) e.g. "/proc/$PID/ns/net" or "/run/netns/$NAME".
As an example:
>>> inode = path_lookup(prog, "/run/netns/foo").dentry.d_inode
>>> net = get_net_ns_by_inode(inode)
>>> netdev = netdev_get_by_name(net, "eth3")
>>> netdev.ifindex.value_()
5
Conventionally ip netns files can be found under "/var/run/netns/",
while Docker netns files can be found under "/var/run/docker/netns".
However, as pointed out by Omar, path_lookup() doesn't know how to
deal with symlinks; resolve it using something like "pwd -P" before
passing it to path_lookup().
Also add a get_net_ns_by_fd() wrapper around it as suggested by Omar.
Example:
>>> import os
>>> pid = os.getpid()
>>> task = find_task(prog, pid)
>>> file = open(f"/proc/{pid}/ns/net")
>>> net = get_net_ns_by_fd(task, file.fileno())
Add a test for get_net_ns_by_inode().
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
Read the Docs defaults to Sphinx 1.8.5. This version was released in
2019 and doesn't know about the :classmethod: option, so the
documentation for Object.from_bytes_() is missing from
drgn.readthedocs.io. Set the required version to the current latest
version as recommended by Read the Docs:
https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html#pinning-dependencies.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper to iterate over all network namespaces in the system. As
an example:
>>> for net in for_each_net(prog):
... if netdev_get_by_name(net, "enp0s3"):
... print(net.ipv4.sysctl_ip_early_demux.value_())
...
1
Also add a test for this new helper to tests/helpers/linux/test_net.py.
Suggested-by: Cong Wang <cong.wang@bytedance.com>
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
QEMU warns about the default 8k msize
(https://wiki.qemu.org/Documentation/9psetup#msize). I wasn't able to
measure any performance difference, but bump it to 1MiB to silence the
warning.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
There are a couple of issues with how we interpret
DW_AT_data_member_location:
1. DW_AT_data_member_location can be a location list, and we shouldn't
interpret the section offset as the member offset.
2. DW_AT_data_member_location can be location description block, and in
DWARF 2, it cannot be a constant. We should handle constant offset
expressions as generated by GCC and Clang.
Closes#13.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
I run tests with setup.py or with the unittest module, but Fedora uses
pytest. pytest assumes that any class or function starting with "test"
is a test case, which is not always the case (e.g.,
drgn.helpers.linux.bitops.test_bit()). We've hit this at least twice, in
#94 and #112.
All of our tests are unittest.TestCase cases, so we can tell pytest to
not match anything else. I'm using pytest.ini instead of pyproject.toml
because pytest only started supporting the latter relatively recently.
Closes#112.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Use the latest version of elfutils (0.185) and apply the fix "libdwfl:
fix potential NULL pointer dereference when reading link map" since that
hasn't been released yet and is needed to avoid crashing when debugging
userspace core dumps.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper to get the network device ("struct net_device *") given an
interface name. As an example:
>>> netdev = netdev_get_by_name(prog["init_net"], "lo")
>>> netdev.ifindex.value_()
1
Or pass a "Program" as the first argument, and let the helper find in
the initial network namespace (i.e. "init_net"):
>>> netdev = netdev_get_by_index(prog, "dummy0")
>>> netdev.ifindex.value_()
2
Also add a test for this new helper to tests/helpers/linux/test_net.py.
This helper simply does a linear search over the name hash table of the
network namespace, since implementing hashing in drgn is non-trivial.
It is obviously slower than net/core/dev.c:netdev_name_node_lookup() in
the kernel, but still useful.
Linux kernel commit ff92741270bf ("net: introduce name_node struct to be
used in hashlist") introduced struct netdev_name_node for name lookups.
Start by assuming that the kernel has this commit, and fall back to the
old path if that fails.
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
Mainly unused imports, unused variables, unnecessary f-strings, and
regex literals missing the r prefix. I'm not adding it to the CI linter
because it's too noisy, though.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper to find the corresponding "struct net_device *" object
given an interface index number. As an example:
>>> netdev = netdev_get_by_index(prog["init_net"], 1)
>>> netdev.name.string_().decode()
'lo'
Or pass a "Program" as the first argument, and let the helper find in
its initial network namespace (i.e. "init_net"):
>>> netdev = netdev_get_by_index(prog, 3)
>>> netdev.name.string_().decode()
'enp0s3'
Also add a test for this new helper to tests/helpers/linux/test_net.py.
For now, a user may combine this new helper with socket.if_nametoindex()
to look up by interface name:
>>> netdev = find_netdev_by_index(prog, socket.if_nametoindex("dummy0"))
>>> netdev.name.string_().decode()
'dummy0'
However, as mentioned by Cong, one should keep in mind that
socket.if_nametoindex() is based on system's current name-to-index
mapping, which may be different from that of e.g. a kdump.
Thus, as suggested by Omar, a better way to do name lookups would be
simply linear-searching the name hash table, which is slower, but less
erorr-prone.
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
Extract for_each_set_bit() that was added internally for the cpumask and
nodemask helpers, and add for_each_clear_bit() and test_bit() to go with
it.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
The API reference has all of the details, but add a short example to the
user guide (and move it before symbols, as stack traces are probably
more interesting/important).
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add an example of stack traces and parameters/local variables and use
some more interesting helpers.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Commit a97f6c4fa2 ("Associate types with program") changed repr() for
drgn.Type to include a "prog." prefix, but it didn't update the
documentation to reflect that. It also forgot to update a global type
constructor to the new Program methods.
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Sometimes we want to traverse numa nodes in the system,
so add kernel nodemask helpers to support this.
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
