* Fix a couple of misc errors in build scripts.
* Tweak scripts/kconfigs to make bpftrace work.
* Update how CI caching works to make builds faster (6 minute turnaround
time)
* Update CI config to generate per-scheduler debug archives w/ guest
dmesg/scheduler stdout, guest stdout, bpftrace script output,
veristat output.
* Update build scripts to accept the following:
** VNG RW -- write to host filesystem (better caching, logging).
* For stress tests in particular (via ini config):
** QEMU Opts -- to facilitate reproducing bugs (i.e. high core count).
** bpftrace scripts -- specify bpftrace scripts to run during stress
tests.
* enable bpftrace when using stress tests
update meson/stress test runner to enable
running bpftrace scripts while running
stress tests.
* disable layered stats output on ci
enable build and test outside src dir
add a option `build_outside_src`, defaulted
to false, to undo a tweak to enable meson to work
well with builds within src dir.
this makes some tools work better + enables
moving builds etc. to /tmp (faster for iteration if
ramdisk)
Rust build was using two separate workspaces - rust/ and scheds/rust.
There's no reason to separate them and it makes doc generation tricky. Use
single top level workspace so that we can drive all rust building from
cargo.
split build and test jobs to reduce ci turnaround time
and make it clear what is failing when something fails.
also add virtiofsd to deps to make test compilation faster
(most test time is compliation) and remove all force 9ps.
meson build script was building each rust sub-project under rust/ and
scheds/rust/ separately. This means that each rust project is built
independently which leads to a couple problems - 1. There are a lot of
shared dependencies but they have to be built over and over again for each
proejct. 2. Concurrency management becomes sad - we either have to unleash
multiple cargo builds at the same time possibly thrashing the system or
build one by one.
We've been trying to solve this from meson side in vain. Thankfully, in
issue #546, @vimproved suggested using cargo workspace which makes the
sub-projects share the same target directory and built together by the same
cargo instance while still allowing each project to behave independently for
development and publishing purposes.
Make the following changes:
- Create two cargo workspaces - one under rust/, the other under
scheds/rust/. Each contains all rust projects underneath it.
- Don't let meson descend into rust/. These are libraries used by the rust
schedulers. No need to build them from meson. Cargo will build them as
needed.
- Change the rust_scheds build target to invoke `cargo build` in
scheds/rust/ and let cargo do its thing.
- Remove per-scheduler meson.build files and instead generate custom_targets
in scheds/rust/meson.build which invokes `cargo build -p $SCHED`.
- This changes rust binary directory. Update README and
meson-scripts/install_rust_user_scheds accordingly.
- Remove per-scheduler Cargo.lock as scheds/rust/Cargo.lock is shared by all
schedulers now.
- Unify .gitignore handling.
The followings are build times on Ryzen 3975W:
Before:
________________________________________________________
Executed in 165.93 secs fish external
usr time 40.55 mins 2.71 millis 40.55 mins
sys time 3.34 mins 36.40 millis 3.34 mins
After:
________________________________________________________
Executed in 36.04 secs fish external
usr time 336.42 secs 0.00 millis 336.42 secs
sys time 36.65 secs 43.95 millis 36.61 secs
Wallclock time is reduced 5x and CPU time 7x.
Add a github action to run
[`veristat`](https://github.com/libbpf/veristat) during PRs and merges.
This uses a [action cache](https://github.com/actions/cache) to store
the veristat values when a PR is merged.
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
Add a meson script to run veristat. This can later be used to generate
reports for BPF program complexity at PR time.
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
sched_ext is about to be merged upstream. There are some compatibility
breaking changes and we're making the current sched_ext/for-6.11
1edab907b57d ("sched_ext/scx_qmap: Pick idle CPU for direct dispatch on
!wakeup enqueues") the baseline.
Tag everything except scx_mitosis as 1.0.0. As scx_mitosis is still in early
development and is currently temporarily disabled, only the patchlevel is
bumped.
Sync to the latest. Right now, it's in an awkward place where receiving
AUTOATTACH compat updatees from kernel breaks build as the libbpf version is
marked as 1.5 but bpf_map__set_autoattach() is not available yet. Sync to
the latest.
The upstrem bpftool git repo (https://github.com/libbpf/bpftool.git) is
periodically force pushed and the specific commit that we needed is not
available anymore.
Instead of failing we are actually fetching the latest bpftool (HEAD)
that introduced some breakage initially fixed by commit e59c48a6
("Update libbpf commit hash").
However, updating libbpf seems to introduce a run-time problem and all
the schedulers are failing to start:
libbpf: failed to find skeleton map ''
libbpf: failed to populate skeleton maps for 'bpf_bpf': -3
So, revert libbpf to the previous version and update the commit for
bpftool to use a version that still allows to generate a compatible BPF
skel.
Fixes: e59c48a6 ("Update libbpf commit hash")
Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
With commit 5d20f89a ("scheds-rust: build rust schedulers in sequence"),
schedulers are now built serially one after the other to prevent meson
and cargo from forking NxN parallel tasks.
However, this change has made building a single scheduler much more
cumbersome, due to the chain of dependencies.
For example, building scx_rusty using the specific meson target would
still result in all schedulers being built, because they all depend on
each other.
To address this issue, introduce the new meson build option
`serialize=true|false` (default is false).
This option allows to disable the schedulers' build chain, restoring the
old behavior.
With this option enabled, it is now possible to build just a single
scheduler, parallelizing the cargo build properly, without triggering
the build of the others. Example:
$ meson setup build -Dbuildtype=release -Dserialize=false
$ meson compile -C build scx_rusty
Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
libbpf has added a member `link` in the struct `bpf_map_skeleton`, which causes
a build failure in SCX. This commit updates the libbpf commit hash to the latest
version to avoid this error.
Please refer to commit be998aa in libbpf repo.
Signed-off-by: Yu-Cheng Chen <otteryc210@gmail.com>
This change adds stress-ng as a load test for schedulers when running in
CI. It will run stress-ng while schedulers are being tested with a
reasonable amount of work. At the end of the run the stress-ng metrics
are collected for later analysis. However, since these results may be
running in a VM they may not be super robust.
It seems that libbpf and bpftool have some places where -Werror triggers
when compiled with gcc. Let's make it non-default. Users can always
enable it locally if they want to by invoking `meson setup` with the
--werror flag.
Signed-off-by: David Vernet <void@manifault.com>
Fix error:
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /home/lucjan/Pobrane/scx-scheds-git/src/scx/build
ninja: Entering directory `/home/lucjan/Pobrane/scx-scheds-git/src/scx/build'
[2/31] Generating bpftool_target with a custom command
FAILED: bpftool_dummy.c.__PHONY__
/bin/bash /home/lucjan/Pobrane/scx-scheds-git/src/scx/meson-scripts/build_bpftool /usr/bin/jq /usr/bin/make /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src 4
libbpf.c: In function ‘bpf_object__gen_loader’:
libbpf.c: In function ‘bpf_object__gen_loader’:
libbpf.c:8994:28: error: ‘calloc’ sizes specified with ‘sizeof’ in the earlier argument and not in the later argument [-Werror=calloc-transposed-args]
8994 | gen = calloc(sizeof(*gen), 1);
| ^
libbpf.c:8994:28: error: ‘calloc’ sizes specified with ‘sizeof’ in the earlier argument and not in the later argument [-Werror=calloc-transposed-args]
8994 | gen = calloc(sizeof(*gen), 1);
| ^
libbpf.c:8994:28: note: earlier argument should specify number of elements, later size of each element
libbpf.c:8994:28: note: earlier argument should specify number of elements, later size of each element
At top level:
cc1: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1: all warnings being treated as errors
make[1]: *** [Makefile:134: /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src/bootstrap/libbpf/staticobjs/libbpf.o] Error 1
make: *** [Makefile:52: /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src/bootstrap/libbpf/libbpf.a] Error 2
make: *** Waiting for unfinished jobs....
At top level:
cc1: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1: all warnings being treated as errors
make[1]: *** [Makefile:134: /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src/libbpf/staticobjs/libbpf.o] Error 1
make: *** [Makefile:44: /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src/libbpf/libbpf.a] Error 2
ninja: build stopped: subcommand failed.
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
Add '-Wno-maybe-uninitialized' option since gcc spits a warning (as an
error) at release build. Following is the error message without the
option.
❯ meson setup build -Dbuildtype=release --prefix ~
❯ meson compile -C build --jobs=$(nproc)
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /home/changwoo/ws-multics69/dev/scx-github/build -j 16
ninja: Entering directory `/home/changwoo/ws-multics69/dev/scx-github/build'
[3/40] Generating libbpf with a custom command
FAILED: cc_cflags_probe.c.__PHONY__
/home/changwoo/ws-multics69/dev/scx-github/meson-scripts/build_libbpf /usr/bin/jq /usr/bin/make /home/changwoo/ws-multics69/dev/scx-github/build/libbpf/src 16
In function ‘elf_close’,
inlined from ‘elf_close’ at elf.c:53:6,
inlined from ‘elf_find_func_offset_from_file’ at elf.c:384:2:
elf.c:57:9: error: ‘elf_fd.elf’ may be used uninitialized [-Werror=maybe-uninitialized]
57 | elf_end(elf_fd->elf);
| ^~~~~~~~~~~~~~~~~~~~
elf.c: In function ‘elf_find_func_offset_from_file’:
elf.c:377:23: note: ‘elf_fd.elf’ was declared here
377 | struct elf_fd elf_fd;
| ^~~~~~
In function ‘elf_close’,
inlined from ‘elf_close’ at elf.c:53:6,
inlined from ‘elf_find_func_offset_from_file’ at elf.c:384:2:
elf.c:58:9: error: ‘elf_fd.fd’ may be used uninitialized [-Werror=maybe-uninitialized]
58 | close(elf_fd->fd);
| ^~~~~~~~~~~~~~~~~
elf.c: In function ‘elf_find_func_offset_from_file’:
elf.c:377:23: note: ‘elf_fd.fd’ was declared here
377 | struct elf_fd elf_fd;
| ^~~~~~
At top level:
cc1: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1: all warnings being treated as errors
make: *** [Makefile:133: staticobjs/elf.o] Error 1
make: *** Waiting for unfinished jobs....
libbpf.c: In function ‘bpf_map__init_kern_struct_ops’:
libbpf.c:1107:18: error: ‘mod_btf’ may be used uninitialized [-Werror=maybe-uninitialized]
1107 | kern_btf = mod_btf ? mod_btf->btf : obj->btf_vmlinux;
| ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libbpf.c:1092:28: note: ‘mod_btf’ was declared here
1092 | struct module_btf *mod_btf;
| ^~~~~~~
In function ‘find_struct_ops_kern_types’,
inlined from ‘bpf_map__init_kern_struct_ops’ at libbpf.c:1100:8:
libbpf.c:980:21: error: ‘btf’ may be used uninitialized [-Werror=maybe-uninitialized]
980 | kern_type = btf__type_by_id(btf, kern_type_id);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libbpf.c: In function ‘bpf_map__init_kern_struct_ops’:
libbpf.c:965:21: note: ‘btf’ was declared here
965 | struct btf *btf;
| ^~~
At top level:
cc1: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1: all warnings being treated as errors
make: *** [Makefile:133: staticobjs/libbpf.o] Error 1
[4/40] Generating bpftool_target with a custom command
ninja: build stopped: subcommand failed.
Signed-off-by: Changwoo Min <changwoo@igalia.com>
meson has a builtin -werror option that can be passed to meson setup. So
as to allow users to specify if they want to treat warnings as errors,
let's add this flag to meson.build.
We elect to make the flag the default behavior for now, as warnings in
BPF scheds are not surfaced unless there is an actual compiler error.
The warning can be turned off by specifying `-D werror=false` when
invoking `meson setup`.
Signed-off-by: David Vernet <void@manifault.com>
The build system included linux/btf.h from system even there is one in
libbpf. Adding libbpf/include/uapi to libbpf_local_h, the build
system will include linux/btf.h provided by libbpf.
Signed-off-by: Kui-Feng Lee <thinker.li@gmail.com>
If we try to cross-build scx on builders with older versions of system's
linux headers (such as those provided by linux-libc-headers in older
releases of Ubuntu), we may hit build failures, due to the different
kernel ABI, such as:
error: invalid use of undefined type ‘struct btf_enum64’
To address this, introduce a new build option called "kernel_headers"
that allows to specify a custom path for the kernel headers required
during the build process.
Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
