Commit Graph

54 Commits

Author SHA1 Message Date
Tejun Heo
eeb2151804
Merge pull request #300 from sirlucjan/pacman-hooks-openrc
Add pacman hooks for openrc
2024-05-20 06:47:25 -10:00
Tejun Heo
c4bed3602f
Merge pull request #301 from sirlucjan/libbpf-werror
meson: fix a build error for libbpf at release build
2024-05-20 06:46:15 -10:00
David Vernet
4b052c1489
Build: Make -Werror non-default
It seems that libbpf and bpftool have some places where -Werror triggers
when compiled with gcc. Let's make it non-default. Users can always
enable it locally if they want to by invoking `meson setup` with the
--werror flag.

Signed-off-by: David Vernet <void@manifault.com>
2024-05-20 08:30:30 -05:00
Piotr Gorski
2bad1dd58e
meson: fix a build error for libbpf at release build
Fix error:
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /home/lucjan/Pobrane/scx-scheds-git/src/scx/build
ninja: Entering directory `/home/lucjan/Pobrane/scx-scheds-git/src/scx/build'
[2/31] Generating bpftool_target with a custom command
FAILED: bpftool_dummy.c.__PHONY__
/bin/bash /home/lucjan/Pobrane/scx-scheds-git/src/scx/meson-scripts/build_bpftool /usr/bin/jq /usr/bin/make /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src 4
libbpf.c: In function ‘bpf_object__gen_loader’:
libbpf.c: In function ‘bpf_object__gen_loader’:
libbpf.c:8994:28: error: ‘calloc’ sizes specified with ‘sizeof’ in the earlier argument and not in the later argument [-Werror=calloc-transposed-args]
 8994 |         gen = calloc(sizeof(*gen), 1);
      |                            ^
libbpf.c:8994:28: error: ‘calloc’ sizes specified with ‘sizeof’ in the earlier argument and not in the later argument [-Werror=calloc-transposed-args]
 8994 |         gen = calloc(sizeof(*gen), 1);
      |                            ^
libbpf.c:8994:28: note: earlier argument should specify number of elements, later size of each element
libbpf.c:8994:28: note: earlier argument should specify number of elements, later size of each element
At top level:
cc1: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1: all warnings being treated as errors
make[1]: *** [Makefile:134: /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src/bootstrap/libbpf/staticobjs/libbpf.o] Error 1
make: *** [Makefile:52: /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src/bootstrap/libbpf/libbpf.a] Error 2
make: *** Waiting for unfinished jobs....
At top level:
cc1: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1: all warnings being treated as errors
make[1]: *** [Makefile:134: /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src/libbpf/staticobjs/libbpf.o] Error 1
make: *** [Makefile:44: /home/lucjan/Pobrane/scx-scheds-git/src/scx/build/bpftool/src/libbpf/libbpf.a] Error 2
ninja: build stopped: subcommand failed.

Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
2024-05-20 10:03:48 +02:00
Piotr Gorski
951a3d66bc
Add pacman hooks for openrc
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
2024-05-20 08:32:05 +02:00
Changwoo Min
d3339166c0 meson: fix a build error for libbpf at release build
Add '-Wno-maybe-uninitialized' option since gcc spits a warning (as an
error) at release build. Following is the error message without the
option.

❯ meson setup build -Dbuildtype=release  --prefix ~
❯ meson compile -C build  --jobs=$(nproc)
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /home/changwoo/ws-multics69/dev/scx-github/build -j 16
ninja: Entering directory `/home/changwoo/ws-multics69/dev/scx-github/build'
[3/40] Generating libbpf with a custom command
FAILED: cc_cflags_probe.c.__PHONY__
/home/changwoo/ws-multics69/dev/scx-github/meson-scripts/build_libbpf /usr/bin/jq /usr/bin/make /home/changwoo/ws-multics69/dev/scx-github/build/libbpf/src 16
In function ‘elf_close’,
    inlined from ‘elf_close’ at elf.c:53:6,
    inlined from ‘elf_find_func_offset_from_file’ at elf.c:384:2:
elf.c:57:9: error: ‘elf_fd.elf’ may be used uninitialized [-Werror=maybe-uninitialized]
   57 |         elf_end(elf_fd->elf);
      |         ^~~~~~~~~~~~~~~~~~~~
elf.c: In function ‘elf_find_func_offset_from_file’:
elf.c:377:23: note: ‘elf_fd.elf’ was declared here
  377 |         struct elf_fd elf_fd;
      |                       ^~~~~~
In function ‘elf_close’,
    inlined from ‘elf_close’ at elf.c:53:6,
    inlined from ‘elf_find_func_offset_from_file’ at elf.c:384:2:
elf.c:58:9: error: ‘elf_fd.fd’ may be used uninitialized [-Werror=maybe-uninitialized]
   58 |         close(elf_fd->fd);
      |         ^~~~~~~~~~~~~~~~~
elf.c: In function ‘elf_find_func_offset_from_file’:
elf.c:377:23: note: ‘elf_fd.fd’ was declared here
  377 |         struct elf_fd elf_fd;
      |                       ^~~~~~
At top level:
cc1: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1: all warnings being treated as errors
make: *** [Makefile:133: staticobjs/elf.o] Error 1
make: *** Waiting for unfinished jobs....
libbpf.c: In function ‘bpf_map__init_kern_struct_ops’:
libbpf.c:1107:18: error: ‘mod_btf’ may be used uninitialized [-Werror=maybe-uninitialized]
 1107 |         kern_btf = mod_btf ? mod_btf->btf : obj->btf_vmlinux;
      |         ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libbpf.c:1092:28: note: ‘mod_btf’ was declared here
 1092 |         struct module_btf *mod_btf;
      |                            ^~~~~~~
In function ‘find_struct_ops_kern_types’,
    inlined from ‘bpf_map__init_kern_struct_ops’ at libbpf.c:1100:8:
libbpf.c:980:21: error: ‘btf’ may be used uninitialized [-Werror=maybe-uninitialized]
  980 |         kern_type = btf__type_by_id(btf, kern_type_id);
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libbpf.c: In function ‘bpf_map__init_kern_struct_ops’:
libbpf.c:965:21: note: ‘btf’ was declared here
  965 |         struct btf *btf;
      |                     ^~~
At top level:
cc1: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics
cc1: all warnings being treated as errors
make: *** [Makefile:133: staticobjs/libbpf.o] Error 1
[4/40] Generating bpftool_target with a custom command
ninja: build stopped: subcommand failed.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-05-19 19:37:43 +09:00
David Vernet
4a88008cbe
build: Allow specifying --werror build option
meson has a builtin -werror option that can be passed to meson setup. So
as to allow users to specify if they want to treat warnings as errors,
let's add this flag to meson.build.

We elect to make the flag the default behavior for now, as warnings in
BPF scheds are not surfaced unless there is an actual compiler error.
The warning can be turned off by specifying `-D werror=false` when
invoking `meson setup`.

Signed-off-by: David Vernet <void@manifault.com>
2024-05-18 07:50:33 -05:00
Piotr Gorski
1fbf4f4f9b
Add pacman hooks for systemd
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
2024-05-13 15:52:06 +02:00
Tejun Heo
e5e88b7e18 Bump versions to prepare for a release 2024-04-29 09:07:27 -10:00
Jordan Rome
c1925f07b4 Sync libbpf_h and libbpf_local_h
Make sure these include the same header directories.
2024-04-04 17:32:37 -07:00
Tejun Heo
4a77c8f8fb meson.build: Update libbpf and bpftool version requirements
The recent compat additions require new libbpf and bpftool. Update the
requirements.

- libbpf >= 1.4
- bpftool >= 7.4
2024-04-04 13:16:08 -10:00
Kui-Feng Lee
b61dec6815 meson: Add missing path to libbpf_local_h.
The build system included linux/btf.h from system even there is one in
libbpf.  Adding libbpf/include/uapi to libbpf_local_h, the build
system will include linux/btf.h provided by libbpf.

Signed-off-by: Kui-Feng Lee <thinker.li@gmail.com>
2024-04-04 14:01:59 -07:00
Piotr Gorski
ef60559cb5
systemd: Move services to separate directory
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
2024-04-04 20:29:54 +02:00
Andrea Righi
eca7ecd24e build: introduce kernel_headers build option
If we try to cross-build scx on builders with older versions of system's
linux headers (such as those provided by linux-libc-headers in older
releases of Ubuntu), we may hit build failures, due to the different
kernel ABI, such as:

 error: invalid use of undefined type ‘struct btf_enum64’

To address this, introduce a new build option called "kernel_headers"
that allows to specify a custom path for the kernel headers required
during the build process.

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
2024-04-04 10:53:36 +02:00
Damian N.
e7b4a03034
fixed option name in meson.build 2024-04-03 19:49:37 +02:00
Damian N.
3e6bcab337
mv scx.initrd openrc/scx.initrd 2024-04-03 19:45:55 +02:00
Damian N.
6e02c8eb5c
main meson: added openrc options 2024-04-03 19:38:28 +02:00
Tejun Heo
6f81409df4 Bump versions
- scx_utils bumped from 0.6.0 to 0.7.0.

- Repo and rust schedulers get a PATCH level bump.
2024-04-02 10:58:50 -10:00
Tejun Heo
1ceb97276e meson.build: Update libbpf/bpftool commits to latest 2024-04-02 10:46:02 -10:00
Breno Leitao
4a18e9c7f7 meson: fail if get_clang_ver fails
In my dev enviroment, bpf_clang_ver is coming as NULL, since I am using
upstream Clang. Fails graceful in this case.

	#  /usr/local/bin/clang --version
	clang version 18.0.0git (https://github.com/llvm/llvm-project.git c458f928fad7bbcf08ab1da9949eb2969fc9f89c)
	Target: x86_64-unknown-linux-gnu
	Thread model: posix
	InstalledDir: /usr/local/bin

But the command below returns nothing
	/home/leit/Devel/scx/meson-scripts/get_clang_ver /usr/local/bin/clang

Signed-off-by: Breno Leitao <leitao@debian.org>
2024-03-14 03:27:19 -07:00
Jordan Rome
ffc7b7dc4a Fetch and build bpftool by default
This pairs with the new default behavior to fetch and build libbpf
and is mostly being used so we can use the latest bpftool and libbpf.
2024-03-11 10:00:01 -07:00
Jordan Rome
1769dece7d Remove libbpf as a submodule
Instead clone the libbpf repo at a specific hash during setup.
This is to fix an issue whereby submodules are not included
in the tarball and therefore won't be updated/fetched during
setup after unzipping the tarball.
2024-03-07 18:31:09 -08:00
Jordan Rome
38dab12459 Always build libbpf as a PIE
This is to fix an error sometimes seen when compiling with gcc,
whereby the position independent sched ext rust libraries
don't play nicely with the libbpf library if it's not also built
as position independent.

This also explicitly sets the rustc relocation-mode to "pic",
which is the default (just so this doesn't accidentally change
out from under us).
2024-03-05 12:56:09 -08:00
Jordan Rome
499924ead8 Add libbpf as a submodule
This is to potentinally reduce issues with folks
using different versions of libbpf at runtime.

This also:
- makes static linking of libbpf the default
- adds steps in `meson setup` to fetch libbpf and make it
2024-03-01 12:39:35 -08:00
Tejun Heo
c3c71781f1 Revert "Merge pull request #165 from sched-ext/reduce-rust-build-load"
This reverts commit a7b39f24e2, reversing
changes made to cf7404fb03.

The PR doesn't do what the description says. It instead limits the number of
rustc instances to 1 for each cargo build making rust builds extremely slow.
Let's revert and try again.
2024-02-29 07:46:37 -10:00
Tejun Heo
438373a8cc Revert "Merge pull request #163 from jordalgo/libbpf-submodule"
This reverts commit 5b9b953e3c, reversing
changes made to a7b39f24e2.

The current git submodule approach is a bit cumbersome and doesn't provide a
unified build environment for both libbpf and scx scheds. Also, the build
instruction doesn't seem to work. Let's revert it for now.
2024-02-29 07:39:01 -10:00
David Vernet
5b9b953e3c
Merge pull request #163 from jordalgo/libbpf-submodule
Add libbpf as a submodule
2024-02-29 09:31:40 -06:00
Jordan Rome
626e66686a Add libbpf as a submodule
This is to potentinally reduce issues with folks using
different versions of libbpf at runtime.
2024-02-29 07:31:13 -08:00
Andrea Righi
274eb8b4d8 build: limit the maximum amount of parallel cargo build
Each cargo build is already parallelized, spreading multiple rustc
across all the available CPUs by default.

Allowing to run multiple instances of cargo at the same time doesn't
provide any benefit and it can only increase the risk of triggering OOM
conditions or overloading the build system.

Therefore, limit the amount of parallel cargo build instances to 1.

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
2024-02-29 15:25:32 +01:00
Jordan Rome
9cedac45ee Gate cargo build options behind 'enable_rust'
This is to not have to require cargo when building the c schedulers.
2024-02-25 17:51:43 -08:00
Piotr Gorski
cb11aecbd0
Bump to 0.1.7
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
2024-02-13 08:20:38 +01:00
Jordan Rome
dd11d97cdb Include libbpf_h path in c sched compilation
If a user wants to use an external libbpf source (via libbpf_h opt)
we need to also pass this to c sched compilation.
2024-02-02 19:46:24 -05:00
Tejun Heo
988b7d13c1 Bump versions
scx_exit_info change doesn't require code to be updated but breaks binary
compatbility. Bump versions and cut a new release.
2024-01-25 09:01:23 -10:00
Piotr Gorski
d618a06d92
Add systemd services for scx schedulers
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
2024-01-15 23:41:59 +01:00
Tejun Heo
942b0269b8 Bump versions
After updates to reflect the updated init and direct dispatch API, the
schedulers aren't compatible with older kernels. Bump versions and publish
releases.
2024-01-08 18:49:54 -10:00
Andrea Righi
05f5c69747 ci: use virtme-ng to test the schedulers
Use virtme-ng to run the schedulers after they're built; virtme-ng
allows to pick an arbitrary sched-ext enabled kernel and run it
virtualizing the entire user-space root filesystem, so we can basically
exceute the recompiled schedulers inside such kernel.

This should allow to catch potential run-time issue in advance (both in
the kernel and the schedulers).

The sched-ext kernel is taken from the Ubuntu ppa (ppa:arighi/sched-ext)
at the moment, since it is the easiest / fastest way to get a
precompiled sched-ext kernel to run inside the Ubuntu 22.04 testing
environment.

The schedulers are tested using the new meson target "test_sched", the
specific actions are defined in meson-scripts/test_sched.

By default each test has a timeout of 30 sec, after the virtme-ng
completes the boot (that should be enough to initialize the scheduler
and run the scheduler for some seconds), while the total lifetime of the
virtme-ng guest is set to 60 sec, after this time the guest will be
killed (this allows to catch potential kernel crashes / hangs).

If a single scheduler fails the test, the entire "test_sched" action
will be interrupted and the overall test result will be considered a
failure.

At the moment scx_layered is excluded from the tests, because it
requires a special configuration (we should probably pre-generate a
default config in the workflow actions and change the scheduler to use
the default config if it's executed without any argument).

Moreover, scx_flatcg is also temporarily excluded from the tests,
because of these known issues:
 - https://github.com/sched-ext/scx/issues/49
 - https://github.com/sched-ext/sched_ext/pull/101

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
2023-12-29 15:54:10 +01:00
Tejun Heo
98773131df Bump versions to publish scx_utils fedora compat change 2023-12-29 06:58:45 +09:00
Piotr Gorski
c6eb66616f
Bump to 0.1.3
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
2023-12-22 00:48:50 +01:00
Tejun Heo
8a07bcc31b Bump versions and add LICENSE symlinks for scx_layered and scx_rusty 2023-12-12 11:21:08 -10:00
Andrea Righi
1742daa4b5 build: add Debian arch name mapping for s390
Add the proper mapping from Debian architecture "s390x" to the kernel
architecture name "s390".

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
2023-12-10 12:42:15 +01:00
Andrea Righi
6343bcf360 build: introduce enable_rust build option
Introduce an option to enable/disable the build of all the Rust
sub-projects.

This can be useful to build scx on those systems where Rust is not
fully supported (e.g., armhf).

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
2023-12-09 15:05:23 +01:00
Tejun Heo
9e12238d64 Support offline compilation 2023-12-08 08:45:44 -10:00
Tejun Heo
d8c2321831 Bump overall version to 0.1.1
- scx_nest included.
- scx_rusty bug fix.
- Ubuntu build fixes.
2023-12-07 13:11:28 -10:00
Tejun Heo
9f70108962 v0.1.0 2023-12-05 15:04:23 -10:00
Tejun Heo
ecd485ed3e Add LICENSE (GPL-2.0) 2023-12-03 12:04:02 -10:00
Tejun Heo
e030d0c816 build: Ensure meson version >= 1.2 2023-12-02 08:09:00 -10:00
Tejun Heo
0f1ed894bd build: rust projects now link against libbpf.a if provided 2023-12-02 06:41:26 -10:00
Tejun Heo
30f6a38573 build: Pipe down build options to scx_utils::BuildHelpers using env vars 2023-12-01 14:49:32 -10:00
Tejun Heo
6b9c392bf0 build: "meson install" works now 2023-12-01 13:37:28 -10:00
Tejun Heo
8fb4fbf54d build: Propagate buildtype==release to cargo 2023-12-01 12:15:16 -10:00