Commit Graph

1959 Commits

Author SHA1 Message Date
likewhatevs
8ed7777ecc
enable build and test outside src dir (#685)
enable build and test outside src dir

add a option `build_outside_src`, defaulted
to false, to undo a tweak to enable meson to work
well with builds within src dir.

this makes some tools work better + enables
moving builds etc. to /tmp (faster for iteration if
ramdisk)
2024-09-25 12:00:15 -04:00
Fredrik Lönnegren
bbce932393 scx_utils: Add gpu-topology crate feature
The gpu-topology feature can be enabled to include GPUs when generating
a topology-map. Disabling the feature will remove the nvml-wrapper
dependency as well as GPU-specific code in topology.rs.

Most of the code was moved to a new module in rust/scx_utils/src/gpu.rs
but some of it was kept in topology.rs and hidden behind #[cfg(feature =
"gpu-topology")].

Signed-off-by: Fredrik Lönnegren <fredrik@frelon.se>
2024-09-25 14:54:52 +02:00
Andrea Righi
aea431c0c6
Merge pull request #678 from sched-ext/rustland-core-fix-mm-stall
scx_rustland_core: fix mm stall
2024-09-24 23:26:24 +02:00
likewhatevs
99d1179866
enable ide's etc. to work on bpf.c files (#668)
* enable ide's etc. to work on the bpf.c files
this makes it so that clangd and ide tools which use clangd
can work on the bpf.c code.

nothing should actually be changed outside of that ide/editor
environment, all the changes are ifdef'ed on LSP which is set
in the added .clangd file.

* move intf include out of both sides of ifdef toggle
2024-09-24 16:55:02 -04:00
Daniel Hodges
2805bb77a5
Merge pull request #683 from hodgesds/layered-idle-topo
scx_layered: Make layered idle CPU selection topology aware
2024-09-24 16:37:23 -04:00
likewhatevs
d8246fdcd6
clean up ci/make ci nicer (#682)
* make ci nicer

Replace build scheds and merged with caching build, and rename
caching build to build-and-test.

This should make the CI reports on PRs be nice and specific
(i.e. at a glance, know what passes and what fails).

It also keeps PR CI jobs up to date (as folks edit things) and
has them all use one config/24.04 etc.

* prevent untar permission errors from causing cache misses
2024-09-24 16:36:56 -04:00
Daniel Hodges
0b4a2af87f
Merge pull request #681 from hodgesds/layered-preempt-fix
scx_layered: Restrict preemption to layer cpumask
2024-09-24 15:36:34 -04:00
Daniel Hodges
679dd5920c scx_layered: Fix comment
Fix comment to be more accurate.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-24 12:31:03 -07:00
Daniel Hodges
87c6e276d9 scx_layered: Restrict idle selection to layer cpus
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-24 09:40:38 -07:00
Daniel Hodges
e68fccd26c scx_layered: Update comments on layer preemption
Update comments on layer preemption to be more descriptive.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-24 08:24:04 -07:00
Mitchell Augustin
e4eaed07a8 Change monitor_no_dbus to auto, use enum-string conv, use log::info, use Tokio spawning 2024-09-24 09:36:27 -05:00
likewhatevs
652c923624
Merge pull request #680 from likewhatevs/main
add 'continue on error' to stress tests in ci jobs
2024-09-24 09:43:10 -04:00
Daniel Hodges
6b966cda0c scx_layered: Restrict preemption to layer cpumask
When preempting restrict preemption to the current layer cpumask. This
may reduce the amount of preemption, but cause better cache locality
of preempted tasks.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-24 06:23:01 -07:00
Pat Somaru
878eaf042e
add 'continue on error' to ci jobs
I think I see PRs being harder to write because all parts of a CI job
are cancelled when one fails.

I think I am also starting to see that we have enough largely disjoint
moving pieces that there will often be one that is failing stress
tests at any time.

Make CI run all stress tests always to address this.
2024-09-24 09:17:55 -04:00
Mitchell Augustin
b7e82bdeba
Merge branch 'sched-ext:main' into scx_loader_automatic 2024-09-24 08:16:37 -05:00
Daniel Hodges
39c06d092c
Merge pull request #679 from vax-r/move_cast_mask
scx_common_bpf: Append cast_mask()
2024-09-24 07:40:27 -04:00
I Hsin Cheng
61cb3f7fc5 scx_common_bpf: Append cast_mask()
Remove cast_mask() function distributed throughout different schedulers
and add it in common.bpf.h so every scheduler can reference it once they
need to.

Signed-off-by: I Hsin Cheng <richard120310@gmail.com>
2024-09-24 16:01:19 +08:00
Andrea Righi
0a57b93846 scx_rustland_core: prevent mm stall
Bypass user-space scheduling for tasks currently handling a page fault,
preventing potential deadlock conditions involving VMA lock / mmap_lock
during user-space scheduling.

Signed-off-by: Andrea Righi <andrea.righi@linux.dev>
2024-09-24 08:46:14 +02:00
Andrea Righi
34bc6a2b64 Revert "scx_rustland_core: dispatch all kthreads directly from BPF"
This reverts commit 809d39aa7f.

Dispatching all kthreads directly doesn't really help much at preventing
stalls with the stress-ng fork stressor, so revert this commit. A better
workaround will be provided in the next commit.

Signed-off-by: Andrea Righi <andrea.righi@linux.dev>
2024-09-24 08:24:01 +02:00
Changwoo Min
b1bc4033b4
Merge pull request #673 from multics69/lavd-prop-lat-cri
scx_lavd: propagate waker's latency criticality to its wakee
2024-09-24 07:34:07 +09:00
Mitchell Augustin
ab1c737e9e
Merge branch 'sched-ext:main' into scx_loader_automatic 2024-09-23 17:19:13 -05:00
Mitchell Augustin
d434ab4266 scx_loader: Add initial automatic scheduler switching via --monitor-no-dbus
Exposes an option --monitor-no-dbus in scx_loader that will monitor CPU
utilization and start scx_lavd when any CPU exceeds 90% for more than 5
seconds. scx_lavd will be terminated if all CPUs are below 90% for
more than 30 seconds. When this flag is specified, scx_loader's
dbus functionality is not utilized.
2024-09-23 17:07:43 -05:00
Daniel Hodges
29fb647c93 scx_layered: Refactor idle core selection
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-23 12:01:42 -07:00
Daniel Hodges
380fd1f3b3 scx_layered: Make idle select topology aware
Make idle CPU selection topology aware.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-23 10:10:43 -07:00
Daniel Hodges
1b5d23dfe1
Merge pull request #675 from hodgesds/layered-dsq-dump-cleanup
scx_layered: Cleanup dump format
2024-09-23 13:09:21 -04:00
Daniel Hodges
35477970bd scx_layered: Cleanup dump format
Cleanup the dump format for topology aware dumps in scx_layered.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-23 10:02:49 -07:00
Daniel Hodges
8b14e48994
Merge pull request #671 from hodgesds/layered-last-waker
scx_layered: Add waker stats per layer
2024-09-23 10:58:54 -04:00
Changwoo Min
71fa92cf1c scx_lavd: propagate waker's latency criticality to its wakee
If a waker is more latency critical than a wakee, inherit a waker's
latency criticality for the wakee. This allows the wakee to consider the
context of who wakes me up. For now, we limit such inheritance to one
hop and one schedule.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-09-23 12:56:16 +09:00
Changwoo Min
ad8536b4a4
Merge pull request #670 from multics69/lavd-opt-preemption
scx_lavd: find a victim cpu for preemption within task's compute domain
2024-09-23 10:22:08 +09:00
Daniel Hodges
91d32663bd
scx_layered: Refactor waker tracking to only use last waker
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-22 18:05:54 -04:00
Daniel Hodges
1a2f82b91c
Merge pull request #666 from hodgesds/layered-local-llc
scx_layered: Add topology aware preemption
2024-09-22 17:36:32 -04:00
Daniel Hodges
326f3b7988
Merge pull request #667 from hodgesds/layered-pcore-grow
scx_layered: Add Big/Little core growth algos
2024-09-22 16:59:42 -04:00
Daniel Hodges
1ac9712d2e
scx_layered: Refactor preemption into a separate function
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-22 16:54:11 -04:00
Daniel Hodges
bc34bd867b
scx_layered: Add option to enable XNUMA preemption
Disable XNUMA preemption by default and add an option to enable it.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-22 16:52:57 -04:00
Daniel Hodges
55b185313a
Remove unneeded Cargo lock file
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-22 16:52:57 -04:00
Daniel Hodges
e105d9f8b1
scx_layered: Use cast_mask helper
Use the cast_mask helper to clean up some of the bpf cpumask conversion
code for preemption.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-22 16:52:57 -04:00
Daniel Hodges
5d9d32b65c
scx_layered: Add stats for XLLC/XNUMA preemptions
Add stats for XLLC/XNUMA preemptions.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-22 16:52:57 -04:00
Daniel Hodges
c15ecbb3a4
scx_layered: Add topology aware preemption
Add topology aware preemption that begins in the local LLC and attempts
to preempt from cpus nearest in the topology.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-22 16:52:56 -04:00
Daniel Hodges
6fb2f0b2b4
scx_layered: Clean up waker code
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-22 06:43:10 -04:00
Daniel Hodges
c55b2c6e69
scx_layered: Add waker stats per layered
Update the task context to keep a mask of wakers and add stats for wakes
across layers.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-22 06:43:03 -04:00
Daniel Hodges
140a101874
Merge pull request #449 from hodgesds/layered-dsq-fixes
scx_layered: Add a hi fallback dsq per llc
2024-09-22 06:39:46 -04:00
Changwoo Min
13a68465bf common: add bpf_cpumask_weight() to common.bpf.h
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-09-22 12:47:38 +09:00
Changwoo Min
7321a89724 scx_lavd: find a victim cpu for preemption within task's compute domain
Previously, we found a victim from the entire CPUs, which include remote
or non-compatible CPUs. Now we limit our search for victim finding
within a task's compute domain.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-09-22 12:47:18 +09:00
Changwoo Min
a13082c2b8
Merge pull request #669 from multics69/lavd-opt-select-cpu
scx_lavd: consider waker's CPU when ops.select_cpu()
2024-09-22 09:16:06 +09:00
Andrea Righi
897977bbc1
Merge pull request #663 from vax-r/bpfland_fix
scx_bpfland: Remove the usage of cast_mask in bpfland_enqueue
2024-09-21 22:15:11 +02:00
Changwoo Min
8d8d8f9f61 scx_lavd: consider waker's CPU when ops.select_cpu()
In case of sync wake-up, consider waker's CPU also to improve cache
locality.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-09-22 01:57:49 +09:00
Daniel Hodges
4aa841de0a
scx_layered: Rename HI_FALLBACK_DSQ to HI_FALLBACK_DSQ_BASE
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-20 17:28:38 -04:00
Daniel Hodges
a3d1344293
scx_layered: Add core growth algo for core type
Add core growth algos for Big/Little core support. The algos allow
layers to grow layers by preferring either big or little cores first.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-20 11:50:15 -04:00
Daniel Hodges
a9f3190b5f
scx_utils: Add extra ordering macros for topology
Add extra ordering macros for Core/CPU structs for ease of use with
Rust standard library features. This issue was hit when trying to sort
cores based on the CoreType. See this similar issue for details:
https://github.com/rust-lang/rust/issues/113550

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-09-20 11:41:23 -04:00
Daniel Hodges
a3cc4c223f
Merge pull request #664 from vax-r/layered_fix_cpumask
scx_layered: Refactor match_layer() and implement helper function to access cpumask within bpf_cpumask
2024-09-20 15:20:35 +02:00