Commit Graph

1972 Commits

Author SHA1 Message Date
Daniel Hodges
eb73005d07
Merge pull request #747 from hodgesds/layered-idle-order
scx_layered: Update idle topology selection order
2024-10-07 20:01:38 +00:00
Daniel Hodges
95c0553913
Merge pull request #751 from ryantimwilson/hotplug-crash-bpf
scx_rusty: Fix BPF crash during CPU hotplug
2024-10-07 19:09:21 +00:00
Ryan Wilson
a76778a4ab scx_rusty: Fix BPF crash during CPU hotplug
When hotplugging CPUs in rapid succession, scx_rusty would crash with:
```
scx_bpf_error (Failed to lookup dom[4294967295]
```

The root cause is if the scheduler is restarted fast enough, a task
on a previously hotplugged CPU may not have moved off that CPU yet.
Thus, the CPU -> domain map would contain an invalid domain (u32::max)
and we would fail to lookup the domain correctly in rusty_select_cpu
for prev_cpu.

To fix this, if the CPU is offline, we do not try to allocate to the
same NUMA node (assuming hotplug is a rare operation) beyond domestic
domain. Instead we use greedy allocation - first idle, then busy - then
any CPU.
2024-10-07 11:59:36 -07:00
Tejun Heo
75903f1031
Merge pull request #750 from hodgesds/ebuild-docs
docs: Add Gentoo installation instructions
2024-10-07 18:24:45 +00:00
Daniel Hodges
cf25f887e2 docs: Add Gentoo installation instructions
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-07 11:22:10 -07:00
Daniel Hodges
0b497d6df0 scx_layered: Update idle topology selection order
Update the idle topology selection order, the current logic is:

core architecture (big/little) -> LLC -> NUMA -> Machine

It's probably better to try to keep cache lines clean and do:

LLC -> core architecture (big/little) -> NUMA -> Machine

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-07 10:34:11 -07:00
Tejun Heo
4979cb8762
Merge pull request #739 from CachyOS/feature/scx-loader-switch-sched
scx_loader: Add SwitchScheduler methods to DBUS interface
2024-10-07 16:40:12 +00:00
Tejun Heo
1296de8cb3
Merge pull request #742 from sirlucjan/drop-journald2
Drop journald from libalpn hooks
2024-10-07 16:39:01 +00:00
Daniel Hodges
62b37b6d78
Merge pull request #745 from hodgesds/layered-topo-refactor
scx_layered: Improve perf on non topo aware paths
2024-10-07 16:27:50 +00:00
Daniel Hodges
024a2aa658 scx_layered: Improve perf on non topo aware paths
Improve the performance on non topology aware paths by skipping some map
lookups and uneccessary initializations.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-07 07:56:18 -07:00
Daniel Hodges
198cfce940
Merge pull request #740 from hodgesds/layered-topo-smt
scx_layered: Use idle smt mask for idle selection
2024-10-07 14:13:48 +00:00
Daniel Hodges
24fba4ab8d scx_layered: Add idle smt layer configuration
Add support for layer configuration for idle CPU selection. This allows
layers to choose whether or not to restrict idle CPU selection to SMT
idle CPUs.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-07 06:58:54 -07:00
Daniel Hodges
2f280ac025 scx_layered: Use idle smt mask for idle selection
In the non topology aware code the idle smt mask is used for finding
idle cpus. Update topology aware idle selection to also use the idle
smt mask. In certain benchmarks this can improve performance.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-07 05:40:59 -07:00
Daniel Hodges
30feecc5ae
Merge pull request #743 from hodgesds/layered-big-little-mask
scx_layered: Add big cpumask
2024-10-07 11:05:01 +00:00
Daniel Hodges
d86638ef0b
scx_layered: Add big cpumask
Add big cpumask to scx_layered and prefer selecting big idle cores when
using the BigLittle growth algo.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-06 14:05:12 -04:00
Andrea Righi
2801b109fd
Merge pull request #741 from sched-ext/bpfland-rework-lowlatency-mode
scx_bpfland: rework lowlatency mode
2024-10-06 16:10:16 +00:00
Piotr Gorski
5414d052f1
Drop journald from libalpn hooks
Signed-off-by: Piotr Gorski <lucjan.lucjanov@gmail.com>
2024-10-06 13:12:56 +02:00
Andrea Righi
9a29547e5b scx_bpfland: rework lowlatency mode
In lowlatency mode (option --lowlatency) tasks are ordered using a
deadline that is evaluated as the vruntime minus a certain "bonus",
determined in function of the max time slice and the average amount of
voluntary context switches, to amplify the priority boost of the tasks
that are voluntarily releasing the CPU (which are typically
interactive).

However, this method can be extremely unfair in some cases: tasks with
short bursts of voluntary context switches may receive a huge priority
boost, making the rest of the system almost unresponsive (see massive
hackbench stress tests for example).

To prevent this rework the task's deadline logic to use the vruntime and
a "deadline component" that is a function of the average used time
slice, scaled using a dynamic task priority (evaluated as the static
task priority and the its average amount of voluntary context switches).

This logic seems to prevent excessive prioritization of tasks performing
short intensive bursts of voluntary context switches.

It also makes lowlatency mode in scx_bpfland (somehow) more similar to
the deadline logic used by scx_rusty.

Signed-off-by: Andrea Righi <andrea.righi@linux.dev>
2024-10-05 17:44:09 +02:00
Changwoo Min
a673dcf809
Merge pull request #736 from multics69/scx-futex-v1
scx_lavd: split main.bpf.c into multiple files
2024-10-05 13:11:15 +09:00
Vladislav Nepogodin
7bd61f4334
scx_loader: Add SwitchScheduler methods to DBUS interface
These methods allow switching between different schedulers without requiring manual stopping and starting.
2024-10-05 02:57:17 +04:00
likewhatevs
719e98aed8
Merge pull request #713 from likewhatevs/have-rusty-track-tgid
Replace PID with Task Pointer in Rusty
2024-10-04 22:18:18 +00:00
Pat Somaru
efabcfcdc3
Replace PID with Task Pointer in Rusty
Replace PID with Task Pointer in Rusty

Fixes: #610
2024-10-04 18:06:37 -04:00
Daniel Hodges
fdc5640635
Merge pull request #735 from hodgesds/layered-weights
scx_layered: Add per layer weights
2024-10-04 19:46:58 +00:00
Daniel Hodges
c56e60b86a scx_layered: Add better debug output of iter algo
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 11:36:36 -07:00
Daniel Hodges
e1241d6e52 scx_layered: Cleanup layer growth weight limits
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 11:16:58 -07:00
Daniel Hodges
17f9b3f4f3 scx_layered: Cleanup layer infeasible weight calc
Cleanup the calculation of the infeasible weight to not use an
unneccesary collect.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 10:12:22 -07:00
Daniel Hodges
0476a10f83 scx_layered: Cleanup from code review
Cleanup from code review.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 10:09:38 -07:00
Daniel Hodges
817e310a31 scx_layered: Add default dsq iter algo
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:58:26 -07:00
Daniel Hodges
7ee12091c3 scx_layered: Add DSQ iteration algo
Add DSQ iteration algorithms.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:58:23 -07:00
Daniel Hodges
6929501aea scx_layered: Refactor stats variable names
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
f066580612 scx_layered: Use dcycle for infeasible weights
Fix a bug to use duty cycle for infeasible weights calculations.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
c55d34c319 scx_layered: Cleanup unused metrics
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
c0c4e183f0 scx_layered: Cargo fmt
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
f3b3d4f19c scx_layered: Add weighted layer DSQ iteration
Add a flag to control DSQ iteration across layers by layer weight. This
helps prevent starvation by iterating over layers with the lowest weight
first.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
bd75ac8dbf scx_layered: Add flags for growth and preemption
Add two new flags `layer_preempt_weight_disable` and
`layer_growth_weight_disable` to disabled preemption and layer growth
when weighted layer load exceeds the configured threshold.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
e48e675cff scx_layered: Remove LoadLedger from stats
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
2518c99bf2 scx_layered: Refactor load calculation
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
54dbf35680 scx_layered: Add weights to userspace layer config
Add weights to userspace layer config.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
07be9dcf59 scx_layered: Add stats for adjusted layer weights
Add stats for infeasible weights adjusted layer stats.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
da38d69009 scx_layered: Add layer weights
Add weights to layers and use the infeasible weights crate to properly
apply weights during contention to prevent starvation.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-10-04 09:56:37 -07:00
Daniel Hodges
0eb0a5c4fa
Merge pull request #737 from minosfuture/main
scx_layered: Refactor topology algorithms to a separate module
2024-10-04 16:50:39 +00:00
Ming Yang
d76036b7cb scx_layered: Add Reverse layer growth algo
Add `LayerGrowthAlgo::Reverse` to be the reverse order of Linear.

Signed-off-by: Ming Yang <minos.future@gmail.com>
2024-10-04 09:29:36 -07:00
Ming Yang
35d5c082d5 scx_layered: Break up layer_core_order function
`layer_core_order` provided multiple core growth implementation

Break it up into smaller function. Also, attach the method to
LayerGrowthAlgo. And `LayerCoreOrderGenerator` is added to make future
growth algo extension easy.

Signed-off-by: Ming Yang <minos.future@gmail.com>
2024-10-04 09:18:01 -07:00
Ming Yang
29308d4705 scx_layered: Move layer core growth logic to separate module
Move layer core growth logic to separate module for further refactoring.

Signed-off-by: Ming Yang <minos.future@gmail.com>
2024-10-04 08:51:11 -07:00
Changwoo Min
7c5c83a3a2 scx_lavd: split main.bpf.c into multiple files
As the main.bpf.c file grows, it gets hard to maintain.
So, split it into multiple logical files. There is no
functional change.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-10-05 00:25:40 +09:00
Daniel Hodges
8f0485b9b9
Merge pull request #732 from minosfuture/main
scx_layered: Add per layer time slices to stats
2024-10-04 12:06:30 +00:00
Ming Yang
658a75df73 scx_layered: Add per layer time slices to stats
Quoting issue #720 description from @hodgesds:

> In `scx_layered` the time slice can be [configured per layer](https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_layered/src/main.rs#L493).
> This should be added to the
> [`LayerStats`](https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_layered/src/stats.rs#L51)
> for each layer. During stats
> [refresh](https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_layered/src/main.rs#L852)
> read the time slice duration (from the bpf skel) to the layer and add it
> to the stats. Finally, update the
> [format](https://github.com/sched-ext/scx/blob/main/scheds/rust/scx_layered/src/stats.rs#L218)
> method for `LayerStats` to print the per layer time slices.

Signed-off-by: Ming Yang <minos.future@gmail.com>
2024-10-03 20:56:03 -07:00
Tejun Heo
db0f83ce89
Merge pull request #731 from frelon/scx-rusty-single-dom-bug
scx_rusty: fix single dom short-circuit
2024-10-03 18:55:16 +00:00
Fredrik Lönnegren
4b290a1757 scx_rusty: fix single dom short-circuit
Remove a short-circuit in cpu_to_dom_id that will return domain id 0 for
any input.

This fixes a crash of scx_rusty when running with a single domain and
any CPU is offline.

Signed-off-by: Fredrik Lönnegren <fredrik@frelon.se>
2024-10-03 20:34:18 +02:00
likewhatevs
58fc3be5fc
Merge pull request #726 from likewhatevs/main
Run integration tests against bpf-next kernel every 6 hrs.
2024-10-03 12:25:18 +00:00