Add a helper for returning the appropriate slice duration for a layer
and replace a various instances where the slice value was being
recalculated.
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
add timer based antistall to scx_layered and new flags
to enable/disable and specify seconds of delay before
it turns on.
also update ci config to make sure this verifies/runs.
Verifier in older kernels choke on function calls from sleepable progs
triggering non-sensical RCU state error:
frame1: R1_w=scalar(id=674,smin=smin32=0,smax=umax=smax32=umax32=51,var_off=(0x0; 0x3f)) R10=; return *llc_ptr;
1072: (61) r0 = *(u32 *)(r2 +0) ; frame1: R0_w=scalar(smin=0,smax=umax=0xffffffff,var_off=(0x0; 0xffffffff)) R2_w=map_value(map=bpf_bpf.rodata,ks=4,vs=9570,off=4400,smin=smin32=0,smax=umax=smax32=umax32=204,var_off=(0x0; 0xfc)) refs=13,647
; }
1073: (95) exit
bpf_rcu_read_unlock is missing
processed 10663 insns (limit 1000000) max_states_per_insn 8 total_states 615 peak_states 281 mark_read 20
-- END PROG LOAD LOG --
Work around by adding and using __always_inline variant of cpu_to_llc_id()
from layered_init(). Note that we can't switch everyone to __always_inline
as that can lead to verification failure due to ins limit.
When selecting the idle CPU use the idle_smt option on the layer. This
may improve cache locality in some cases by placing tasks on CPUs that
are on closer cache lines.
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
On some older kernels layered fails to validate. Prevent certain helpers
from being inlined to pass the verifier.
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
The loops in topology aware mode were recently refactored to place the -per-LLC
loops inside the per-layer loops. However, the layer specific checks were left
in the inner loops, slowing this down unnecessarily.
Pull the layer specific checks from the inner loop into the outer loop.
Also changes these functions to `__weak` to ensure they don't get inlined -
they're expected to be verified as global functions.
Note to reviewers: this looks good to me, but I'd appreciate if you reviewed
the De Morgan applications in detail.
Test plan:
- `cargo build --release && sudo target/release/scx_layered --run-example` on a
machine with multiple LLCs. It's possible to stall it quite easily with
stress-ng but I believe this is the case on main.
With the recent rework of scx_bpfland the default options for the
different profiles in scx_loader are not valid anymore.
Update them with some appropriate options.
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Add fallback DSQ cost accounting so that fallback DSQ costs are
accounted for and so that dispatch of fallback DSQs can be done in a
standardized way.
Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
The verifier error seems to stem from the wrong vmlinux.h.
Also, PR #889 seems to completely fix the problem.
So, drop the workaround.
Signed-off-by: Changwoo Min <changwoo@igalia.com>
Previously, the cur_logical_clk is updated with WIRTE_ONCE(),
which does not guarantee the atomicity when concurrent writes happen
-- which is possible. So change it using CAS (compare-and-swap).
Signed-off-by: Changwoo Min <changwoo@igalia.com>
