scx-upstream/scheds
Kumar Kartikeya Dwivedi c4c994c9ce
scx_central: Break dispatch_to_cpu loop when running out of buffer slots
For the case where many tasks being popped from the central queue cannot
be dispatched to the local DSQ of the target CPU, we will keep bouncing
them to the fallback DSQ and continue the dispatch_to_cpu loop until we
find one which can be dispatch to the local DSQ of the target CPU.

In a contrived case, it might be so that all tasks pin themselves to
CPUs != target CPU, and due to their affinity cannot be dispatched to
that CPU's local DSQ. If all of them are filling up the central queue,
then we will keep looping in the dispatch_to_cpu loop and eventually run
out of slots for the dispatch buffer. The nr_mismatched counter will
quickly rise and sched-ext will notice the error and unload the BPF
scheduler.

To remedy this, ensure that we break the dispatch_to_cpu loop when we
can no longer perform a dispatch operation. The outer loop in
central_dispatch for the central CPU should ensure the loop breaks when
we run out of these slots and schedule a self-IPI to the central core,
and allow sched-ext to consume the dispatch buffer before restarting the
dispatch loop again.

A basic way to reproduce this scenario is to do:
taskset -c 0 perf bench sched messaging

The error in the kernel will be:
sched_ext: BPF scheduler "central" errored, disabling
sched_ext: runtime error (dispatch buffer overflow)
bpf_prog_6a473147db3cec67_dispatch_to_cpu+0xc2/0x19a
bpf_prog_c9e51ba75372a829_central_dispatch+0x103/0x1a5

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2023-12-12 07:50:46 +00:00
..
c-user scx_nest: use proper format string for u64 types 2023-12-09 14:49:50 +01:00
include scheds: Rearrange include files to match kernel/tools/sched_ext/include 2023-12-03 12:47:23 -10:00
kernel-examples scx_central: Break dispatch_to_cpu loop when running out of buffer slots 2023-12-12 07:50:46 +00:00
rust-user scx_rusty: fix "subtract with overflow" error 2023-12-09 14:47:35 +01:00
meson.build build: introduce enable_rust build option 2023-12-09 15:05:23 +01:00
README.md scx_nest: Add scx_nest scheduler 2023-12-07 13:28:09 -06:00
sync-to-kernel.sh sync-to-kernel.sh: Drop unused shopt globstar 2023-12-03 15:35:15 -10:00

SCHED_EXT SCHEDULERS

Introduction

This directory contains the repo's schedulers.

Some of these schedulers are simply examples of different types of schedulers that can be built using sched_ext. They can be loaded and used to schedule on your system, but their primary purpose is to illustrate how various features of sched_ext can be used.

Other schedulers are actually performant, production-ready schedulers. That is, for the correct workload and with the correct tuning, they may be deployed in a production environment with acceptable or possibly even improved performance. Some of the examples could be improved to become production schedulers.

Please see the following README files for details on each of the various types of schedulers:

  • kernel-examples describes all of the example schedulers that are also shipped with the Linux kernel tree.
  • rust-user describes all of the schedulers with rust user space components. All of these schedulers are production ready.
  • c-user describes all of the schedulers with C user space components. All of these schedulers are production ready.

Note on syncing

Note that there is a sync-to-kernel.sh script in this directory. This is used to sync any changes to the kernel-examples/ schedulers with the Linux kernel tree. If you've made any changes to a scheduler in kernel-examples/, please use the script to synchronize with the sched_ext Linux kernel tree:

$ ./sync-to-kernel.sh /path/to/kernel/tree