Commit Graph

233 Commits

Author SHA1 Message Date
Changwoo Min
3c6d86b342 scx_lavd: upgrade nix package from 0.28.0 to 0.29.0
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-14 22:31:05 +09:00
Changwoo Min
b7b8c8de90 scx_lavd: fix build errors
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 14:10:40 +09:00
Changwoo Min
182b0bd249 scx_lavd: make the verifier in 6.8 kernel happy
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:04:04 +09:00
Changwoo Min
4ecf3fc94e scx_lavd: build cpdom map from rust
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:03:18 +09:00
Changwoo Min
1f1a3dc4f1 scx_lavd: sort cores in descending order of max freq
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
c213a3e44f scx_lavd: make core compaction core type aware
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
c35b6b27ff scx_lavd: consider task pinning for core-type-aware ops.enqueue()
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
25bf98d2a0 scx_lavd: make ops.select_cpu() core type aware
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
fa87e1c593 scx_lavd: make ops.dispatch() core type aware
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
c1cf11f7b1 scx_lavd: make ops.enqueue() core type aware
Put a performance-critical task to a performance critical queue and a
regular task to a regular queue.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
03a8c10ece scx_lavd: add cpdom_ctx to abstract compute domain and its DSQ
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
623b05a282 scx_lavd: revise perf_cri factor to reflect wakeup, runtime, and run_freq
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
15871fd032 scx_lavd: turn off pinned core less aggressively
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
9dc7f94cb6 scx_lavd: unifiy the deadline calculation and ineligibility calculation
The unified version is not only simpler but also works better.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:40 +09:00
Changwoo Min
4705520d40 scx_lavd: remove unnecessary options which has never been used
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-12 13:01:34 +09:00
Tejun Heo
63c4a0191f
Merge branch 'main' into topic/inlined-skeleton-members 2024-08-08 14:23:37 -10:00
Tejun Heo
cd6a4d72c7 Bump versions for 1.0.2 release 2024-08-08 14:10:16 -10:00
Tejun Heo
7c3ffe96e1 Unify crate dependency versions
Different sub-projects are using different versions for the same crates.
Synchronize them to the latest.
2024-08-08 13:26:47 -10:00
Tejun Heo
b226865b96 scx_lavd: Make FlatTopology::new() a bit prettier
- Use .enumerate() consistently while building the cpu_fids vector.

- Use .then_with() to chain .cmp() when sorting cpu_fids.

Both reduce visual clutter.
2024-08-04 11:16:19 -10:00
Changwoo Min
f3fd6e9cb3 scx_lavd: drop 2-level-scheduling
With optimizations of calculatring ineligibility duration, now the
scheduler works well under heavy load without 2-level scheduling, so we
drop it for simplicitiy.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-02 21:46:07 +09:00
Changwoo Min
c38e749c36 scx_lavd: improve the equation for calculating ineligibility duration
This commit include a few changes:
- treat a new forked task more conservatively
- defer the execution of more tasks for longer time using ineligibility duration
- consider if a task is waken up in calculating ineligibility duration
2024-08-02 21:08:29 +09:00
Changwoo Min
5e194330f0 scx_lavd: consider task's wakeup and vruntime (starvation) more aggressively
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-08-02 12:25:29 +09:00
Changwoo Min
fc0ffeb45b scx_lavd: print the overall status of a scheduled task
L or R: Latency-critical, Regular
H or I: performance-Hungry, performance-Insensitive
B or T: Big, liTtle
E or G: Eligible, Greedy
P or N: Preemption, Not

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-31 19:00:35 +09:00
Changwoo Min
22d4b13e8e scx_lavd: classify CPUs into BIG and little ones based on their average capacity
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-31 19:00:35 +09:00
Changwoo Min
0ad2f30fa8
Merge pull request #460 from multics69/lavd-misc
scx_lavd: misc updates
2024-07-31 08:55:04 +09:00
Changwoo Min
9b455cf010
Merge pull request #458 from sched-ext/lavd-fix-cpu-ctx-size
scx_lavd: set correct size for cpu_ctx_stor
2024-07-31 00:39:13 +09:00
Changwoo Min
6136cbee65 scx_lavd: tuning the time slice and preemption margins
Tuning the time slice under high load and change the kick/tick margins
for preemption more conservative. Especially, aggressive IPI-based
preemption (kick) causes performance unstability.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-31 00:30:59 +09:00
Changwoo Min
35b0d9f3c2 scx_lavd: improve starvation factor equation
Instead of using coarse-grained log(), let's directly use the ratio of
task's service time. Also, the virtual dealine equation is also updated
to reflect this change.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-31 00:27:17 +09:00
Changwoo Min
f9657a549f scx_lavd: fix bpf verification error in old kernel versions
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-31 00:22:43 +09:00
Changwoo Min
d2615b4975 scx_lavd: fix warnings from the rust code
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-31 00:21:32 +09:00
Andrea Righi
2015faa745 scx_lavd: set correct size for cpu_ctx_stor
The max_entries parameter in BPF_MAP_TYPE_PERCPU_ARRAY defines the
number of values per CPU and for cpu_ctx_stor we only need one item: the
CPU context.

Set max_entries to 1 to avoid allocating unnecessary memory and slightly
reduce the memory footprint.

Signed-off-by: Andrea Righi <righi.andrea@gmail.com>
2024-07-30 09:32:55 +02:00
Changwoo Min
643edb5431
Merge pull request #457 from multics69/lavd-amp-v2
scx_lavd: support two-level scheduling for heavy-loaded cases (like bpfland)
2024-07-30 10:39:06 +09:00
Changwoo Min
b91c1e4759 scx_lavd: add more comments on no_2_level_scheduling implementation
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-29 12:22:28 +09:00
Changwoo Min
f71fff9bbe scx_lavd: print a warning message when system does not provide a proper freq info
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-28 15:53:02 +09:00
Changwoo Min
4449d8e31c scx_lavd: incorporate a task's static priority in calculating its latency criticality
That's because static (nice) priority is a strong hint to distinguish
latency-critical tasks.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-28 15:41:43 +09:00
Changwoo Min
221f1fe12a scx_lavd: further prioritize producers over consumers
That is because many latency-critical tasks are producers.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-28 15:38:54 +09:00
Changwoo Min
7106e8cdca scx_lavd: support two-level scheduling for heavy-loaded cases
We introduce two-level scheduling similar to scx_bpfland. The two-level
scheduling consists of two DSQs: 1) latency-critical run queue and 2)
regular run queue. The scheduler prioritizes scheduling tasks on the
latency-critical queue but makes its best effort to schedule tasks on
the regular queue. The scheduler could be more resilient under heavy
load by segregating regular, non-latency-critical tasks from
latency-critical tasks.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-28 15:33:17 +09:00
Changwoo Min
9236c3e57c scx_lavd: increase the targeted latency for heavy loaded cases
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-28 15:30:01 +09:00
Changwoo Min
230512208d scx_lavd: fix div by zero error in some installations
The max frequency information from topology (from sysfs) seems not
always true. In some installations, it returns zero for all CPUs. In
this case, let's just consider all CPUs have the same capacity (1024),
hoping the kernel can give more preceise information.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-28 12:47:00 +09:00
Changwoo Min
59e54f4972 scx_lavd: print how to disable logging
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-28 12:31:51 +09:00
Changwoo Min
df1108ec6c scx_lavd: segregate starvation factor from the latency criticality (refactoring)
Latency criticality is a task's inherent property, but the starvation
factor is its dynamic status for the urgency of scheduling. Hence, we
segregate the starvation factor out. Also, cleaned up unnecessary
arguments and struct fields related.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-27 17:25:39 +09:00
Changwoo Min
eeea847697 scx_lavd: adjust time slice based on CPU's capacity
When a task is running on more performant core, the scheduler will give
a longer time slice. On the other hand, on a less performant core, a
shorter time slice will be assigned. The longer time slice helps
boosting clock frequency on a performant core. Also, the shorter time
slice gives more chance the performant core being utilized.

Regarding the CPU capacity, we first check if kernel-provided capacitiy values
are trustworthy or not. If not (i.e., all the same values), we rely on
the user-provided value, based on each CPU's maximum clock frequency.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-26 18:46:21 +09:00
Changwoo Min
e7b6ed1838 scx_lavd: add --prefer-smt-core option
With the --prefer-smt-core option is on, the core compaction prefers to
 utilizae hyper-twin first before utilizing the other physical CPUs. By
 default, the option is off.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-26 18:46:21 +09:00
Changwoo Min
19e337cd9b scx_lavd: make the core compaction AMP-aware
Previously, the core compaction assumed that each core's capacity was
the same. Now, we additionally consider each core's max clock frequency.
So, it always tries to use the higher-frequency cores first.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-26 18:46:21 +09:00
Changwoo Min
dbb3957eb1 scx_lavd: add a missing no_freq_scaling option check
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-26 18:46:21 +09:00
Changwoo Min
90b57a3fd7 scx_lavd: put a pinned kernel task to an overflow set
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-26 18:46:21 +09:00
Changwoo Min
e76bf999df scx_lavd: clean up constants (no functional changes)
Remove unused constants and rename outdated constants to proper names
(LAVD_TC_* to LAVC_CC_* and LAVD_ELIGIBLE_DSQ to LAVD_GLOBAL_DSQ).

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-26 18:46:21 +09:00
Changwoo Min
a9aab6b229 scx_lavd: fix typo
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-21 17:58:44 +09:00
Changwoo Min
add96f0e18 scx_lavd: do not maintain ineligible runnable tasks separately
With all the other optimizations and tunings, it turns out that maintaining
two runqueues has more harm than good.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-20 17:49:12 +09:00
Changwoo Min
827187d213 scx_lavd: adjust ineligible duration according to task's lat_cri
Further depenalize above-average latency-critical tasks and penalize
further below-avergage latency-critical tasks in ineligibility duration.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-20 17:37:27 +09:00
Changwoo Min
c653622ed9 scx_lavd: add LAVD_VDL_LOOSENESS_FT in calculating virtual deadline
LAVD_VDL_LOOSENESS_FT represents how loose the deadline is. The smaller
value means the deadline is tighter. While it is unlikely to be tuned,
let's keep it as a tunable for now.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-20 12:00:50 +09:00
Changwoo Min
e94070d5ca scx_lavd: remove LAVD_BOOST_*
These are no longer necessary after directly using latency criticality.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-20 11:53:20 +09:00
Changwoo Min
43f0fcb87c scx_lavd: removed unused LAVD_LOAD_FACTOR_*
These are no longer necessary after remnoving load factor calculation.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-20 11:51:12 +09:00
Changwoo Min
3924ebaa4d scx_lavd: properly synchronize taskc->vdeadline_log_clk
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-20 01:41:29 +09:00
Changwoo Min
02ad43d116 scx_lavd: directly use p->scx.weight instead load_ideal
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-20 00:25:11 +09:00
Changwoo Min
c955caefd8 scx_lavd: drop sys_load_factor
In theory, sys_load_factor should not be necessary since we do not
stretch the time space anymore.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-20 00:10:29 +09:00
Changwoo Min
67a6deb983 scx_lavd: use lat_cri instead of lat_prio universally
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-19 23:56:51 +09:00
Changwoo Min
6f10d6907c scx_lavd: drop sched_prio_to_slice_weight[] table
Use p->scx.weight instead.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-19 22:39:01 +09:00
Changwoo Min
034303f00f scx_lavd: consider starvation factor in determining latency criticality
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-19 22:17:50 +09:00
Changwoo Min
99e0d21c3c scx_lavd: drop the runtime factor in calculating latency criticality
That is okay since the runtime is considered in calculating a virtual
deadline. A shorter runtime will result in a tighter deadline linearly.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-19 17:28:40 +09:00
Changwoo Min
b90599e967 scx_lavd: do not inherit parent's properties
If inheriting the parent's properties, a new fork task tends to be too
prioritized. That is, many parent processes, such as `make,` are a bit
more latency-critical than average.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-19 15:29:13 +09:00
Changwoo Min
78d96a6fb6 scx_lavd: advance clock by reverse proportional to the system load
Advancing the clock slower when overloaded gives more opportunities for
latency-critical tasks to cut in the run queue. Controlling the clock
better reflects the actual load than the prior approach of stretching
the time-space when overloaded.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-18 15:53:38 +09:00
Changwoo Min
9bc20f9160 scx_lavd: maintain ineligible runnable tasks separately
We now maintain two run queues—an eligible run queue (DSQ) and an
ineligible run queue (rbtree)—sorted by the task's virtual deadline.
When the eligible run queue is empty, or the ineligible run queue has
not been consumed for too long (e.g., 15 msec), a task in the ineligible
run queue is moved to the eligible run queue for execution. With these
two queues, we have a better admission control.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-17 23:46:11 +09:00
Changwoo Min
55e19ea5df scx_lavd: do not prioritize a wake-up task in ops.select_cpu()
This is a prep for adding an ineligible DSQ.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-17 11:16:02 +09:00
Changwoo Min
c84b73e971 scx_lavd: rename LAVD_GLOBAL_DSQ to LAVD_ELIGIBLE_DSQ
This is a prep to add a global ineligible dsq.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-17 10:34:34 +09:00
Daniel Müller
565aec3662 rust: Update libbpf-rs & libbpf-cargo to 0.24
Update libbpf-rs & libbpf-cargo to 0.24. Among other things, generated
skeletons now contain directly accessible map and program objects, no
longer necessitating the use of accessor methods. As a result, the risk
for mutability conflicts is reduced greatly.

Signed-off-by: Daniel Müller <deso@posteo.net>
2024-07-16 11:48:52 -07:00
Changwoo Min
971bb2e024 scx_lavd: pretty formatting for ineligible duration
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-16 23:54:15 +09:00
Changwoo Min
adfbf3934c scx_lavd: tuning the max ineligible duration
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-16 23:52:23 +09:00
Changwoo Min
eff444516f scx_lavd: directly measure service time for eligibility enforcement
Estimating the service time from run time and frequency is not
incorrect. However, it reacts slowly to sudden changes since it relies
on the moving average. Hence, we directly measure the service time to
enforce fairness.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-16 23:48:26 +09:00
Tejun Heo
51334b5c4d Bump versions for 1.0.1 release 2024-07-15 13:21:52 -10:00
Tejun Heo
3ae76acd12
Merge pull request #424 from sched-ext/sync-upstream-kernel-and-bump-to-1.0
Sync to upstream kernel and bump to 1.0
2024-07-14 07:00:38 -10:00
Tejun Heo
761ec142ce Bump most versions to 1.0.0
sched_ext is about to be merged upstream. There are some compatibility
breaking changes and we're making the current sched_ext/for-6.11
1edab907b57d ("sched_ext/scx_qmap: Pick idle CPU for direct dispatch on
!wakeup enqueues") the baseline.

Tag everything except scx_mitosis as 1.0.0. As scx_mitosis is still in early
development and is currently temporarily disabled, only the patchlevel is
bumped.
2024-07-12 11:34:14 -10:00
Tejun Heo
f261d0f037 Sync from kernel - 1edab907b57d
Sync from sched_ext/for-6.11 1edab907b57d ("sched_ext/scx_qmap: Pick idle
CPU for direct dispatch on !wakeup enqueues")

 git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git for-6.11

- cgroup support hasn't landed in the upstream kernel yet. This most likely
  will happen in a few weeks. For the time being, disable scx_flatcg,
  scx_pair and scx_mitosis.

- Compat macro for DSQ task iterator dropped. This is now a part of
  the baseline.

- scx_bpf_consume() isn't upstream yet. BPF interfacing side is still being
  discussed. Dropped example usage from tools/sched_ext. None of the
  practical schedulers use it, so this should be fine for now.

- scx_bpf_cpu_rq() added.

- AUTOATTACH workaround for newer libbpf versions added.
2024-07-12 11:08:41 -10:00
Changwoo Min
512bd143a5 scx_lavd: count only related tasks in calculating waker_freq
A task can become a runnable on any task's context not only its waker
task.  Thus, we should not count wake-up on unrelated task's context.
With this commit, the scheduler can (much more) accurately detect
waker-wakee relationsships.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-12 22:51:09 +09:00
Changwoo Min
95733f63ab scx_lavd: calculate time slice as a function of run queue length
The prior approach using the sum of weights gives too much penalty to
nice tasks with large nice values. With this commit, the time slice is
determined by the number of runnable tasks regardless of nice priority.
Note that the fairness will still be enforced based on tasks' nice
priorities (weights).

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-12 22:45:22 +09:00
Changwoo Min
d4bc92bea7 scx_lavd: print lat_cri to output
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-12 13:23:56 +09:00
Changwoo Min
4c5c564523 scx_lavd: initial current logical clock to zero
To easily distinguish, let's initialize the current logical clock to
zero (not the current physical time). Also, avoid the deadline
calculation being zero by adding +1 here and there.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-12 10:15:54 +09:00
Changwoo Min
bdbfeb9fd1 scx_lavd: use logical current clock for virtual deadlines
This commit changes the use of a physical clock to a virtual, logical
clock in calculating deadlines.

- The virtual current clock advances upon a task's running to its
  virtual deadline.

- When enqueuing a task, its virtual deadline from the virtual current
  clock is calculated.

With the above two changes, this guarantees that there is no such task
whose virtual deadline is smaller than the virtual current clock. This
means any enqueuing task can compete with any other already enqueued
tasks. This allows a latency-critical task to be immediately scheduled
if needed.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-11 22:41:56 +09:00
Changwoo Min
408ea7892c scx_lavd: induce sched_prio_to_latency_weight from slice weight
So sched_prio_to_latency_weight is removed.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-11 21:37:21 +09:00
Changwoo Min
bd964acff6 scx_lavd: deprioritize a newly forked task in latency
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-11 21:36:32 +09:00
Changwoo Min
48debe416e scx_lavd: tuning the deadline equation under high load
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-11 21:35:54 +09:00
Changwoo Min
c72e063680 scx_lavd: do not use lat_prio_to_greedy_thresholds
With other optimizations, lat_prio_to_greedy_thresholds is not effective
any more.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-11 21:35:01 +09:00
Changwoo Min
9ed488798e scx_lavd: use task's runtime to determine its deaddline
It has an effect of further perferring shorter jobs.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-11 21:34:25 +09:00
Changwoo Min
e081b2a294 scx_lavd: rename LAVD_MAX_CAS_RETRY to LAVD_MAX_RETRY
Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-07-11 21:33:56 +09:00
Andrea Righi
cf4883fbf8 meson: introduce serialize build option
With commit 5d20f89a ("scheds-rust: build rust schedulers in sequence"),
schedulers are now built serially one after the other to prevent meson
and cargo from forking NxN parallel tasks.

However, this change has made building a single scheduler much more
cumbersome, due to the chain of dependencies.

For example, building scx_rusty using the specific meson target would
still result in all schedulers being built, because they all depend on
each other.

To address this issue, introduce the new meson build option
`serialize=true|false` (default is false).

This option allows to disable the schedulers' build chain, restoring the
old behavior.

With this option enabled, it is now possible to build just a single
scheduler, parallelizing the cargo build properly, without triggering
the build of the others. Example:

  $ meson setup build -Dbuildtype=release -Dserialize=false
  $ meson compile -C build scx_rusty

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
2024-06-28 10:17:37 +02:00
Changwoo Min
24a238846e scx_lavd: optimizing deadline related tunables
The competition window was 7.5 msec, half of the targeted latency.
However, it is too wide for some workloads, so unrelated tasks may
compete with each other. Hence, it is tightened to about 1 msec with
LAVD_LAT_WEIGHT_SHIFT to avoid unnecessary competition.

Also, when a system is overloaded, now the time space is stretched more
aggressively (i.e., lat_prio^2) when a task's latency priority is low
(high value).

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-06-28 09:00:45 +09:00
Changwoo Min
f86d564d89 scx_lavd: fast path for ops.dispatch() when fully loaded
When fully loaded so all CPUs are using, skip checking the cpumask.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-06-27 18:00:39 +09:00
Changwoo Min
abc6e31fef scx_lavd: for a forked task, inherit its parent's statistics
The old approach was too conservative in running a new task, so when a
fork-heavy workload competes with a CPU-bound workload, the fork-heavy
one is starved. The new approach solves the starvation problem by
inheriting parent's statistics. It seems a good (at least better than
old) guess how a new task will behave.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-06-26 19:00:10 +09:00
Changwoo Min
ac9c49f5b5 scx_lavd: loosen the deadline when overloaded
When the system is highly loaded with compute-intensive tasks, the old
setting chokes latensive-intensive tasks, so loosen the dealine when the
system is overloaded (> 100% utilization).

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-06-26 15:06:31 +09:00
Changwoo Min
b32734168b scx_lavd: print build ID when lavd is loaded
When the lavd is loaded, it prints out its build id. It helps to easily
identify what version it is when testing.

```
01:56:54 [INFO] scx_lavd scheduler is initialized (build ID: 0.8.1-g98a5fa8595430414115c504857cea1a458393838-dirty x86_64-unknown-linux-gnu)
```

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-06-26 10:57:19 +09:00
Changwoo Min
5d0db5c5fe scx_lavd: revising tunables to reduce micro-stutters
This is a second attempt to optimize tunables for a wider range of
games.

1) LAVD_BOOST_RANGE increased from 14 (35%) to 40 (100% of nice range).
   Now the latency priority (biased by nice value) will decide which
   task should run first . The nice value will decide the time slice.

2) The first change will give higher priority to latency-critical task
   compared to before. For compensation, the slice boost also increased
   (2x -> 3x).

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-06-25 16:13:32 +09:00
Changwoo Min
016229cbcf scx_lavd: revising tunables for less-preemptive games
In some games (e.g., Elden Ring), it was observed that preemption
happens much less frequently. The reason is that tasks' runtime per
schedule is similar, so it does not meet the existing criteria. To
alleviate the problem, the following three tunables are revised:

1) Smaller LAVD_PREEMPT_KICK_MARGIN and LAVD_PREEMPT_TICK_MARGIN help to
   trigger more preemption.

2) Smaller LAVD_SLICE_MAX_NS works better especially 250 or 300Hz
   kernels.

3) Longer LAVD_ELIGIBLE_TIME_MAX purturbes time lines less frequently.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-06-24 00:27:33 +09:00
Changwoo Min
9c21ace276
Merge pull request #373 from vax-r/lavd_reuse
scx_lavd: Reuse can_task1_kick_task2
2024-06-19 15:29:05 +09:00
I Hsin Cheng
99960ad960 scx_lavd: Reuse can_task1_kick_task2
Use the function can_task1_kick_task2() to replace places which also
checking the comp_preemption_info between two cpus for better
consistency.

Signed-off-by: I Hsin Cheng <richard120310@gmail.com>
2024-06-19 11:01:31 +08:00
Changwoo Min
691869e83f
Merge pull request #369 from sched-ext/lavd-fix-pick-cpu
scx_lavd: properly check for idle CPUs in pick_cpu()
2024-06-19 09:23:17 +09:00
Andrea Righi
bad9ed13ef scx_lavd: properly check for idle CPUs in pick_cpu()
It seems that we are not updating `is_idle` when we find an idle CPU
with pick_cpu(), causing unnecessary rescheduling events when
select_cpu() is called.

To resolve this, ensure that the is_idle state is correctly set.
Additionally, always ensure that the task is dispatched to the local DSQ
immediately upon finding (and reserving) an idle CPU.

Signed-off-by: Andrea Righi <andrea.righi@canonical.com>
2024-06-18 17:36:39 +02:00
Changwoo Min
632fa9e4f2 scx_lavd: misc code clean up
- clean up u63 and u32 usages in structures to reduce struct size
- refactoring pick_cpu() for readability

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-06-18 18:11:49 +09:00
Changwoo Min
5165bf5a03 scx_lavd: tuning CPU frequency scaling
The required CPU performance (cpuperf) was set to 1024 (100%) when the
CPU utilization was 100%. When a sudden load spike happens, it makes the
system adapt slowly in the next interval.

The new scheme always reserves some headroom in advance, so it sets
cpuperf to 1024 when the CPU utilization reaches to 85%. This gives some
room to adapt in advance.

Signed-off-by: Changwoo Min <changwoo@igalia.com>
2024-06-18 18:11:49 +09:00
Tejun Heo
5b5e5be906 compat: Drop __COMPAT_SCX_KICK_IDLE
In preparation of upstreaming, let's set the min version requirement at the
released v6.9 kernels. Drop __COMPAT_SCX_KICK_IDLE. The open helper macros
now check the existence of SCX_KICK_IDLE and abort if not.
2024-06-15 20:24:15 -10:00
Tejun Heo
7c9aedaefe compat: Drop __COMPAT_scx_bpf_switch_all()
In preparation of upstreaming, let's set the min version requirement at the
released v6.9 kernels. Drop __COMPAT_scx_bpf_switch_call(). The open helper
macros now check the existence of SCX_OPS_SWITCH_PARTIAL and abort if not.
2024-06-15 20:03:37 -10:00