Commit Graph

1316 Commits

Author SHA1 Message Date
Tejun Heo
3a688cfde7 scx_stats: Add support for no-value user attributes and a bunch of other changes
- Allow no-value user attributes which are automatically assigned "true"
  when specified.

- Make "top" attribute string "true" instead of bool true for consistency.
  Testing for existence is always enough for value-less attributes.

- Don't drop leading "_" from user attribute names when storing in dicts.
  Dropping makes things more confusing.

- Add "_om_skip" to scx_layered fields which don't jive well with OM.
  scxstats_to_openmetrics.py is updated accordignly and no longer generates
  warnings on those fields.

- Examples and README updated accordingly.
2024-08-16 07:52:02 -10:00
Tejun Heo
ee77090a2b
Merge pull request #501 from CachyOS/install/fedora
INSTALL: Update Fedora installation docs
2024-08-16 07:14:01 -10:00
Tejun Heo
e2b8525fbe
Merge pull request #504 from vax-r/fix_typo
scx_rusty: Fix typo
2024-08-16 07:13:23 -10:00
I Hsin Cheng
5d85937842 scx_rusty: Fix typo
Signed-off-by: I Hsin Cheng <richard120310@gmail.com>
2024-08-16 22:03:59 +08:00
Tejun Heo
7c924e001b
Merge pull request #503 from sched-ext/htejun/cargo-update
scheds/rust: Include Cargo.lock in the repo
2024-08-15 23:10:03 -10:00
Tejun Heo
c16b48d7b2 scheds/rust: Include Cargo.lock in the repo
Binary packages are expected to include Cargo.lock in the repo so that the
produced binaries match across different builds.
2024-08-15 23:08:35 -10:00
Tejun Heo
22167aeb14
Merge pull request #502 from sched-ext/htejun/scx_stats
scx_stats: Refine scx_stats and implement scxstats_to_openmetrics.py
2024-08-15 22:55:11 -10:00
Tejun Heo
870a262713 scx_stats: Add scripts/scxstats_to_openmetrics.py
This is a generic tool to pipe from scx_stats to OpenMetrics. This is a
barebone implmentation and the current output may not match what scx_layered
was outputting before. Will be updated later.
2024-08-15 22:51:22 -10:00
Eric Naim
a0bfd011bd
INSTALL: Fix Chaotic Nyx hyperlink
Signed-off-by: Eric Naim <dnaim@proton.me>
2024-08-16 14:53:14 +07:00
Eric Naim
91fcbb17bd
INSTALL: Update Fedora installation docs
The guide that is currently available for Fedora sched-ext is outdated. To remedy this,
I have opted to update the guide to use CachyOS's kernel that is also available on Fedora.
The scx schedulers that are available on Fedora's repositories are also outdated and doesn't work
with the current patchset. I have also updated the scheduler installation to use our package
in the CachyOS Addons COPR.

Signed-off-by: Eric Naim <dnaim@proton.me>
2024-08-16 14:53:08 +07:00
Tejun Heo
570ca56c57 scx_layered: s/_om_field_prefix/_om_prefix/ 2024-08-15 21:29:58 -10:00
Tejun Heo
af01dd19ec
Merge pull request #500 from sched-ext/htejun/scx_stats
scx_stats, scx_layered: Add `om_prefix` attribute and fix s/stat/stats/ stragglers
2024-08-15 21:27:38 -10:00
Tejun Heo
ea453e51d3 scx_stats: Rename "all" attribute to "top" and clean up examples a bit 2024-08-15 21:24:55 -10:00
Tejun Heo
a910fa451a scx_layered: Add _om attributes to LayerStats for OpenMetrics piping 2024-08-15 19:11:49 -10:00
Tejun Heo
6a5d6f7c27 scx_stats: Replace field_prefix attribute with '_' prefixed user attributes 2024-08-15 19:09:59 -10:00
Tejun Heo
f7c5a598bc scx_stats: Store ScxStatsMeta in BTreeMap instead of Vec
This makes the metadata easier to use.
2024-08-15 18:32:53 -10:00
Tejun Heo
834ce62b95 scx_stats: Fields ScxStatsMeta should be a BTreeMap not vec
Also simplify trait bound assertion.
2024-08-15 18:21:19 -10:00
Tejun Heo
f345e83c05 scx_stats: Replace om_prefix field attribute with field_prefix struct attribute
And strongly distinguish between field and struct attributes while parsing.
2024-08-15 18:05:24 -10:00
Tejun Heo
a9922deaa2 scx_stats: Add "all" attribute and rename metadata type strings 2024-08-15 14:50:00 -10:00
Tejun Heo
ebc1a89c34 scx_stats: s/stat/stats/ stragglers 2024-08-15 14:00:00 -10:00
Tejun Heo
bafd67b568 scx_stats: Fix parsing for multiple stat attributes
The code was assuming single attribute per #[stat()] block. Update it so
that there can be multiple comma separated attributes in a single block.
2024-08-15 13:46:20 -10:00
Tejun Heo
8f361af077 scx_layered: Shorten stat field descriptions 2024-08-15 13:25:48 -10:00
Tejun Heo
db0ddbe249 scx_utils: Add support for om_prefix stat attribute
This will be used to build generic bridge to openmetrics.
2024-08-15 13:08:44 -10:00
Tejun Heo
1912e05f0b
Merge pull request #499 from sched-ext/htejun/scx_stats
scx_stats: Misc changes to sync dep versions and publish on crates.io
2024-08-15 12:32:44 -10:00
Tejun Heo
ba2e0be899 scx_stats: Add .gitignore 2024-08-15 12:31:04 -10:00
Tejun Heo
0b9c8b5cbd scx_stats: Update versions to 0.2.0 to republish 2024-08-15 12:29:27 -10:00
Tejun Heo
d6ef2c2a1f scx_stats: Synchronize crate dependency versions 2024-08-15 12:28:29 -10:00
Tejun Heo
d037fcd223 scx_stats: Drop version from Cargo.toml::dev-dependencies
Otherwise, the cyclic dependency prevents publishing.
2024-08-15 12:26:17 -10:00
Tejun Heo
ec51c567fe scx_stats: Adding missing license tag in Cargo.toml's 2024-08-15 12:23:46 -10:00
Tejun Heo
cef143210a
Merge pull request #498 from sched-ext/htejun/scx_stats
scx_stats: Add package metadata and some documentation
2024-08-15 12:20:00 -10:00
Tejun Heo
47c774d011 scx_stats: Add README and cleanup examples 2024-08-15 12:17:30 -10:00
Daniel Hodges
1213d1b9d7
Merge pull request #497 from hodgesds/layer-stats-fixes
scx_layered: Update nr_cpus when resizing layers
2024-08-15 18:01:42 -04:00
Tejun Heo
616285baa8 scx_stats: Add package metadata 2024-08-15 11:09:26 -10:00
Daniel Hodges
0319afc88e scx_layered: Update nr_cpus when resizing layers
After updating scx_layered to be topology aware the nr_cpus field on the
layer was not being updated properly. Update layer growing/shrinking
logic to correctly update the nr_cpus count.

Signed-off-by: Daniel Hodges <hodges.daniel.scott@gmail.com>
2024-08-15 13:22:26 -07:00
Tejun Heo
cc73b6a826
Merge pull request #496 from sched-ext/htejun/scx_stat
scx_stat: Initial commit
2024-08-15 09:24:55 -10:00
Tejun Heo
b614cf848f scx_layered: Make monitor time based iterations dumber
This makes ctrl-c a bit more responsive without complicating code.
2024-08-15 09:23:29 -10:00
Tejun Heo
45fb724ee2 scx_layered: Restore cpumask reporting 2024-08-15 09:12:29 -10:00
Tejun Heo
751a38e34e scx_layered: Refactor stats printing code 2024-08-15 08:53:19 -10:00
Tejun Heo
a4f424056e scx_layered: Move stats server launching to stats.rs 2024-08-15 06:30:42 -10:00
Tejun Heo
17afc72479 scx_stats: Rename cleanups
- s/stat/stats/ on several stragglers.

- Rename traits so that they are more distinctive from struct and other
  names and follow the convention.
2024-08-15 06:24:56 -10:00
Tejun Heo
a091d5ea7d scx_layered: s/monitor.rs/stats.rs/ and make stats refresh code struct ops 2024-08-15 06:13:05 -10:00
Tejun Heo
8aae9a5de2 scx_stats: s/scx_stat/scx_stats/
Use plural form which is more widespread and also used in scheduler
implementations. No functional changes.
2024-08-15 05:31:34 -10:00
Changwoo Min
452838910c
Merge pull request #495 from sched-ext/htejun/lavd-build-fix
scx_lavd: Build fix
2024-08-15 11:03:41 +09:00
Tejun Heo
6e466d18df scx_layered: Initial switch to scx_stat
- This makes the scheduler side simpler and allows on-demand monitoring.

- OpenMetrics support is dropped for now. Will add a generic tool for it.

- This is a naive conversion. Will be further refined.

scx_layered no longer prints statistics by default. To watch statistics, run
`scx_layered --monitor` while the scheduler is running.
2024-08-14 13:48:41 -10:00
Tejun Heo
7820ec9b46 scx_stat, scx_layered: cargo fmt 2024-08-14 11:47:37 -10:00
Tejun Heo
03495cd37b scx_stat: Initial commit
scx_stat is a statistics reporting library with the following goals:

- Minimal boilerplate code. Statistics are defined as a regular rust struct
  which can contain integers, floats, strings, structs, and array or
  BTreeMap of them. The struct and each field can be annotated using the
  "stat" attribute. The "Stat" derive macro generates the necessary
  metadata.

- On-demand reporting. ScxStatServer implements a UNIX domain socket server
  which communicates using a simple json protocol. Stat structs can be added
  to the server with closures to output them. There can be any number of
  readers. ScxStatServer can be queried for the metadata of all stat structs
  that it may report so that a generic tool can be written to pipe the
  reported statistics to other frameworks (e.g. openmetrics).

- The interface protocol is pretty simple and it isn't difficult to interact
  directly in json. ScxStatClient is provided to further simplify client
  implementation.

- See rust/scx_stat/examples/{server,client}.rs for usage.
2024-08-14 11:37:06 -10:00
Tejun Heo
099b6c266a scx_lavd: Build fix
Add "signal" feature to nix dependency; otherwise, build fails.
2024-08-14 07:55:04 -10:00
Andrea Righi
0f018c5fff
Merge pull request #484 from vax-r/rustland_unused
scx: Remove unused variables, imports and functions
2024-08-14 19:03:26 +02:00
Andrea Righi
d2ef2fc1fb
Merge pull request #491 from sched-ext/bpfland-primary-domain
scx_bpfland: primary domain
2024-08-14 18:45:48 +02:00
Andrea Righi
f9a994412d scx_bpfland: introduce primary scheduling domain
Allow to specify a primary scheduling domain via the new command line
option `--primary-domain CPUMASK`, where CPUMASK can be a hex number of
arbitrary length, representing the CPUs assigned to the domain.

If this option is not specified the scheduler will use all the available
CPUs in the system as primary domain (no behavior change).

Otherwise, if a primary scheduling domain is defined, the scheduler will
try to dispatch tasks only to the CPUs assigned to the primary domain,
until these CPUs are saturated, at which point tasks may overflow to
other available CPUs.

This feature can be used to prioritize certain cores over others and it
can be really effective in systems with heterogeneous cores (e.g.,
hybrid systems with P-cores and E-cores).

== Example (hybrid architecture) ==

Hardware:
 - Dell Precision 5480 with 13th Gen Intel(R) Core(TM) i7-13800H
   - 6 P-cores 0..5  with 2 CPUs each (CPU from  0..11)
   - 8 E-cores 6..13 with 1 CPU  each (CPU from 12..19)

== Test ==

WebGL application (https://webglsamples.org/aquarium/aquarium.html):
this allows to generate a steady workload in the system without
over-saturating the CPUs.

Use different scheduler configurations:

 - EEVDF (default)
 - scx_bpfland using P-cores only (--primary-domain 0x00fff)
 - scx_bpfland using E-cores only (--primary-domain 0xff000)

Measure performance (fps) and power consumption (W).

== Result ==

                  +-----+-----+------+-----+----------+
                  | min | max | avg  |       |        |
                  | fps | fps | fps  | stdev | power  |
+-----------------+-----+-----+------+-------+--------+
| EEVDF           | 28  | 34  | 31.0 |  1.73 |  3.5W  |
| bpfland-p-cores | 33  | 34  | 33.5 |  0.29 |  3.5W  |
| bpfland-e-cores | 25  | 26  | 25.5 |  0.29 |  2.2W  |
+-----------------+-----+-----+------+-------+--------+

Using a primary scheduling domain of only P-cores with scx_bpfland
allows to achieve a more stable and predictable level of performance,
with an average of 33.5 fps and an error of ±0.5 fps.

In contrast, using EEVDF results in an average frame rate of 31.0 fps
with an error of ±3.0 fps, indicating slightly less consistency, due to
the fact that tasks are evenly distributed across all the cores in the
system (both slow and fast cores).

On the other hand, using a scheduling domain solely of E-cores with
scx_bpfland results in a lower average frame rate (25.5 fps), though it
maintains a stable performance (error of ±0.5 fps), but the power
consumption is also reduced, averaging 2.2W, compared to 3.5W with
either of the other configurations.

== Conclusion ==

In summary, with this change users have the flexibility to prioritize
scheduling on performance cores for better performance and consistency,
or prioritize energy efficient cores for reduced power consumption, on
hybrid architectures.

Moreover, this feature can also be used to minimize the number of cores
used by the scheduler, until they reach full capacity. This capability
can be useful for reducing power consumption even in homogeneous systems
or for conducting scheduling experiments with smaller sets of cores,
provided the system is not overcommitted.

Signed-off-by: Andrea Righi <andrea.righi@linux.dev>
2024-08-14 16:17:54 +02:00