Commit Graph

226 Commits

Author SHA1 Message Date
Tejun Heo
6a2faf2e17
Merge pull request #530 from sched-ext/htejun/misc
scx_utils::topology: Use lazy_static instead of LazyLock
2024-08-21 06:04:14 -10:00
Andrea Righi
9f7a11bba6
Merge pull request #528 from sched-ext/bpfland-turbo-boost
scx_bpfland: properly classify Intel Turbo Boost CPUs
2024-08-21 17:40:25 +02:00
Tejun Heo
081b999361 scx_utils::topology: Use lazy_static instead of LazyLock
LazyLock is stable but has become so only very recently and can trigger
build errors on not-too-old stable rustc's which are still in wide use.
Let's use lazy_static instead for now.

Signed-off-by: Tejun Heo <tj@kernel.org>
2024-08-21 05:34:39 -10:00
Andrea Righi
bbe388e3bc scx_utils: topology: add base_freq() method to Cpu
With Intel Turbo Boost enabled, some CPUs might show a higher maximum
frequency than others, even if they are not actually faster cores. This
can potentially confuse some auto-detection logic for distinguishing
between fast and slow cores in certain schedulers.

The base CPU frequency reported in
/sys/devices/system/cpu/cpuN/cpufreq/base_frequency represents a more
reliable indicator for identifying truly fast and slow cores.

To address this, provide a new base_freq() method in the struct Cpu,
which will return the base operational frequency of a CPU when Turbo
Boost is present. If Turbo Boost is not available, base_freq() will
return the maximum frequency, functioning the same as max_freq().

Signed-off-by: Andrea Righi <andrea.righi@linux.dev>
2024-08-21 10:07:57 +02:00
Tejun Heo
1da249f063 scx_utils::topology: Always use NR_CPU_IDS and NR_CPUS_POSSIBLE
Always use the LazyLock versions and drop the counterparts from Topology.
2024-08-20 21:57:56 -10:00
Tejun Heo
1ae4655b3c scx_utils::cpumask: Default to displaying in hex
There isn't much to gain by displaying cpumasks in binary. Drop separate
Display implementation just default to 'x' formatting.
2024-08-20 21:50:23 -10:00
Tejun Heo
3ca2f0b6f9 scx_utils/cpumask: Use nr_cpu_ids instead of num_possible_cpus
- Add static NR_CPU_IDS and NR_CPUS_POSSIBLE to topology.

- Fix comment for Topology::nr_cpu_ids(). Was missing a negation.

- cpumaks should be sized by nr_cpus_ids, not num_possible_cpus and the
  number can't change while the system is running. Drop cpumask.nr_cpus and
  use *NR_CPU_IDS everywhere.
2024-08-20 21:25:40 -10:00
Tejun Heo
0cc59a5243 scx_utils: cargo fmt 2024-08-20 21:25:40 -10:00
Tejun Heo
f7c193e528 scx_utils, scx_rusty: Minor updates to version handling
- Update scx_utils/build.rs so that 12 char SHA1 is generated instead of
  full one.

- Add --version to scx_rusty. Use custom one as we don't want to use the
  default cargo version one.
2024-08-20 21:03:05 -10:00
Andrea Righi
235f19fdf1 cpumask: implement hex string formatter
Allow to format a Cpumask as an hex string, implementing the proper
formatter LowerHex / UpperHex traits.

Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Andrea Righi <andrea.righi@linux.dev>
2024-08-21 07:21:22 +02:00
Andrea Righi
33b6ada98e
Merge pull request #509 from sched-ext/bpfland-topology
scx_bpfland: topology awareness
2024-08-20 14:37:23 +02:00
Andrea Righi
0b2dc6b9fc scx_utils: Add L2 / L3 cache id to CPU
Add the L2 / L3 cache id to the Cpu struct, to quickly determine the
cache nodes associated to each CPU.

Signed-off-by: Andrea Righi <andrea.righi@linux.dev>
2024-08-20 10:16:02 +02:00
Tejun Heo
4e859d067e scx_stats/scripts/scxstats_to_openmetrics: Retry connection
It now retries until told to exit. This is a bit easier to use and matches
`scx_layered --monitor`.
2024-08-19 12:52:57 -10:00
Tejun Heo
6cba8d786a scx_stats: server: open_ops must be kept throughout a client session
open_ops tracks which ops have been opened by the client session; however,
it was being created on each handle_request() making every request to open
each time. Fix it by moving it to the caller.
2024-08-19 08:38:13 -10:00
Tejun Heo
17a460c179 scx_stats: ScxStatsOps fields must be public 2024-08-19 07:51:05 -10:00
Tejun Heo
27c530e17e scx_stats: Add missing trait exports 2024-08-19 07:16:43 -10:00
Tejun Heo
1e89184ba7 scx_stats: server: s/Tx/Req/ and s/Rx/Res/ for clarity 2024-08-19 07:11:26 -10:00
Tejun Heo
4d88c9aec7 scx_stats: Add channel arguments to open and close ops too 2024-08-19 06:56:14 -10:00
Tejun Heo
a77fe372d6 scx_stats: Make server shutdown when connection is dropped and add communication channel
This will make implementing connection sessions easier where each stats
client connection maintains a set of states.
2024-08-19 06:23:16 -10:00
Tejun Heo
689d380db1 scx_stats: Implement ScxStatsServer::add_stats_ops()
This allows stats reader to maintain persistent states per connection.
2024-08-16 13:24:17 -10:00
Tejun Heo
da2e014d15 scx_stats: Add more documentation on user attributes in README 2024-08-16 09:11:23 -10:00
Tejun Heo
3a688cfde7 scx_stats: Add support for no-value user attributes and a bunch of other changes
- Allow no-value user attributes which are automatically assigned "true"
  when specified.

- Make "top" attribute string "true" instead of bool true for consistency.
  Testing for existence is always enough for value-less attributes.

- Don't drop leading "_" from user attribute names when storing in dicts.
  Dropping makes things more confusing.

- Add "_om_skip" to scx_layered fields which don't jive well with OM.
  scxstats_to_openmetrics.py is updated accordignly and no longer generates
  warnings on those fields.

- Examples and README updated accordingly.
2024-08-16 07:52:02 -10:00
Tejun Heo
870a262713 scx_stats: Add scripts/scxstats_to_openmetrics.py
This is a generic tool to pipe from scx_stats to OpenMetrics. This is a
barebone implmentation and the current output may not match what scx_layered
was outputting before. Will be updated later.
2024-08-15 22:51:22 -10:00
Tejun Heo
ea453e51d3 scx_stats: Rename "all" attribute to "top" and clean up examples a bit 2024-08-15 21:24:55 -10:00
Tejun Heo
6a5d6f7c27 scx_stats: Replace field_prefix attribute with '_' prefixed user attributes 2024-08-15 19:09:59 -10:00
Tejun Heo
f7c5a598bc scx_stats: Store ScxStatsMeta in BTreeMap instead of Vec
This makes the metadata easier to use.
2024-08-15 18:32:53 -10:00
Tejun Heo
834ce62b95 scx_stats: Fields ScxStatsMeta should be a BTreeMap not vec
Also simplify trait bound assertion.
2024-08-15 18:21:19 -10:00
Tejun Heo
f345e83c05 scx_stats: Replace om_prefix field attribute with field_prefix struct attribute
And strongly distinguish between field and struct attributes while parsing.
2024-08-15 18:05:24 -10:00
Tejun Heo
a9922deaa2 scx_stats: Add "all" attribute and rename metadata type strings 2024-08-15 14:50:00 -10:00
Tejun Heo
ebc1a89c34 scx_stats: s/stat/stats/ stragglers 2024-08-15 14:00:00 -10:00
Tejun Heo
bafd67b568 scx_stats: Fix parsing for multiple stat attributes
The code was assuming single attribute per #[stat()] block. Update it so
that there can be multiple comma separated attributes in a single block.
2024-08-15 13:46:20 -10:00
Tejun Heo
db0ddbe249 scx_utils: Add support for om_prefix stat attribute
This will be used to build generic bridge to openmetrics.
2024-08-15 13:08:44 -10:00
Tejun Heo
ba2e0be899 scx_stats: Add .gitignore 2024-08-15 12:31:04 -10:00
Tejun Heo
0b9c8b5cbd scx_stats: Update versions to 0.2.0 to republish 2024-08-15 12:29:27 -10:00
Tejun Heo
d6ef2c2a1f scx_stats: Synchronize crate dependency versions 2024-08-15 12:28:29 -10:00
Tejun Heo
d037fcd223 scx_stats: Drop version from Cargo.toml::dev-dependencies
Otherwise, the cyclic dependency prevents publishing.
2024-08-15 12:26:17 -10:00
Tejun Heo
ec51c567fe scx_stats: Adding missing license tag in Cargo.toml's 2024-08-15 12:23:46 -10:00
Tejun Heo
47c774d011 scx_stats: Add README and cleanup examples 2024-08-15 12:17:30 -10:00
Tejun Heo
616285baa8 scx_stats: Add package metadata 2024-08-15 11:09:26 -10:00
Tejun Heo
17afc72479 scx_stats: Rename cleanups
- s/stat/stats/ on several stragglers.

- Rename traits so that they are more distinctive from struct and other
  names and follow the convention.
2024-08-15 06:24:56 -10:00
Tejun Heo
8aae9a5de2 scx_stats: s/scx_stat/scx_stats/
Use plural form which is more widespread and also used in scheduler
implementations. No functional changes.
2024-08-15 05:31:34 -10:00
Tejun Heo
7820ec9b46 scx_stat, scx_layered: cargo fmt 2024-08-14 11:47:37 -10:00
Tejun Heo
03495cd37b scx_stat: Initial commit
scx_stat is a statistics reporting library with the following goals:

- Minimal boilerplate code. Statistics are defined as a regular rust struct
  which can contain integers, floats, strings, structs, and array or
  BTreeMap of them. The struct and each field can be annotated using the
  "stat" attribute. The "Stat" derive macro generates the necessary
  metadata.

- On-demand reporting. ScxStatServer implements a UNIX domain socket server
  which communicates using a simple json protocol. Stat structs can be added
  to the server with closures to output them. There can be any number of
  readers. ScxStatServer can be queried for the metadata of all stat structs
  that it may report so that a generic tool can be written to pipe the
  reported statistics to other frameworks (e.g. openmetrics).

- The interface protocol is pretty simple and it isn't difficult to interact
  directly in json. ScxStatClient is provided to further simplify client
  implementation.

- See rust/scx_stat/examples/{server,client}.rs for usage.
2024-08-14 11:37:06 -10:00
Tejun Heo
45f7fd13b7 versions: Synchronize crate dependency versions 2024-08-08 14:45:46 -10:00
Tejun Heo
63c4a0191f
Merge branch 'main' into topic/inlined-skeleton-members 2024-08-08 14:23:37 -10:00
Tejun Heo
cd6a4d72c7 Bump versions for 1.0.2 release 2024-08-08 14:10:16 -10:00
Tejun Heo
7c3ffe96e1 Unify crate dependency versions
Different sub-projects are using different versions for the same crates.
Synchronize them to the latest.
2024-08-08 13:26:47 -10:00
Andrea Righi
9d808ae206
Merge pull request #468 from sched-ext/rustland-refactoring
scx_rustland refactoring
2024-08-07 11:38:21 +02:00
Andrea Righi
51cfb69199 scx_rustland_core: re-introduce partial mode
Re-add the partial mode option that was dropped during the refactoring.

The partial option allows to apply the scheduler only to the tasks which
have their scheduling policy set to SCHED_EXT via sched_setscheduler().

Signed-off-by: Andrea Righi <andrea.righi@linux.dev>
2024-08-07 08:41:06 +02:00
Andrea Righi
e1f2b3822e scx_rustland_core: drop CPU ownership API
The API for determining which PID is running on a specific CPU is racy
and is unnecessary since this information can be obtained from user
space.

Additionally, it's not reliable for identifying idle CPUs.  Therefore,
it's better to remove this API and, in the future, provide a cpumask
alternative that can export the idle state of the CPUs to user space.

As a consequence also change scx_rustland to dispatch one task a time,
instead of dispatching tasks in batches of idle cores (that are usually
not accurate due to the racy nature of the CPU ownership interaface).

Dispatching one task at a time even makes the scheduler more performant,
due to the vruntime scheduling being applied to more tasks sitting in
the scheduler's queue.

Signed-off-by: Andrea Righi <andrea.righi@linux.dev>
2024-08-07 08:41:06 +02:00