layered: add RandomTopo layer growth algorithm

Add an additional layer growth algorithm, named 'RandomTopo'. It follows these
rules:
- Randomise NUMA nodes. List each core in each NUMA node before a core from
  another NUMA node.
- Randomise LLCs within each NUMA node. List each core in each LLC before a
  core in a different LLC.
- Randomise the core order within each LLC.

This attempts to provide a relatively evenly distributed set of cores while
considering topology. Unlike `Topo`, it does not require you to specify the
ordering and instead generates it from the hardware, making desyncs between the
config and the hardware less likely.

Currently `RandomTopo` considers topology even with `--disable-topology=true`.
I can see the arguments for this going both ways. On one hand requesting
disable topology suggests you want no consideration of machine topology, and
`RandomTopo` should decay to `Random` (which it does on single node/LLC machines
anyway). On the other hand, the config explicitly specifies `RandomTopo` and
should consider the topology. If anyone feels strongly I can change this to
respect `disable_topology`.

Test plan:
```sh
$ sudo target/release/scx_layered -v f:/tmp/test.json
...
14:31:19 [DEBUG] layer: batch algo: RandomTopo core order: [47, 44, 43, 42, 40, 45, 46, 41, 38, 37, 36, 39, 34, 32, 35, 33, 54, 49, 50, 52, 51, 48, 55, 53, 68, 64, 66, 67, 70, 69, 71, 65, 9, 10, 12, 15, 14, 11, 8, 13, 59, 60, 57, 63, 62, 56, 58, 61, 2, 3, 5, 4, 0, 6, 7, 1, 86, 83, 85, 87, 84, 81, 80, 82, 20, 22, 19, 23, 21, 18, 17, 16, 30, 25, 26, 31, 28, 27, 29, 24, 78, 73, 74, 79, 75, 77, 76, 72]
14:31:19 [DEBUG] layer: immediate algo: RandomTopo core order: [45, 40, 46, 42, 47, 43, 41, 44, 80, 82, 83, 84, 85, 86, 81, 87, 13, 10, 9, 15, 14, 12, 11, 8, 36, 38, 39, 32, 34, 35, 33, 37, 7, 3, 1, 0, 2, 5, 4, 6, 53, 52, 54, 48, 50, 49, 55, 51, 76, 77, 79, 78, 73, 74, 72, 75, 71, 66, 64, 67, 70, 69, 65, 68, 24, 26, 31, 25, 28, 30, 27, 29, 58, 56, 59, 61, 57, 62, 60, 63, 16, 19, 17, 23, 22, 20, 18, 21]
...
```

This is a machine with 1 NUMA/11 LLCs with 8 cores per LLC and you can see the
results are grouped by LLC but random within.
This commit is contained in:
Jake Hillion 2024-10-17 15:32:44 +01:00
parent b01ff79080
commit a0fe303b61
2 changed files with 30 additions and 0 deletions

View File

@ -162,6 +162,7 @@ enum layer_growth_algo {
GROWTH_ALGO_ROUND_ROBIN,
GROWTH_ALGO_BIG_LITTLE,
GROWTH_ALGO_LITTLE_BIG,
GROWTH_ALGO_RANDOM_TOPO,
};
struct layer {

View File

@ -36,6 +36,10 @@ pub enum LayerGrowthAlgo {
/// LittleBig attempts to first grow across all little cores and then
/// allocates onto big cores after all little cores are allocated.
LittleBig,
/// RandomTopo is sticky to NUMA nodes/LLCs but randomises the order in which
/// it visits each. The layer will select a random NUMA node, then a random LLC
/// within it, then randomly iterate the cores in that LLC.
RandomTopo,
}
const GROWTH_ALGO_STICKY: i32 = bpf_intf::layer_growth_algo_GROWTH_ALGO_STICKY as i32;
@ -46,6 +50,7 @@ const GROWTH_ALGO_TOPO: i32 = bpf_intf::layer_growth_algo_GROWTH_ALGO_TOPO as i3
const GROWTH_ALGO_ROUND_ROBIN: i32 = bpf_intf::layer_growth_algo_GROWTH_ALGO_ROUND_ROBIN as i32;
const GROWTH_ALGO_BIG_LITTLE: i32 = bpf_intf::layer_growth_algo_GROWTH_ALGO_BIG_LITTLE as i32;
const GROWTH_ALGO_LITTLE_BIG: i32 = bpf_intf::layer_growth_algo_GROWTH_ALGO_LITTLE_BIG as i32;
const GROWTH_ALGO_RANDOM_TOPO: i32 = bpf_intf::layer_growth_algo_GROWTH_ALGO_RANDOM_TOPO as i32;
impl LayerGrowthAlgo {
pub fn as_bpf_enum(&self) -> i32 {
@ -58,6 +63,7 @@ impl LayerGrowthAlgo {
LayerGrowthAlgo::RoundRobin => GROWTH_ALGO_ROUND_ROBIN,
LayerGrowthAlgo::BigLittle => GROWTH_ALGO_BIG_LITTLE,
LayerGrowthAlgo::LittleBig => GROWTH_ALGO_LITTLE_BIG,
LayerGrowthAlgo::RandomTopo => GROWTH_ALGO_RANDOM_TOPO,
}
}
@ -83,6 +89,7 @@ impl LayerGrowthAlgo {
LayerGrowthAlgo::BigLittle => generator.grow_big_little(),
LayerGrowthAlgo::LittleBig => generator.grow_little_big(),
LayerGrowthAlgo::Topo => generator.grow_topo(),
LayerGrowthAlgo::RandomTopo => generator.grow_random_topo(),
}
}
}
@ -237,6 +244,28 @@ impl<'a> LayerCoreOrderGenerator<'a> {
core_order
}
}
fn grow_random_topo(&self) -> Vec<usize> {
fastrand::seed(self.layer_idx.try_into().unwrap());
let mut nodes: Vec<_> = self.topo.nodes().into_iter().collect();
fastrand::shuffle(&mut nodes);
nodes
.into_iter()
.flat_map(|node| {
let mut llcs: Vec<_> = node.llcs().values().collect();
fastrand::shuffle(&mut llcs);
llcs.into_iter()
})
.flat_map(|llc| {
let mut cores: Vec<_> = llc.cores().values().collect();
fastrand::shuffle(&mut cores);
cores.into_iter()
})
.map(|c| self.cpu_pool.get_core_topological_id(c))
.collect()
}
}
struct IteratorInterleaver<T>