scx_rustland_core: always dispatch per-cpu kthreads directly

Do not send per-CPU kthreads to the user-space scheduler, but always
dispatch them directly from BPF.

In specific environments, sending critical per-CPU kthreads to the
user-space scheduler can lead to potential stalls. This occurs because
the user-space scheduler might be blocked by an action that these
per-CPU kthreads need to perform, but they cannot complete their action
if the scheduler needs to schedule them, hence the deadlock.

To prevent this deadlock, always dispatch the per-CPU kthreads directly
from the BPF component, ensuring that the user-space scheduler does not
get blocked by these events.

Fixes: c0a2cfb ("scx_rustland_core: always schedule per-CPU kthreads to user-space")
Signed-off-by: Andrea Righi <andrea.righi@linux.dev>

rust/scx_rustland_core/assets/bpf/main.bpf.c

@@ -266,6 +266,14 @@ static inline bool is_usersched_task(const struct task_struct *p)
 	return p->pid == usersched_pid;
 }
 
+/*
+ * Return true if the target task @p is a kernel thread.
+ */
+static inline bool is_kthread(const struct task_struct *p)
+{
+	return p->flags & PF_KTHREAD;
+}
+
 /*
  * Flag used to wake-up the user-space scheduler.
  */
@@ -640,6 +648,22 @@ void BPF_STRUCT_OPS(rustland_enqueue, struct task_struct *p, u64 enq_flags)
 	if (is_usersched_task(p))
 		return;
 
+	/*
+	 * Always dispatch per-CPU kthreads directly on their target CPU.
+	 *
+	 * This allows to prioritize critical kernel threads that may
+	 * potentially stall the entire system if they are blocked for too long
+	 * (i.e., ksoftirqd/N, rcuop/N, etc.).
+	 */
+	if (is_kthread(p) && p->nr_cpus_allowed == 1) {
+		s32 cpu = scx_bpf_task_cpu(p);
+		u64 dsq_id = cpu_to_dsq(cpu);
+
+		scx_bpf_dispatch_vtime(p, dsq_id, SCX_SLICE_DFL, 0, 0);
+		__sync_fetch_and_add(&nr_kernel_dispatches, 1);
+		return;
+	}
+
 	/*
 	 * Add tasks to the @queued list, they will be processed by the
 	 * user-space scheduler.