From e6d1c597f137ee8a4439d7d9c3d24e22b86eb03c Mon Sep 17 00:00:00 2001
From: Euan Kemp <euank@euank.com>
Date: Sat, 5 Mar 2022 00:43:47 -0800
Subject: [PATCH] nixos/k3s: use the systemd driver for docker + unified
 cgroups

This is necessary for it to work at all. The single-node-docker test
will fail without this change.

Also add a release note for it.
---
 .../from_md/release-notes/rl-2205.section.xml      | 14 ++++++++++++++
 nixos/doc/manual/release-notes/rl-2205.section.md  |  4 ++++
 nixos/modules/services/cluster/k3s/default.nix     |  1 +
 3 files changed, 19 insertions(+)

diff --git a/nixos/doc/manual/from_md/release-notes/rl-2205.section.xml b/nixos/doc/manual/from_md/release-notes/rl-2205.section.xml
index 2bcfc86b432b..33cc929b466f 100644
--- a/nixos/doc/manual/from_md/release-notes/rl-2205.section.xml
+++ b/nixos/doc/manual/from_md/release-notes/rl-2205.section.xml
@@ -364,6 +364,20 @@
           relying on the insecure behaviour before upgrading.
         </para>
       </listitem>
+      <listitem>
+        <para>
+          <literal>services.k3s.enable</literal> no longer implies
+          <literal>systemd.enableUnifiedCgroupHierarchy = false</literal>,
+          and will default to the <quote>systemd</quote> cgroup driver
+          when using <literal>services.k3s.docker = true</literal>. This
+          change may require a reboot to take effect, and k3s may not be
+          able to run if the boot cgroup hierarchy does not match its
+          configuration. The previous behavior may be retained by
+          explicitly setting
+          <literal>systemd.enableUnifiedCgroupHierarchy = false</literal>
+          in your configuration.
+        </para>
+      </listitem>
       <listitem>
         <para>
           The DHCP server (<literal>services.dhcpd4</literal>,
diff --git a/nixos/doc/manual/release-notes/rl-2205.section.md b/nixos/doc/manual/release-notes/rl-2205.section.md
index 650ace8d9d2a..b5d5574a9889 100644
--- a/nixos/doc/manual/release-notes/rl-2205.section.md
+++ b/nixos/doc/manual/release-notes/rl-2205.section.md
@@ -121,6 +121,10 @@ In addition to numerous new and upgraded packages, this release has the followin
 
 - `services.kubernetes.scheduler.{port,address}` now set `--secure-port` and `--bind-address` instead of `--port` and `--address`, since the former have been deprecated and are no longer functional in kubernetes>=1.23. Ensure that you are not relying on the insecure behaviour before upgrading.
 
+- `services.k3s.enable` no longer implies `systemd.enableUnifiedCgroupHierarchy = false`, and will default to the 'systemd' cgroup driver when using `services.k3s.docker = true`.
+  This change may require a reboot to take effect, and k3s may not be able to run if the boot cgroup hierarchy does not match its configuration.
+  The previous behavior may be retained by explicitly setting `systemd.enableUnifiedCgroupHierarchy = false` in your configuration.
+
 - The DHCP server (`services.dhcpd4`, `services.dhcpd6`) has been hardened.
   The service is now using the systemd's `DynamicUser` mechanism to run as an unprivileged dynamically-allocated user with limited capabilities.
   The dhcpd state files are now always stored in `/var/lib/dhcpd{4,6}` and the `services.dhcpd4.stateDir` and `service.dhcpd6.stateDir` options have been removed.
diff --git a/nixos/modules/services/cluster/k3s/default.nix b/nixos/modules/services/cluster/k3s/default.nix
index 7bda11afedfb..3a36cfa3f37b 100644
--- a/nixos/modules/services/cluster/k3s/default.nix
+++ b/nixos/modules/services/cluster/k3s/default.nix
@@ -114,6 +114,7 @@ in
           [
             "${cfg.package}/bin/k3s ${cfg.role}"
           ] ++ (optional cfg.docker "--docker")
+          ++ (optional (cfg.docker && config.systemd.enableUnifiedCgroupHierarchy) "--kubelet-arg=cgroup-driver=systemd")
           ++ (optional cfg.disableAgent "--disable-agent")
           ++ (optional (cfg.serverAddr != "") "--server ${cfg.serverAddr}")
           ++ (optional (cfg.token != "") "--token ${cfg.token}")