128 lines
5.3 KiB
XML
128 lines
5.3 KiB
XML
<chapter xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude"
|
|
version="5.0"
|
|
xml:id="sec-kubernetes">
|
|
<title>Kubernetes</title>
|
|
|
|
<para>
|
|
The NixOS Kubernetes module is a collective term for a handful of
|
|
individual submodules implementing the Kubernetes cluster components.
|
|
</para>
|
|
|
|
<para>
|
|
There are generally two ways of enabling Kubernetes on NixOS.
|
|
One way is to enable and configure cluster components appropriately by hand:
|
|
<programlisting>
|
|
services.kubernetes = {
|
|
apiserver.enable = true;
|
|
controllerManager.enable = true;
|
|
scheduler.enable = true;
|
|
addonManager.enable = true;
|
|
proxy.enable = true;
|
|
flannel.enable = true;
|
|
};
|
|
</programlisting>
|
|
Another way is to assign cluster roles ("master" and/or "node") to the host.
|
|
This enables apiserver, controllerManager, scheduler, addonManager,
|
|
kube-proxy and etcd:
|
|
<programlisting>
|
|
<xref linkend="opt-services.kubernetes.roles"/> = [ "master" ];
|
|
</programlisting>
|
|
While this will enable the kubelet and kube-proxy only:
|
|
<programlisting>
|
|
<xref linkend="opt-services.kubernetes.roles"/> = [ "node" ];
|
|
</programlisting>
|
|
Assigning both the master and node roles is usable if you want a single
|
|
node Kubernetes cluster for dev or testing purposes:
|
|
<programlisting>
|
|
<xref linkend="opt-services.kubernetes.roles"/> = [ "master" "node" ];
|
|
</programlisting>
|
|
Note: Assigning either role will also default both
|
|
<xref linkend="opt-services.kubernetes.flannel.enable"/> and
|
|
<xref linkend="opt-services.kubernetes.easyCerts"/> to true.
|
|
This sets up flannel as CNI and activates automatic PKI bootstrapping.
|
|
</para>
|
|
|
|
<para>
|
|
As of kubernetes 1.10.X it has been deprecated to open
|
|
non-tls-enabled ports on kubernetes components. Thus, from NixOS 19.03 all
|
|
plain HTTP ports have been disabled by default.
|
|
While opening insecure ports is still possible, it is recommended not to
|
|
bind these to other interfaces than loopback.
|
|
|
|
To re-enable the insecure port on the apiserver, see options:
|
|
<xref linkend="opt-services.kubernetes.apiserver.insecurePort"/>
|
|
and
|
|
<xref linkend="opt-services.kubernetes.apiserver.insecureBindAddress"/>
|
|
</para>
|
|
|
|
<note>
|
|
<para>
|
|
As of NixOS 19.03, it is mandatory to configure:
|
|
<xref linkend="opt-services.kubernetes.masterAddress"/>.
|
|
The masterAddress must be resolveable and routeable by all cluster nodes.
|
|
In single node clusters, this can be set to <literal>localhost</literal>.
|
|
</para>
|
|
</note>
|
|
|
|
<para>
|
|
Role-based access control (RBAC) authorization mode is enabled by default.
|
|
This means that anonymous requests to the apiserver secure port will
|
|
expectedly cause a permission denied error. All cluster components must
|
|
therefore be configured with x509 certificates for two-way tls communication.
|
|
The x509 certificate subject section determines the roles and permissions
|
|
granted by the apiserver to perform clusterwide or namespaced operations.
|
|
See also:
|
|
<link
|
|
xlink:href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">
|
|
Using RBAC Authorization</link>.
|
|
</para>
|
|
|
|
<para>
|
|
The NixOS kubernetes module provides an option for automatic certificate
|
|
bootstrapping and configuration,
|
|
<xref linkend="opt-services.kubernetes.easyCerts"/>.
|
|
The PKI bootstrapping process involves setting up a certificate authority
|
|
(CA) daemon (cfssl) on the kubernetes master node. cfssl generates a CA-cert
|
|
for the cluster, and uses the CA-cert for signing subordinate certs issued to
|
|
each of the cluster components. Subsequently, the certmgr daemon monitors
|
|
active certificates and renews them when needed. For single node Kubernetes
|
|
clusters, setting <xref linkend="opt-services.kubernetes.easyCerts"/> = true
|
|
is sufficient and no further action is required. For joining extra node
|
|
machines to an existing cluster on the other hand, establishing initial trust
|
|
is mandatory.
|
|
</para>
|
|
|
|
<para>
|
|
To add new nodes to the cluster:
|
|
On any (non-master) cluster node where
|
|
<xref linkend="opt-services.kubernetes.easyCerts"/> is enabled, the helper
|
|
script <literal>nixos-kubernetes-node-join</literal> is available on PATH.
|
|
Given a token on stdin, it will copy the token to the kubernetes
|
|
secrets directory and restart the certmgr service. As requested
|
|
certificates are issued, the script will restart kubernetes cluster
|
|
components as needed for them to pick up new keypairs.
|
|
</para>
|
|
|
|
<note>
|
|
<para>
|
|
Multi-master (HA) clusters are not supported by the easyCerts module.
|
|
</para>
|
|
</note>
|
|
|
|
<para>
|
|
In order to interact with an RBAC-enabled cluster as an administrator, one
|
|
needs to have cluster-admin privileges. By default, when easyCerts is
|
|
enabled, a cluster-admin kubeconfig file is generated and linked into
|
|
<literal>/etc/kubernetes/cluster-admin.kubeconfig</literal> as determined by
|
|
<xref linkend="opt-services.kubernetes.pki.etcClusterAdminKubeconfig"/>.
|
|
<literal>export KUBECONFIG=/etc/kubernetes/cluster-admin.kubeconfig</literal>
|
|
will make kubectl use this kubeconfig to access and authenticate the cluster.
|
|
The cluster-admin kubeconfig references an auto-generated keypair owned by
|
|
root. Thus, only root on the kubernetes master may obtain cluster-admin
|
|
rights by means of this file.
|
|
</para>
|
|
|
|
</chapter>
|