---
# k3s_server role defaults (ADR-0001).

# Best-effort version pin (kept near the ci/versions kubectl 1.36.x line). Bump
# deliberately — a version change is a code change. Empty string => latest stable.
k3s_version: "v1.36.4+k3s1"

k3s_install_url: "https://get.k3s.io"
k3s_install_script: "/usr/local/bin/k3s-install.sh"

# Components disabled at install so Cilium + Gateway API own networking (ADR-0001/0002).
# flannel + kube-proxy + network-policy are disabled via dedicated flags below;
# traefik + servicelb via --disable.
# local-storage: k3s ships local-path as a DEFAULT StorageClass, which collides with Longhorn's
# default (two default SCs = ambiguous PVC binding). Longhorn (ADR-0003) is the cluster's storage,
# so disable k3s local-path entirely → `longhorn` is the sole default. (Effective on fresh installs;
# the k3s install task is idempotent, so this does not restart an existing cluster — see DECISIONS-LOG.)
k3s_disable: "traefik,servicelb,local-storage"

# --- k3s --data-dir (R-ENV-02) ------------------------------------------------------
# EVERYTHING k3s persists lives under this one directory: the embedded etcd database and
# WAL (server/db), the node token, the TLS CA and every issued cert, AND the containerd
# image store (agent/containerd) — which is by far the largest tenant.
#
# THE DEFAULT IS k3s'S OWN DEFAULT, restated explicitly rather than left implicit, so the
# LIVE dev cluster's install command is byte-for-byte what it already was: dev keeps
# /var/lib/rancher/k3s, where it also has a DEDICATED etcd device mounted at
# server/db (docs/plans/ssd-etcd-isolation.md). Changing this value on dev would move a
# running cluster's entire state directory out from under it and orphan that etcd mount.
#
# PROD OVERRIDES IT, in inventory/group_vars/k3s_prod/vars.yml — the same seam
# kube_vip_address / k3s_api_host / node_baseline_backports_kernel use, so a FOURTH prod
# node inherits it from group membership rather than from anyone remembering. It is set
# there as the CLUSTER-WIDE `k3s_data_dir` because TWO roles must agree on it: this role
# turns it into --data-dir, and roles/node_baseline creates the directory on the claimed
# data partition and derives etcd's location from it (node_baseline_k3s_data_dir defers to
# this same variable). One line moves both.
#
# WHY PROD MOVES IT (infra/PROD-PROVISIONING.md §2, revised 2026-09-04). Prod's /var is a
# separate partition of 11.8 GiB. Measured on the fully-deployed dev cluster, containerd
# alone is 21-25 GB per node and /var totals 58-70 GB — so prod would fill /var during its
# FIRST image pulls, and a full /var means etcd cannot write its WAL. That is cluster death,
# not degradation. Prod points --data-dir at /data/k3s on the claimed ~424 GB partition.
#
# NOT ALSO WRITTEN TO /etc/rancher/k3s/config.yaml, deliberately. --data-dir is consumed at
# INSTALL time (it is baked into the systemd unit's ExecStart); adding a `data-dir:` line to
# config.yaml would make that file `changed` on the LIVE dev cluster and fire the rolling
# Restart handler for a value that is not changing. The install flags are the correct and
# sufficient place, and prod has no k3s installed yet.
k3s_data_dir: "/var/lib/rancher/k3s"

# Set per-play: 'primary' (runs --cluster-init) or 'secondary' (joins the primary).
k3s_server_mode: "secondary"

# Subject Alternative Names baked into the kube-apiserver serving cert. Must include
# the kube-vip control-plane VIP (kube_vip_address) so TLS via the VIP validates,
# every server's real node IP (so the cert is valid whichever node answers behind
# the VIP), and 127.0.0.1 (local kubeconfig / health probes). k3s regenerates the
# serving cert on restart when this set changes. Deduplicated (the primary IP is
# also k3s_api_host).
#
# THIS LIST COVERS BOTH API ADDRESSES, WHICH IS WHY THE 2026-09-06 SPLIT NEEDED NO CERT
# CHANGE AND NO k3s RESTART. group_vars/all.yml now distinguishes k3s_api_host (the
# founder, for the cold join) from k3s_apiserver_endpoint (= kube_vip_address, what Cilium
# and every steady-state client dial). Both are already in this set, so repointing a
# consumer from one to the other cannot produce a TLS failure. VERIFIED LIVE 2026-09-06,
# not assumed: prod's apiserver answered through https://192.168.2.9:6443 with full CA
# verification and tls-server-name=192.168.2.9. Do not remove either entry — dropping
# kube_vip_address breaks every client of the VIP; dropping k3s_api_host breaks nothing
# today only because it is also in the k3s_servers_group node-IP list below, and that
# coincidence is not a contract.
#
# k3s_servers_group — NOT a literal group name — is what makes this role environment-
# agnostic (R-ENV-02): group_vars/all.yml sets it to `k3s_servers` (dev) and
# group_vars/k3s_prod/vars.yml to `k3s_prod`, so each environment SANs its own servers.
k3s_tls_sans: >-
  {{ ([k3s_api_host, kube_vip_address, '127.0.0.1']
      + (groups[k3s_servers_group] | map('extract', hostvars, 'k3s_node_ip') | list))
     | unique }}

# --- Keycloak OIDC issuer for kubectl SSO (R-IDP-03, R-ENV-02) -----------------------
# THE ANSIBLE-SIDE HALF OF ${BASE_DOMAIN}, and it cannot be the Flux-side half.
#
# The kube-apiserver's `oidc-issuer-url` is a k3s SERVER FLAG written into
# /etc/rancher/k3s/config.yaml by this role — it is NOT a Kubernetes manifest, so Flux never
# sees it and `postBuild.substituteFrom: cluster-vars` cannot reach it. The platform hostname
# suffix therefore exists as ONE FACT IN TWO PLACES, and they must agree:
#
#   Flux side     clusters/base/cluster-vars.yaml   BASE_DOMAIN      (per-overlay)
#   Ansible side  THIS variable                     k3s_base_domain  (per-inventory-group)
#
# [parity] k3s_base_domain MUST equal the BASE_DOMAIN the SAME cluster's Flux overlay sets.
# If they disagree, the apiserver validates ID tokens against a DIFFERENT Keycloak than the
# one the cluster actually runs: every `kubectl` SSO login fails with an issuer mismatch,
# while the cert-based admin kubeconfig keeps working — so it looks like "SSO is broken",
# not like a config skew. The break-glass cert kubeconfig is why this is recoverable.
#
# THE DEFAULT IS THE DEV VALUE, restated explicitly rather than left implicit — the same
# convention as k3s_data_dir above and cluster-vars' GATEWAY_VIP / LONGHORN_DATA_PATH: the
# base default is the known-good LIVE value, so dev's rendered config.yaml is byte-for-byte
# what it already is and the Restart handler does NOT fire on a re-run.
#
# PROD OVERRIDES IT in inventory/group_vars/k3s_prod/vars.yml — the same inventory seam
# k3s_data_dir / k3s_api_host / kube_vip_address use, so a FOURTH prod server inherits it
# from group membership rather than from anyone remembering.
k3s_base_domain: "home.keithjackson.dev"

# Derived, so the realm path is stated once. Override k3s_base_domain, not this.
k3s_oidc_issuer_url: "https://keycloak.{{ k3s_base_domain }}/realms/homelab"

# --- Graceful node shutdown (kubelet) ----------------------------------------------------------
# MEASURED 2026-10-17, during the UPS move: graceful node shutdown was NOT configured on any prod
# node, so `shutdown -h` did NOT drain pods. systemd stops k3s, containerd goes with it, and every
# pod is SIGKILLed where it stands -- including 9 CloudNativePG instances mid-checkpoint, the UniFi
# VM, and 25 attached Longhorn volumes. The whole-cluster shutdown that day had to quiesce
# everything BY HAND first (runbooks/cluster-maintenance-shutdown.md), and the one volume that did
# not finish detaching came back `degraded` and had to rebuild.
#
# With these set, the kubelet takes a systemd inhibitor lock and, on shutdown, evicts ordinary pods
# first and critical ones last, so a plain `shutdown -h`/reboot drains the node. That makes the
# manual scale-down in the runbook a belt-and-braces step for a PLANNED whole-cluster shutdown
# rather than the only thing standing between a reboot and a hard kill.
#
# 90s/20s, not larger: the budget is spent on every reboot and node upgrade, and the kubelet waits
# the FULL ordinary period before it starts on critical pods. Postgres does a fast shutdown in
# well under 90s here, and Longhorn needs only enough time to flush and detach.
k3s_kubelet_shutdown_grace_period: "90s"
k3s_kubelet_shutdown_grace_period_critical_pods: "20s"

# *** THE HALF THAT IS EASY TO MISS, AND SILENTLY BREAKS THE OTHER HALF. ***
# Graceful node shutdown works by the kubelet holding a logind INHIBITOR DELAY lock. systemd caps
# how long any inhibitor may delay shutdown at `InhibitDelayMaxSec`, which DEFAULTS TO 5 SECONDS.
# If that cap is below the grace period above, systemd proceeds anyway and the drain is truncated
# to ~5s -- i.e. the feature appears configured, reports no error, and still hard-kills pods. So
# this MUST be >= k3s_kubelet_shutdown_grace_period; the role asserts it rather than trusting it.
# Set a little above the kubelet's total (90s + 20s) to leave headroom.
k3s_logind_inhibit_delay_max_sec: 120