jakman85andClaude Opus 4.8
60a58f01ad
Add an HA control-plane API VIP for the k3s cluster with kube-vip, and add the VIP + all node IPs to the kube-apiserver serving-cert SANs so TLS via the VIP validates. Prepares the stable failover endpoint the operator fronts with a pfSense port-forward to 192.168.1.49:6443. roles/kube_vip + playbooks/kube-vip.yml: - Templates a single manifest (RBAC + DaemonSet kube-vip-ds) into each server's k3s auto-deploy dir (/var/lib/rancher/k3s/server/manifests) so it is reproducible and independent of Flux; k3s auto-applies it. - Control-plane ONLY (cp_enable=true, svc_enable=false): Cilium keeps ownership of Service LoadBalancing (L2 pool .53-.60) — no ARP conflict. - Runs on control-plane nodes (nodeSelector node-role.kubernetes.io/control-plane), hostNetwork, NET_ADMIN/NET_RAW, image pinned ghcr.io/kube-vip/kube-vip:v0.8.9. - vip_interface auto-detected per node (the iface holding 192.168.1.5x = eth0). - Leader election (plndr-cp-lock) with 15/10/2s lease timings. - This cluster runs without kube-proxy (Cilium replacement), so a hostNetwork pod cannot reach the kubernetes ClusterIP; kube-vip is pointed at the node-local apiserver (KUBERNETES_SERVICE_HOST=127.0.0.1:6443) so leader election works. roles/k3s_server tls-san: - New k3s_tls_sans (VIP + all three node IPs + 127.0.0.1, deduped); install args now emit one --tls-san per entry. - Persist the SAN list to /etc/rancher/k3s/config.yaml (read by a running k3s) and Restart k3s handler so an existing cluster regenerates its serving cert. - k3s.yml secondaries run serial:1 so a SAN change rolls one node at a time, keeping etcd quorum (kube-vip keeps the API reachable during each restart). Verified live: DS 3/3 ready, VIP answers on 192.168.1.49:6443, all 3 certs and an s_client against the VIP carry 192.168.1.49 + .50/.51/.52 + 127.0.0.1, all nodes Ready throughout the rolling restart. Co-Authored-By: Claude Opus 4.8 <[email protected]>
33 lines
1.7 KiB
YAML
33 lines
1.7 KiB
YAML
---
|
|
# k3s_server role defaults (ADR-0001).
|
|
|
|
# Best-effort version pin (kept near the ci/versions kubectl 1.31.x line). Bump
|
|
# deliberately — a version change is a code change. Empty string => latest stable.
|
|
k3s_version: "v1.31.5+k3s1"
|
|
|
|
k3s_install_url: "https://get.k3s.io"
|
|
k3s_install_script: "/usr/local/bin/k3s-install.sh"
|
|
|
|
# Components disabled at install so Cilium + Gateway API own networking (ADR-0001/0002).
|
|
# flannel + kube-proxy + network-policy are disabled via dedicated flags below;
|
|
# traefik + servicelb via --disable.
|
|
# local-storage: k3s ships local-path as a DEFAULT StorageClass, which collides with Longhorn's
|
|
# default (two default SCs = ambiguous PVC binding). Longhorn (ADR-0003) is the cluster's storage,
|
|
# so disable k3s local-path entirely → `longhorn` is the sole default. (Effective on fresh installs;
|
|
# the k3s install task is idempotent, so this does not restart an existing cluster — see DECISIONS-LOG.)
|
|
k3s_disable: "traefik,servicelb,local-storage"
|
|
|
|
# Set per-play: 'primary' (runs --cluster-init) or 'secondary' (joins the primary).
|
|
k3s_server_mode: "secondary"
|
|
|
|
# Subject Alternative Names baked into the kube-apiserver serving cert. Must include
|
|
# the kube-vip control-plane VIP (kube_vip_address) so TLS via the VIP validates,
|
|
# every server's real node IP (so the cert is valid whichever node answers behind
|
|
# the VIP), and 127.0.0.1 (local kubeconfig / health probes). k3s regenerates the
|
|
# serving cert on restart when this set changes. Deduplicated (the primary IP is
|
|
# also k3s_api_host).
|
|
k3s_tls_sans: >-
|
|
{{ ([k3s_api_host, kube_vip_address, '127.0.0.1']
|
|
+ (groups['k3s_servers'] | map('extract', hostvars, 'k3s_node_ip') | list))
|
|
| unique }}
|