feat(infra): kube-vip control-plane HA VIP (192.168.1.49) + API tls-san

Add an HA control-plane API VIP for the k3s cluster with kube-vip, and add
the VIP + all node IPs to the kube-apiserver serving-cert SANs so TLS via the
VIP validates. Prepares the stable failover endpoint the operator fronts with
a pfSense port-forward to 192.168.1.49:6443.

roles/kube_vip + playbooks/kube-vip.yml:
- Templates a single manifest (RBAC + DaemonSet kube-vip-ds) into each server's
  k3s auto-deploy dir (/var/lib/rancher/k3s/server/manifests) so it is
  reproducible and independent of Flux; k3s auto-applies it.
- Control-plane ONLY (cp_enable=true, svc_enable=false): Cilium keeps ownership
  of Service LoadBalancing (L2 pool .53-.60) — no ARP conflict.
- Runs on control-plane nodes (nodeSelector node-role.kubernetes.io/control-plane),
  hostNetwork, NET_ADMIN/NET_RAW, image pinned ghcr.io/kube-vip/kube-vip:v0.8.9.
- vip_interface auto-detected per node (the iface holding 192.168.1.5x = eth0).
- Leader election (plndr-cp-lock) with 15/10/2s lease timings.
- This cluster runs without kube-proxy (Cilium replacement), so a hostNetwork
  pod cannot reach the kubernetes ClusterIP; kube-vip is pointed at the
  node-local apiserver (KUBERNETES_SERVICE_HOST=127.0.0.1:6443) so leader
  election works.

roles/k3s_server tls-san:
- New k3s_tls_sans (VIP + all three node IPs + 127.0.0.1, deduped); install
  args now emit one --tls-san per entry.
- Persist the SAN list to /etc/rancher/k3s/config.yaml (read by a running k3s)
  and Restart k3s handler so an existing cluster regenerates its serving cert.
- k3s.yml secondaries run serial:1 so a SAN change rolls one node at a time,
  keeping etcd quorum (kube-vip keeps the API reachable during each restart).

Verified live: DS 3/3 ready, VIP answers on 192.168.1.49:6443, all 3 certs and
an s_client against the VIP carry 192.168.1.49 + .50/.51/.52 + 127.0.0.1, all
nodes Ready throughout the rolling restart.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed 2026-08-25 21:52:43 -05:00
1 parent d99d171468
commit 60a58f01ad
9 files changed
+276 -1

No files matched your search

@@ -12,6 +12,13 @@ ansible_user: k3s
k3s_api_host: "{{ hostvars[groups['k3s_primary'][0]]['k3s_node_ip'] }}"
k3s_api_port: 6443
# Control-plane HA VIP floated by kube-vip (roles/kube_vip). A stable failover
# endpoint for the kube-apiserver, fronted by an operator pfSense port-forward at
# 192.168.1.49:6443. Shared here so BOTH kube_vip AND the k3s_server tls-san list
# agree on the address (the serving cert must carry this IP or TLS via the VIP
# fails to validate).
kube_vip_address: "192.168.1.49"
# Controller (dev container) reaches the kube-API through an SSH tunnel
# (local 127.0.0.1:16443 -> primary 127.0.0.1:6443), because the API port is not
# NAT-forwarded and 192.168.1.50:6443 is unroutable from the dev host. Only the