jakman85andClaude Opus 4.8
60a58f01ad
Add an HA control-plane API VIP for the k3s cluster with kube-vip, and add the VIP + all node IPs to the kube-apiserver serving-cert SANs so TLS via the VIP validates. Prepares the stable failover endpoint the operator fronts with a pfSense port-forward to 192.168.1.49:6443. roles/kube_vip + playbooks/kube-vip.yml: - Templates a single manifest (RBAC + DaemonSet kube-vip-ds) into each server's k3s auto-deploy dir (/var/lib/rancher/k3s/server/manifests) so it is reproducible and independent of Flux; k3s auto-applies it. - Control-plane ONLY (cp_enable=true, svc_enable=false): Cilium keeps ownership of Service LoadBalancing (L2 pool .53-.60) — no ARP conflict. - Runs on control-plane nodes (nodeSelector node-role.kubernetes.io/control-plane), hostNetwork, NET_ADMIN/NET_RAW, image pinned ghcr.io/kube-vip/kube-vip:v0.8.9. - vip_interface auto-detected per node (the iface holding 192.168.1.5x = eth0). - Leader election (plndr-cp-lock) with 15/10/2s lease timings. - This cluster runs without kube-proxy (Cilium replacement), so a hostNetwork pod cannot reach the kubernetes ClusterIP; kube-vip is pointed at the node-local apiserver (KUBERNETES_SERVICE_HOST=127.0.0.1:6443) so leader election works. roles/k3s_server tls-san: - New k3s_tls_sans (VIP + all three node IPs + 127.0.0.1, deduped); install args now emit one --tls-san per entry. - Persist the SAN list to /etc/rancher/k3s/config.yaml (read by a running k3s) and Restart k3s handler so an existing cluster regenerates its serving cert. - k3s.yml secondaries run serial:1 so a SAN change rolls one node at a time, keeping etcd quorum (kube-vip keeps the API reachable during each restart). Verified live: DS 3/3 ready, VIP answers on 192.168.1.49:6443, all 3 certs and an s_client against the VIP carry 192.168.1.49 + .50/.51/.52 + 127.0.0.1, all nodes Ready throughout the rolling restart. Co-Authored-By: Claude Opus 4.8 <[email protected]>
35 lines
1.5 KiB
YAML
35 lines
1.5 KiB
YAML
---
|
|
# kube_vip role defaults — control-plane HA VIP for the k3s kube-API.
|
|
#
|
|
# kube-vip runs as a DaemonSet on the control-plane nodes and floats a single
|
|
# virtual IP (kube_vip_address) that always points at the current leader, giving
|
|
# the kube-apiserver a stable failover endpoint (an operator port-forward fronts
|
|
# it at 192.168.1.49:6443). It is CONTROL-PLANE ONLY (svc_enable=false): Cilium
|
|
# owns Service LoadBalancing (L2 pool .53-.60), so kube-vip must not touch
|
|
# Services or the two would fight over ARP for LoadBalancer IPs.
|
|
|
|
# The floating control-plane VIP. Defined in group_vars/all.yml so k3s_server's
|
|
# tls-san list and this role stay in parity; this default is the fallback.
|
|
kube_vip_address: "192.168.1.49"
|
|
|
|
# Image pinned deliberately — a version bump is a code change (verified tag on
|
|
# ghcr.io). v0.8.9 is a current v0.8.x release.
|
|
kube_vip_version: "v0.8.9"
|
|
kube_vip_image: "ghcr.io/kube-vip/kube-vip:{{ kube_vip_version }}"
|
|
|
|
kube_vip_namespace: "kube-system"
|
|
kube_vip_port: 6443
|
|
|
|
# Empty => auto-detected per host (the interface holding 192.168.1.5x); override
|
|
# only if detection is wrong.
|
|
kube_vip_interface: ""
|
|
|
|
# Leader-election tuning (seconds) — a snappy failover without thrashing.
|
|
kube_vip_lease_duration: 15
|
|
kube_vip_renew_deadline: 10
|
|
kube_vip_retry_period: 2
|
|
|
|
# k3s auto-deploy manifests dir: k3s applies anything here on start, so kube-vip
|
|
# is reproducible and independent of Flux/Helm.
|
|
kube_vip_manifest_path: "/var/lib/rancher/k3s/server/manifests/kube-vip.yaml"
|