jakman85andClaude Opus 4.8
60a58f01ad
Add an HA control-plane API VIP for the k3s cluster with kube-vip, and add the VIP + all node IPs to the kube-apiserver serving-cert SANs so TLS via the VIP validates. Prepares the stable failover endpoint the operator fronts with a pfSense port-forward to 192.168.1.49:6443. roles/kube_vip + playbooks/kube-vip.yml: - Templates a single manifest (RBAC + DaemonSet kube-vip-ds) into each server's k3s auto-deploy dir (/var/lib/rancher/k3s/server/manifests) so it is reproducible and independent of Flux; k3s auto-applies it. - Control-plane ONLY (cp_enable=true, svc_enable=false): Cilium keeps ownership of Service LoadBalancing (L2 pool .53-.60) — no ARP conflict. - Runs on control-plane nodes (nodeSelector node-role.kubernetes.io/control-plane), hostNetwork, NET_ADMIN/NET_RAW, image pinned ghcr.io/kube-vip/kube-vip:v0.8.9. - vip_interface auto-detected per node (the iface holding 192.168.1.5x = eth0). - Leader election (plndr-cp-lock) with 15/10/2s lease timings. - This cluster runs without kube-proxy (Cilium replacement), so a hostNetwork pod cannot reach the kubernetes ClusterIP; kube-vip is pointed at the node-local apiserver (KUBERNETES_SERVICE_HOST=127.0.0.1:6443) so leader election works. roles/k3s_server tls-san: - New k3s_tls_sans (VIP + all three node IPs + 127.0.0.1, deduped); install args now emit one --tls-san per entry. - Persist the SAN list to /etc/rancher/k3s/config.yaml (read by a running k3s) and Restart k3s handler so an existing cluster regenerates its serving cert. - k3s.yml secondaries run serial:1 so a SAN change rolls one node at a time, keeping etcd quorum (kube-vip keeps the API reachable during each restart). Verified live: DS 3/3 ready, VIP answers on 192.168.1.49:6443, all 3 certs and an s_client against the VIP carry 192.168.1.49 + .50/.51/.52 + 127.0.0.1, all nodes Ready throughout the rolling restart. Co-Authored-By: Claude Opus 4.8 <[email protected]>
46 lines
1.6 KiB
YAML
46 lines
1.6 KiB
YAML
---
|
|
# Deploy kube-vip (control-plane HA VIP) via the k3s auto-deploy manifests dir.
|
|
# k3s applies /var/lib/rancher/k3s/server/manifests/*.yaml on start, so this is
|
|
# reproducible and independent of Flux. The same manifest is written on every
|
|
# server so the DaemonSet survives the loss of any single node.
|
|
|
|
- name: Detect the LAN interface holding the node's 192.168.1.5x address
|
|
ansible.builtin.shell: >-
|
|
set -o pipefail;
|
|
ip -o -4 addr show | awk '/192\.168\.1\.5[0-2]\//{print $2; exit}'
|
|
args:
|
|
executable: /bin/bash
|
|
register: kube_vip_iface_detect
|
|
changed_when: false
|
|
|
|
- name: Resolve the kube-vip VIP interface (override wins, else detected)
|
|
ansible.builtin.set_fact:
|
|
kube_vip_vip_interface: >-
|
|
{{ kube_vip_interface if (kube_vip_interface | length > 0)
|
|
else (kube_vip_iface_detect.stdout | trim) }}
|
|
|
|
- name: Fail early if no LAN interface could be resolved
|
|
ansible.builtin.assert:
|
|
that:
|
|
- kube_vip_vip_interface is defined
|
|
- kube_vip_vip_interface | length > 0
|
|
fail_msg: >-
|
|
Could not detect the 192.168.1.5x LAN interface for kube-vip on
|
|
{{ inventory_hostname }} — set kube_vip_interface explicitly.
|
|
|
|
- name: Ensure the k3s auto-deploy manifests directory exists
|
|
ansible.builtin.file:
|
|
path: "{{ kube_vip_manifest_path | dirname }}"
|
|
state: directory
|
|
owner: root
|
|
group: root
|
|
mode: "0700"
|
|
|
|
- name: Template the kube-vip control-plane manifest (RBAC + DaemonSet)
|
|
ansible.builtin.template:
|
|
src: kube-vip.yaml.j2
|
|
dest: "{{ kube_vip_manifest_path }}"
|
|
owner: root
|
|
group: root
|
|
mode: "0600"
|