--- # Shared connection + cluster facts for every k3s node. # cloud-init created this login user (tofu var login_user); it has NOPASSWD sudo. ansible_user: k3s # --- environment group NAMES (R-ENV-02: one set of roles, scoped overrides) --------- # The GROUP NAMES a role uses to answer "who is MY environment's primary / MY # environment's servers?". Roles must never hardcode `groups['k3s_primary']`, or they # only ever work against dev. These two are the dev values; group_vars/k3s_prod/vars.yml # overrides them with k3s_prod_primary / k3s_prod, so a host in the prod group resolves # its own primary automatically — exactly the way k3s_api_host below is overridden there. # # These are GROUP NAMES (keys into `groups`), NOT host patterns. The play-level target # pattern is a separate, extra-vars-only knob (`k3s_hosts` / `k3s_primary_hosts` / # `k3s_secondary_hosts`, see playbooks/*.yml) because Ansible templates a play's `hosts:` # before it has any host context and so cannot see group_vars at all. k3s_primary_group: k3s_primary k3s_servers_group: k3s_servers # Filename of the kubeconfig the controller-side roles (cilium, cluster_verify, # flux_bootstrap) fetch from the primary into infra/ansible/.cache/. Scoped per # environment so pointing those plays at prod can NEVER overwrite the dev kubeconfig # that infra/devcontainer/dev.sh exports as $KUBECONFIG — an operator's `kubectl get # nodes` silently answering from prod would be a genuinely dangerous mix-up. # group_vars/k3s_prod/vars.yml overrides this to prod.kubeconfig. (.cache/ is gitignored; # a kubeconfig is a credential.) cluster_kubeconfig_name: dev.kubeconfig # --- TWO API ADDRESSES, AND THEY ARE NOT INTERCHANGEABLE (split 2026-09-06) ---------- # This used to be ONE variable, k3s_api_host, consumed by BOTH the cold-bootstrap join and # Cilium's k8sServiceHost. That conflation is what pointed every cilium-agent on the LIVE # prod cluster at 192.168.2.10 — k3s-prod-1's own address — on a cluster whose entire # purpose is HA: verified 2026-09-06, all three agents carried # KUBERNETES_SERVICE_HOST=192.168.2.10 while kube-vip held 192.168.2.9. Lose that one node # and no service/policy update reaches the CNI until it returns (the existing datapath # survives on Cilium's cache, so it fails SILENTLY). The two consumers have genuinely # different requirements, so they now have two variables. # # k3s_api_host THE FOUNDER'S OWN ADDRESS. Bootstrap-time only. # k3s_apiserver_endpoint THE HA ENDPOINT (the kube-vip VIP). Steady-state. # # DO NOT "simplify" these back into one. See each block below for why each half must be # the address it is. # THE FOUNDER'S OWN NODE IP — a BOOTSTRAP-TIME address, deliberately NOT the VIP. # NOTE: ansible_host is now the pfSense NAT SSH target (192.168.2.102:225x), NOT the node # IP — so we read the real 192.168.1.x from k3s_node_ip instead. # # EXACTLY TWO CONSUMERS, and both of them need the founder specifically: # 1. roles/k3s_server — the secondary `--server https://:` JOIN URL. On a COLD # build the VIP DOES NOT EXIST YET at join time: kube-vip is a k3s auto-deploy manifest # that only lands once a cluster exists, so a join aimed at the VIP is a join aimed at # an unanswered address and the cold build is unjoinable. The founder is up (it just ran # --cluster-init), which is the whole reason this variable is the founder. # 2. roles/k3s_server — one entry of k3s_tls_sans, so the founder's own address is a SAN # alongside the VIP and every other server's IP. # # ANYTHING THAT RUNS AGAINST A CLUSTER THAT ALREADY EXISTS MUST NOT USE THIS. It is a # single point of failure by construction. Use k3s_apiserver_endpoint below. k3s_api_host: "{{ hostvars[groups[k3s_primary_group][0]]['k3s_node_ip'] }}" # Shared by both halves: k3s serves the API on 6443 in every environment, so the port is # not a per-environment knob and is not duplicated per address. k3s_api_port: 6443 # THE HA kube-apiserver ENDPOINT — a VIRTUAL address that reaches the cluster wherever the # control plane currently is, not a machine. OPERATOR DECISION 2026-09-06: in-cluster # components dial the VIP, as kube-vip was always intended to provide. # # DERIVED from kube_vip_address (below), never restated: an environment overrides the VIP in # ONE place (group_vars/k3s_prod/vars.yml sets kube_vip_address: 192.168.2.9) and this # follows automatically, so prod needs no override of its own and a fourth environment # cannot set one of the two and forget the other. # # NAMED FOR THE ROLE, NOT THE MECHANISM. Consumers want "the address that reaches the # apiserver, HA"; that it is currently floated by kube-vip is an implementation detail. If # the mechanism is ever something else (an external LB, a DNS name), this one line moves and # no consumer changes. # # CONSUMERS: # - roles/cilium -> cilium-values.yaml.j2 -> k8sServiceHost. [parity] with the Flux side: # clusters/base/cluster-vars.yaml K3S_API_HOST (and each overlay's patch) MUST equal # THIS value for that environment, or Flux and the bootstrap fight over the CNI's # control-plane path. See platform/cilium/helmrelease.yaml's PARITY CONTRACT header. # # TLS IS ALREADY COVERED — verified, not assumed: k3s_tls_sans (roles/k3s_server/defaults) # has always included kube_vip_address, and on 2026-09-06 the prod apiserver was reached # through https://192.168.2.9:6443 with FULL CA verification and tls-server-name=192.168.2.9 # (infra/ansible/.cache/prod.kubeconfig, no insecure-skip-tls-verify). So the serving cert # needs no change and no k3s restart for this switch. # # ORDERING CONSEQUENCE, and it is why playbooks/site.yml changed too: Cilium now needs the # VIP to be UP before it installs. site.yml therefore imports kube-vip.yml BETWEEN k3s.yml # and cilium.yml, and roles/kube_vip waits for the VIP to answer before returning. kube-vip # can precede the CNI because its DaemonSet is hostNetwork with # KUBERNETES_SERVICE_HOST=127.0.0.1 — it needs neither a CNI nor kube-proxy. k3s_apiserver_endpoint: "{{ kube_vip_address }}" # Control-plane HA VIP floated by kube-vip (roles/kube_vip). A stable failover # endpoint for the kube-apiserver, fronted by an operator pfSense port-forward at # 192.168.1.49:6443. Shared here so THREE things agree on the one address: # - roles/kube_vip — what to ARP and lease-elect for; # - roles/k3s_server — the tls-san entry (the serving cert must carry this IP or # TLS via the VIP fails to validate); # - k3s_apiserver_endpoint — above; what in-cluster components actually dial. # THIS is the one line an environment overrides (prod: 192.168.2.9, in # group_vars/k3s_prod/vars.yml); all three follow. kube_vip_address: "192.168.1.49" # Controller (dev container) reaches the kube-API DIRECTLY via the pfSense NAT # (192.168.2.50:6443 -> kube-vip VIP 192.168.1.49:6443) — no SSH tunnel. TLS is # validated against the VIP via tls-server-name (the VIP IP is in the serving cert's # tls-san list, so TLS through the NAT validates). Only the controller kubeconfig uses # these; k8sServiceHost (in-cluster) stays the real node IP. cluster_controller_api_server: "https://192.168.2.50:6443" cluster_controller_tls_server_name: "{{ kube_vip_address }}"