feat(env): tokenize hardcoded dev LAN addresses into cluster-vars (R-ENV-02)

R-ENV-02 promises dev and prod differ ONLY by scoped cluster-vars overrides. In fact dev LAN addresses were hardcoded into base manifests, so clusters/prod would have come up ANNOUNCING DEV ADDRESSES (192.168.1.x) on the prod LAN (192.168.2.x). New cluster-vars knobs (base default = dev literal, per the file's existing convention): K3S_API_HOST platform/cilium/helmrelease.yaml (+ the values.yaml mirror) CILIUM_LB_POOL_START platform-config/cilium/l2-ip-pool.yaml CILIUM_LB_POOL_STOP " UNIFI_L4_VIP apps/vms/unifi/50-service-l4.yaml CRAFTY_L4_VIP platform/crafty/service-l4.yaml S3_BACKUP_ENDPOINT platform/keycloak/postgres-cluster.yaml, apps/wave-0/{nextcloud, semaphore}-db.yaml, and platform/velero/helmrelease.yaml Also fixes GATEWAY_VIP's base default, which was still the RETIRED libvirt address 10.73.0.240, and adds the GATEWAY_VIP patch that clusters/prod was missing entirely. clusters/base/apps-vms.yaml gains postBuild.substituteFrom (it was the one consuming Kustomization without it, so a ${TOKEN} there would have shipped as a literal string). The envsubst contract is documented there: ${VAR} substitutes and escapes as $${VAR}; $VAR and $(cmd) are left alone; ${ARR[0]} hard-errors. The three UOS_URL expansions in the unifi cloud-init are escaped accordingly. PROD ADDRESS PLAN (operator, 2026-08-31): nodes .10/.11/.12 · control-plane VIP .9 (Ansible-side) · GATEWAY_VIP .20 Cilium LB pool .20-.30 CONTAINMENT INVARIANT, now documented at the pool and in the prod overlay: GATEWAY_VIP MUST equal CILIUM_LB_POOL_START. The shared Gateway pins no address, so Cilium LB-IPAM gives it the first free address of a matching pool block. An initial prod proposal put .20 outside a .21-.30 pool; the Gateway would have come up on .21 while split-horizon coredns-custom kept answering *.home.keithjackson.dev with .20 — an address nothing announces — blackholing in-cluster OIDC hairpin traffic while reporting healthy. The pool was widened to .20-.30. The control-plane VIP (.9) stays outside the pool so kube-vip and LB-IPAM can never contend. Values prod still cannot know stay INVALID `TBD-PROD-*` placeholders on purpose: base defaults are dev values, so an omitted patch means prod silently inherits a dev address — far worse than a loud failed apply. DEV IS BYTE-IDENTICAL. Verified by kustomize build over 13 paths before/after, then flux envsubst with the dev cluster-vars values applied to both sides: zero drift. The only deltas are the intended additive keys in the overlay ConfigMap itself. No unresolved ${TOKEN} survives. render-golden green (10 goldens); yamllint clean. Serves: R-ENV-02. Task: T-7.1 (prerequisite). Gate: MERGE GATE: PASS. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-31 22:43:36 -05:00
1 parent e3c44368c8
commit 3f980d2a17
@@ -89,7 +89,13 @@ spec:
            # userDataSecretRef only if needed). qemu-guest-agent gives the KubeVirt AgentConnected
            # signal (proves cloud-init ran); install-uos resolves the latest UOS linux-x64 build from
            # Ubiquiti's PUBLIC firmware API (no auth) and installs it non-interactively (UI on :11443).
-            # NOTE: apps-vms has NO postBuild.substituteFrom, so shell ${...} below is NOT envsubst'd.
+            # NOTE: apps-vms NOW carries postBuild.substituteFrom cluster-vars (needed for
+            # ${UNIFI_L4_VIP} in 50-service-l4.yaml), so Flux envsubst DOES run over this userData.
+            # Every shell brace-expansion below is therefore escaped as $${...}, which envsubst emits
+            # as a literal ${...} — verified with `flux envsubst`. Bare $VAR and $(...) are left alone
+            # by envsubst and need no escaping. NEVER write an unescaped ${...} here, and never write
+            # ${ARR[0]} even escaped-adjacent: envsubst hard-fails on `[` ("missing closing brace")
+            # and would break the whole apps-vms apply.
            userData: |-
              #cloud-config
              hostname: unifi
@@ -103,11 +109,11 @@ spec:
                  content: |
                    #!/usr/bin/env bash
                    set -euo pipefail
-                    if [ -z "${UOS_URL:-}" ]; then
+                    if [ -z "$${UOS_URL:-}" ]; then
                      UOS_URL="$(curl -fsSL https://fw-update.ui.com/api/firmware-latest | jq -r '._embedded.firmware|map(select(.product=="unifi-os-server" and .platform=="linux-x64"))|sort_by(.version_major,.version_minor,.version_patch)|last._links.data.href')"
                    fi
-                    [ -n "${UOS_URL}" ] && [ "${UOS_URL}" != "null" ] || { echo "no UOS URL"; exit 1; }
-                    t="$(mktemp)"; echo "downloading ${UOS_URL}"; curl -fL --retry 5 --retry-delay 10 -o "$t" "$UOS_URL"; chmod +x "$t"; echo "installing"; yes | "$t"; echo "done"
+                    [ -n "$${UOS_URL}" ] && [ "$${UOS_URL}" != "null" ] || { echo "no UOS URL"; exit 1; }
+                    t="$(mktemp)"; echo "downloading $${UOS_URL}"; curl -fL --retry 5 --retry-delay 10 -o "$t" "$UOS_URL"; chmod +x "$t"; echo "installing"; yes | "$t"; echo "done"
              runcmd:
                - [ systemctl, enable, --now, qemu-guest-agent ]
                - [ systemctl, enable, --now, podman.socket ]
@@ -8,8 +8,9 @@
 # CANNOT express: raw TCP on non-HTTP ports (8080 inform, 6789), UDP at all (3478 STUN, 10001
 # discovery), or the device-facing protocols that never speak HTTP. UniFi device ADOPTION rides
 # exactly those L4 ports. So the controller is exposed as a type=LoadBalancer Service whose VIP
-# is pulled from the CiliumLoadBalancerIPPool `default-pool` (192.168.1.53-.60) and ARP-announced
-# on the flat LAN by the existing CiliumL2AnnouncementPolicy `default-l2` — the SAME L2/LB-IPAM
+# is pulled from the CiliumLoadBalancerIPPool `default-pool` (dev 192.168.1.53-.60 — the
+# CILIUM_LB_POOL_START/STOP cluster-vars knobs) and ARP-announced on the flat LAN by the existing
+# CiliumL2AnnouncementPolicy `default-l2` — the SAME L2/LB-IPAM
 # machinery the Gateway ingress VIP already uses (platform-config/cilium/*). No new Cilium config
 # is required; these Services just consume the pool.
 #
@@ -23,8 +24,11 @@
 # into the guest. externalTrafficPolicy: Local preserves the LAN client source IP and keeps the
 # VIP pinned to the node running the VM (also what L2 announcement does).
 #
-# FLAG: confirm 192.168.1.54 is free, inside default-pool, and reserved in tofu (the Gateway
-# already holds one pool IP, typically .53). Adjust if it collides.
+# VIP: ${UNIFI_L4_VIP} (cluster-vars, R-ENV-02; dev 192.168.1.54, confirmed free, inside
+# default-pool and reserved in tofu — the Gateway already holds the pool's first IP, dev .53).
+# Substituted by the `apps-vms` Flux Kustomization, which carries postBuild.substituteFrom for
+# exactly this token; every shell ${...} in 30-virtualmachine.yaml's cloud-init is escaped as $${...}
+# accordingly. A new environment MUST set UNIFI_L4_VIP to a free address in ITS pool.
 #
 # REUSABLE PATTERN (the deferred `minecraft` :25565/L4 problem is the same shape): expose an
 # L4-only workload as a Cilium LoadBalancer on a pinned, L2-announced VIP with a per-app sharing
@@ -39,8 +43,8 @@ metadata:
    app.kubernetes.io/name: unifi
  annotations:
    lbipam.cilium.io/sharing-key: "unifi"
-    lbipam.cilium.io/ips: "192.168.1.54"
-    # external-dns (source: service) publishes A unifi.home.keithjackson.dev -> 192.168.1.54.
+    lbipam.cilium.io/ips: "${UNIFI_L4_VIP}"
+    # external-dns (source: service) publishes A unifi.home.keithjackson.dev -> the pinned VIP.
    # This is the browser + device target; UI is https://unifi.home.keithjackson.dev:11443.
    external-dns.alpha.kubernetes.io/hostname: "unifi.home.keithjackson.dev"
 spec:
@@ -65,7 +69,7 @@ metadata:
    app.kubernetes.io/name: unifi
  annotations:
    lbipam.cilium.io/sharing-key: "unifi"     # same key + same IP => shares the VIP with -tcp
-    lbipam.cilium.io/ips: "192.168.1.54"
+    lbipam.cilium.io/ips: "${UNIFI_L4_VIP}"
 spec:
  type: LoadBalancer
  externalTrafficPolicy: Local