fix(vpn-ingress): the VPN ingress node must announce no L2 VIPs, or clients route nothing

A node cannot reach a LoadBalancer VIP whose Cilium L2 announcement lease it
holds itself: the announcer does not answer its own ARP request, so on the
lease holder the neighbour entry settles at FAILED and packets to that VIP die
before leaving the node. Measured on prod 2026-10-08, symmetric across 3 nodes
x 5 VIPs -- each node reached every VIP EXCEPT the ones it alone announced.

platform/vpn-ingress is uniquely exposed: it runs WireGuard on hostNetwork and
MASQUERADEs remote clients to the node's own address, so a VPN client takes
the node's LAN path and inherits its ARP failures. The router's udp/51820
forward pointed at k3s-prod-1, which held the shared Gateway lease, so all 16
Gateway hostnames were dead over the VPN -- while PiHole DNS kept answering
(its VIP has two Services and k3s-prod-2 held the other lease). The tunnel
handshook, names resolved, every app hung, and nothing reported it.

  on k3s-prod-1:  ip neigh show 192.168.2.20 -> FAILED      (its own VIP)
  on prod-2/3:    ip neigh show 192.168.2.20 -> lladdr ...  (prod-1), 443 OK

Mark the node and exclude it from announcing:

* node_baseline labels the forwarded node (node_baseline_vpn_ingress_node,
  k3s-prod-1 on prod) homelab.keithjackson.dev/vpn-ingress=true, and removes
  the label from any node that no longer matches so moving it MOVES the label.
* l2-announcement-policy.yaml excludes that label, so the node ARP-resolves
  every VIP. Fixes all VIPs for VPN clients, not just the one that collided.
  Cost: announcement candidacy drops from 3 nodes to 2.
* VpnIngressNodeAnnouncesL2VIP joins kube_lease_owner against kube_node_labels
  (label added to the kube-state-metrics allowlist), with
  VpnIngressNodeLabelMissing as its watchdog -- NotIn on an absent key matches,
  so a missing label would silently re-arm the whole failure AND leave the
  critical alert inert. Four promtool cases, including that inertness.

A node LABEL rather than a cluster-vars knob because the alert must read it
too, and extract_rules.py rightly refuses a ${...} placeholder in an alert
expr -- the rule unit-tested would not be the rule deployed. One source of
truth, and it touches no sensitive path.

Runbook/docs: step 3 of vpn-remote-access.md said "pick any node" and
"failover is this one line". Both false; the first caused this.

UNAPPLIED and unproven end to end; the Ansible tasks have never run. See the
handoff entry's third section.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

platform/monitoring/kube-prometheus-stack.yaml

@@ -281,8 +281,20 @@ spec:
       # without anyone having to remember to re-enable it later. Scoped to one label on one resource:
       # 3 extra series, no cardinality concern. Label name is sanitized by KSM to
       # `label_homelab_keithjackson_dev_egress_gateway` (`label_` + non-alphanumerics -> `_`).
+      # `vpn-ingress` (added 2026-10-17) marks the node the SOHO router forwards WAN udp/51820 to.
+      # platform-config/cilium/l2-announcement-policy.yaml excludes that node from announcing any
+      # LoadBalancer VIP, because a node cannot reach a VIP whose L2 announcement lease it holds
+      # itself and vpn-ingress MASQUERADEs remote clients to the node's own address (measured
+      # 2026-10-08: all 16 Gateway hostnames dead over the VPN while DNS still resolved). It is
+      # allowlisted HERE because that is the only way the invariant can be WATCHED:
+      # `VpnIngressNodeAnnouncesL2VIP` joins `kube_lease_owner` against this label, and the alert
+      # unit-test harness refuses a dollar-brace placeholder in an alert expr (written out in words:
+      # a literal one here is unparseable to envsubst and fails the whole Kustomization), so a knob
+      # could not have been used. Without this line BOTH that alert and its
+      # `VpnIngressNodeLabelMissing` watchdog are structurally inert. Same reasoning, and the same
+      # 3-extra-series cost, as the two egress labels below.
       metricLabelsAllowlist:
-        - "nodes=[homelab.keithjackson.dev/egress-gateway,homelab.keithjackson.dev/egress-uplink]"
+        - "nodes=[homelab.keithjackson.dev/egress-gateway,homelab.keithjackson.dev/egress-uplink,homelab.keithjackson.dev/vpn-ingress]"
       # --- VELERO SCHEDULE OBJECTS AS METRICS (added 2026-09-13) ---------------------------------
       # Velero exports NOTHING that enumerates Schedule objects and nothing that exposes
       # `spec.paused`. Verified on prod against velero v1.18.1: all 35 velero_* metric names that