feat(pihole): authoritative LAN DNS for the zone on udp/53 + tcp/53 at a pinned VIP

platform/coredns/coredns-custom.yaml has always said its split-horizon is for
PODS and deferred LAN clients to "PiHole split-horizon". PiHole served no DNS:
its Service was ClusterIP with ports 80 and 4180 only. The LAN half of R-DNS-03
that docs/09 and the README's Phase 2 row both claim did not exist, so LAN
clients ran on hand-maintained per-client dnsmasq files - one went stale after
the dev decommission and cost the operator sonarr.home.keithjackson.dev. The
SOHO router that owns 192.168.2.0/24 cannot cover for it either: it strips
RFC1918 answers for public domains as rebind protection and returns NOERROR
with no answer.

EXPOSURE (apps/wave-0/pihole-dns.yaml, hand-written, placed into wave 0 by
render.py's EXTRA_UNITS - the schema has no field for a LoadBalancer Service, a
pinned lbipam VIP, a sharing key, source ranges or UDP, the same reason
apps/vms/unifi and platform/crafty hand-write theirs):
  - two Services, udp/53 and tcp/53, sharing one pinned VIP
    ${PIHOLE_DNS_VIP} = 192.168.2.23 on prod, the next free address in the
    .20-.30 pool (pool status IPsUsed 3 of 11; .20 Gateway, .21 unifi, .22
    crafty; arping from a LAN host got no reply).
  - no external-dns hostname annotation: nothing public is created, and
    platform/externaldns/ is untouched.

ANSWERS (catalog/pihole.yaml env - declarative, no reconciler): PiHole v6 takes
every FTL setting as an FTLCONF_ environment variable and treats it as
env-forced, applied ahead of pihole.toml on every cold start and read-only in
the UI. That is a real declarative entry point, which the Keycloak and CNPG
CronJob reconcilers exist precisely because their state lacks, so a reconciler
here would be ceremony. misc.dnsmasq_lines carries a wildcard
address=/<zone>/<GATEWAY_VIP> plus one longer-domain override per L4 service
(unifi, crafty-mc, pihole-dns); dnsmasq resolves overlaps by longest matching
domain, so precedence is structural and a newly onboarded Gateway-fronted
service needs no DNS change. dns.upstreams is declared too - the live pod
carried hand-set upstreams that no manifest knew about and that every restart
discarded.

externalTrafficPolicy Cluster, NOT Local, and loadBalancerSourceRanges for the
LAN restriction. Measured on prod, Cilium 1.20.1: the L2 announcer does not
filter lease candidates by which nodes hold a backend - the two Services landed
on k3s-prod-3 and k3s-prod-2 while the pod ran on k3s-prod-1 - and it keeps
renewing after the pod moves away, so with Local a reschedule leaves LAN DNS
dead indefinitely. Cluster serves the VIP from any node; the LAN restriction
moves to loadBalancerSourceRanges, which Cilium enforces in eBPF on the VIP
before load-balancing (verified by narrowing the range to exclude the test
client and back). The cost is SNAT: forwarded queries reach PiHole from the
ingress node, so PiHole's per-client rate limit is disabled in the catalog or
the whole LAN would count as one client against its 1000-per-60s default.

Proven from a LAN host (192.168.2.132) against 192.168.2.23 with the pod and
both leases on three DIFFERENT nodes: sonarr/grafana/nextcloud/keycloak/apex
-> 192.168.2.20, unifi -> .21, crafty-mc -> .22, pihole-dns -> .23, recursion
(cloudflare.com, github.com, debian.org) intact over both udp and tcp, AAAA
NODATA, NXDOMAIN outside the zone, and still answering after a forced
prod-1 -> prod-2 reschedule.

Co-Authored-By: Claude Opus 5 <[email protected]>

clusters/prod/kustomization.yaml

@@ -448,6 +448,23 @@ patches:
       - op: replace
         path: /data/CRAFTY_L4_VIP
         value: "192.168.2.22"
+      # PiHole's DNS listener VIP — udp/53 + tcp/53 for LAN clients (apps/wave-0/pihole-dns.yaml).
+      # 192.168.2.23 is the NEXT FREE address in the prod pool 192.168.2.20-.30, verified live
+      # 2026-09-14: the CiliumLoadBalancerIPPool `default-pool` reported IPsTotal 11 / IPsUsed 3 /
+      # IPsAvailable 8, and the only three allocations are .20 (cilium-gateway-shared), .21
+      # (unifi-l4-tcp/-udp) and .22 (crafty-l4-tcp/-udp). ARP-probed from a LAN host in the same
+      # broadcast domain (arping 192.168.2.23, 3 probes, 0 responses), so nothing off-cluster holds
+      # it either, and it is inside the pool block the operator already asserted sits outside the
+      # SOHO router's DHCP scope.
+      - op: replace
+        path: /data/PIHOLE_DNS_VIP
+        value: "192.168.2.23"
+      # The prod flat LAN. This is the 192.168.2.0/24 segment the SOHO ROUTER at 192.168.2.1 owns —
+      # NOT pfSense's 192.168.1.0/24, which is a different network and not in the LAN-DNS path.
+      # It scopes the ingress allow for PiHole's DNS listener to LAN clients only.
+      - op: replace
+        path: /data/LAN_CIDR
+        value: "192.168.2.0/24"
       # ------------------------------------------------------------------------------------------
       # BULK (EXTERNAL NFS) STORAGE HOST — *** NOT YET DECIDED. THIS PLACEHOLDER IS LOAD-BEARING. ***
       # ------------------------------------------------------------------------------------------