feat(pihole): authoritative LAN DNS for the zone on udp/53 + tcp/53 at a pinned VIP

platform/coredns/coredns-custom.yaml has always said its split-horizon is for PODS and deferred LAN clients to "PiHole split-horizon". PiHole served no DNS: its Service was ClusterIP with ports 80 and 4180 only. The LAN half of R-DNS-03 that docs/09 and the README's Phase 2 row both claim did not exist, so LAN clients ran on hand-maintained per-client dnsmasq files - one went stale after the dev decommission and cost the operator sonarr.home.keithjackson.dev. The SOHO router that owns 192.168.2.0/24 cannot cover for it either: it strips RFC1918 answers for public domains as rebind protection and returns NOERROR with no answer. EXPOSURE (apps/wave-0/pihole-dns.yaml, hand-written, placed into wave 0 by render.py's EXTRA_UNITS - the schema has no field for a LoadBalancer Service, a pinned lbipam VIP, a sharing key, source ranges or UDP, the same reason apps/vms/unifi and platform/crafty hand-write theirs): - two Services, udp/53 and tcp/53, sharing one pinned VIP ${PIHOLE_DNS_VIP} = 192.168.2.23 on prod, the next free address in the .20-.30 pool (pool status IPsUsed 3 of 11; .20 Gateway, .21 unifi, .22 crafty; arping from a LAN host got no reply). - no external-dns hostname annotation: nothing public is created, and platform/externaldns/ is untouched. ANSWERS (catalog/pihole.yaml env - declarative, no reconciler): PiHole v6 takes every FTL setting as an FTLCONF_ environment variable and treats it as env-forced, applied ahead of pihole.toml on every cold start and read-only in the UI. That is a real declarative entry point, which the Keycloak and CNPG CronJob reconcilers exist precisely because their state lacks, so a reconciler here would be ceremony. misc.dnsmasq_lines carries a wildcard address=/<zone>/<GATEWAY_VIP> plus one longer-domain override per L4 service (unifi, crafty-mc, pihole-dns); dnsmasq resolves overlaps by longest matching domain, so precedence is structural and a newly onboarded Gateway-fronted service needs no DNS change. dns.upstreams is declared too - the live pod carried hand-set upstreams that no manifest knew about and that every restart discarded. externalTrafficPolicy Cluster, NOT Local, and loadBalancerSourceRanges for the LAN restriction. Measured on prod, Cilium 1.20.1: the L2 announcer does not filter lease candidates by which nodes hold a backend - the two Services landed on k3s-prod-3 and k3s-prod-2 while the pod ran on k3s-prod-1 - and it keeps renewing after the pod moves away, so with Local a reschedule leaves LAN DNS dead indefinitely. Cluster serves the VIP from any node; the LAN restriction moves to loadBalancerSourceRanges, which Cilium enforces in eBPF on the VIP before load-balancing (verified by narrowing the range to exclude the test client and back). The cost is SNAT: forwarded queries reach PiHole from the ingress node, so PiHole's per-client rate limit is disabled in the catalog or the whole LAN would count as one client against its 1000-per-60s default. Proven from a LAN host (192.168.2.132) against 192.168.2.23 with the pod and both leases on three DIFFERENT nodes: sonarr/grafana/nextcloud/keycloak/apex -> 192.168.2.20, unifi -> .21, crafty-mc -> .22, pihole-dns -> .23, recursion (cloudflare.com, github.com, debian.org) intact over both udp and tcp, AAAA NODATA, NXDOMAIN outside the zone, and still answering after a forced prod-1 -> prod-2 reschedule. Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-14 00:57:58 -05:00
1 parent ce7819a7da
commit fd43f332d6
@@ -99,7 +99,9 @@ data:
  # (CILIUM_LB_POOL_*, and keycloak-db's S3_BACKUP_ENDPOINT + CNPG_BACKUP_PATH: the Cluster lives at
  # platform/keycloak/postgres-cluster.yaml but is rendered by platform-config, which is the ONLY
  # tier that lists it), crafty (CRAFTY_L4_VIP), apps-wave-0 (nextcloud-db + semaphore-db:
-  # S3_BACKUP_ENDPOINT, CNPG_BACKUP_PATH), velero (S3_BACKUP_ENDPOINT, S3_BACKUP_REGION,
+  # S3_BACKUP_ENDPOINT, CNPG_BACKUP_PATH; pihole + pihole-dns: PIHOLE_DNS_VIP, LAN_CIDR, plus
+  # GATEWAY_VIP/UNIFI_L4_VIP/CRAFTY_L4_VIP read BY pihole to author its LAN split-horizon answers),
+  # velero (S3_BACKUP_ENDPOINT, S3_BACKUP_REGION,
  # VELERO_BACKUP_BUCKET, VELERO_BACKUP_PREFIX), apps-vms (UNIFI_L4_VIP). A ${TOKEN} in an
  # UNSUBSTITUTED path would ship the literal string into the cluster.
  #
@@ -159,6 +161,30 @@ data:
  # HTTPS Gateway cannot express (raw TCP/UDP), so they hold their own stable address:
  UNIFI_L4_VIP: "192.168.1.54"               # UniFi controller VM: inform/STUN/adoption (apps/vms/unifi)
  CRAFTY_L4_VIP: "192.168.1.55"              # Crafty: Minecraft Java (TCP) + Bedrock (UDP) game ports
+  # PiHole's DNS listener: udp/53 + tcp/53 for LAN CLIENTS (apps/wave-0/pihole-dns.yaml). It is a
+  # THIRD pinned L4 VIP for the same reason as the two above — the `shared` Gateway is an L7 HTTPS
+  # terminator and cannot express DNS at all (udp/53 especially). It must NOT be ${GATEWAY_VIP}:
+  # LAN clients are pointed at this address as their resolver, and it has to stay put across
+  # reschedules, so it is pinned by lbipam.cilium.io/ips rather than taken from the pool head.
+  #
+  # WHY A SEPARATE ADDRESS AND NOT THE GATEWAY VIP: every `<svc>.${BASE_DOMAIN}` name resolves TO
+  # the Gateway VIP. Serving DNS from the same address would be legal but conflates "the thing that
+  # answers where services live" with "a service", and it would put LAN resolution behind the
+  # Gateway Service's own lifecycle. Keep them separate.
+  #
+  # dev  -> 192.168.1.56  (base default; next free after crafty .55 in dev's .53-.60 pool)
+  # prod -> 192.168.2.23  (next free after crafty .22 in prod's .20-.30 pool; see clusters/prod)
+  PIHOLE_DNS_VIP: "192.168.1.56"             # PiHole: authoritative LAN DNS for ${BASE_DOMAIN}
+
+  # The FLAT LAN this cluster's nodes and LB VIPs live on, in CIDR form. Consumed as the ingress
+  # peer of the PiHole DNS CiliumNetworkPolicy (apps/wave-0/pihole-dns.yaml): a resolver reachable
+  # from `world` is an open resolver and an amplification reflector, so the allow is scoped to this
+  # CIDR, not to Cilium's `world` entity the way the unifi/crafty game-port allows are. It is the
+  # same /24 the node IPs and ${CILIUM_LB_POOL_START}..${CILIUM_LB_POOL_STOP} come out of.
+  #
+  # dev  -> 192.168.1.0/24  (Proxmox LAN; nodes .50/.51/.52)
+  # prod -> 192.168.2.0/24  (prod LAN; nodes .10/.11/.12)
+  LAN_CIDR: "192.168.1.0/24"                 # ingress scope for the PiHole DNS listener

  # --- Bulk (external NFS) storage host (docs/04 bulk tier) ---------------------------------------
  # THE address of the Proxmox host whose ZFS datasets back every `class: bulk` volume: the media
@@ -448,6 +448,23 @@ patches:
      - op: replace
        path: /data/CRAFTY_L4_VIP
        value: "192.168.2.22"
+      # PiHole's DNS listener VIP — udp/53 + tcp/53 for LAN clients (apps/wave-0/pihole-dns.yaml).
+      # 192.168.2.23 is the NEXT FREE address in the prod pool 192.168.2.20-.30, verified live
+      # 2026-09-14: the CiliumLoadBalancerIPPool `default-pool` reported IPsTotal 11 / IPsUsed 3 /
+      # IPsAvailable 8, and the only three allocations are .20 (cilium-gateway-shared), .21
+      # (unifi-l4-tcp/-udp) and .22 (crafty-l4-tcp/-udp). ARP-probed from a LAN host in the same
+      # broadcast domain (arping 192.168.2.23, 3 probes, 0 responses), so nothing off-cluster holds
+      # it either, and it is inside the pool block the operator already asserted sits outside the
+      # SOHO router's DHCP scope.
+      - op: replace
+        path: /data/PIHOLE_DNS_VIP
+        value: "192.168.2.23"
+      # The prod flat LAN. This is the 192.168.2.0/24 segment the SOHO ROUTER at 192.168.2.1 owns —
+      # NOT pfSense's 192.168.1.0/24, which is a different network and not in the LAN-DNS path.
+      # It scopes the ingress allow for PiHole's DNS listener to LAN clients only.
+      - op: replace
+        path: /data/LAN_CIDR
+        value: "192.168.2.0/24"
      # ------------------------------------------------------------------------------------------
      # BULK (EXTERNAL NFS) STORAGE HOST — *** NOT YET DECIDED. THIS PLACEHOLDER IS LOAD-BEARING. ***
      # ------------------------------------------------------------------------------------------