diff --git a/DECISIONS-LOG.md b/DECISIONS-LOG.md index a9d846a..2f49adc 100644 --- a/DECISIONS-LOG.md +++ b/DECISIONS-LOG.md @@ -3675,13 +3675,21 @@ dnsmasq forwarding the zone, so there was no fallback. A sibling file pinned the **Second cause — and why deleting the override would NOT have fixed it.** I was about to recommend simply removing both files, on the grounds that public DNS already answers correctly (it does: Cloudflare returns `192.168.2.20` for the apex and every HTTP name, `.21` for unifi, `.22` for -crafty-mc). Checking the upstream first showed that is not enough: **pfSense at `192.168.2.1` -returns `NOERROR` with `ANSWER: 0`** for `home.keithjackson.dev`, `sonarr.home.keithjackson.dev` and -even the bare `keithjackson.dev`, while resolving `cloudflare.com` fine and correctly returning the -zone's Cloudflare NS records. That is the signature of **DNS-rebind / private-address filtering**: -the resolver discards RFC1918 answers for public domains, and this zone legitimately points at +crafty-mc). Checking the upstream first showed that is not enough: **`192.168.2.1` returns `NOERROR` +with `ANSWER: 0`** for `home.keithjackson.dev`, `sonarr.home.keithjackson.dev` and even the bare +`keithjackson.dev`, while resolving `cloudflare.com` fine and correctly returning the zone's +Cloudflare NS records. That is the signature of **DNS-rebind / private-address filtering**: the +resolver discards RFC1918 answers for public domains, and this zone legitimately points at `192.168.2.x`. With the override removed, the workstation would have had no source of truth at all. +**CORRECTION (same day).** I first wrote that `192.168.2.1` was pfSense. It is not, and the operator +said so. This repo already records the topology: `192.168.2.0/24` is **pfSense's WAN** — the SOHO +network — while pfSense itself lives on `192.168.1.0/24` (`clusters/prod/kustomization.yaml:458`, +`clusters/base/cluster-vars.yaml:190`, DECISIONS-LOG 1772). `192.168.2.1` is the **SOHO router**, +which this log had ALREADY identified as *"both gateway and only IPv4 resolver"* and flagged as a +SPOF (entries 1318, 1468). pfSense is not in prod's DNS path at all. The measurement stands; the +attribution was wrong, and it invalidated my preferred fix — see below. + **The real gap.** `platform/coredns/coredns-custom.yaml` is explicit that its split-horizon is for pods — *"External/LAN clients keep their own answers (PiHole split-horizon, docs/09)"*. But **Pi-hole serves no DNS**: its Service is `ClusterIP` with ports 80 and 4180 only, no port 53 and no @@ -3693,12 +3701,17 @@ exactly why this broke — and why it will break again on the next address chang VIP `192.168.2.20`, with the two L4 services overriding it (dnsmasq applies the most specific match), all three verified against public DNS. Requires the operator's sudo; the file was staged for them. -**Proper fixes, none applied (operator's call):** -1. **pfSense domain override / `private-domain: "keithjackson.dev"`** on the DNS Resolver — one - change, fixes every LAN client, and removes the need for per-client files. Preferred. -2. **Expose Pi-hole's DNS to the LAN** (port 53 via a LoadBalancer IP) and make it authoritative for - the zone, which is what the design already claims. Bigger, but it is the documented architecture. -3. Keep per-client dnsmasq overrides — works, but it is N hand-maintained copies of one fact, and - this incident is what that costs. +**Proper fixes, none applied (operator's call), REORDERED after the correction:** +1. **Expose Pi-hole's DNS to the LAN** (port 53 on a LoadBalancer IP) and point clients at it. Now + the preferred option: it is the architecture the repo already claims, it does not depend on a + consumer SOHO router's feature set, and it simultaneously retires the single-resolver SPOF this + log flagged at entries 1318/1468. Note `rescued/fix/node-dns-resilient-upstream` (unmerged, stale + tree — cherry-pick only) already does the node-side half: "PiHole primary + pfSense fallback". +2. **Configure the SOHO router** to stop stripping RFC1918 answers for this zone, or to serve a + domain override. One change if the device supports it — many consumer routers do not, and it is + not under GitOps either way. This was my original first choice on the mistaken belief it was + pfSense; pfSense cannot help here because prod's resolver path never reaches it. +3. Keep per-client dnsmasq overrides — works today, but it is N hand-maintained copies of one fact, + and this incident is what that costs. Until 1 or 2 lands, README's Phase 2 "split-horizon CoreDNS" should be read as **in-cluster only**.