jakman85andClaude Opus 5.5
92944486d1
Gateway target -> fc81938b-...cfargotunnel.com, --cloudflare-proxied globally, L4 service publishers (unifi, crafty-mc) pinned DNS-only, Plex web UI and Headlamp disowned (LAN/VPN-only). L0 internal-services-publish-no-public-dns gains assertion (9) tying the Gateway target to the tunnel UUID, requiring cloudflare-proxied=false on every service-source publisher, and keeping the tunnel's 404 list and the unpublished set consistent. L2 dns-exposure fixture opts out of proxying. Co-Authored-By: Claude Opus 5.5 <[email protected]>
193 lines
14 KiB
YAML
193 lines
14 KiB
YAML
---
|
|
# ExternalDNS — publishes public DNS records for `public` services automatically (docs/09, R-DNS-02).
|
|
# Watches Gateway/HTTPRoute (+Service); manages records in Cloudflare. Phase 0 STUB.
|
|
apiVersion: helm.toolkit.fluxcd.io/v2
|
|
kind: HelmRelease
|
|
metadata:
|
|
name: external-dns
|
|
namespace: external-dns
|
|
spec:
|
|
interval: 30m
|
|
chart:
|
|
spec:
|
|
chart: external-dns
|
|
# chart 1.23.x = app v0.23.0. Floor >=1.23.0 because `annotationPrefix` below needs app >=0.22.
|
|
# Renovate maintains — but READ THE APP RELEASE NOTES on every 0.x minor: 0.22 changed the
|
|
# annotation prefix with no fallback and silently re-published every internal name (2026-11-07).
|
|
version: ">=1.23.0 <1.24.0"
|
|
sourceRef:
|
|
kind: HelmRepository
|
|
name: external-dns
|
|
namespace: external-dns
|
|
install:
|
|
remediation:
|
|
retries: 3
|
|
upgrade:
|
|
remediation:
|
|
retries: 3
|
|
values:
|
|
provider:
|
|
name: cloudflare
|
|
# ---------------------------------------------------------------------------------------------
|
|
# ANNOTATION PREFIX — PINNED TO THE LEGACY `alpha` PREFIX (2026-11-07). Do not remove.
|
|
#
|
|
# external-dns v0.22.0 (#6424) changed the default annotation prefix from
|
|
# `external-dns.alpha.kubernetes.io/` to `external-dns.kubernetes.io/` "with no fallback".
|
|
# EVERY external-dns annotation in this repo uses the alpha prefix: the Gateway's `target`
|
|
# (platform/gateway/gateway.yaml), the renderer's `controller: none` on `internal` routes
|
|
# (catalog/render.py -> apps/wave-*), and the L4 Services' `hostname` (crafty-mc, unifi).
|
|
# Renovate PR #129 moved the chart range to `<1.24.0` (app v0.21.0 -> v0.23.0) and on the first
|
|
# reconcile (2026-11-07T12:13Z) v0.23.0, reading only the GA prefix, saw none of them:
|
|
# * `target` ignored -> every public name flipped CNAME wan.<base> -> A 192.168.2.20 (RFC1918)
|
|
# * `controller: none` ignored -> internal jackett, pihole, semaphore PUBLISHED (R-DNS-03 broken)
|
|
# * `hostname` ignored -> unifi and crafty-mc A records DELETED
|
|
# Logged as `action=CREATE ... type=A` / `action=DELETE ... type=CNAME` for every name at once.
|
|
#
|
|
# The chart's `annotationPrefix` renders `--annotation-prefix=`, which v0.23.0 applies to every
|
|
# key it reads (source/annotations/annotations.go SetAnnotationPrefix: target, controller,
|
|
# hostname, ttl, ...), restoring exactly the v0.21 behaviour. Chosen over
|
|
# `--enable-legacy-annotation-prefix` (a dual-read migration aid upstream says will be removed)
|
|
# and over migrating the annotations (renderer + goldens + conformance = sensitive paths, and a
|
|
# flag-day across ~20 objects). Migrating to the GA prefix is the follow-up: change every
|
|
# annotation AND this value in ONE commit, never one without the other.
|
|
annotationPrefix: "external-dns.alpha.kubernetes.io/"
|
|
# Watch Gateway API routes (Cilium Gateway) + Services. docs/09: records rendered from intent.
|
|
#
|
|
# BOTH SOURCES ARE UNFILTERED HERE, AND THAT IS DELIBERATE — see the WHICH RESOURCES block
|
|
# below. Scoping is done PER RESOURCE by the renderer, not cluster-wide by this list.
|
|
sources:
|
|
- gateway-httproute
|
|
- service
|
|
# ---------------------------------------------------------------------------------------------
|
|
# WHICH RESOURCES MAY PUBLISH — `internal` services no longer can (R-DNS-03, 2026-09-14)
|
|
#
|
|
# THE DEFECT. `sources` above admits EVERY HTTPRoute on the shared Gateway, and each one's
|
|
# address is the Gateway VIP. docs/09 R-DNS-03 says `internal` services are unreachable from the
|
|
# public internet and that Pi-hole serves the internal split-horizon answers; the renderer
|
|
# contract said an `internal` service renders "no public DNS record". Neither sentence was
|
|
# enforced by anything, so six `exposure: internal` services (jackett, pihole, qbittorrent,
|
|
# radarr, semaphore, sonarr) published `<name>.home.keithjackson.dev` into this PUBLIC
|
|
# authoritative zone answering 192.168.2.20. That exposed the internal subnet, the Gateway VIP
|
|
# and a complete service inventory to anyone who could guess a label, and made every one of
|
|
# those names a DNS-rebind primitive — which is exactly why the SOHO router at 192.168.2.1
|
|
# strips RFC1918 answers and LAN clients need per-client dnsmasq overrides to reach their own
|
|
# services. Pre-state captured at the AUTHORITY and committed:
|
|
# .standup-dns/dns-20260914-pre-externaldns-scope.txt.
|
|
#
|
|
# THE MECHANISM — per-resource exclusion, emitted by catalog/render.py from `exposure`.
|
|
# An `internal` service's rendered HTTPRoute carries
|
|
# external-dns.alpha.kubernetes.io/controller: none
|
|
# ExternalDNS v0.21.0 (the DEPLOYED app version; read off the live prod Deployment, image
|
|
# registry.k8s.io/external-dns/external-dns:v0.21.0) skips any source object whose `controller`
|
|
# annotation is present and != its own `dns-controller`:
|
|
# source/annotations/processors.go IsControllerMismatch()
|
|
# source/gateway.go Endpoints(): if annotations.IsControllerMismatch(meta, src.rtKind) { continue }
|
|
# source/service.go Endpoints(): the identical check
|
|
# The gateway check runs BEFORE any hostname is resolved, so such a route cannot become an
|
|
# endpoint for any name it declares. PROVEN LIVE on prod before this landed, with a throwaway
|
|
# hostname and no existing record at risk: an annotated route published nothing at all, and
|
|
# ADDING the annotation to a route that already had a record produced
|
|
# action=DELETE record=<name> type=A and action=DELETE record=a-<name> type=TXT
|
|
# one ~60s reconcile later, and moved nothing else in the zone.
|
|
#
|
|
# WHY PER-RESOURCE AND NOT A CLUSTER-WIDE OPT-IN FILTER. The chart's `labelFilter`
|
|
# (--label-filter) is the stronger default posture — publish only what is labelled — and
|
|
# v0.21.0's own --help confirms BOTH configured sources support it ("currently supported by
|
|
# source types crd, gateway-httproute, ... service and ambassador-host"). It is NOT set here
|
|
# because its blast radius is not this defect's. Nine publishing objects in this repo are
|
|
# hand-authored platform/apps manifests with no `exposure` field for the renderer to read, and
|
|
# seven of them hold live records: auth (oauth2-proxy), crafty, crafty-mc, grafana, headlamp,
|
|
# the apex (homepage), unifi. Setting an opt-in filter DELETES all seven on the next reconcile,
|
|
# and whether each should be publicly resolvable at all is an OPERATOR decision per name, not a
|
|
# renderer one. They are enumerated in the shrink-only register in
|
|
# conformance/tests/l0-static/internal-services-publish-no-public-dns/check.sh, so the set
|
|
# cannot grow silently — adjudicate them there, then this filter becomes the right hardening.
|
|
#
|
|
# DELIBERATELY UNTOUCHED: `policy: sync`, `txtOwnerId`, `domainFilters`. Nothing above widens
|
|
# what this instance can see or own; per-resource exclusion only ever REMOVES an object from
|
|
# this instance's view, so it cannot make prod capable of deleting a record it does not own.
|
|
# ---------------------------------------------------------------------------------------------
|
|
# THE PUBLIC TARGET — THE CLOUDFLARE TUNNEL, PROXIED (2026-11-08; supersedes 2026-10-03)
|
|
#
|
|
# History: until 2026-10-03 every gateway-sourced record answered the Gateway VIP (RFC1918);
|
|
# from 2026-10-03 they were CNAMEs to `wan.${BASE_DOMAIN}` (DNS-only, the home WAN IP, with the
|
|
# router forwarding tcp/80+443 to the VIP). Both published an origin address of ours. Now:
|
|
# * platform/gateway/gateway.yaml sets `target: <tunnel-uuid>.cfargotunnel.com` on the shared
|
|
# Gateway. `target` is read in v0.23.0's gateway-httproute source ONLY from the GATEWAY
|
|
# (source/gateway.go: `overrides: annotations.TargetsFromTargetAnnotation(gw.Annotations)`),
|
|
# never from an HTTPRoute, so this one value moves every gateway-sourced record at once.
|
|
# * `--cloudflare-proxied` below orange-clouds every record by default. It is GLOBAL because
|
|
# that is the only practical lever: v0.23.0 reads the per-object `cloudflare-proxied`
|
|
# annotation from the ROUTE (or Service), never the Gateway (source/gateway.go:
|
|
# `annotations.ProviderSpecificAnnotations(annots)` with `annots := meta.Annotations` of the
|
|
# route), so a Gateway-level opt-in does not exist. A CNAME to cfargotunnel.com MUST be
|
|
# proxied — DNS-only it resolves to nothing routable — so "proxied by default" is the right
|
|
# default for this zone's gateway names.
|
|
# * The provider honours a per-object `"false"` over the global default (provider/cloudflare
|
|
# shouldBeProxied: ParseBool of the annotation overrides proxiedByDefault). The key it reads
|
|
# is `<annotationPrefix>cloudflare-proxied` (annotations.SetAnnotationPrefix re-derives
|
|
# CloudflarePrefix), i.e. `external-dns.alpha.kubernetes.io/cloudflare-proxied` here.
|
|
#
|
|
# THE `service` SOURCE MUST OPT OUT. The L4 LoadBalancers (crafty-mc, unifi) publish an A record
|
|
# at their own LB VIP; Cloudflare's proxy is HTTP(S)-only, so a proxied A for a game or UniFi
|
|
# inform port is broken by construction. Each carries `cloudflare-proxied: "false"` and stays
|
|
# DNS-only, exactly as before this change. conformance/tests/l0-static/
|
|
# internal-services-publish-no-public-dns (assertion 9b) FAILS on any `service`-source publisher
|
|
# without it, so a new L4 Service cannot silently become a proxied record.
|
|
#
|
|
# Internal services are unchanged: their routes carry `controller: none` (above), which is
|
|
# checked BEFORE the target is read, so they still publish nothing. Plex's web UI and Headlamp
|
|
# joined that set on 2026-11-08 (operator: LAN/VPN-only). `wan.${BASE_DOMAIN}` (ddns.yaml) has no
|
|
# external-dns TXT registry entry and is untouched; it now serves only WireGuard udp/51820.
|
|
#
|
|
# THE RISK THIS ADDS, AND WHAT GUARDS IT. If the annotation prefix ever stops being honoured
|
|
# again (the 2026-11-07 v0.23.0 incident), `target` is ignored and every name falls back to the
|
|
# Gateway's own address — and with this flag it would be published as a PROXIED A record at
|
|
# ${GATEWAY_VIP} (RFC1918; Cloudflare 1003/522s for every public name) while `controller: none`
|
|
# is also ignored and internal names publish. Assertion (8) of the L0 check above ties the
|
|
# prefix this tree writes to `values.annotationPrefix`; (8b) forbids overriding it here.
|
|
extraArgs:
|
|
- --cloudflare-proxied
|
|
# ---------------------------------------------------------------------------------------------
|
|
# CROSS-CLUSTER DNS OWNERSHIP. Read before changing either knob below.
|
|
#
|
|
# On 2026-09-04 prod's external-dns DELETED EVERY ONE of dev's public records. Both clusters ran
|
|
# `policy: sync` against the SAME Cloudflare zone, served the SAME hostnames, and — because
|
|
# NEITHER set txtOwnerId — both inherited the chart default owner id `default`. Prod therefore
|
|
# read dev's TXT registry entries as its own, found no local resource claiming those names, and
|
|
# garbage-collected them. `policy: sync` is correct and stays; what was missing was identity.
|
|
# Full write-up: infra/PROD-PROVISIONING.md §10.
|
|
#
|
|
# TWO INDEPENDENT GUARDS NOW, either of which alone prevents a recurrence:
|
|
# 1. txtOwnerId — per-environment identity; a cluster only ever touches records it owns.
|
|
# 2. domainFilters — per-environment name scope; prod cannot even see names outside its
|
|
# own subtree. Defence in depth, added after the incident.
|
|
# ---------------------------------------------------------------------------------------------
|
|
# Which names this instance may manage at all. Cloudflare-provider caveat, empirically hit
|
|
# 2026-08: this value is used for ZONE selection as well as record matching, and a filter that is
|
|
# a strict subdomain of the zone dropped the parent zone entirely ("no hosted zone matching record
|
|
# DNS Name") — so DEV keeps the zone apex `keithjackson.dev`, which is also what dev serves the
|
|
# apex of. PROD narrows to `prod.home.keithjackson.dev`. If prod's first reconcile logs no
|
|
# matching zone, that caveat still applies: prod creates NOTHING (fail-closed, never destructive)
|
|
# and the fallback is to widen prod's EXTERNALDNS_DOMAIN_FILTER back to the zone apex, leaving
|
|
# txtOwnerId as the sole guard. See clusters/base/cluster-vars.yaml.
|
|
domainFilters:
|
|
- "${EXTERNALDNS_DOMAIN_FILTER}"
|
|
# The TXT-registry owner id. MUST be distinct per environment — this is the knob whose absence
|
|
# caused the incident. Dev renders `default` (its live records are already registered under that
|
|
# owner; changing it would orphan them and duplicate every record). Prod renders `prod`.
|
|
txtOwnerId: "${EXTERNALDNS_TXT_OWNER_ID}"
|
|
policy: sync # create + update + delete records as services come and go — but ONLY
|
|
# records this owner id owns AND this domainFilter admits (see above).
|
|
# Cloudflare API token: SOPS-encrypted secret, referenced BY NAME only (OQ-4, ADR-0007).
|
|
env:
|
|
- name: CF_API_TOKEN
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: cloudflare-api-token
|
|
key: api-token
|
|
# R-DNS-02 PROVEN via the `service` source: the conformance/l2-behavioral/dns-exposure test
|
|
# creates a LoadBalancer Service with an external-dns hostname and asserts the record auto-appears
|
|
# in public DNS (then teardown + policy=sync removes it). gateway-httproute source wiring is the
|
|
# same provider/zone path; a route-based test can follow when app routes exist.
|