Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 3 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

feat(dns): exposure: internal can no longer publish a public record

Browse Source 
docs/09 R-DNS-03 and schemas/renderer-contract.md both said an `internal`
service renders no public DNS record. Nothing enforced either sentence.
ExternalDNS's `gateway-httproute` source admitted EVERY HTTPRoute on the shared
Gateway, so six `exposure: internal` services — jackett, pihole, qbittorrent,
radarr, semaphore, sonarr — published `<name>.home.keithjackson.dev` into the
public authoritative Cloudflare zone answering with the Gateway VIP
192.168.2.20. That published the internal subnet, the Gateway VIP and a full
service inventory to anyone who could guess a label, and made each of those
names a DNS-rebind primitive — which is why the SOHO router strips RFC1918
answers and LAN clients need per-client dnsmasq overrides to reach their own
services.

THE MECHANISM, established against the DEPLOYED version rather than from
memory. The live prod Deployment runs
registry.k8s.io/external-dns/external-dns:v0.21.0. In v0.21.0,
source/annotations/processors.go IsControllerMismatch() skips a source object
whose `external-dns.alpha.kubernetes.io/controller` annotation is present and
!= its own `dns-controller`, and source/gateway.go applies that check inside
Endpoints() BEFORE any hostname is resolved; source/service.go applies the
identical check, covering the second configured source. catalog/render.py now
emits that annotation on every HTTPRoute it renders for an `internal` service,
derived from `exposure` alone — no per-service list — so a service that omits
`exposure` inherits the internal default and is unpublishable too. Verified
LIVE on prod on a throwaway hostname with no existing record at risk: the
annotated route published nothing, and adding the annotation to a route that
already had a record produced action=DELETE for both its A record and its `a-`
TXT registry entry one ~60s reconcile later, moving nothing else in the zone.

WHY PER-RESOURCE AND NOT A CLUSTER-WIDE OPT-IN FILTER. `labelFilter` is the
stronger default posture and v0.21.0 supports it for both configured sources,
but nine publishing objects here are hand-authored platform/apps manifests with
no `exposure` field, seven of which hold live records (auth, crafty, crafty-mc,
grafana, headlamp, the apex, unifi). An opt-in filter deletes all seven at
once, and whether each should be publicly resolvable is an operator decision
per name. Per-resource exclusion has exactly the blast radius of the defect;
the seven are enumerated in a shrink-only register so the unadjudicated set
cannot grow silently, and the filter stays available as hardening afterwards.

`policy: sync`, `txtOwnerId` and `domainFilters` are UNTOUCHED. Per-resource
exclusion only ever removes an object from this instance's view, so nothing
here makes prod capable of deleting a record it does not own.

WHAT THIS DELETES, and it is not reversible by a revert alone: 12 records — the
A record and the `a-`-prefixed TXT registry entry for each of jackett, pihole,
qbittorrent, radarr, semaphore and sonarr. The zone has no
`*.home.keithjackson.dev` wildcard and `*.keithjackson.dev` does not match two
labels, so each becomes a public NXDOMAIN with no fallback. The complete
pre-state is captured at the authority and committed as a recovery baseline
(.standup-dns/dns-20260914-pre-externaldns-scope.txt), with the restore
procedure for both halves of a record.

MERGE ORDER IS NOT OPTIONAL: feat/pihole-lan-dns-authority must land and be
verified first. Removing these records while the LAN still depends on them,
with the router stripping RFC1918 and no internal authority yet, breaks name
resolution outright.

The five `exposure: public` services are LEFT UNCHANGED and still answer with
the Gateway VIP, so they remain publicly unreachable. That is two open operator
decisions, both written up in platform/externaldns/helmrelease.yaml: what
address they should answer with (136.51.115.252 is the legacy non-k3s stack
serving a placeholder cert, not established as correct for this cluster), and
how to separate them from the non-public names that share the Gateway —
because in v0.21.0 the target annotation is read only from the GATEWAY and a
ListenerSet, never from an HTTPRoute, so a per-service override is not
expressible while they all attach to `shared`. No knob is added for it yet: one
that cannot take effect invites someone to set it and expect one.

Guard: conformance/tests/l0-static/internal-services-publish-no-public-dns, a
whole-repo-scoped L0 static check (no ci/l0.sh stage — the
whole-repo-invariants-wired meta-test forbids one). It asserts both directions
(present on internal, absent on public), the `service`-source half, that the
renderer DERIVES the annotation from `exposure` and cannot emit external-dns's
own claiming value, proves the shipped renderer end-to-end on
internal/public/omitted-field fixtures, and ratchets the non-catalog register.
Proven to fail on the pre-fix tree (9 of 53 assertions, naming all six
services) and pass after (56/56).

Schema 1.6.0 -> 1.7.0: no field changes, a mapping change to `exposure`.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed 2026-09-14 00:43:12 -05:00
1 parent ce7819a7da
commit 4080065aa7
13 files changed
+992 -4
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

catalog/render.py
+164 -1
View File
@@ -80,6 +80,18 @@ BULK_NFS_SERVER = "${BULK_NFS_SERVER}"
BULK_NFS_MOUNT_OPTIONS = ["nfsvers=4.1", "hard", "noatime", "timeo=600", "retrans=2"]
GATEWAY_NAME = "shared"
GATEWAY_NS = "gateway"
# ExternalDNS per-resource scoping (R-DNS-03; see _externaldns_route_annotations for the full WHY
# and for the v0.21.0 source citation). The KEY is external-dns's own `ControllerKey`
# (annotationPrefix + "controller"); external-dns skips any source object whose value is PRESENT
# and != `ControllerValue` ("dns-controller"), so the VALUE only has to be some other string.
# `none` is chosen because it reads as intent in `kubectl get httproute -o yaml` rather than as a
# typo — "no DNS controller owns this route" — and because the one value that must never be
# emitted here is external-dns's own, which a literal `none` cannot drift into.
EXTERNALDNS_CONTROLLER_KEY = "external-dns.alpha.kubernetes.io/controller"
EXTERNALDNS_CONTROLLER_DISOWN = "none"
# The value external-dns claims for itself. Held here ONLY so the renderer and the L0 guard can
# assert the disown value is not it; never emitted.
EXTERNALDNS_CONTROLLER_CLAIMED = "dns-controller"
KP_RELEASE = "kube-prometheus-stack" # ServiceMonitor/PrometheusRule discovery label
MONITORING_NS = "monitoring" # where Prometheus runs; the only admitted scrape source
@@ -836,12 +848,15 @@ def r_in_cluster_client_policies(s: dict) -> list:
def _route_meta(s: dict, route: dict) -> dict:
"""HTTPRoute metadata: auth wiring hints PLUS the ExternalDNS scoping derived from
`exposure` (_externaldns_route_annotations — R-DNS-03: an `internal` service renders no
public DNS record, and now cannot)."""
meta = {
"name": f"{s['name']}-{route['port']}",
"namespace": s["name"],
"labels": {"app.kubernetes.io/name": s["name"]},
}
ann = _auth_annotations(s)
ann = {**_auth_annotations(s), **_externaldns_route_annotations(s)}
if ann:
meta["annotations"] = ann
return meta
@@ -1039,6 +1054,64 @@ def _auth_annotations(s: dict) -> dict:
return {}
def _externaldns_route_annotations(s: dict) -> dict:
"""ExternalDNS scoping, rendered FROM `exposure` (renderer-contract §exposure; R-DNS-03).
`internal` renders NO public DNS record. That sentence was in the contract from the start and
nothing enforced it: ExternalDNS watched EVERY HTTPRoute the shared Gateway carried, so an
`internal` service's route published `<name>.${BASE_DOMAIN}` into the PUBLIC authoritative
Cloudflare zone, answering with the Gateway VIP — an RFC1918 address. Six of them did
(jackett, pihole, qbittorrent, radarr, semaphore, sonarr), which published the internal
subnet, the Gateway VIP and a full service inventory to anyone who asked, and made every one
of those names a DNS-rebind primitive that the SOHO router strips on the way back in.
THE MECHANISM, verified against the DEPLOYED version (chart >=1.21.0 <1.22.0, app v0.21.0 —
`registry.k8s.io/external-dns/external-dns:v0.21.0`, read off the live prod Deployment), not
from memory. external-dns v0.21.0 `source/annotations/processors.go`:
func IsControllerMismatch(entity metav1.ObjectMetaAccessor, rType string) bool {
value, ok := entity.GetObjectMeta().GetAnnotations()[ControllerKey]
if ok && value != ControllerValue { ...; return true }
return false
}
with `ControllerKey = "external-dns.alpha.kubernetes.io/controller"` and
`ControllerValue = "dns-controller"`. `source/gateway.go` calls it per route inside
`Endpoints()`, BEFORE any hostname is resolved:
if annotations.IsControllerMismatch(meta, src.rtKind) { continue }
so an HTTPRoute carrying any other value is not merely unpublished — it never becomes an
endpoint at all, for any hostname it names. `source/service.go` applies the identical check,
so the same annotation scopes the `service` source too if a rendered Service ever grows an
external-dns hostname.
WHY THIS IS STRUCTURAL AND NOT INCIDENTAL. The annotation is derived HERE, from `exposure`,
with no per-service list anywhere: every HTTPRoute this renderer emits for an `internal`
service carries it, and ci/render-golden.sh proves `apps/` reproduces byte-for-byte from
`catalog/`, so an `internal` service cannot EXIST in the applied tree without it. Flipping
`exposure: internal` to `public` is the only edit that can publish a name, which is exactly
the operator-facing declaration R-DNS-03 is written in terms of. The L0 check
conformance/tests/l0-static/internal-services-publish-no-public-dns re-proves all of that on
every gate run, including that this function reads `exposure` rather than naming services.
WHY NOT A CLUSTER-WIDE OPT-IN FILTER (`--label-filter`, chart `labelFilter`). It exists, and
v0.21.0's own `--help` names both sources in use as supporting it ("currently supported by
source types crd, gateway-httproute, ... service and ambassador-host"), so a "publish only
what is labelled" posture is available and is STRONGER by default. It is not taken here
because its blast radius is not this defect's: seven published names come from hand-authored
platform/apps manifests that carry no `exposure` field and that this renderer never sees
(auth/oauth2-proxy, crafty + crafty-mc, grafana, headlamp, the homepage apex, unifi). An
opt-in filter DELETES all seven the moment it is set, and whether each should stay published
is an operator decision, not a renderer one. Per-resource exclusion has exactly the blast
radius of the reported defect. The filter remains the available hardening once those seven
are adjudicated — see platform/externaldns/helmrelease.yaml.
"""
if s["exposure"] == "internal":
return {EXTERNALDNS_CONTROLLER_KEY: EXTERNALDNS_CONTROLLER_DISOWN}
return {}
def r_servicemonitor(s: dict):
"""ServiceMonitor — rendered ONLY when the SD says WHERE metrics are served.
@@ -2078,6 +2151,96 @@ def _selftest() -> int:
"a non-exposed service must not render a gateway-ingress allow (least privilege)",
)
# 4b-dns) exposure -> ExternalDNS SCOPING on the HTTPRoute (R-DNS-03). `internal` renders no
# public DNS record and must be INCAPABLE of rendering one: its route carries
# external-dns's own controller annotation set to something other than `dns-controller`,
# which v0.21.0's gateway-httproute source (source/gateway.go ->
# annotations.IsControllerMismatch) uses to skip the route before any hostname is
# resolved. `public` must NOT carry it, or the five public names would vanish instead.
# Both directions are asserted here because only asserting the internal half would pass
# just as happily if the renderer stamped EVERY route and silently unpublished the zone.
dns_int_docs = render(
{
"name": "di",
"source": {"image": "a:1"},
"exposure": "internal",
"ingressRoutes": [{"host": "di.home.keithjackson.dev", "port": 80}],
"auth": {"mode": "none"},
}
)
dns_int_routes = [d[1] for d in dns_int_docs if d[1]["kind"] == "HTTPRoute"]
check(
len(dns_int_routes) == 1,
"an internal exposed service must still render its HTTPRoute (LAN reachability is "
"unchanged — only the PUBLIC record is withdrawn)",
)
if dns_int_routes:
iann = dns_int_routes[0]["metadata"].get("annotations", {})
check(
iann.get(EXTERNALDNS_CONTROLLER_KEY) == EXTERNALDNS_CONTROLLER_DISOWN,
"an internal service's HTTPRoute must carry the ExternalDNS disown annotation",
)
check(
iann.get(EXTERNALDNS_CONTROLLER_KEY) != EXTERNALDNS_CONTROLLER_CLAIMED,
"the disown value must never be external-dns's own `dns-controller` (that CLAIMS "
"the route rather than releasing it — the one value that inverts the guarantee)",
)
dns_pub_docs = render(
{
"name": "dp",
"source": {"image": "a:1"},
"exposure": "public",
"ingressRoutes": [{"host": "dp.home.keithjackson.dev", "port": 80}],
"auth": {"mode": "native"},
}
)
dns_pub_routes = [d[1] for d in dns_pub_docs if d[1]["kind"] == "HTTPRoute"]
check(len(dns_pub_routes) == 1, "a public service must render its HTTPRoute")
if dns_pub_routes:
check(
EXTERNALDNS_CONTROLLER_KEY
not in dns_pub_routes[0]["metadata"].get("annotations", {}),
"a public service's HTTPRoute must NOT carry the ExternalDNS disown annotation",
)
# The DEFAULT posture is `internal` (renderer-contract "Default posture"), so a
# ServiceDefinition that simply omits `exposure` must also be unpublishable. Omitting the
# field is the likeliest way a new service arrives, and it is the case a per-service
# allow-list would miss.
dns_dflt_docs = render(
{
"name": "dd",
"source": {"image": "a:1"},
"ingressRoutes": [{"host": "dd.home.keithjackson.dev", "port": 80}],
"auth": {"mode": "none"},
}
)
dns_dflt_route = next(d[1] for d in dns_dflt_docs if d[1]["kind"] == "HTTPRoute")
check(
dns_dflt_route["metadata"]["annotations"].get(EXTERNALDNS_CONTROLLER_KEY)
== EXTERNALDNS_CONTROLLER_DISOWN,
"a ServiceDefinition that OMITS `exposure` defaults to internal and must also be "
"unpublishable (default posture, renderer-contract)",
)
# The scoping annotation must COEXIST with the auth hints rather than replace them: both are
# merged into one `annotations` map, and a forward-auth internal app is the overlap case.
dns_fa_docs = render(
{
"name": "df",
"source": {"image": "a:1"},
"exposure": "internal",
"ingressRoutes": [{"host": "df.home.keithjackson.dev", "port": 80}],
"auth": {"mode": "forward-auth", "groups": ["platform-admins"]},
}
)
dns_fa_ann = next(
d[1] for d in dns_fa_docs if d[1]["kind"] == "HTTPRoute"
)["metadata"]["annotations"]
check(
dns_fa_ann.get(EXTERNALDNS_CONTROLLER_KEY) == EXTERNALDNS_CONTROLLER_DISOWN
and dns_fa_ann.get(f"{LBL}/auth-mode") == "forward-auth",
"DNS scoping and auth-mode hints must COEXIST on one internal forward-auth route",
)
# 4c) inClusterClients -> one ADDITIVE, least-privilege ingress allow per entry that admits a
# PLATFORM controller's NAMESPACE (not the reserved `ingress` identity, not `all`) to ONE
# declared port. This is what keeps a cluster whose GitOps source is an in-cluster Service