Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 3 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

merge: internal services stop advertising themselves in public DNS

Browse Source 
Six exposure:internal services published A records into the public Cloudflare
zone answering 192.168.2.20, disclosing the internal subnet, the Gateway VIP
and a full service inventory, and forming a DNS-rebind primitive that is why
the SOHO router strips these answers in the first place.

The renderer now stamps external-dns.alpha.kubernetes.io/controller: none on
every HTTPRoute it renders for an internal service, derived from the exposure
field alone — so a service that omits exposure defaults to internal and is
likewise unpublishable. Established against the deployed v0.21.0's own source,
where the controller check runs before hostname resolution in both configured
sources, then proven live on a throwaway hostname including that adding the
annotation to an existing record triggers the DELETE.

Deletes exactly 12 records: the A and TXT pair for jackett, pihole,
qbittorrent, radarr, semaphore and sonarr. A 39-record recovery baseline is
committed alongside.

Safe to land before the DHCP change, contrary to my earlier caution: the
router returns NOTHING for all six names today, so these records serve nobody
on the LAN, while Pi-hole at 192.168.2.23 already answers for anyone pointed
at it.

The five exposure:public services are untouched. They remain unreachable
publicly, which is a separate problem: the legacy edge at 136.51.115.252
returns HTTP 403 for our hostnames and serves a placeholder certificate, so
nothing is reverse-proxying them today.
This commit is contained in:
jakman85 committed 2026-09-18 20:50:43 -05:00
2 parents 3173254cc4 4080065aa7
commit b184a42729
13 files changed
+992 -4
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

catalog/render.py
+164 -1
View File
@@ -80,6 +80,18 @@ BULK_NFS_SERVER = "${BULK_NFS_SERVER}"
BULK_NFS_MOUNT_OPTIONS = ["nfsvers=4.1", "hard", "noatime", "timeo=600", "retrans=2"]
GATEWAY_NAME = "shared"
GATEWAY_NS = "gateway"
# ExternalDNS per-resource scoping (R-DNS-03; see _externaldns_route_annotations for the full WHY
# and for the v0.21.0 source citation). The KEY is external-dns's own `ControllerKey`
# (annotationPrefix + "controller"); external-dns skips any source object whose value is PRESENT
# and != `ControllerValue` ("dns-controller"), so the VALUE only has to be some other string.
# `none` is chosen because it reads as intent in `kubectl get httproute -o yaml` rather than as a
# typo — "no DNS controller owns this route" — and because the one value that must never be
# emitted here is external-dns's own, which a literal `none` cannot drift into.
EXTERNALDNS_CONTROLLER_KEY = "external-dns.alpha.kubernetes.io/controller"
EXTERNALDNS_CONTROLLER_DISOWN = "none"
# The value external-dns claims for itself. Held here ONLY so the renderer and the L0 guard can
# assert the disown value is not it; never emitted.
EXTERNALDNS_CONTROLLER_CLAIMED = "dns-controller"
KP_RELEASE = "kube-prometheus-stack" # ServiceMonitor/PrometheusRule discovery label
MONITORING_NS = "monitoring" # where Prometheus runs; the only admitted scrape source
@@ -836,12 +848,15 @@ def r_in_cluster_client_policies(s: dict) -> list:
def _route_meta(s: dict, route: dict) -> dict:
"""HTTPRoute metadata: auth wiring hints PLUS the ExternalDNS scoping derived from
`exposure` (_externaldns_route_annotations — R-DNS-03: an `internal` service renders no
public DNS record, and now cannot)."""
meta = {
"name": f"{s['name']}-{route['port']}",
"namespace": s["name"],
"labels": {"app.kubernetes.io/name": s["name"]},
}
ann = _auth_annotations(s)
ann = {**_auth_annotations(s), **_externaldns_route_annotations(s)}
if ann:
meta["annotations"] = ann
return meta
@@ -1039,6 +1054,64 @@ def _auth_annotations(s: dict) -> dict:
return {}
def _externaldns_route_annotations(s: dict) -> dict:
"""ExternalDNS scoping, rendered FROM `exposure` (renderer-contract §exposure; R-DNS-03).
`internal` renders NO public DNS record. That sentence was in the contract from the start and
nothing enforced it: ExternalDNS watched EVERY HTTPRoute the shared Gateway carried, so an
`internal` service's route published `<name>.${BASE_DOMAIN}` into the PUBLIC authoritative
Cloudflare zone, answering with the Gateway VIP — an RFC1918 address. Six of them did
(jackett, pihole, qbittorrent, radarr, semaphore, sonarr), which published the internal
subnet, the Gateway VIP and a full service inventory to anyone who asked, and made every one
of those names a DNS-rebind primitive that the SOHO router strips on the way back in.
THE MECHANISM, verified against the DEPLOYED version (chart >=1.21.0 <1.22.0, app v0.21.0 —
`registry.k8s.io/external-dns/external-dns:v0.21.0`, read off the live prod Deployment), not
from memory. external-dns v0.21.0 `source/annotations/processors.go`:
func IsControllerMismatch(entity metav1.ObjectMetaAccessor, rType string) bool {
value, ok := entity.GetObjectMeta().GetAnnotations()[ControllerKey]
if ok && value != ControllerValue { ...; return true }
return false
}
with `ControllerKey = "external-dns.alpha.kubernetes.io/controller"` and
`ControllerValue = "dns-controller"`. `source/gateway.go` calls it per route inside
`Endpoints()`, BEFORE any hostname is resolved:
if annotations.IsControllerMismatch(meta, src.rtKind) { continue }
so an HTTPRoute carrying any other value is not merely unpublished — it never becomes an
endpoint at all, for any hostname it names. `source/service.go` applies the identical check,
so the same annotation scopes the `service` source too if a rendered Service ever grows an
external-dns hostname.
WHY THIS IS STRUCTURAL AND NOT INCIDENTAL. The annotation is derived HERE, from `exposure`,
with no per-service list anywhere: every HTTPRoute this renderer emits for an `internal`
service carries it, and ci/render-golden.sh proves `apps/` reproduces byte-for-byte from
`catalog/`, so an `internal` service cannot EXIST in the applied tree without it. Flipping
`exposure: internal` to `public` is the only edit that can publish a name, which is exactly
the operator-facing declaration R-DNS-03 is written in terms of. The L0 check
conformance/tests/l0-static/internal-services-publish-no-public-dns re-proves all of that on
every gate run, including that this function reads `exposure` rather than naming services.
WHY NOT A CLUSTER-WIDE OPT-IN FILTER (`--label-filter`, chart `labelFilter`). It exists, and
v0.21.0's own `--help` names both sources in use as supporting it ("currently supported by
source types crd, gateway-httproute, ... service and ambassador-host"), so a "publish only
what is labelled" posture is available and is STRONGER by default. It is not taken here
because its blast radius is not this defect's: seven published names come from hand-authored
platform/apps manifests that carry no `exposure` field and that this renderer never sees
(auth/oauth2-proxy, crafty + crafty-mc, grafana, headlamp, the homepage apex, unifi). An
opt-in filter DELETES all seven the moment it is set, and whether each should stay published
is an operator decision, not a renderer one. Per-resource exclusion has exactly the blast
radius of the reported defect. The filter remains the available hardening once those seven
are adjudicated — see platform/externaldns/helmrelease.yaml.
"""
if s["exposure"] == "internal":
return {EXTERNALDNS_CONTROLLER_KEY: EXTERNALDNS_CONTROLLER_DISOWN}
return {}
def r_servicemonitor(s: dict):
"""ServiceMonitor — rendered ONLY when the SD says WHERE metrics are served.
@@ -2097,6 +2170,96 @@ def _selftest() -> int:
"a non-exposed service must not render a gateway-ingress allow (least privilege)",
)
# 4b-dns) exposure -> ExternalDNS SCOPING on the HTTPRoute (R-DNS-03). `internal` renders no
# public DNS record and must be INCAPABLE of rendering one: its route carries
# external-dns's own controller annotation set to something other than `dns-controller`,
# which v0.21.0's gateway-httproute source (source/gateway.go ->
# annotations.IsControllerMismatch) uses to skip the route before any hostname is
# resolved. `public` must NOT carry it, or the five public names would vanish instead.
# Both directions are asserted here because only asserting the internal half would pass
# just as happily if the renderer stamped EVERY route and silently unpublished the zone.
dns_int_docs = render(
{
"name": "di",
"source": {"image": "a:1"},
"exposure": "internal",
"ingressRoutes": [{"host": "di.home.keithjackson.dev", "port": 80}],
"auth": {"mode": "none"},
}
)
dns_int_routes = [d[1] for d in dns_int_docs if d[1]["kind"] == "HTTPRoute"]
check(
len(dns_int_routes) == 1,
"an internal exposed service must still render its HTTPRoute (LAN reachability is "
"unchanged — only the PUBLIC record is withdrawn)",
)
if dns_int_routes:
iann = dns_int_routes[0]["metadata"].get("annotations", {})
check(
iann.get(EXTERNALDNS_CONTROLLER_KEY) == EXTERNALDNS_CONTROLLER_DISOWN,
"an internal service's HTTPRoute must carry the ExternalDNS disown annotation",
)
check(
iann.get(EXTERNALDNS_CONTROLLER_KEY) != EXTERNALDNS_CONTROLLER_CLAIMED,
"the disown value must never be external-dns's own `dns-controller` (that CLAIMS "
"the route rather than releasing it — the one value that inverts the guarantee)",
)
dns_pub_docs = render(
{
"name": "dp",
"source": {"image": "a:1"},
"exposure": "public",
"ingressRoutes": [{"host": "dp.home.keithjackson.dev", "port": 80}],
"auth": {"mode": "native"},
}
)
dns_pub_routes = [d[1] for d in dns_pub_docs if d[1]["kind"] == "HTTPRoute"]
check(len(dns_pub_routes) == 1, "a public service must render its HTTPRoute")
if dns_pub_routes:
check(
EXTERNALDNS_CONTROLLER_KEY
not in dns_pub_routes[0]["metadata"].get("annotations", {}),
"a public service's HTTPRoute must NOT carry the ExternalDNS disown annotation",
)
# The DEFAULT posture is `internal` (renderer-contract "Default posture"), so a
# ServiceDefinition that simply omits `exposure` must also be unpublishable. Omitting the
# field is the likeliest way a new service arrives, and it is the case a per-service
# allow-list would miss.
dns_dflt_docs = render(
{
"name": "dd",
"source": {"image": "a:1"},
"ingressRoutes": [{"host": "dd.home.keithjackson.dev", "port": 80}],
"auth": {"mode": "none"},
}
)
dns_dflt_route = next(d[1] for d in dns_dflt_docs if d[1]["kind"] == "HTTPRoute")
check(
dns_dflt_route["metadata"]["annotations"].get(EXTERNALDNS_CONTROLLER_KEY)
== EXTERNALDNS_CONTROLLER_DISOWN,
"a ServiceDefinition that OMITS `exposure` defaults to internal and must also be "
"unpublishable (default posture, renderer-contract)",
)
# The scoping annotation must COEXIST with the auth hints rather than replace them: both are
# merged into one `annotations` map, and a forward-auth internal app is the overlap case.
dns_fa_docs = render(
{
"name": "df",
"source": {"image": "a:1"},
"exposure": "internal",
"ingressRoutes": [{"host": "df.home.keithjackson.dev", "port": 80}],
"auth": {"mode": "forward-auth", "groups": ["platform-admins"]},
}
)
dns_fa_ann = next(
d[1] for d in dns_fa_docs if d[1]["kind"] == "HTTPRoute"
)["metadata"]["annotations"]
check(
dns_fa_ann.get(EXTERNALDNS_CONTROLLER_KEY) == EXTERNALDNS_CONTROLLER_DISOWN
and dns_fa_ann.get(f"{LBL}/auth-mode") == "forward-auth",
"DNS scoping and auth-mode hints must COEXIST on one internal forward-auth route",
)
# 4c) inClusterClients -> one ADDITIVE, least-privilege ingress allow per entry that admits a
# PLATFORM controller's NAMESPACE (not the reserved `ingress` identity, not `all`) to ONE
# declared port. This is what keeps a cluster whose GitOps source is an in-cluster Service