feat(dns): publish public names as proxied CNAMEs to the Cloudflare Tunnel
Gateway target -> fc81938b-...cfargotunnel.com, --cloudflare-proxied globally, L4 service publishers (unifi, crafty-mc) pinned DNS-only, Plex web UI and Headlamp disowned (LAN/VPN-only). L0 internal-services-publish-no-public-dns gains assertion (9) tying the Gateway target to the tunnel UUID, requiring cloudflare-proxied=false on every service-source publisher, and keeping the tunnel's 404 list and the unpublished set consistent. L2 dns-exposure fixture opts out of proxying. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5.5 committed
1 parent
2e777e13ca
commit
92944486d1
16 files changed
+417
-88
No files matched your search
+4
-3
@@ -92,9 +92,10 @@
|
||||
# the NAME axis only. R-NET-04 / the edge port-forward posture owns the other.
|
||||
# - Anything about the seven non-catalog publishers' own correctness. The register records WHAT
|
||||
# each publishes so the claim is reviewable, which is all a static check can offer.
|
||||
# - Anything about the five `public` services' TARGET. They still resolve to the Gateway VIP
|
||||
# rather than a routable address; that is an open operator decision recorded in
|
||||
# platform/externaldns/helmrelease.yaml, and asserting a target here would hard-code a guess.
|
||||
# - That the public names RESOLVE to Cloudflare at runtime. check.sh (9) pins the TREE: the
|
||||
# Gateway target is the tunnel UUID in platform/cloudflared/config.yaml, proxying is on, the L4
|
||||
# `service` publishers opt out, and every name the tunnel 404s is unpublished. Whether Cloudflare
|
||||
# then answers anycast for them is a live read (`dig @1.1.1.1`), not a static one.
|
||||
# - It does not read ../home-network-k3s. The spec is read-only (R-TEST-06).
|
||||
# =============================================================================
|
||||
apiVersion: chainsaw.kyverno.io/v1alpha1
|
||||
|
||||
@@ -29,7 +29,17 @@
|
||||
# ci/l0.sh stage (whole-repo-invariants-wired fails the gate if one appears);
|
||||
# (8) the external-dns HelmRelease declares `annotationPrefix` explicitly and it matches the
|
||||
# prefix of EVERY external-dns annotation in the tree and in catalog/render.py — added after
|
||||
# the 2026-11-07 v0.23.0 default-prefix change silently voided (1)-(3) at runtime.
|
||||
# the 2026-11-07 v0.23.0 default-prefix change silently voided (1)-(3) at runtime;
|
||||
# (9) THE CLOUDFLARE TUNNEL WIRING (2026-11-08) — every public name is a PROXIED CNAME to the tunnel:
|
||||
# (a) the shared Gateway's `target` is EXACTLY `<tunnel:>.cfargotunnel.com`, the UUID read from
|
||||
# platform/cloudflared/config.yaml, and the HelmRelease sets `--cloudflare-proxied`;
|
||||
# (b) every `service`-source publisher carries `cloudflare-proxied: "false"` (L4 A records stay
|
||||
# DNS-only), and no HTTPRoute opts out of proxying (a DNS-only tunnel CNAME is dead);
|
||||
# (c) no hostname the tunnel maps to `http_status:404` has a public record — no undisowned route
|
||||
# names it, and the only `service`-source publishers of one are the declared L4 DNS-only
|
||||
# names (unifi, crafty-mc); conversely every disowned route's hostname IS 404'd by the
|
||||
# tunnel, so the "unpublished" and "refused at the tunnel" lists cannot drift apart;
|
||||
# (d) the operator's LAN/VPN-only names (plex, headlamp) are disowned everywhere they appear.
|
||||
#
|
||||
# WHAT THIS DOES *NOT* PROVE (honest gaps — it is STATIC, offline, cluster-free):
|
||||
# - That the record is gone from Cloudflare. Live read; belongs beside
|
||||
@@ -39,8 +49,11 @@
|
||||
# both the A and its `a-` TXT within ~60s) — but a manual proof is not a gate.
|
||||
# - That an internal name is UNREACHABLE. A port-forward plus a literal IP needs no DNS record.
|
||||
# This check owns the NAME axis only.
|
||||
# - Anything about the five public services' TARGET. They still answer with the Gateway VIP; the
|
||||
# routable-target decision is open and recorded in platform/externaldns/helmrelease.yaml.
|
||||
# - That the tunnel actually SERVES the public names, or that Cloudflare honoured `proxied`. (9)
|
||||
# proves the tree asks for a proxied CNAME to the right tunnel; the live answer (`dig @1.1.1.1`
|
||||
# returning Cloudflare anycast) is a runtime read. Nor does (8)/(9) prove the RUNNING controller
|
||||
# honours the prefix — a future chart bump that drops it would publish PROXIED A records at the
|
||||
# Gateway VIP; (8) only keeps the tree and the HelmRelease value consistent.
|
||||
# - It does not read ../home-network-k3s. The spec is read-only (R-TEST-06).
|
||||
#
|
||||
# Exit 0 = all assertions pass. Exit 1 = a failure. Exit 3 (honest SKIP, which the runner turns
|
||||
@@ -141,9 +154,12 @@ def docs_of(p: Path):
|
||||
# unadjudicated set cannot grow without a review event. Same ratchet reasoning as
|
||||
# KNOWN_EXTERNAL_MONITORS in servicemonitor-ports-resolve.
|
||||
#
|
||||
# EVERY ENTRY HERE IS A NAME THAT STILL ANSWERS WITH AN RFC1918 ADDRESS IN A PUBLIC ZONE. That is
|
||||
# the honest state after this change, and it is deliberate: converting them is an operator
|
||||
# decision per name (is grafana meant to be publicly resolvable at all?), not a renderer change,
|
||||
# WHAT THE ENTRIES PUBLISH NOW (2026-11-08). Every HTTPRoute entry that is not disowned publishes a
|
||||
# PROXIED CNAME to the Cloudflare Tunnel (the shared Gateway's `target`, assertion 9a), so no origin
|
||||
# address of ours is in the public zone for it. Headlamp is disowned (operator: LAN/VPN-only, 9d).
|
||||
# The two `service`-source entries still publish their RFC1918 LB VIP as a DNS-only A record —
|
||||
# deliberate (L4 ports cannot ride Cloudflare's HTTP proxy), and pinned DNS-only by 9b. Whether each
|
||||
# remaining name should be public at all stays an operator decision per name, not a renderer change,
|
||||
# and a cluster-wide ExternalDNS opt-in filter — the available hardening, see
|
||||
# platform/externaldns/helmrelease.yaml — would DELETE all of them at once the moment it is set.
|
||||
#
|
||||
@@ -151,27 +167,28 @@ def docs_of(p: Path):
|
||||
# =============================================================================================
|
||||
KNOWN_NON_CATALOG_PUBLISHERS: dict[str, str] = {
|
||||
"platform/oauth2-proxy/httproute.yaml:oauth2-proxy/oauth2-proxy":
|
||||
"publishes auth.${BASE_DOMAIN} -> the Gateway VIP. The oauth2-proxy login endpoint is a "
|
||||
"publishes auth.${BASE_DOMAIN} -> the tunnel (proxied CNAME). The oauth2-proxy login endpoint is a "
|
||||
"platform component, not a catalog service; it must be reachable by every browser that "
|
||||
"any forward-auth app redirects, so whether it is PUBLICLY resolvable is coupled to "
|
||||
"whether the five public services are — an operator decision, not a renderer one.",
|
||||
"platform/keycloak/httproute.yaml:keycloak/keycloak":
|
||||
"publishes keycloak.${BASE_DOMAIN} -> the Gateway VIP, and it is the route that actually "
|
||||
"publishes keycloak.${BASE_DOMAIN} -> the tunnel (proxied CNAME), and it is the route that actually "
|
||||
"OWNS that record (the TXT registry entry reads resource=httproute/keycloak/keycloak, not "
|
||||
"the catalog-rendered keycloak-8080 route, because external-dns merges duplicate "
|
||||
"hostnames and this one won). Keycloak is `exposure: public` in the catalog, so the name "
|
||||
"is meant to be public either way; it is listed because the PUBLISHING object is the "
|
||||
"hand-authored platform route, which carries no exposure field.",
|
||||
"platform/monitoring/grafana-httproute.yaml:monitoring/grafana":
|
||||
"publishes grafana.${BASE_DOMAIN} -> the Gateway VIP. An admin dashboard; almost "
|
||||
"certainly should not be publicly resolvable, but it is not a catalog service and "
|
||||
"withdrawing it is an operator call with the same LAN-resolution prerequisite as the six "
|
||||
"internal services (Pi-hole must answer first).",
|
||||
"publishes grafana.${BASE_DOMAIN} -> the tunnel (proxied CNAME). An admin dashboard "
|
||||
"behind Keycloak; it is not a catalog service, so withdrawing it is an operator call per "
|
||||
"name (it stayed public in the 2026-11-08 tunnel decision).",
|
||||
"platform/headlamp/httproute.yaml:headlamp/headlamp":
|
||||
"publishes headlamp.${BASE_DOMAIN} -> the Gateway VIP. A Kubernetes web UI with native "
|
||||
"Keycloak OIDC; same adjudication as grafana.",
|
||||
"publishes NOTHING since 2026-11-08: the route carries `controller: none` (operator "
|
||||
"decision: Headlamp is LAN/VPN-only) and the tunnel 404s the name. A Kubernetes web UI "
|
||||
"with native Keycloak OIDC; listed because it is a hand-authored route, and pinned "
|
||||
"disowned by (9d).",
|
||||
"platform/homepage/httproute.yaml:homepage/homepage":
|
||||
"publishes the APEX ${BASE_DOMAIN} itself -> the Gateway VIP, via the https-apex "
|
||||
"publishes the APEX ${BASE_DOMAIN} itself -> the tunnel (proxied CNAME), via the https-apex "
|
||||
"listener. The apex is not a per-service name and cannot be withdrawn by an exposure "
|
||||
"declaration.",
|
||||
"runbooks/serving/httproute.yaml:runbooks/runbooks":
|
||||
@@ -180,16 +197,16 @@ KNOWN_NON_CATALOG_PUBLISHERS: dict[str, str] = {
|
||||
"entry names the homepage route). Deliberately un-authenticated — see the manifest's own "
|
||||
"header for why gating runbooks on Keycloak is a loop with no exit.",
|
||||
"platform/crafty/httproute.yaml:crafty/crafty":
|
||||
"publishes crafty.${BASE_DOMAIN} -> the Gateway VIP (the Minecraft admin UI, gated by an "
|
||||
"publishes crafty.${BASE_DOMAIN} -> the tunnel (proxied CNAME) (the Minecraft admin UI, gated by an "
|
||||
"oauth2-proxy sidecar). Platform component, no exposure field.",
|
||||
"platform/crafty/service-l4.yaml:crafty/crafty-l4-tcp":
|
||||
"the `service` SOURCE, not a route: publishes crafty-mc.${BASE_DOMAIN} -> "
|
||||
"${CRAFTY_L4_VIP} (192.168.2.22) for the raw Minecraft TCP/UDP game ports the HTTPS "
|
||||
"Gateway cannot express. A game server is the one case where a public name may be "
|
||||
"genuinely wanted; the address it answers with is still RFC1918.",
|
||||
"genuinely wanted; the address it answers with is still RFC1918, DNS-only (9b).",
|
||||
"apps/vms/unifi/50-service-l4.yaml:unifi/unifi-l4-tcp":
|
||||
"the `service` source: publishes unifi.${BASE_DOMAIN} -> ${UNIFI_L4_VIP} "
|
||||
"(192.168.2.21) for the UniFi controller VM's inform/STUN/adoption ports. Hand-authored "
|
||||
"(192.168.2.21), DNS-only (9b), for the UniFi controller VM's inform/STUN/adoption ports. Hand-authored "
|
||||
"under apps/vms (a VM, not a rendered container service); the catalog's only VM entry is "
|
||||
"vm-demo, which declares no ingressRoutes and publishes nothing.",
|
||||
}
|
||||
@@ -515,6 +532,113 @@ check(not bad_render,
|
||||
f"(8d) catalog/render.py emits external-dns keys under {bad_render}, but the controller "
|
||||
f"reads {ANNOTATION_PREFIX!r}")
|
||||
|
||||
# =============================================================================================
|
||||
# (9) THE CLOUDFLARE TUNNEL WIRING (2026-11-08). The public names are PROXIED CNAMEs to the tunnel.
|
||||
# =============================================================================================
|
||||
CF_CONFIG = ROOT / "platform/cloudflared/config.yaml"
|
||||
GATEWAY_FILE = ROOT / "platform/gateway/gateway.yaml"
|
||||
TARGET_KEY = ANNOTATION_PREFIX + "target"
|
||||
PROXIED_KEY = ANNOTATION_PREFIX + "cloudflare-proxied"
|
||||
UUID_RE = re.compile(r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$")
|
||||
|
||||
# The L4 names the tunnel 404s that are NONETHELESS published — as DNS-only A records from the
|
||||
# `service` source, at their own LB VIP, because they are not HTTP and never touch the tunnel
|
||||
# (operator decision 2026-11-08). Shrink-only, and must agree with the tunnel's 404 list (9c).
|
||||
L4_DNS_ONLY_NAMES = {"unifi.${BASE_DOMAIN}", "crafty-mc.${BASE_DOMAIN}"}
|
||||
# The operator's LAN/VPN-only HTTP names (2026-11-08). Pinned here as well as in the tunnel config, so
|
||||
# deleting one from config.yaml cannot quietly also release it from (9c).
|
||||
OPERATOR_LAN_ONLY_NAMES = {"plex.${BASE_DOMAIN}", "headlamp.${BASE_DOMAIN}"}
|
||||
|
||||
cf = next(iter(docs_of(CF_CONFIG)), None)
|
||||
check(cf is not None and isinstance(cf.get("ingress"), list),
|
||||
f"(9) {CF_CONFIG.relative_to(ROOT)} unreadable or has no `ingress` list — cannot prove what "
|
||||
f"the tunnel refuses or which tunnel the Gateway must target")
|
||||
cf = cf or {}
|
||||
tunnel = str(cf.get("tunnel") or "")
|
||||
check(bool(UUID_RE.match(tunnel)),
|
||||
f"(9a) platform/cloudflared/config.yaml `tunnel:` is {tunnel!r}, not a literal tunnel UUID — "
|
||||
f"the Gateway target is derived from it")
|
||||
gw = next((d for d in docs_of(GATEWAY_FILE) if d.get("kind") == "Gateway"
|
||||
and ident(d) == ("gateway", "shared")), None)
|
||||
check(gw is not None, f"(9a) {GATEWAY_FILE.relative_to(ROOT)} holds no Gateway gateway/shared")
|
||||
want_target = f"{tunnel}.cfargotunnel.com"
|
||||
got_target = annotations_of(gw or {}).get(TARGET_KEY)
|
||||
check(got_target == want_target,
|
||||
f"(9a) Gateway gateway/shared `{TARGET_KEY}` is {got_target!r}; it must be EXACTLY "
|
||||
f"{want_target!r} (the tunnel in platform/cloudflared/config.yaml). Any other value publishes "
|
||||
f"every public name at the wrong place — a dead cfargotunnel.com name (Cloudflare 1016), or "
|
||||
f"an origin address of ours.")
|
||||
check("--cloudflare-proxied" in [str(a) for a in (extra if isinstance(extra, list) else [])]
|
||||
or (isinstance(extra, dict) and str(extra.get("cloudflare-proxied", "")).lower() == "true"),
|
||||
"(9a) external-dns HelmRelease extraArgs must include `--cloudflare-proxied`: a DNS-only "
|
||||
"CNAME to <uuid>.cfargotunnel.com resolves to nothing routable, so every public name is dead")
|
||||
|
||||
for rel, d in dns_services:
|
||||
if is_disowned(d):
|
||||
continue
|
||||
if HOSTNAME_KEY not in annotations_of(d):
|
||||
continue
|
||||
v = annotations_of(d).get(PROXIED_KEY)
|
||||
check(str(v).lower() == "false",
|
||||
f"(9b) {rel} Service {'/'.join(ident(d))} publishes via the `service` source but carries "
|
||||
f"`{PROXIED_KEY}: {v!r}`, not \"false\". With --cloudflare-proxied global it would become "
|
||||
f"an orange-clouded A record, which cannot carry L4 traffic and hides nothing (the LB VIP "
|
||||
f"is RFC1918). Every `service`-source publisher must opt out explicitly.")
|
||||
for rel, d in routes:
|
||||
v = annotations_of(d).get(PROXIED_KEY)
|
||||
check(v is None or str(v).lower() == "true",
|
||||
f"(9b) {rel} HTTPRoute {'/'.join(ident(d))} sets `{PROXIED_KEY}: {v!r}`. A gateway-sourced "
|
||||
f"record is a CNAME to the tunnel; DNS-only, it is dead. Disown the route instead "
|
||||
f"(`{CONTROLLER_KEY}: none`) if the name should not be public.")
|
||||
|
||||
refused = {str(r.get("hostname")) for r in cf.get("ingress") or []
|
||||
if isinstance(r, dict) and r.get("hostname") and r.get("service") == "http_status:404"}
|
||||
check(len(refused) > 0, "(9c) the tunnel config 404s no hostname — parse contract broken (the "
|
||||
"operator's LAN/VPN-only list is known to be non-empty)")
|
||||
published_l4: set[str] = set()
|
||||
for rel, d in routes:
|
||||
hosts = {str(h) for h in (d.get("spec") or {}).get("hostnames") or []}
|
||||
leaked = sorted(hosts & refused)
|
||||
if is_disowned(d):
|
||||
unrefused = sorted(hosts - refused)
|
||||
check(not unrefused,
|
||||
f"(9c) {rel} HTTPRoute {'/'.join(ident(d))} is disowned (no public record) but the "
|
||||
f"tunnel does not 404 {unrefused}: add them to platform/cloudflared/config.yaml above "
|
||||
f"the wildcard, or the two halves of \"LAN/VPN-only\" have drifted apart.")
|
||||
continue
|
||||
check(not leaked,
|
||||
f"(9c) {rel} HTTPRoute {'/'.join(ident(d))} PUBLISHES {leaked}, which the tunnel maps to "
|
||||
f"http_status:404 (LAN/VPN-only). The name would be a public proxied record that serves "
|
||||
f"only 404s — disown the route (`{CONTROLLER_KEY}: none`) or adjudicate the name.")
|
||||
for rel, d in dns_services:
|
||||
if is_disowned(d):
|
||||
continue
|
||||
hosts = {h.strip() for h in str(annotations_of(d).get(HOSTNAME_KEY) or "").split(",")
|
||||
if h.strip()}
|
||||
for h in sorted(hosts & refused):
|
||||
published_l4.add(h)
|
||||
check(h in L4_DNS_ONLY_NAMES,
|
||||
f"(9c) {rel} Service {'/'.join(ident(d))} publishes {h}, which the tunnel 404s, and it "
|
||||
f"is not a declared L4 DNS-only name {sorted(L4_DNS_ONLY_NAMES)}")
|
||||
check(L4_DNS_ONLY_NAMES <= refused,
|
||||
f"(9c) L4 DNS-only names {sorted(L4_DNS_ONLY_NAMES - refused)} are not 404'd by the tunnel; a "
|
||||
f"stray CNAME to the tunnel for them would hit the Gateway's catch-all instead of a 404")
|
||||
check(published_l4 == L4_DNS_ONLY_NAMES,
|
||||
f"(9c) L4_DNS_ONLY_NAMES is shrink-only: {sorted(L4_DNS_ONLY_NAMES - published_l4)} no longer "
|
||||
f"published by any `service`-source publisher — delete them from the list")
|
||||
|
||||
check(OPERATOR_LAN_ONLY_NAMES <= refused,
|
||||
f"(9d) operator LAN/VPN-only names {sorted(OPERATOR_LAN_ONLY_NAMES - refused)} are missing "
|
||||
f"from the tunnel's http_status:404 rules (platform/cloudflared/config.yaml)")
|
||||
for name in sorted(OPERATOR_LAN_ONLY_NAMES):
|
||||
naming = [(rel, d) for rel, d in routes
|
||||
if name in {str(h) for h in (d.get("spec") or {}).get("hostnames") or []}]
|
||||
check(len(naming) > 0, f"(9d) no HTTPRoute names {name} — stale OPERATOR_LAN_ONLY_NAMES entry")
|
||||
for rel, d in naming:
|
||||
check(is_disowned(d),
|
||||
f"(9d) {rel} HTTPRoute {'/'.join(ident(d))} names LAN/VPN-only {name} but is not "
|
||||
f"disowned, so external-dns publishes it publicly (operator decision 2026-11-08)")
|
||||
|
||||
# =============================================================================================
|
||||
if FAILURES:
|
||||
for f in FAILURES:
|
||||
|
||||
@@ -64,6 +64,12 @@ spec:
|
||||
annotations:
|
||||
# conformance-dns.<this cluster's BASE_DOMAIN> — never a literal apex.
|
||||
external-dns.alpha.kubernetes.io/hostname: (join('.', ['conformance-dns', $baseDomain]))
|
||||
# DNS-ONLY (2026-11-08). external-dns runs with --cloudflare-proxied globally (the
|
||||
# gateway names point at the Cloudflare Tunnel). Without this opt-out the probe
|
||||
# record is orange-clouded, 1.1.1.1 answers Cloudflare anycast instead of the LB IP,
|
||||
# and the exact-IP poll below can never match. Same opt-out every real
|
||||
# `service`-source publisher carries (unifi, crafty-mc).
|
||||
external-dns.alpha.kubernetes.io/cloudflare-proxied: "false"
|
||||
spec:
|
||||
type: LoadBalancer
|
||||
# No real backend needed — Cilium LB-IPAM assigns a VIP regardless, and
|
||||
|
||||
Reference in new issue
Block a user