feat(dns): publish public names as proxied CNAMEs to the Cloudflare Tunnel
Gateway target -> fc81938b-...cfargotunnel.com, --cloudflare-proxied globally, L4 service publishers (unifi, crafty-mc) pinned DNS-only, Plex web UI and Headlamp disowned (LAN/VPN-only). L0 internal-services-publish-no-public-dns gains assertion (9) tying the Gateway target to the tunnel UUID, requiring cloudflare-proxied=false on every service-source publisher, and keeping the tunnel's 404 list and the unpublished set consistent. L2 dns-exposure fixture opts out of proxying. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5.5 committed
1 parent
2e777e13ca
commit
92944486d1
16 files changed
+417
-88
No files matched your search
@@ -20,29 +20,44 @@ A wildcard `*.keithjackson.dev` matches only one label deep, so `<svc>.home.keit
|
||||
its own record. **`internal` services need NO public record** — PiHole resolves them to the internal
|
||||
Gateway VIP (split-horizon). ExternalDNS manages **`public`** services' records only.
|
||||
|
||||
## Public target + in-cluster DDNS (2026-10-03)
|
||||
## Public target: the Cloudflare Tunnel, proxied (2026-11-08)
|
||||
|
||||
Gateway-published records are **CNAMEs to `wan.home.keithjackson.dev`**, not the Gateway VIP:
|
||||
Gateway-published records are **proxied CNAMEs to the Cloudflare Tunnel**
|
||||
(`platform/cloudflared`), not the WAN IP and not the Gateway VIP:
|
||||
|
||||
```
|
||||
plex.home.keithjackson.dev CNAME wan.home.keithjackson.dev <- external-dns (Gateway `target` annotation)
|
||||
wan.home.keithjackson.dev A <current WAN IP> <- cloudflare-ddns (ddns.yaml), every 5m
|
||||
gitea.home.keithjackson.dev CNAME (proxied) fc81938b-5855-47f1-8a27-6286f56dda93.cfargotunnel.com <- external-dns
|
||||
unifi.home.keithjackson.dev A (DNS-only) 192.168.2.21 <- external-dns, `service` source (L4)
|
||||
crafty-mc.home... A (DNS-only) 192.168.2.22 <- external-dns, `service` source (L4)
|
||||
wan.home.keithjackson.dev A (DNS-only) <current WAN IP> <- cloudflare-ddns (ddns.yaml): WireGuard udp/51820 ONLY
|
||||
```
|
||||
|
||||
- The `target` annotation is on the **shared Gateway** (`platform/gateway/gateway.yaml`), so it
|
||||
applies to every route that publishes at all. `internal` routes (`controller: none`) still publish
|
||||
nothing.
|
||||
- `ddns.yaml` runs `favonia/cloudflare-ddns` in this namespace with the same `cloudflare-api-token`.
|
||||
It detects the WAN IP through Cloudflare's trace endpoint, so the pod **must not** carry
|
||||
`egress-class: vpn`.
|
||||
- The `target` annotation is on the **shared Gateway** (`platform/gateway/gateway.yaml`); external-dns
|
||||
reads it from the Gateway only, so it applies to every route that publishes at all. Its UUID must
|
||||
equal `tunnel:` in `platform/cloudflared/config.yaml` (L0 check
|
||||
`internal-services-publish-no-public-dns` (9a)).
|
||||
- `--cloudflare-proxied` (HelmRelease `extraArgs`) orange-clouds every record by default — the only
|
||||
practical lever, because v0.23.0 reads the per-object `cloudflare-proxied` annotation from the
|
||||
route/Service, never the Gateway. **Every `service`-source publisher must carry
|
||||
`external-dns.alpha.kubernetes.io/cloudflare-proxied: "false"`** (L4 ports cannot ride the HTTP
|
||||
proxy); (9b) fails the gate otherwise.
|
||||
- `internal` routes (`controller: none`) still publish nothing. Plex's web UI and Headlamp joined that
|
||||
set on 2026-11-08 (LAN/VPN-only); the tunnel also 404s every such name (9c/9d).
|
||||
- Public answers are Cloudflare anycast **A and AAAA** — Cloudflare's edge, not an origin address.
|
||||
- LAN and pods resolve `*.home.keithjackson.dev` to the Gateway VIP via PiHole and CoreDNS
|
||||
(split-horizon) and never use the public path.
|
||||
- **Nodes resolve through public DNS** (`node_baseline_dns_servers`), so from a node, the apiserver's
|
||||
OIDC issuer `keycloak.home.keithjackson.dev` resolves to the WAN IP. kubectl SSO therefore relies on
|
||||
the router hairpinning (NAT reflection) WAN:443 back to the Gateway.
|
||||
- **Nodes resolve through public DNS** (`node_baseline_dns_servers`), so from a node the apiserver's
|
||||
OIDC issuer `keycloak.home.keithjackson.dev` now resolves to Cloudflare and its discovery/JWKS
|
||||
fetches go through the tunnel (publicly trusted edge cert; no `oidc-ca-file` is configured). The
|
||||
router NAT hairpin is no longer on that path.
|
||||
|
||||
Check: `dig +short @1.1.1.1 plex.home.keithjackson.dev` → `wan.home.keithjackson.dev.` then the WAN IP;
|
||||
`kubectl -n external-dns logs deploy/cloudflare-ddns`.
|
||||
Check: `dig +short @1.1.1.1 gitea.home.keithjackson.dev` → Cloudflare anycast addresses (no CNAME is
|
||||
visible for a proxied record); `kubectl -n external-dns logs deploy/external-dns`.
|
||||
|
||||
### History
|
||||
|
||||
2026-10-03 → 2026-11-08: records were DNS-only CNAMEs to `wan.home.keithjackson.dev`, with the router
|
||||
forwarding WAN tcp/80+443 to the Gateway VIP. Before 2026-10-03 they answered the Gateway VIP itself.
|
||||
|
||||
## Secrets
|
||||
|
||||
|
||||
@@ -2,9 +2,15 @@
|
||||
# In-cluster dynamic DNS for the home WAN (2026-10-03).
|
||||
#
|
||||
# Keeps ONE record, `wan.${BASE_DOMAIN}` (A, IPv4 only, not proxied), pointed at the home WAN's
|
||||
# current public IP. Every public name external-dns publishes from the shared Gateway is a CNAME to
|
||||
# it (platform/gateway/gateway.yaml `external-dns.alpha.kubernetes.io/target`), so a WAN IP change
|
||||
# moves all of them through this one record, with no Git change and no external-dns involvement.
|
||||
# current public IP.
|
||||
#
|
||||
# WIREGUARD-ONLY SINCE 2026-11-08. From 2026-10-03 every public name external-dns published from the
|
||||
# shared Gateway was a CNAME to this record. They are now proxied CNAMEs to the Cloudflare Tunnel
|
||||
# (platform/gateway/gateway.yaml `external-dns.alpha.kubernetes.io/target`), so NOTHING in the zone
|
||||
# points here any more. The record stays for exactly one consumer: the WireGuard VPN endpoint
|
||||
# (udp/51820, platform/vpn-ingress, infra/vpn-ingress/new-peer.sh), which cannot ride Cloudflare's
|
||||
# HTTP proxy and needs the real WAN address. It must stay DNS-only (`PROXIED` unset/false below) —
|
||||
# an orange-clouded A record would hand peers a Cloudflare anycast address that drops UDP.
|
||||
#
|
||||
# OWNERSHIP SPLIT WITH EXTERNAL-DNS. This record carries no external-dns TXT registry entry, so
|
||||
# external-dns (policy: sync, owner `prod`) never treats it as its own and never deletes it.
|
||||
@@ -64,7 +70,7 @@ spec:
|
||||
- name: IP6_PROVIDER
|
||||
value: "none" # IPv4 WAN only; leaves any AAAA alone
|
||||
- name: PROXIED
|
||||
value: "false" # DNS-only: Plex/Gitea/etc. terminate TLS on the Gateway
|
||||
value: "false" # DNS-only: WireGuard udp/51820 needs the real WAN IP
|
||||
- name: UPDATE_CRON
|
||||
value: "@every 5m"
|
||||
- name: DELETE_ON_STOP
|
||||
|
||||
@@ -107,29 +107,47 @@ spec:
|
||||
# what this instance can see or own; per-resource exclusion only ever REMOVES an object from
|
||||
# this instance's view, so it cannot make prod capable of deleting a record it does not own.
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
# THE PUBLIC TARGET — DECIDED 2026-10-03 (operator): every Gateway-published name goes public
|
||||
# THE PUBLIC TARGET — THE CLOUDFLARE TUNNEL, PROXIED (2026-11-08; supersedes 2026-10-03)
|
||||
#
|
||||
# Until 2026-10-03 every gateway-sourced record answered with the Gateway VIP (192.168.2.20,
|
||||
# RFC1918), so nothing published here was reachable from the internet. Two facts constrained the
|
||||
# fix, and both still hold:
|
||||
# * `external-dns.alpha.kubernetes.io/target` is read in v0.21.0's gateway-httproute source ONLY
|
||||
# from the GATEWAY (source/gateway.go: `overrides: annotations.TargetsFromTargetAnnotation(
|
||||
# gw.Annotations)`) and from a ListenerSet listener — NEVER from an HTTPRoute. A target on the
|
||||
# shared Gateway therefore moves EVERY gateway-sourced record at once.
|
||||
# * The home WAN IP is residential and can change, so a literal IP in Git would rot.
|
||||
# History: until 2026-10-03 every gateway-sourced record answered the Gateway VIP (RFC1918);
|
||||
# from 2026-10-03 they were CNAMEs to `wan.${BASE_DOMAIN}` (DNS-only, the home WAN IP, with the
|
||||
# router forwarding tcp/80+443 to the VIP). Both published an origin address of ours. Now:
|
||||
# * platform/gateway/gateway.yaml sets `target: <tunnel-uuid>.cfargotunnel.com` on the shared
|
||||
# Gateway. `target` is read in v0.23.0's gateway-httproute source ONLY from the GATEWAY
|
||||
# (source/gateway.go: `overrides: annotations.TargetsFromTargetAnnotation(gw.Annotations)`),
|
||||
# never from an HTTPRoute, so this one value moves every gateway-sourced record at once.
|
||||
# * `--cloudflare-proxied` below orange-clouds every record by default. It is GLOBAL because
|
||||
# that is the only practical lever: v0.23.0 reads the per-object `cloudflare-proxied`
|
||||
# annotation from the ROUTE (or Service), never the Gateway (source/gateway.go:
|
||||
# `annotations.ProviderSpecificAnnotations(annots)` with `annots := meta.Annotations` of the
|
||||
# route), so a Gateway-level opt-in does not exist. A CNAME to cfargotunnel.com MUST be
|
||||
# proxied — DNS-only it resolves to nothing routable — so "proxied by default" is the right
|
||||
# default for this zone's gateway names.
|
||||
# * The provider honours a per-object `"false"` over the global default (provider/cloudflare
|
||||
# shouldBeProxied: ParseBool of the annotation overrides proxiedByDefault). The key it reads
|
||||
# is `<annotationPrefix>cloudflare-proxied` (annotations.SetAnnotationPrefix re-derives
|
||||
# CloudflarePrefix), i.e. `external-dns.alpha.kubernetes.io/cloudflare-proxied` here.
|
||||
#
|
||||
# THE `service` SOURCE MUST OPT OUT. The L4 LoadBalancers (crafty-mc, unifi) publish an A record
|
||||
# at their own LB VIP; Cloudflare's proxy is HTTP(S)-only, so a proxied A for a game or UniFi
|
||||
# inform port is broken by construction. Each carries `cloudflare-proxied: "false"` and stays
|
||||
# DNS-only, exactly as before this change. conformance/tests/l0-static/
|
||||
# internal-services-publish-no-public-dns (assertion 9b) FAILS on any `service`-source publisher
|
||||
# without it, so a new L4 Service cannot silently become a proxied record.
|
||||
#
|
||||
# The operator's answers: (a) yes, ALL of them — the five `exposure: public` apps AND the admin
|
||||
# UIs (auth, crafty, grafana, headlamp, the apex), which sit behind Keycloak / oauth2-proxy;
|
||||
# (b) the address is kept current IN-CLUSTER. Implementation:
|
||||
# * platform/gateway/gateway.yaml sets `target: wan.${BASE_DOMAIN}` on the shared Gateway, so
|
||||
# each published name becomes a CNAME to that one hostname;
|
||||
# * platform/externaldns/ddns.yaml (favonia/cloudflare-ddns) keeps `wan.${BASE_DOMAIN}` an A
|
||||
# record at the current WAN IP. That record has no external-dns TXT registry entry, so this
|
||||
# instance never owns or garbage-collects it.
|
||||
# Internal services are unchanged: their routes carry `controller: none` (above), which is
|
||||
# checked BEFORE the target is read, so they still publish nothing. The `service` source (the L4
|
||||
# LoadBalancers crafty-mc, unifi) does not read the Gateway's annotation and still publishes its
|
||||
# LB VIP. Whether those should go public is a separate, still-open question.
|
||||
# checked BEFORE the target is read, so they still publish nothing. Plex's web UI and Headlamp
|
||||
# joined that set on 2026-11-08 (operator: LAN/VPN-only). `wan.${BASE_DOMAIN}` (ddns.yaml) has no
|
||||
# external-dns TXT registry entry and is untouched; it now serves only WireGuard udp/51820.
|
||||
#
|
||||
# THE RISK THIS ADDS, AND WHAT GUARDS IT. If the annotation prefix ever stops being honoured
|
||||
# again (the 2026-11-07 v0.23.0 incident), `target` is ignored and every name falls back to the
|
||||
# Gateway's own address — and with this flag it would be published as a PROXIED A record at
|
||||
# ${GATEWAY_VIP} (RFC1918; Cloudflare 1003/522s for every public name) while `controller: none`
|
||||
# is also ignored and internal names publish. Assertion (8) of the L0 check above ties the
|
||||
# prefix this tree writes to `values.annotationPrefix`; (8b) forbids overriding it here.
|
||||
extraArgs:
|
||||
- --cloudflare-proxied
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
# CROSS-CLUSTER DNS OWNERSHIP. Read before changing either knob below.
|
||||
#
|
||||
|
||||
Reference in new issue
Block a user