jakman85andClaude Opus 5.5
28d0e0a0f2
Record the operator's 2026-11-08 decision to move public HTTP(S) ingress from edge port-forwarding of 80/443 to an outbound-only Cloudflare Tunnel (in-cluster cloudflared, proxied CNAMEs via ExternalDNS). Motivation: the GFiber router cannot hairpin NAT and advertises Google IPv6 DNS via RA, so LAN clients bypass Pi-hole split horizon and fail. Docs 00/01/03/07/08/09/13/14, GLOSSARY and definition-of-done brought into line. R-DNS-01..03 and R-TLS-01..03 wording unchanged. New requirements: R-DNS-04 (no 80/443 forward, resolver-independent LAN reachability), R-DNS-05 (only `public` services at the tunnel; 404 catch-all; no public record for `internal`), R-SEC-04 (Kubernetes API never publicly reachable). Co-Authored-By: Claude Opus 5.5 <[email protected]>
135 lines
8.4 KiB
Markdown
135 lines
8.4 KiB
Markdown
# 09 — DNS & Ingress
|
|
|
|
Serves: **R-DNS-01/02/03/04/05**, **R-C2-01**, **R-SEC-04**, works with **R-TLS-***. Related:
|
|
[ADR-0002 (Cilium/Gateway API)](../decisions/adr-0002-cni-cilium.md),
|
|
[ADR-0016 (Cloudflare Tunnel public ingress)](../decisions/adr-0016-public-ingress-cloudflare-tunnel.md).
|
|
|
|
## Naming model
|
|
|
|
- All services live at `<service>.home.keithjackson.dev` (e.g. `plex.home.keithjackson.dev`).
|
|
- The **root** `home.keithjackson.dev` serves the **Command & Control UI** ([12](12-command-center-ui.md)).
|
|
- `*.home.keithjackson.dev` is the platform's zone, in Cloudflare DNS. `public` services are
|
|
published as **proxied CNAMEs to the Cloudflare Tunnel** (`<tunnel-uuid>.cfargotunnel.com`,
|
|
ADR-0016), not as records pointing at the network's public IP.
|
|
|
|
## DNS automation (R-DNS-02)
|
|
|
|
- **ExternalDNS** watches Gateway/HTTPRoute (and Service) resources and manages records in the
|
|
authoritative DNS provider (Cloudflare, per OQ-4) automatically — a new service's DNS appears from
|
|
its declared intent, no manual record editing.
|
|
- Records for `*.home.keithjackson.dev` are created/updated/removed as services come and go.
|
|
- Public records are created **only for `exposure: public` services** (R-DNS-05). The DNS-only
|
|
(unproxied) `wan.` record for WireGuard is kept current by cloudflare-ddns, outside this flow.
|
|
|
|
## Split-horizon resolution (R-DNS-03)
|
|
|
|
Internal and external answers differ:
|
|
- **External** (`<service>.home.keithjackson.dev` from the internet): a `public` service resolves to
|
|
**Cloudflare's edge** (proxied CNAME to the tunnel). Cloudflare forwards the request down the
|
|
outbound-only tunnel to cloudflared in the cluster, which forwards it to the Gateway (ADR-0016).
|
|
`internal` services have no public answer.
|
|
- **Internal** (LAN clients): Pi-hole answers with the **internal Gateway address** (or the service's
|
|
L4 VIP) for the names it holds local records for.
|
|
- **Correctness does not depend on split horizon for `public` web apps** (R-DNS-04). The ISP router
|
|
cannot hairpin NAT and advertises a public IPv6 resolver via RA, so some LAN clients will get the
|
|
public answer. That answer goes through Cloudflare and works from the LAN too.
|
|
- **PiHole** (itself an onboarded service) provides internal LAN DNS. It keeps explicit local records
|
|
for:
|
|
- the `internal` names;
|
|
- the L4 VIPs;
|
|
- the names that must avoid Cloudflare's 100 MB body / 100 s timeout limits or must work during an
|
|
internet outage: **gitea**, **nextcloud**, **keycloak/auth**.
|
|
|
|
A trusted certificate is served either way: Cloudflare's edge certificate on the tunnel path, the
|
|
Let's Encrypt wildcard on the LAN path ([08](08-tls-and-certificates.md)).
|
|
- **In-cluster** resolution keeps its own split horizon (CoreDNS) and the node `/etc/hosts` pins (OIDC
|
|
issuer, Gitea registry), so cluster-internal callers never traverse Cloudflare.
|
|
|
|
> DNS is thus both a platform capability *and* a hosted service. Bootstrapping order matters: the
|
|
> platform needs working DNS before PiHole is fully managed — handled by the bring-up sequence in
|
|
> [`roadmap/phases.md`](../roadmap/phases.md).
|
|
|
|
## Ingress (north-south) via Gateway API
|
|
|
|
- **Gateway API implemented by Cilium** is the ingress layer (no separate ingress controller;
|
|
reuses the CNI, [ADR-0002](../decisions/adr-0002-cni-cilium.md)).
|
|
- A shared **Gateway** defines HTTPS listeners on `*.home.keithjackson.dev` terminating TLS with the
|
|
cert-manager cert ([08](08-tls-and-certificates.md)).
|
|
- Each service contributes an **HTTPRoute** mapping its hostname to its Service — rendered from the
|
|
ServiceDefinition, not hand-authored.
|
|
- **Exposure class** is declared per service:
|
|
- `internal` — reachable only on the LAN/VPN (no tunnel route, no public record; internal DNS only).
|
|
- `public` — additionally reachable from the internet through the **Cloudflare Tunnel**:
|
|
cloudflared → the Gateway over HTTPS (ADR-0016).
|
|
- **Never `public`:** jackett, pihole, semaphore, qbittorrent, radarr, sonarr, **headlamp** (a front
|
|
end to the Kubernetes API), plex's web UI, and the non-HTTP L4 services unifi and crafty-mc.
|
|
- **The Kubernetes API is never published** — no tunnel route, no edge forward, no public record
|
|
(R-SEC-04).
|
|
|
|
## Public ingress: Cloudflare Tunnel (ADR-0016)
|
|
|
|
- **cloudflared** runs in-cluster as a Flux-managed workload and holds an **outbound-only**
|
|
connection to Cloudflare. No inbound 80/443 reaches the WAN IP.
|
|
- Its ingress rules list one route per `public` hostname → the Gateway (HTTPS, origin certificate
|
|
validated with the hostname as SNI). The rules end in a **catch-all `http_status:404`**, so an
|
|
unlisted hostname is refused at the tunnel (R-DNS-05).
|
|
- **Edge TLS** is Cloudflare ACM/Total TLS (Universal SSL does not cover two-label-deep names);
|
|
**origin TLS** is the Let's Encrypt wildcard at the Gateway ([08](08-tls-and-certificates.md)).
|
|
- **Limits and trade-offs:**
|
|
- 100 MB request bodies and 100 s responses through the proxy;
|
|
- Cloudflare terminates TLS and sees plaintext;
|
|
- a Cloudflare outage takes public access down (the LAN keeps working through Pi-hole's local
|
|
records);
|
|
- the client IP arrives as `CF-Connecting-IP` / `X-Forwarded-For`, not as the TCP source.
|
|
|
|
## Edge integration (single SOHO router — OQ-1 resolved)
|
|
|
|
- The environment has **one SOHO edge router**: no VLANs, no VPN, no services, no API/IaC. It
|
|
**does not forward 80/443** (public web ingress is the tunnel, ADR-0016). Its only remaining
|
|
inbound forwards are **WireGuard udp/51820** (remote access, via the DNS-only `wan.` record) and
|
|
**Plex tcp/32400** (remote streaming via plex.tv; Cloudflare's terms restrict proxied video). This
|
|
is a documented manual edge step (the SOHO device has no automation), the analogue of the old
|
|
pfSense manual-config exception. The WAN IP therefore stays public.
|
|
- No multi-WAN inbound rules — single WAN ([03](03-networking-isolation-and-egress.md)).
|
|
- `internal` services are never port-forwarded or tunnelled; they rely on internal DNS + Gateway (or
|
|
their L4 VIP) on the LAN, or WireGuard from outside.
|
|
- **Ingress VIP mechanism — resolved: Cilium L2 announcement (ARP/NDP)** on the flat LAN. BGP is not
|
|
available on a SOHO router, and a single flat subnet makes L2 announcement the natural choice.
|
|
|
|
## DNS record depth (note for `*.home.keithjackson.dev`)
|
|
|
|
A wildcard `*.keithjackson.dev` matches only **one** label deep — it covers `home.keithjackson.dev`
|
|
but **not** `plex.home.keithjackson.dev`. Consequences:
|
|
- **`internal` services need no public record** — PiHole resolves `<svc>.home.keithjackson.dev` to
|
|
the internal Gateway VIP (split-horizon). This is the majority of services.
|
|
- **`public` services** need a public record for `<svc>.home.keithjackson.dev`: a **proxied CNAME to
|
|
the tunnel** (ADR-0016). Per OQ-4 (resolved), **ExternalDNS manages these automatically** via the
|
|
scoped Cloudflare API token — no manual record editing (see [08](08-tls-and-certificates.md)). The
|
|
same depth limit is why edge TLS needs ACM/Total TLS rather than Universal SSL.
|
|
|
|
## Interaction with default-deny
|
|
|
|
Ingress opens **north-south** paths to the specific service; it does **not** relax **east-west**
|
|
default-deny ([03](03-networking-isolation-and-egress.md)). A service being publicly reachable does
|
|
not let other in-cluster services reach it unless an explicit allow rule exists.
|
|
|
|
## Acceptance gates
|
|
- A new service's `<service>.home.keithjackson.dev` record appears automatically and routes to it.
|
|
- `internal` services are unreachable from the public internet; `public` services are reachable and
|
|
TLS-valid.
|
|
- Internal clients resolving through Pi-hole reach the internal Gateway for the names it holds local
|
|
records for, and get a trusted cert.
|
|
- A LAN client resolving a `public` name through a **public** resolver reaches it, TLS-valid
|
|
(R-DNS-04). The WAN IP does not answer on 80/443.
|
|
- A request at the tunnel for any hostname that is not a `public` service returns **404**, and no
|
|
`internal` service has a public DNS record (R-DNS-05).
|
|
- The Kubernetes API is unreachable from the internet: no tunnel route, no edge forward, no public
|
|
record (R-SEC-04).
|
|
- The root `home.keithjackson.dev` serves the C2 UI.
|
|
|
|
## Resolved
|
|
- **OQ-1** — single SOHO edge; ingress VIP via **Cilium L2 announcement**. Public web ingress amended
|
|
2026-11-08 to a **Cloudflare Tunnel** (ADR-0016); the edge keeps only udp/51820 and tcp/32400.
|
|
- **OQ-4** — scoped **Cloudflare API token** drives ExternalDNS (records) and cert-manager DNS-01
|
|
(certs) automatically; see [08](08-tls-and-certificates.md).
|