jakman85andClaude Opus 5.5
250fb54154
The operator ruled the implementation repo's main (50c8719) the latest and accepted state. Sync the spec to it. Accepted changes are reflected: one bare-metal environment, TrueNAS bulk storage, R2 off-cluster backup, CloudNativePG, single-node VPN gateway with Cilium fail-closed, inbound WireGuard, per-app auth sidecars, Renovate, the CI gate as approver, and the pm-agent. New ADR-0017..0025; ADR-0016 corrected and accepted; R-* IDs unchanged, eight amended. Where the implementation has simply not met a requirement, the requirement stands and the shortfall is recorded in docs/16-known-gaps.md. Co-Authored-By: Claude Opus 5.5 <[email protected]>
154 lines
10 KiB
Markdown
154 lines
10 KiB
Markdown
# 09 — DNS & Ingress
|
|
|
|
Serves: **R-DNS-01/02/03/04/05**, **R-C2-01**, **R-SEC-04**, works with **R-TLS-***. Related:
|
|
[ADR-0002 (Cilium/Gateway API)](../decisions/adr-0002-cni-cilium.md),
|
|
[ADR-0016 (Cloudflare Tunnel public ingress)](../decisions/adr-0016-public-ingress-cloudflare-tunnel.md),
|
|
[ADR-0021 (inbound remote access)](../decisions/adr-0021-inbound-remote-access-wireguard.md).
|
|
|
|
## Naming model
|
|
|
|
- All services live at `<service>.home.keithjackson.dev` (e.g. `plex.home.keithjackson.dev`).
|
|
- The **root** `home.keithjackson.dev` serves the **Command & Control UI** ([12](12-command-center-ui.md)).
|
|
- `*.home.keithjackson.dev` is the platform's zone, in Cloudflare DNS. `public` services are
|
|
published as **proxied CNAMEs to the Cloudflare Tunnel** (`<tunnel-uuid>.cfargotunnel.com`,
|
|
ADR-0016), not as records pointing at the network's public IP.
|
|
|
|
## DNS automation (R-DNS-02)
|
|
|
|
- **ExternalDNS** watches Gateway/HTTPRoute (and Service) resources and manages records in the
|
|
authoritative DNS provider (Cloudflare, per OQ-4) automatically — a new service's DNS appears from
|
|
its declared intent, no manual record editing.
|
|
- Records for `*.home.keithjackson.dev` are created/updated/removed as services come and go.
|
|
- Public records are created **only for services declared public** (see the exposure class below,
|
|
R-DNS-05). The non-HTTP L4 names **unifi** and **crafty-mc** are the one exception: they hold DNS-only
|
|
records at their LAN addresses, and are never proxied or tunnelled. The DNS-only (unproxied) `wan.`
|
|
record for WireGuard is kept current by cloudflare-ddns, outside this flow.
|
|
|
|
## Split-horizon resolution (R-DNS-03)
|
|
|
|
Internal and external answers differ:
|
|
- **External** (`<service>.home.keithjackson.dev` from the internet): a `public` service resolves to
|
|
**Cloudflare's edge** (proxied CNAME to the tunnel). Cloudflare forwards the request down the
|
|
outbound-only tunnel to cloudflared in the cluster, which forwards it to the Gateway (ADR-0016).
|
|
`internal` services have no public answer, apart from the L4 exception above.
|
|
- **Internal** (LAN clients): Pi-hole answers with the **internal Gateway address** (or the service's
|
|
L4 VIP) for the names it holds local records for.
|
|
- **Correctness does not depend on split horizon for `public` web apps** (R-DNS-04). The ISP router
|
|
cannot hairpin NAT and advertises a public IPv6 resolver via RA, so some LAN clients will get the
|
|
public answer. That answer goes through Cloudflare and works from the LAN too.
|
|
- **PiHole** (itself an onboarded service) provides internal LAN DNS. It keeps explicit local records
|
|
for:
|
|
- the `internal` names;
|
|
- the L4 VIPs;
|
|
- the names that must avoid Cloudflare's 100 MB body / 100 s timeout limits or must work during an
|
|
internet outage: **gitea**, **nextcloud**, **keycloak/auth**.
|
|
|
|
It holds no zone-wide entry. Every other name is forwarded upstream, so the remaining `public` names
|
|
resolve to Cloudflare from the LAN too and depend on the internet and the tunnel. Its DNS address
|
|
accepts queries only from the LAN and WireGuard clients. Pi-hole runs in the cluster, so LAN DNS
|
|
depends on cluster health; an off-cluster resolver is an operator choice not taken.
|
|
|
|
A trusted certificate is served either way: Cloudflare's edge certificate on the tunnel path, the
|
|
Let's Encrypt wildcard on the LAN path ([08](08-tls-and-certificates.md)).
|
|
- **In-cluster** resolution keeps its own split horizon (CoreDNS) and the node `/etc/hosts` pins (OIDC
|
|
issuer, Gitea registry), so cluster-internal callers never traverse Cloudflare.
|
|
|
|
> DNS is thus both a platform capability *and* a hosted service. Bootstrapping order matters: the
|
|
> platform needs working DNS before PiHole is fully managed — handled by the bring-up sequence in
|
|
> [`roadmap/phases.md`](../roadmap/phases.md).
|
|
|
|
## Ingress (north-south) via Gateway API
|
|
|
|
- **Gateway API implemented by Cilium** is the ingress layer (no separate ingress controller;
|
|
reuses the CNI, [ADR-0002](../decisions/adr-0002-cni-cilium.md)).
|
|
- A shared **Gateway** defines HTTPS listeners on `*.home.keithjackson.dev` terminating TLS with the
|
|
cert-manager cert ([08](08-tls-and-certificates.md)).
|
|
- Each service contributes an **HTTPRoute** mapping its hostname to its Service — rendered from the
|
|
ServiceDefinition, not hand-authored.
|
|
- **Exposure class** is declared per service:
|
|
- `internal` — reachable only on the LAN/VPN (refused at the tunnel, no public record; internal DNS only).
|
|
- `public` — additionally reachable from the internet through the **Cloudflare Tunnel**:
|
|
cloudflared → the Gateway over HTTPS (ADR-0016). A service is public by catalog
|
|
`exposure: public`, or as a **registered platform route**: `auth`, Grafana, Crafty's web UI, and
|
|
the apex with its `/runbooks`. Platform routes have no catalog entry, so they are held in a
|
|
shrink-only register, enforced by the static check
|
|
`conformance/tests/l0-static/internal-services-publish-no-public-dns` in the
|
|
[implementation repo](https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl).
|
|
- **Never `public`:** jackett, pihole, semaphore, qbittorrent, radarr, sonarr, **headlamp** (a front
|
|
end to the Kubernetes API), plex's web UI, and the non-HTTP L4 services unifi and crafty-mc (which
|
|
hold only the DNS-only LAN records above).
|
|
- **The Kubernetes API is never published** — no tunnel route, no edge forward, no public record
|
|
(R-SEC-04).
|
|
|
|
## Public ingress: Cloudflare Tunnel (ADR-0016)
|
|
|
|
- **cloudflared** runs in-cluster as a Flux-managed workload and holds an **outbound-only**
|
|
connection to Cloudflare. No inbound 80/443 reaches the WAN IP.
|
|
- Its ingress rules are a **deny-list plus a zone wildcard**. Each LAN/VPN-only hostname maps to
|
|
`http_status:404`; the apex and `*.home.keithjackson.dev` then go to the Gateway (HTTPS, origin
|
|
certificate validated with the hostname as SNI); a **catch-all `http_status:404`** closes the list
|
|
for hosts outside the zone (R-DNS-05). A static check keeps the deny-list and the set of unpublished
|
|
routes identical in both directions, so a LAN/VPN-only name cannot be published or tunnelled by
|
|
omission.
|
|
- **Edge TLS** is Cloudflare ACM/Total TLS (Universal SSL does not cover two-label-deep names);
|
|
**origin TLS** is the Let's Encrypt wildcard at the Gateway ([08](08-tls-and-certificates.md)).
|
|
- **Limits and trade-offs:**
|
|
- 100 MB request bodies and 100 s responses through the proxy;
|
|
- Cloudflare terminates TLS and sees plaintext;
|
|
- a Cloudflare outage takes public access down (the LAN keeps working through Pi-hole's local
|
|
records);
|
|
- the client IP arrives as `CF-Connecting-IP` / `X-Forwarded-For`, not as the TCP source.
|
|
|
|
## Edge integration (single SOHO router — OQ-1 resolved)
|
|
|
|
- The environment has **one SOHO edge router**: no VLANs, no VPN, no services, no API/IaC. It
|
|
**does not forward 80/443** (public web ingress is the tunnel, ADR-0016). Its only remaining
|
|
inbound forwards are **WireGuard udp/51820** (remote access, ADR-0021, via the DNS-only `wan.` record) and
|
|
**Plex tcp/32400** (remote streaming via plex.tv; Cloudflare's terms restrict proxied video). This
|
|
is a documented manual edge step (the SOHO device has no automation), the analogue of the old
|
|
pfSense manual-config exception. The WAN IP therefore stays public.
|
|
- No multi-WAN inbound rules — single WAN ([03](03-networking-isolation-and-egress.md)).
|
|
- `internal` services are never port-forwarded or tunnelled; they rely on internal DNS + Gateway (or
|
|
their L4 VIP) on the LAN, or WireGuard from outside.
|
|
- **Ingress VIP mechanism — resolved: Cilium L2 announcement (ARP/NDP)** on the flat LAN. BGP is not
|
|
available on a SOHO router, and a single flat subnet makes L2 announcement the natural choice.
|
|
|
|
## DNS record depth (note for `*.home.keithjackson.dev`)
|
|
|
|
A wildcard `*.keithjackson.dev` matches only **one** label deep — it covers `home.keithjackson.dev`
|
|
but **not** `plex.home.keithjackson.dev`. Consequences:
|
|
- **`internal` services need no public record** — PiHole resolves `<svc>.home.keithjackson.dev` to
|
|
the internal Gateway VIP (split-horizon). This is the majority of services.
|
|
- **`public` services** need a public record for `<svc>.home.keithjackson.dev`: a **proxied CNAME to
|
|
the tunnel** (ADR-0016). Per OQ-4 (resolved), **ExternalDNS manages these automatically** via the
|
|
scoped Cloudflare API token — no manual record editing (see [08](08-tls-and-certificates.md)). The
|
|
same depth limit is why edge TLS needs ACM/Total TLS rather than Universal SSL.
|
|
|
|
## Interaction with default-deny
|
|
|
|
Ingress opens **north-south** paths to the specific service; it does **not** relax **east-west**
|
|
default-deny ([03](03-networking-isolation-and-egress.md)). A service being publicly reachable does
|
|
not let other in-cluster services reach it unless an explicit allow rule exists. Cilium's Gateway
|
|
re-evaluates the calling pod's egress policy against the real backend, so any in-cluster client of the
|
|
Gateway, cloudflared included, needs an explicit egress allow to the backends it reaches.
|
|
|
|
## Acceptance gates
|
|
- A new service's `<service>.home.keithjackson.dev` record appears automatically and routes to it.
|
|
- `internal` services are unreachable from the public internet; `public` services are reachable and
|
|
TLS-valid.
|
|
- Internal clients resolving through Pi-hole reach the internal Gateway for the names it holds local
|
|
records for, and get a trusted cert.
|
|
- A LAN client resolving a `public` name through a **public** resolver reaches it, TLS-valid
|
|
(R-DNS-04). The WAN IP does not answer on 80/443.
|
|
- A request at the tunnel for a LAN/VPN-only hostname returns **404**, and no `internal` service has
|
|
a public DNS record, except the DNS-only L4 names (R-DNS-05).
|
|
- The Kubernetes API is unreachable from the internet: no tunnel route, no edge forward, no public
|
|
record (R-SEC-04).
|
|
- The root `home.keithjackson.dev` serves the C2 UI.
|
|
|
|
## Resolved
|
|
- **OQ-1** — single SOHO edge; ingress VIP via **Cilium L2 announcement**. Public web ingress amended
|
|
2026-11-08 to a **Cloudflare Tunnel** (ADR-0016); the edge keeps only udp/51820 and tcp/32400.
|
|
- **OQ-4** — scoped **Cloudflare API token** drives ExternalDNS (records) and cert-manager DNS-01
|
|
(certs) automatically; see [08](08-tls-and-certificates.md).
|