jakman85andClaude Opus 5.5
28d0e0a0f2
Record the operator's 2026-11-08 decision to move public HTTP(S) ingress from edge port-forwarding of 80/443 to an outbound-only Cloudflare Tunnel (in-cluster cloudflared, proxied CNAMEs via ExternalDNS). Motivation: the GFiber router cannot hairpin NAT and advertises Google IPv6 DNS via RA, so LAN clients bypass Pi-hole split horizon and fail. Docs 00/01/03/07/08/09/13/14, GLOSSARY and definition-of-done brought into line. R-DNS-01..03 and R-TLS-01..03 wording unchanged. New requirements: R-DNS-04 (no 80/443 forward, resolver-independent LAN reachability), R-DNS-05 (only `public` services at the tunnel; 404 catch-all; no public record for `internal`), R-SEC-04 (Kubernetes API never publicly reachable). Co-Authored-By: Claude Opus 5.5 <[email protected]>
90 lines
5.3 KiB
Markdown
90 lines
5.3 KiB
Markdown
# 08 — TLS & Certificates
|
|
|
|
Serves: **R-TLS-01/02/03**, **R-DNS-01**. Related: [ADR-0006 (cert-manager + Let's Encrypt)](../decisions/adr-0006-tls-cert-manager-letsencrypt.md),
|
|
[ADR-0016 (Cloudflare Tunnel public ingress)](../decisions/adr-0016-public-ingress-cloudflare-tunnel.md).
|
|
|
|
## Requirement recap
|
|
|
|
Every externally reachable service presents TLS from a **credible, trusted, non-self-signed**
|
|
authority, with certificates **auto issued, renewed, and revoked** — including for **internal-only**
|
|
services (R-TLS-01/02/03).
|
|
|
|
## Solution: cert-manager + Let's Encrypt via DNS-01
|
|
|
|
- **cert-manager** runs in the cluster and manages the full certificate lifecycle.
|
|
- **Let's Encrypt** is the trusted, non-self-signed CA (publicly trusted; free; automatable) —
|
|
satisfies R-TLS-01.
|
|
- **ACME DNS-01 challenge** against the `keithjackson.dev` DNS zone (Cloudflare or equivalent, per
|
|
OQ-4). DNS-01 is the key choice because it validates domain control **without exposing the service
|
|
publicly** — so internal-only services get real, trusted certs (R-TLS-03), and **wildcard**
|
|
certificates (`*.home.keithjackson.dev`) are possible.
|
|
|
|
## Issuance model
|
|
|
|
- A `ClusterIssuer` for Let's Encrypt production (and one for staging, used during bring-up to avoid
|
|
rate limits).
|
|
- Certificates are requested declaratively — either a **wildcard** `*.home.keithjackson.dev` cert
|
|
shared by the Gateway, or **per-service** certs, decided per ADR-0006. Recommendation: a wildcard
|
|
cert for `*.home.keithjackson.dev` terminated at the Gateway simplifies onboarding (a new service
|
|
needs no new cert), with per-service certs available where isolation warrants.
|
|
- The DNS provider API token is a SOPS-encrypted secret ([07](07-security-and-secrets.md)).
|
|
|
|
## DNS-01 challenge access (OQ-4 — RESOLVED: scoped Cloudflare API token)
|
|
|
|
Cloudflare already has a wildcard `*.keithjackson.dev` → public-IP record, which does not cover
|
|
certificate issuance (ACME DNS-01 must **write `_acme-challenge` TXT records**,
|
|
and that wildcard does not match two-label-deep names like `plex.home.keithjackson.dev`).
|
|
|
|
**Decision (OQ-4):** use a **scoped Cloudflare API token** — `Zone·DNS·Edit` limited to the
|
|
`keithjackson.dev` zone — stored **SOPS-encrypted** ([07](07-security-and-secrets.md)). With it:
|
|
- **cert-manager** solves DNS-01 by writing/removing TXT challenge records automatically (issue,
|
|
renew, revoke — R-TLS-02), including the wildcard `*.home.keithjackson.dev` cert and internal-only
|
|
services (R-TLS-03).
|
|
- **ExternalDNS** manages the `*.home.keithjackson.dev` records automatically ([09](09-dns-and-ingress.md),
|
|
R-DNS-02) — no manual record editing for public services.
|
|
|
|
**Operator prerequisite:** create the scoped token in Cloudflare and hand it over once; it is then
|
|
committed SOPS-encrypted and never touched manually again. The token is least-privilege (single zone,
|
|
DNS edit only) so its blast radius is bounded (R-SEC-02). Rotation = re-encrypt + commit.
|
|
|
|
## Renewal & revocation (R-TLS-02)
|
|
|
|
- cert-manager renews well before expiry automatically; no human action.
|
|
- Revocation: removing/rotating a certificate is a declarative change; cert-manager handles
|
|
issuance of replacements. Expiry monitoring is an alert condition ([10](10-observability-and-alerting.md), R-OBS-04).
|
|
|
|
## Where TLS terminates
|
|
|
|
- **North-south**: at the **Gateway API** listener (implemented by Cilium, [09](09-dns-and-ingress.md)),
|
|
using the cert-manager-issued cert. Internal services resolve to the Gateway on the LAN.
|
|
- **Public services via the Cloudflare Tunnel (ADR-0016)** are TLS twice:
|
|
- **Edge:** Cloudflare terminates the client's TLS with an ACM/Total TLS certificate. It is
|
|
publicly trusted and auto-renewed (R-TLS-01/02). Universal SSL is not enough, because it covers
|
|
only one subdomain level.
|
|
- **Origin:** cloudflared re-encrypts to the Gateway and **validates** the Let's Encrypt wildcard,
|
|
using the service hostname as SNI. Verification is never disabled.
|
|
|
|
Cloudflare sees plaintext between the two legs, a trade-off accepted in ADR-0016.
|
|
- Internal service-to-service traffic relies primarily on network policy for security; mTLS between
|
|
workloads is **optional** future hardening (could be added via Cilium/service mesh) and is not a
|
|
launch requirement — record as a possible future ADR.
|
|
|
|
## Trust chain for internal clients
|
|
|
|
Because certs come from Let's Encrypt (publicly trusted), internal clients trust them with no custom
|
|
CA distribution. Split-horizon DNS ([09](09-dns-and-ingress.md)) resolves
|
|
`<service>.home.keithjackson.dev` to the internal Gateway while the same trusted cert is served.
|
|
|
|
## Acceptance gates
|
|
- `https://<service>.home.keithjackson.dev` serves a valid, non-self-signed, unexpired certificate
|
|
(verified by an external TLS checker), including for internal-only services.
|
|
- Certificates renew automatically ahead of expiry with no manual step (observed across a renewal).
|
|
- Certificate expiry and issuance failures raise alerts.
|
|
- Onboarding a new service requires no manual certificate step.
|
|
|
|
## Resolved
|
|
- **OQ-4** — DNS-01 via a **scoped Cloudflare API token** (`Zone·DNS·Edit`, `keithjackson.dev`),
|
|
SOPS-encrypted; drives both cert-manager DNS-01 and ExternalDNS. Phase 2 TLS/DNS is unblocked once
|
|
the operator provides the token.
|
|
- Wildcard vs. per-service certs default — **wildcard** `*.home.keithjackson.dev` (ADR-0006).
|