Operator answers folded into the specs: - OQ-3 VPN: ExpressVPN, cluster-controlled SINGLE shared in-cluster tunnel (OpenVPN UDP / non-TCP), rules enforced outside the service. New ADR-0012 supersedes ADR-0008 (Gluetun-sidecar default dropped). Central kill-switch. - OQ-1 WAN/edge: one commercial WAN, no multi-ISP plans; dumb SOHO edge (no VLAN/VPN/services). Multi-WAN out of scope; segmentation + VPN in-cluster; edge = NAT + manual port-forward. Gateway VIP resolved to Cilium L2 announcement. T-3.3 marked won't-do. - OQ-2 storage: bulk media lives outside the cluster (backing TBD, non-blocking) via a 'bulk' external-mount indirection; Plex metadata DB stays on Longhorn. Split out a separate OQ-BKP for the off-cluster DR backup target (Phase 4). - OQ-4 refined: wildcard record covers reachability but NOT ACME DNS-01 (needs TXT-write access) and does not match *.home.* depth. Still OPEN: Cloudflare API token (Option A) vs acme-dns delegation (Option B). T-2.4/T-2.5 blocked. Updated: docs 00,01,03,04,08,09; ADR-0008/0012 + index; task board. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
4.7 KiB
08 — TLS & Certificates
Serves: R-TLS-01/02/03, R-DNS-01. Related: ADR-0006 (cert-manager + Let's Encrypt).
Requirement recap
Every externally reachable service presents TLS from a credible, trusted, non-self-signed authority, with certificates auto issued, renewed, and revoked — including for internal-only services (R-TLS-01/02/03).
Solution: cert-manager + Let's Encrypt via DNS-01
- cert-manager runs in the cluster and manages the full certificate lifecycle.
- Let's Encrypt is the trusted, non-self-signed CA (publicly trusted; free; automatable) — satisfies R-TLS-01.
- ACME DNS-01 challenge against the
keithjackson.devDNS zone (Cloudflare or equivalent, per OQ-4). DNS-01 is the key choice because it validates domain control without exposing the service publicly — so internal-only services get real, trusted certs (R-TLS-03), and wildcard certificates (*.home.keithjackson.dev) are possible.
Issuance model
- A
ClusterIssuerfor Let's Encrypt production (and one for staging, used during bring-up to avoid rate limits). - Certificates are requested declaratively — either a wildcard
*.home.keithjackson.devcert shared by the Gateway, or per-service certs, decided per ADR-0006. Recommendation: a wildcard cert for*.home.keithjackson.devterminated at the Gateway simplifies onboarding (a new service needs no new cert), with per-service certs available where isolation warrants. - The DNS provider API token is a SOPS-encrypted secret (07).
DNS-01 challenge access (OQ-4 — the one open item)
Cloudflare already has a wildcard *.keithjackson.dev → public-IP record. That covers
reachability but not certificate issuance: ACME DNS-01 requires cert-manager to write
_acme-challenge TXT records, and the existing wildcard does not match two-label-deep names like
plex.home.keithjackson.dev. So a challenge-access decision is still required:
- Option A — scoped Cloudflare API token (Zone·DNS·Edit for
keithjackson.dev), stored SOPS-encrypted (07). cert-manager writes TXT challenges and ExternalDNS manages the*.home.keithjackson.devrecords — fully automated. Recommended (simplest). - Option B — acme-dns delegation: a one-time
_acme-challengeCNAME in Cloudflare points to an in-cluster acme-dns server; cert-manager writes only to acme-dns, so no standing Cloudflare credential lives in the cluster. More moving parts; public*.homerecords then need the one-time wildcard record instead of ExternalDNS.
Either option supports the wildcard *.home.keithjackson.dev cert and internal-only issuance
(R-TLS-03). This is the remaining blocker for the TLS/DNS workstream (Phase 2).
Renewal & revocation (R-TLS-02)
- cert-manager renews well before expiry automatically; no human action.
- Revocation: removing/rotating a certificate is a declarative change; cert-manager handles issuance of replacements. Expiry monitoring is an alert condition (10, R-OBS-04).
Where TLS terminates
- North-south: at the Gateway API listener (implemented by Cilium, 09), using the cert-manager-issued cert. Public services are reached via the edge router forwarding to the Gateway; internal services resolve to the Gateway on the LAN.
- Internal service-to-service traffic relies primarily on network policy for security; mTLS between workloads is optional future hardening (could be added via Cilium/service mesh) and is not a launch requirement — record as a possible future ADR.
Trust chain for internal clients
Because certs come from Let's Encrypt (publicly trusted), internal clients trust them with no custom
CA distribution. Split-horizon DNS (09) resolves
<service>.home.keithjackson.dev to the internal Gateway while the same trusted cert is served.
Acceptance gates
https://<service>.home.keithjackson.devserves a valid, non-self-signed, unexpired certificate (verified by an external TLS checker), including for internal-only services.- Certificates renew automatically ahead of expiry with no manual step (observed across a renewal).
- Certificate expiry and issuance failures raise alerts.
- Onboarding a new service requires no manual certificate step.
Open questions
- OQ-4 — DNS-01 challenge access: Option A (scoped Cloudflare API token, recommended) vs Option B (acme-dns delegation). Provider is Cloudflare (confirmed). This is the only item still blocking Phase 2 TLS.
- Wildcard vs. per-service certs default — resolved: wildcard
*.home.keithjackson.dev(ADR-0006).