Files
home-network-k3s/docs/08-tls-and-certificates.md
T
jakman85andClaude Opus 5.5 28d0e0a0f2 ADR-0016 (Proposed): public ingress via Cloudflare Tunnel
Record the operator's 2026-11-08 decision to move public HTTP(S) ingress
from edge port-forwarding of 80/443 to an outbound-only Cloudflare Tunnel
(in-cluster cloudflared, proxied CNAMEs via ExternalDNS). Motivation: the
GFiber router cannot hairpin NAT and advertises Google IPv6 DNS via RA, so
LAN clients bypass Pi-hole split horizon and fail.

Docs 00/01/03/07/08/09/13/14, GLOSSARY and definition-of-done brought into
line. R-DNS-01..03 and R-TLS-01..03 wording unchanged. New requirements:
R-DNS-04 (no 80/443 forward, resolver-independent LAN reachability),
R-DNS-05 (only `public` services at the tunnel; 404 catch-all; no public
record for `internal`), R-SEC-04 (Kubernetes API never publicly reachable).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 12:38:16 -06:00

5.3 KiB

08 — TLS & Certificates

Serves: R-TLS-01/02/03, R-DNS-01. Related: ADR-0006 (cert-manager + Let's Encrypt), ADR-0016 (Cloudflare Tunnel public ingress).

Requirement recap

Every externally reachable service presents TLS from a credible, trusted, non-self-signed authority, with certificates auto issued, renewed, and revoked — including for internal-only services (R-TLS-01/02/03).

Solution: cert-manager + Let's Encrypt via DNS-01

  • cert-manager runs in the cluster and manages the full certificate lifecycle.
  • Let's Encrypt is the trusted, non-self-signed CA (publicly trusted; free; automatable) — satisfies R-TLS-01.
  • ACME DNS-01 challenge against the keithjackson.dev DNS zone (Cloudflare or equivalent, per OQ-4). DNS-01 is the key choice because it validates domain control without exposing the service publicly — so internal-only services get real, trusted certs (R-TLS-03), and wildcard certificates (*.home.keithjackson.dev) are possible.

Issuance model

  • A ClusterIssuer for Let's Encrypt production (and one for staging, used during bring-up to avoid rate limits).
  • Certificates are requested declaratively — either a wildcard *.home.keithjackson.dev cert shared by the Gateway, or per-service certs, decided per ADR-0006. Recommendation: a wildcard cert for *.home.keithjackson.dev terminated at the Gateway simplifies onboarding (a new service needs no new cert), with per-service certs available where isolation warrants.
  • The DNS provider API token is a SOPS-encrypted secret (07).

DNS-01 challenge access (OQ-4 — RESOLVED: scoped Cloudflare API token)

Cloudflare already has a wildcard *.keithjackson.dev → public-IP record, which does not cover certificate issuance (ACME DNS-01 must write _acme-challenge TXT records, and that wildcard does not match two-label-deep names like plex.home.keithjackson.dev).

Decision (OQ-4): use a scoped Cloudflare API token — Zone·DNS·Edit limited to the keithjackson.dev zone — stored SOPS-encrypted (07). With it:

  • cert-manager solves DNS-01 by writing/removing TXT challenge records automatically (issue, renew, revoke — R-TLS-02), including the wildcard *.home.keithjackson.dev cert and internal-only services (R-TLS-03).
  • ExternalDNS manages the *.home.keithjackson.dev records automatically (09, R-DNS-02) — no manual record editing for public services.

Operator prerequisite: create the scoped token in Cloudflare and hand it over once; it is then committed SOPS-encrypted and never touched manually again. The token is least-privilege (single zone, DNS edit only) so its blast radius is bounded (R-SEC-02). Rotation = re-encrypt + commit.

Renewal & revocation (R-TLS-02)

  • cert-manager renews well before expiry automatically; no human action.
  • Revocation: removing/rotating a certificate is a declarative change; cert-manager handles issuance of replacements. Expiry monitoring is an alert condition (10, R-OBS-04).

Where TLS terminates

  • North-south: at the Gateway API listener (implemented by Cilium, 09), using the cert-manager-issued cert. Internal services resolve to the Gateway on the LAN.

  • Public services via the Cloudflare Tunnel (ADR-0016) are TLS twice:

    • Edge: Cloudflare terminates the client's TLS with an ACM/Total TLS certificate. It is publicly trusted and auto-renewed (R-TLS-01/02). Universal SSL is not enough, because it covers only one subdomain level.
    • Origin: cloudflared re-encrypts to the Gateway and validates the Let's Encrypt wildcard, using the service hostname as SNI. Verification is never disabled.

    Cloudflare sees plaintext between the two legs, a trade-off accepted in ADR-0016.

  • Internal service-to-service traffic relies primarily on network policy for security; mTLS between workloads is optional future hardening (could be added via Cilium/service mesh) and is not a launch requirement — record as a possible future ADR.

Trust chain for internal clients

Because certs come from Let's Encrypt (publicly trusted), internal clients trust them with no custom CA distribution. Split-horizon DNS (09) resolves <service>.home.keithjackson.dev to the internal Gateway while the same trusted cert is served.

Acceptance gates

  • https://<service>.home.keithjackson.dev serves a valid, non-self-signed, unexpired certificate (verified by an external TLS checker), including for internal-only services.
  • Certificates renew automatically ahead of expiry with no manual step (observed across a renewal).
  • Certificate expiry and issuance failures raise alerts.
  • Onboarding a new service requires no manual certificate step.

Resolved

  • OQ-4 — DNS-01 via a scoped Cloudflare API token (Zone·DNS·Edit, keithjackson.dev), SOPS-encrypted; drives both cert-manager DNS-01 and ExternalDNS. Phase 2 TLS/DNS is unblocked once the operator provides the token.
  • Wildcard vs. per-service certs default — wildcard *.home.keithjackson.dev (ADR-0006).