fix(node): pin gitea to the Gateway VIP — containerd was resolving it to the non-hairpinned WAN
gate / gate (pull_request) Successful in 1m52s
gate / conformance (pull_request) Has been skipped

The pm-agent image pull failed TLS verification, and the previous entry on this branch
recorded it honestly as not diagnosed. Diagnosed now, and it is not about the image.

containerd pulls as a NODE process, so it resolves through node_baseline_dns_servers (the
LAN gateway, deliberately not PiHole) and gets the public answer: a CNAME to
wan.home.keithjackson.dev -> 136.51.115.252, measured via both 1.1.1.1 and 8.8.8.8. This
router does not hairpin, so WAN:443 answers with its own placeholder certificate —
measured with the right SNI from the workstation:

    subject=C=ZZ, ST=Somewhere, L=Unknown, O=%D342e7e6b, CN=%D
    No extensions in certificate

No extensions means no subjectAltName, which is exactly 'certificate is not valid for any
names'. Meanwhile 192.168.2.20 IS the shared Gateway's VIP and its https listener serves
the real Let's Encrypt *.home.keithjackson.dev. The Gateway, the wildcard cert and the
digest were never implicated — only the node's answer.

This file already documents the identical failure for the Keycloak issuer, measured from
k3s-prod-2/-3 on 2026-10-03 (curl exit 60, ssl_verify_result=18), and already carries the
mechanism that fixes it. gitea is simply the second node process to need it; containerd is
a node process and nobody had added it. An operator must run the node_baseline play.

Also records, where an operator reads them, that the agent authenticates to Gitea with a
local account and a scoped PAT by decision — and why the federated alternative is not
merely unchosen but unavailable: Gitea implements only the Authorization Code Grant and
validates no third-party OIDC token, so a Keycloak service-account token cannot
authenticate to its API or to git-over-HTTP.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed 2026-10-06 00:35:18 -05:00
1 parent a209a618fd
commit 523a463fd7
6 files changed
+189 -18

No files matched your search

@@ -327,6 +327,29 @@ node_baseline_hosts_pins:
- ip: "192.168.2.20"
names:
- "keycloak.{{ k3s_base_domain }}"
# gitea: the SECOND node process to need this pin, for the identical reason, found
# 2026-10-06. containerd pulls images as a NODE process, so it resolves through the
# public upstreams above and gets the CNAME to wan.home.keithjackson.dev — the WAN IP,
# which this router does not hairpin. The pull then fails on the router's own
# placeholder certificate rather than on anything to do with the image:
#
# failed to resolve reference "gitea.home.keithjackson.dev/home-network/agent-runner@sha256:…":
# tls: failed to verify certificate: x509: certificate is not valid for any names,
# but wanted to match gitea.home.keithjackson.dev
#
# Measured from the workstation the same day, against the WAN address with that SNI:
# subject=C=ZZ, ST=Somewhere, L=Unknown, O=%D342e7e6b, CN=%D
# No extensions in certificate <- no SANs at all, hence "not valid for ANY names"
# while the same name via PiHole answers 192.168.2.20 and serves the real
# Let's Encrypt *.home.keithjackson.dev. So the certificate and the Gateway are fine;
# only the node's ANSWER is wrong. Same root cause, same fix, as the issuer pin above.
#
# WHO NEEDS IT: any workload whose image lives in the in-cluster Gitea registry —
# today platform/pm-agent (agent-runner) and conformance/incluster
# (conformance-runner), neither of which has ever been pulled. A pull is still
# UNAUTHENTICATED after this pin, which is a separate thing to settle; this only makes
# the node reach the right endpoint.
- "gitea.{{ k3s_base_domain }}"
# --- platform hostname suffix, ANSIBLE SIDE (R-ENV-02, R-DNS-01) --------------------
# PROD SERVES ITS OWN SUBDOMAIN. [parity] MUST equal BASE_DOMAIN in clusters/prod/