jakman85andClaude Opus 5
523a463fd7
The pm-agent image pull failed TLS verification, and the previous entry on this branch
recorded it honestly as not diagnosed. Diagnosed now, and it is not about the image.
containerd pulls as a NODE process, so it resolves through node_baseline_dns_servers (the
LAN gateway, deliberately not PiHole) and gets the public answer: a CNAME to
wan.home.keithjackson.dev -> 136.51.115.252, measured via both 1.1.1.1 and 8.8.8.8. This
router does not hairpin, so WAN:443 answers with its own placeholder certificate —
measured with the right SNI from the workstation:
subject=C=ZZ, ST=Somewhere, L=Unknown, O=%D342e7e6b, CN=%D
No extensions in certificate
No extensions means no subjectAltName, which is exactly 'certificate is not valid for any
names'. Meanwhile 192.168.2.20 IS the shared Gateway's VIP and its https listener serves
the real Let's Encrypt *.home.keithjackson.dev. The Gateway, the wildcard cert and the
digest were never implicated — only the node's answer.
This file already documents the identical failure for the Keycloak issuer, measured from
k3s-prod-2/-3 on 2026-10-03 (curl exit 60, ssl_verify_result=18), and already carries the
mechanism that fixes it. gitea is simply the second node process to need it; containerd is
a node process and nobody had added it. An operator must run the node_baseline play.
Also records, where an operator reads them, that the agent authenticates to Gitea with a
local account and a scoped PAT by decision — and why the federated alternative is not
merely unchosen but unavailable: Gitea implements only the Authorization Code Grant and
validates no third-party OIDC token, so a Keycloak service-account token cannot
authenticate to its API or to git-over-HTTP.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
infra/ — node provisioning and the toolchain
OpenTofu (Proxmox VE) + cloud-init + Ansible bring up the 3-node k3s/Cilium dev cluster.
Spec: ../home-network-k3s/docs/02-environments-and-provisioning.md.
| Path | What |
|---|---|
tofu/ |
The "Create" stage: 3 cloud-init'd VMs on the Proxmox box (bpg/proxmox provider) |
cloud-init/ |
First-boot baseline shipped as a Proxmox user-data snippet |
ansible/ |
The "Baseline"/"Configure" stages: k3s HA, Cilium, kube-vip, the VPN egress gateway |
devcontainer/ |
The pinned toolchain (tofu, ansible, sops, age, flux, kubectl, helm and the whole CI gate toolchain) |
cluster.sh |
RETIRED — drove the old libvirt VMs; does not work against Proxmox |
PHASE1-STATUS.md |
Current state of the cluster and how to reconnect to it |
PHASE1-SCOPE.md |
Historical Phase-1 scoping doc (libvirt era — superseded) |
Where the cluster actually lives
The dev cluster is not on this workstation. It runs on a Proxmox VE host; the nodes sit on the Proxmox LAN and are reached across a pfSense NAT:
| Nodes | k3s-dev-1/2/3 at 192.168.1.50/.51/.52 — 6 vCPU, 24 GiB, Debian 13, k3s v1.36.4 |
| kube-API | kube-vip VIP 192.168.1.49:6443, reached from the workstation via NAT 192.168.2.50:6443 |
| Node SSH | pfSense NAT 192.168.2.102:2250/2251/2252 → node :22 (Ansible only) |
| Ingress VIP | 192.168.1.53 (Cilium L2, shared Gateway); .53–.60 reserved for LB VIPs |
The earlier libvirt/KVM cluster on 10.73.0.0/24 is retired. Any doc or command referencing
virsh, qemu:///system, or 10.73.0.x for the cluster is historical.