fix(node-baseline): pin the OIDC issuer to the Gateway VIP on prod nodes #41

2026-10-03T22:40:57Z

gitea-admin commented

2026-10-03 22:40:57 +00:00

Follow-up to #40. Public DNS now sends keycloak.home.keithjackson.dev to the WAN IP. The prod nodes resolve through public DNS, and the router doesn't hairpin, so the kube-apiserver can't reach its OIDC issuer. Measured on k3s-prod-2/-3: the name resolves to 136.51.115.252 and curl exits 60 because of the router's self-signed cert.

No OIDC errors are logged yet because the apiserver is still using keys it cached before the flip. kubectl SSO will break on the next k3s restart or Keycloak key rotation.

Change

node_baseline: a new node_baseline_hosts_pins list (default [], so dev is unaffected) and one blockinfile task (tags dns, hosts) that writes a marked /etc/hosts block.
group_vars/k3s_prod: pins keycloak.{{ k3s_base_domain }} → 192.168.2.20 (parity with GATEWAY_VIP). The stale "no pin needed" comment is replaced with the measured reason.

Verified

--check --diff --tags hosts on k3s-prod-2/-3 adds exactly the three-line marked block.
Toolbox gate rc=3: spec-immutability, decisions-log, lint, l0, render-golden, l1 and approval-policy all pass. gitea-settings was skipped (no token on the workstation).

After merge (operator)

cd infra/ansible
ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags hosts

k3s-prod-1 is currently unreachable from the workstation because its SSH host key changed (likely the 2026-09-11 reinstall). Confirm its ED25519 fingerprint SHA256:zgUS8OeWznzCX9hlN7XdZ8iLNkU1G2loXjtJFfEUYpE and update known_hosts first, or run with --limit k3s-prod-2,k3s-prod-3 for now.

🤖 Generated with Claude Code

Follow-up to #40. Public DNS now sends `keycloak.home.keithjackson.dev` to the WAN IP. The prod nodes resolve through public DNS, and the router doesn't hairpin, so the kube-apiserver can't reach its OIDC issuer. Measured on k3s-prod-2/-3: the name resolves to `136.51.115.252` and `curl` exits 60 because of the router's self-signed cert. No OIDC errors are logged yet because the apiserver is still using keys it cached before the flip. kubectl SSO will break on the next k3s restart or Keycloak key rotation. ## Change - `node_baseline`: a new `node_baseline_hosts_pins` list (default `[]`, so dev is unaffected) and one `blockinfile` task (tags `dns`, `hosts`) that writes a marked `/etc/hosts` block. - `group_vars/k3s_prod`: pins `keycloak.{{ k3s_base_domain }}` → `192.168.2.20` (parity with `GATEWAY_VIP`). The stale "no pin needed" comment is replaced with the measured reason. ## Verified - `--check --diff --tags hosts` on k3s-prod-2/-3 adds exactly the three-line marked block. - Toolbox gate `rc=3`: spec-immutability, decisions-log, lint, l0, render-golden, l1 and approval-policy all pass. `gitea-settings` was skipped (no token on the workstation). ## After merge (operator) ``` cd infra/ansible ansible-playbook playbooks/node-baseline.yml -e k3s_hosts=k3s_prod --tags hosts ``` **k3s-prod-1 is currently unreachable from the workstation** because its SSH host key changed (likely the 2026-09-11 reinstall). Confirm its ED25519 fingerprint `SHA256:zgUS8OeWznzCX9hlN7XdZ8iLNkU1G2loXjtJFfEUYpE` and update `known_hosts` first, or run with `--limit k3s-prod-2,k3s-prod-3` for now. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

gitea-admin added 1 commit 2026-10-03 22:40:57 +00:00

fix(node-baseline): pin the OIDC issuer to the Gateway VIP on prod nodes

gate / gate (pull_request) Successful in 1m46s

Details

gate / conformance (pull_request) Has been skipped

Details

788eaeae62

After public DNS moved to the WAN IP (#40), prod nodes resolve
keycloak.home.keithjackson.dev to a WAN address the router does not
hairpin, so the apiserver could not reach its OIDC issuer. Adds an opt-in
node_baseline_hosts_pins /etc/hosts block; prod pins keycloak to
GATEWAY_VIP.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jakman85 merged commit 7ef2af205d into main

2026-10-03 22:46:38 +00:00

jakman85 referenced this issue from a commit

2026-10-03 22:46:39 +00:00

Merge pull request 'fix(node-baseline): pin the OIDC issuer to the Gateway VIP on prod nodes' (#41) from fix/node-keycloak-hosts-pin into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Home-Network/home-network-k3s-impl#41