Commit Graph
10 Commits
Author SHA1 Message Date
jakman85andClaude Opus 5 f7d3884ad2 fix(ci): scope the requirement matcher to the diff and to non-doc files
`ci/touched-requirements.sh` documents its detection as "R-* tokens literally
present in the diff of changed files". It did not do that — it grepped the ENTIRE
CONTENT of every changed file. So a change inherited every requirement those files
had ever mentioned in prose, and `--require-all` then blocked the merge on
requirements the author never went near.

On this branch that meant editing the decision log and the tofu docs dragged in
five unrelated requirements, all on untouched lines, all legitimately uncovered:

  [coverage] REQUIRE: FAILED — required requirement(s) uncovered: [5 IDs]

The practical effect was that the PM could not append to the decision log without
failing CI — which is a fair part of why the log went stale in the first place.
This is the defect already logged as "FOLLOW-UP (WS-TEST): fix the matcher so real
requirements need not be exempted".

Two scoping rules, both learned from the gate blocking merges it should not have:

  1. Scan the ADDED/REMOVED lines of the diff, as the header always claimed.
     Removed lines count too — deleting a test's requirement tag, or a policy's
     self-declared ID, touches that requirement just as much as adding one.

  2. Skip `*.md`. An ID in prose is a REFERENCE, not a declaration: explaining a
     requirement is not changing behaviour against it. Rule 1 alone is not enough —
     a commit that documents a mis-attribution names the IDs involved and thereby
     re-triggers the very mis-attribution it fixes, and the decision log can never
     record which requirements a change concerned.

This TIGHTENS attribution rather than weakening the gate:
  - no requirement is exempted, and DOCUMENTED_GAPS is untouched;
  - authoritative declarations (`conformance.home.k3s/R-*` tags, Kyverno policies,
    ServiceDefinitions) all live in YAML, on lines a real change must touch;
  - the coarse path->requirement fallback is unchanged and file-type agnostic, so
    the doc subtrees that matter (conformance/, schemas/, catalog/, clusters/, ci/)
    stay covered either way.

Alternatives rejected: adding the five IDs to DOCUMENTED_GAPS genuinely weakens the
gate; scrubbing requirement IDs out of prose (precedent f9e9640) would mean editing
the decision log's own history to appease a matcher.

Gate semantics change — flagged in DECISIONS-LOG.md for review.

Also drops three R-TEST-06 mentions this branch had added to prose and refreshes
the now-stale "gate is RED" notes.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-27 00:06:28 -05:00
jakman85andClaude Opus 5 12c40d8c9c docs: reconcile the documentation with the deployed reality
The docs had drifted badly behind ~85 commits of delivery. The root README still
said "Phase 0 — Foundations. No cluster exists yet"; the infra docs still
described a libvirt/KVM cluster that has been retired for two days; and
DECISIONS-LOG.md — the PM's authoritative record — stopped at 2026-08-25 AM.
Anyone following these docs would have been working from a false map.

Verified against the live cluster (3/3 nodes, 7/7 Flux Kustomizations Ready,
10/10 HelmReleases Ready) rather than inferred from commit messages.

README.md
  - Phase 0 -> a per-phase status table: 0-6 delivered and deployed, 7 not open.
  - Layout now shows the wave split, platform-config, apps/vms, infra, bootstrap.
  - Real conformance number (36/51, 70.6%) and how to regenerate it.
  - Devcontainer workflow; note that the gate was red and why.

infra/PHASE1-STATUS.md — rewritten. Was a `virsh`/libvirt resume guide for a
  cluster that no longer exists. Now: the Proxmox substrate, the pfSense NAT
  paths, both kubectl access paths (Keycloak OIDC normal / cert break-glass),
  the toolchain, off-repo state that must not be lost, and the fact that Flux's
  git source is an app inside the cluster (`git push origin main` IS the deploy).

infra/PHASE1-SCOPE.md — historical banner. The exit criteria and sizing
  rationale are still worth keeping; the libvirt execution steps are not.

infra/README.md — Proxmox, plus a map of the subtree and where the cluster
  actually lives (it is not on this workstation).

infra/devcontainer/README.md — the "run tofu and ansible on the HOST" limitation
  is OBSOLETE: it was a dmacvicar/libvirt polkit/PID problem, and bpg/proxmox
  talks to an HTTP API. Everything runs in the container now. Documents the full
  CI-gate toolchain and what each bind mount is load-bearing for (the age keyring
  and the read-only spec repo especially — without the latter, spec-immutability
  silently skips instead of verifying).

infra/tofu/README.md — was "Authoring only — nothing here has been applied". It
  is applied and live. Disk sizes corrected 40/100 -> 64/128 GiB (confirmed
  against the running nodes), pfSense NAT marked done, and a warning that
  terraform.tfstate is absent on a fresh workstation — an apply with empty state
  would try to recreate the three live nodes.

infra/tofu/variables.tf — section comment disagreed with its own defaults.

infra/cluster.sh — retired-with-instructions header. It silently drives libvirt
  domains that do not exist; behaviour unchanged, but it no longer lies.

apps/vms/unifi/README.md — was "NOT WIRED IN", 60Gi, and listed a file that has
  since been dropped. Now: live status, 40Gi, resolved open questions marked
  resolved, and the two constraints the next person will otherwise rediscover —
  KubeVirt's 2048B inline userData cap, and that apps-vms must never carry
  postBuild.substituteFrom (envsubst corrupts the cloud-init).

DECISIONS-LOG.md — backfilled the gap as one structured entry: the Proxmox
  migration, NAT + kube-vip, the twice-proven oauth2-proxy cold-start deadlock,
  the Flux source cutover, CI going live in-cluster, ten app onboardings and the
  renderer bugs they exposed, the wave split (verified: DBs adopted, not
  recreated — original ages intact, prune restored), KubeVirt storage, the UniFi
  VM, and kubectl SSO. Plus verified live state and the carried-forward open list.

Two corrections to the record while verifying:
  - kubectl OIDC is LIVE (`kubectl auth whoami` -> oidc:<user>,
    [oidc:platform-admins]), so the rolling restart 3472f1d deferred has
    happened.
  - UniFi OS Server is NOT installed in the guest. The VM is healthy (Ubuntu
    24.04.4, guest agent connected, live-migratable) but :11443 answers nothing,
    and the CNP explicitly permits cluster -> :11443, so it is genuinely not
    listening rather than a policy artifact.

Docs and comments only; no manifest semantics change.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-27 00:00:05 -05:00
jakman85andClaude Opus 4.8 81ef78295a infra: direct kube-API access via kube-vip VIP + NAT (retire SSH tunnel)
Controller now reaches the kube-API directly through the pfSense NAT
(192.168.2.50:6443 -> kube-vip VIP 192.168.1.49:6443), validating TLS
against the VIP via tls-server-name. Removes the dev.sh SSH-tunnel
scaffolding and updates the ansible source-of-truth so a rebuild
regenerates a direct-access kubeconfig.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-25 22:10:27 -05:00
jakman85andClaude Opus 4.8 bb48f675ee ci: bake pinned test toolchain into devcontainer + fix conformance layer/spec-immutability seams
Bake the conformance harness toolchain into infra/devcontainer so `make -C ci gate`
runs identically here and in CI:
- Containerfile: install kyverno CLI (1.13.4), kyverno-json, chainsaw (0.2.12),
  yamllint (1.35.1), check-jsonschema (0.29.4), yq (4.44.6) via new pinned RUN block;
  add diffutils + python3-pip; drop stale LIBVIRT_DEFAULT_URI; refresh header for the
  Proxmox(bpg)+conformance toolbox; extend /etc/dev-toolchain-versions provenance.
- versions.env: add KYVERNO/KYVERNO_JSON/CHAINSAW/YAMLLINT/YQ/CHECK_JSONSCHEMA;
  align KUBECTL 1.32.0 -> 1.31.4. dev.sh build(): pass the new --build-args.
- ci/versions.env: pin CHECK_JSONSCHEMA_VERSION=0.29.4 (single source of truth).

Fix harness seams so the gate can run:
- dev.sh run(): mount the sibling spec repo (home-network-k3s) read-only at
  /home-network-k3s for ci/spec-immutability.sh (R-TEST-06); conditional so the
  container still runs without it.
- conformance/lib/common.sh conf_dirs_for_layer(): accept a comma/space list of
  layers and union the dirs, so ci/conformance.sh `run.sh --layer l2,l3` selects
  both layers in one pass (backward compatible for single layers).

NOTE: ci/versions.env pins KYVERNO_JSON_VERSION=0.0.5, which has no upstream release
(max tag is v0.0.3; platform/kyverno/scripts/run-l0.sh already defaults to v0.0.3).
Baked v0.0.3 and flagged for PM to correct ci/versions.env. kyverno-json is not used
by any static gate stage.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-25 15:43:19 -05:00
jakman85andClaude Opus 4.8 0d7495486e devcontainer: wire dev.sh to the Proxmox k3s cluster
- drop stale libvirt socket mount + LIBVIRT_DEFAULT_URI (old KVM cluster)
- auto-open an SSH tunnel (local :16443 -> primary 127.0.0.1:6443 over the
  pfSense SSH NAT) on each run so kubectl/flux/helm reach the cluster with no
  manual setup; DEV_NO_TUNNEL=1 opts out, K3S_SSH_* env overrides the NAT front
- refresh header/usage comments (bpg/proxmox, not dmacvicar/libvirt)

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-25 14:59:19 -05:00
jakman85andClaude Opus 4.8 1ff25318c9 ansible(proxmox): route Ansible via pfSense NAT; decouple k3s node-ip from ansible_host; mount vault-pass into dev container
- inventory routes SSH via 192.168.2.102:2250/2251/2252; new k3s_node_ip carries
  each node's real 192.168.1.x for --node-ip / k3s_api_host / Cilium parity
- dev.sh bind-mounts ~/.config/home-k3s/ansible-vault-pass (ro) so vaulted
  playbooks can run in the pinned container (ansible.cfg vault_password_file)

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-25 11:55:37 -05:00
jakman85andClaude Opus 4.8 4bb9c8fcbd infra(tofu): docs + example for the Proxmox migration
Update tofu README (Proxmox targets table, operator prerequisites:
enable Snippets content on `local`, endpoint scheme fix, pfSense NAT),
terraform.tfvars.example (Proxmox vars + env-only auth, token never a
tofu var), and the devcontainer versions.env provider note.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-25 10:23:28 -05:00
jakman85andClaude Opus 4.8 f089adf261 devcontainer: set KUBECONFIG + persist SOPS age key to host
Two dev.sh gaps surfaced while running bootstrap/sops/install-age-key.sh:
- kubectl defaulted to localhost:8080 (connection refused) because KUBECONFIG
  was never set in the toolbox. Set KUBECONFIG=/work/infra/ansible/.cache/dev.kubeconfig
  (override via DEV_KUBECONFIG); --network host already reaches 10.73.0.11:6443.
- age-keygen.sh writes the PRIVATE key to ~/.config/sops/age/keys.txt, but that
  path was not mounted in the --rm container, so the key was lost on exit. Mount
  ${HOME}/.config/sops -> /root/.config/sops (created first) so the key persists
  on the host and SOPS_AGE_KEY_FILE resolves to it.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-16 22:31:51 -05:00
jakman85andClaude Opus 4.8 017ed0fb9c Phase 1 checkpoint: status/resume doc + cluster helper + toolchain note
Cluster is UP (3/3 Ready, Cilium OK); Phase 1 paused before Flux bootstrap
(Option A: suspend platform/apps until Phase 2). Add infra/PHASE1-STATUS.md
(what's done, state file locations, reconnect steps, remaining T-1.4/1.5,
Phase-2 platform prereq), infra/cluster.sh (status/start/stop/kubeconfig), and
document that tofu+ansible run on the host (libvirt provider limitation).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-16 02:18:39 -05:00
jakman85andClaude Opus 4.8 5d2b248b68 Phase 1 toolchain: pinned dev container (prereqs not all Python)
Per operator rule: OpenTofu/age/sops/flux are Go binaries, so bake the toolchain
into a pinned Fedora-44 dev container (matches host libvirt 12) instead of a venv.
infra/devcontainer/: Containerfile + versions.env (tofu 1.9.0, age 1.2.1, sops
3.9.4, flux 2.4.0, kubectl 1.32.0, helm 3.16.0) + dev.sh (runs with host libvirt
socket + repo + ssh mounted) + README. Built & validated: all tools resolve;
virsh reaches host libvirt from inside the container.

Also record resolved Phase 1 decisions in PHASE1-SCOPE.md (6GiB/4vCPU x3;
k3s-dev NAT 10.73.0.0/24; container toolchain). Gate green.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-16 00:59:47 -05:00