docs: PM handoff — Phase-7 prep session complete (PR #4, 13 commits, CI green)

Records what landed, the numbers (coverage 37->40/51, safety-critical scaffolds 1->0,
PROVEN still 0 by design), the corrections made mid-session so they are not re-litigated
(prod NOPASSWD rejected; the '$i envsubst bug' disproven by running the real tool; the
stranded DNS branch dropped rather than landed), three near-misses worth remembering, the
six items blocked on the operator, and what is knowingly left undone.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed 2026-09-01 00:15:28 -05:00
1 parent c1f24e4a6d
commit ce1966acb0
1 file changed
+79
+79
View File
@@ -1407,3 +1407,82 @@ definition of done, only a green requirement-tagged test is acceptance — so th
to make `dr-restore/chainsaw-test.yaml` and the node-loss test actually assert the behaviour the
operator has already demonstrated, and let the runner enforce it. Recording the attestation here
does not close the requirement, and the gate must not be softened to make it appear closed.
---
## 2026-09-01 — Phase-7 prep session complete. **PM HANDOFF.**
Everything below is on branch `docs/phase-7-open-prod-node-1` / **PR #4**, 13 commits, the
enforced Gitea gate **green**. Nothing has been installed on prod hardware. `main` is untouched
until the operator merges.
### Delivered
1. **Phase 7 opened (prep only)** with the entry-gate shortfall recorded, not waived.
2. **T-7.1 checklist** (`infra/PROD-PROVISIONING.md`) written against the real hardware.
3. **R-ENV-02 tokenization** — 6 `cluster-vars` knobs; prod would otherwise have come up
announcing DEV addresses. Dev proven byte-identical.
4. **Prod Ansible**: inventory, vaulted `become` (prod gets NO passwordless sudo), and
**target-group parameterisation** so prod reuses the roles unchanged.
5. **`node_health` role** — read-only, portable dev+prod, run against both.
6. **Repo-derived spec status report** (`run.sh --status`).
7. **CI false-green killed** + L0 static tests for R-SEC-01/02 and R-TEST-06.
8. **dind readiness fix**; **DNS resilience re-authored** for the current stack.
9. **`dr-restore` now asserts** — safety-critical scaffolds 0.
### Numbers
Coverage 37/51 → **40/51**. Executable tests 32 → **36**. Scaffold-only 5 → **4**.
Safety-critical without an executable L3 1 → **0**. **PROVEN: 0 — still not derivable**, by
design; proof-of-pass is not recoverable from repo state, so a green-run ledger is the only
thing that ever closes a requirement.
### Corrections made during the session (recorded so they are not re-litigated)
- **NOPASSWD sudo on prod: REJECTED** by the operator. Prod uses vaulted `become`. My original
checklist copied dev's convenience onto prod; that was wrong.
- **The "`$i` envsubst bug" was FALSE.** An assessment claimed Flux envsubst ate bare `$i`,
making the dind cert-wait bound a no-op. Running the real `flux envsubst` disproved it: bare
`$VAR` passes through; only `${VAR}` expands — matching this repo's own contract in
`clusters/base/apps-vms.yaml`, itself written after empirical probing. **No escaping was
applied**; `$${i}` would have been an active regression (correct only on the envsubst path —
on a plain `kustomize build`, `sh` reads `$$` as its PID). Agent reports are not evidence;
the tool is.
- **The stranded DNS branch was NOT landed.** Its netplan mechanism no longer exists on Debian
13; landing it would have broken `node-baseline.yml` on every node.
### Near-misses worth remembering
- A YAML **comment** nearly broke every merge in the repo: Flux envsubst runs over comments too,
so a braced name with punctuation is a fatal parse error that fails the whole `apps-wave-1`
Kustomization.
- `roles/cilium`, `cluster_verify`, `flux_bootstrap` all hardcoded `.cache/dev.kubeconfig` — the
file `dev.sh` exports as `$KUBECONFIG`. The moment prod became targetable, a prod run would
have silently overwritten it and left the operator's `kubectl` answering from **prod**.
- Prod's `GATEWAY_VIP` outside the LB pool would have brought the Gateway up on `.21` while
split-horizon DNS answered `.20` — blackholing in-cluster SSO while reporting healthy.
### BLOCKED ON THE OPERATOR — nothing else moves without these
1. **Create `inventory/group_vars/k3s_prod/vault.yml`** (`ansible_become_password`). The
passphrase file already exists. This gates SMART, the packages-only baseline, and every
future prod play.
2. **Drive health decision** — run `smartctl -a /dev/sda` once `smartmontools` lands via
`node-baseline.yml -e k3s_hosts=k3s_prod --tags packages`. Prod state on an unmeasured
~2011 consumer SSD is the open risk.
3. **Prod's second DNS upstream** — `192.168.2.1` is both gateway and only IPv4 resolver.
4. **Prod backup target** — `S3_BACKUP_ENDPOINT` is still `TBD-PROD-*`; dev's versitygw is on
the far side of the pfSense boundary.
5. **`UNIFI_L4_VIP` / `CRAFTY_L4_VIP`** — allocatable from `.20-.30`, not invented here.
6. **Merge PR #4.**
### Known, documented, NOT fixed
- `roles/flux_bootstrap` shells to `bootstrap/flux/bootstrap.sh` with a hardcoded
`--path=clusters/dev` — retargeting prod needs a `bootstrap/` change.
- Prod has no VPN egress group (needs its own vaulted PIA credentials).
- **`dr-restore` is AUTHORED, NEVER EXECUTED** (`conformance.home.k3s/never-executed`). It is
destructive; converting it to proven needs an operator maintenance-window run. The log must
show `DR_RESTORED_FROM_OFF_CLUSTER_BACKUP_OK`, **not** `DR_RESTORE_SKIPPED_OK` — green but
proving nothing.
- `dind` → native sidecar `initContainer` with a `docker info` startupProbe (NOT a bare
tcpSocket probe) is the right shape on k3s 1.36; deferred so a merge-gate failure is
unambiguous.
- **ECC is unknowable on this hardware** — no EDAC controller bound; non-ECC consumer RAM.
Silent memory corruption on a prod node would be invisible. Hardware property, not fixable.
- L2/L3 conformance still SKIPs in CI (`docs/plans/l2-l3-conformance-ci-wiring.md`); it needs
the trigger-credential decision — a scoped in-cluster SA, never a broad kubeconfig in CI.