0 Fork 0

Files

T

jakman85andClaude Opus 4.8 e20f17b896 K2 DONE: L2 network behavior tests pass against the live cluster

R-NET-01 default-deny + R-NET-02 deny-undeclared PASS via run.sh (JUnit+coverage,
exit 0). Fixed 3 real bugs found by running them:
- chainsaw strict list-matching: assert scalar 'phase: Running' not a conditions[] element.
- default-deny blocks the kubelet readiness probe -> drop probes (dest pods just need to run).
- chainsaw report path ../_artifacts resolved to repo-root -> fix to _artifacts (was exit 1).
- swap agnhost->busybox httpd listeners (cached everywhere; cache-timing-proof).
Logged the kubelet-probe finding for WS-NET's T-2.2 default-deny baseline design.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

2026-08-16 15:46:00 -05:00

5.4 KiB

Raw Blame History

Autonomous Session Decision Log (PM)

Program Manager working independently while the operator is AFK (started 2026-08-16). Every best-effort decision and assumption is recorded here for later review. Specs were NOT modified (PM cannot change ../home-network-k3s; that requires the System Architect hat). Blocked items that need the operator are called out.

Guardrails I'm holding to

No spec changes. No fabricated secrets (Cloudflare token, PIA creds, SMTP — all await hand-off).
Don't break the running cluster: platform/ stays suspended in Flux until each component is filled in + validated; platform/cilium must carry k8sServiceHost=10.73.0.11 before it ever reconciles.
Everything author-first, make -C ci gate green before merge, feature-branch→main.

Decisions / assumptions (append-only)

#	Decision / assumption	Why	Review flag
A1	Complete Phase 1 T-1.4 via Option A (Flux bootstrapped, `platform`+`apps` suspended)	You picked Option A earlier as the safe path; "keep going" authorizes it	Confirm scope
A2	Advance Phase 2 only for components needing no operator secret: Longhorn, monitoring, Kyverno-admission, Cilium-parity	cert-manager/externaldns/velero/VPN are blocked on token/PIA/OQ-BKP	—
A3	Dev resource caps for monitoring (short retention, small PVCs, single replicas) from `cluster-vars`	22 GiB RAM ceiling	Tune later
A4	Host Gitea repo is public (NAT-only) so Flux needs no git creds	dev bootstrap simplicity	Fine for dev

Autonomous session results (2026-08-16)

Phase-2 authoring (author-only, merged): platform/longhorn (3-replica SC + bulk indirection), platform/monitoring (kube-prometheus-stack + Loki + Alloy, dev-trimmed ~1.3 GiB, +8 alert rules), platform/cilium (parity fix: k8sServiceHost=10.73.0.11 + masquerade, safe to adopt in Phase 2), platform/kyverno (verified). Gate green.
Flux INSTALLED on the cluster (4 controllers Running). Host Gitea working (named-volume fix), repo pushed. GitRepository created → BLOCKED: cluster pods can't reach 10.73.0.1:3000 (firewalld libvirt zone drops VM→host:3000).
- RESOLVED (operator added the ansible-vault become password): ran host-prereqs.yml (firewall opened via vault→become), VM→Gitea = 200, and Flux GitRepository is READY=True (main@sha1:a2f3bdc…). Phase-1 T-1.4 core is DONE — Flux installed + reconciling clusters/dev from the host Gitea. platform/apps Kustomizations intentionally NOT created (Option A) — the running cluster is untouched.
- Firewall + Gitea admin/repo/push + host become password are all IaC now (host-prereqs.yml, bootstrap-gitea.sh automation, vaulted group_vars/hypervisor) → destroy/rebuild is hands-off.
NOT deployed: platform/apps Kustomizations were deliberately NOT created — the running cluster (Cilium etc.) is untouched. Only Flux controllers + the GitRepository source exist.

KNOWN ISSUES to fix before Phase-2 un-suspend (logged, not yet fixed)

#	Issue	Impact	Fix
K1	~~Substitution-var case mismatch (UPPERCASE refs vs lowercase cluster-vars keys)~~	—	✅ FIXED: uppercased cluster-vars keys + dev/prod overlay paths; all refs resolve; builds green
K2	~~Conformance L2 tests partial/unverified~~	—	✅ DONE: fixed 3 test bugs (see below), ran via run.sh against the live cluster → R-NET-01 default-deny + R-NET-02 deny-undeclared PASS, JUnit+coverage written, exit 0. chainsaw v0.2.12 installed (host `~/.local/bin`)
K3	`bootstrap/gitea/bootstrap-gitea.sh` used a host bind-mount → rootless-podman perms broke app.ini	initial Gitea start failed	FIXED: switched to a named volume
K4	Kyverno/chainsaw findings during K2 (fixed in the tests): (a) chainsaw does STRICT (length-equal) list matching — asserting one condition against a pod's full `conditions[]` fails → used scalar `phase: Running`; (b) a default-deny baseline blocks the kubelet readiness probe (node→pod), so probe-bearing pods never go Ready → dropped probes; (c) chainsaw report `path: ../_artifacts` resolved to repo-root → exit 1 → fixed to `_artifacts`; (d) swapped agnhost→busybox httpd (cached, cache-timing-proof)	—	FIXED in the test files/config

FINDING for WS-NET (Phase 2, T-2.2 — default-deny baseline design)

The platform's real default-deny baseline must not break kubelet health/liveness/readiness probes (node→pod ingress). A naive ingress: [] NetworkPolicy in Cilium drops the kubelet TCP probe, so any pod with a probe never becomes Ready. The baseline must explicitly allow the health-check path (Cilium host/health-check allowance or an equivalent ingress exception). Surfaced empirically by the R-NET-02 conformance test; carry into T-2.2.

Toolchain note

chainsaw v0.2.12 is installed on the host (~/.local/bin) for local conformance runs; for full reproducibility it should also be baked into infra/devcontainer (pinned) — the in-cluster Chainsaw Job (ADR-0011) already uses the pinned image. Deferred (non-blocking).

Blocked on operator (hand-offs)

T-2.4p Cloudflare API token → cert-manager DNS-01 + ExternalDNS (Phase 2 TLS/DNS).
T-3.2p PIA username/password → VPN egress gateway (Phase 3).
OQ-BKP off-cluster backup target → Velero (Phase 4).
SMTP creds → Alertmanager email (Phase 4).

5.4 KiB Raw Blame History