Operator ruling: "I care about the state of the repository and what the status of each
spec is based on that state" — not a hand-maintained task board. This is that report.
conformance/lib/status.py (stdlib-only, --selftest) sits beside lib/coverage.py and
IMPORTS it rather than forking the registry/test parsers. Driven by a new
`conformance/run.sh --status`; artifacts land in _artifacts/status/ alongside coverage
and are git-ignored by the existing convention. Exit codes follow ci/lib.sh (0/1/3);
--strict-coverage also applies the status hygiene gate.
Signals derived per requirement: is there an implementing artifact; is it RECONCILED
(walk resources: from clusters/dev, expand each Flux Kustomization's spec.path, recurse
— so a commented-out resource correctly does not count); is there a tagged test and is
its body executable or a TODO scaffold (TODO markers OUTRANK an executable: "true"
annotation — ambiguity never upgrades a row).
DELIBERATELY NOT DERIVED, and reported as such rather than defaulting to green:
proof-of-pass (nothing tracked in git records a run result — _artifacts/ and results/
are git-ignored) and cluster liveness. "Reconciled" means Flux WOULD apply it, not that
it is healthy. PROVEN is reported as "not derivable" for all 51.
WHAT IT SHOWS THAT coverage.md DOES NOT. Coverage reports 37/51 covered, 0
safety-critical gaps. Three of those claims do not survive repo state:
1. 5 of the 37 "covered" requirements have only a TODO-body scaffold (R-STATE-03,
R-OBS-02, R-OBS-04, R-GITOPS-03, R-GITOPS-02). Coverage counts a tagged file; it
never looks inside it.
2. R-STATE-03 is SAFETY-CRITICAL and its L3 fault test asserts nothing. Coverage reports
"0 safety-critical gaps" only because dr-restore/chainsaw-test.yaml exists and carries
an L3 label. --strict now fails on this.
3. R-UPD-01/02/03 are NOT reconciled: platform/image-automation/ is commented out of
platform/kustomization.yaml. Coverage shows them covered with executable L3 tests,
which reads as delivered; automated updates are actually 0/3 wired+testable.
Headline: 51 requirements | reconciled 20 (+12 partial) | executable test 32 |
wired+testable 22 (+8 partial) | scaffold-only 5 | PROVEN 0 (not derivable).
Also surfaces 18 requirements with no reconciled manifest citing their R-ID — listed as
a traceability gap (R-TEST-04) rather than guessed either way. Annotating those manifests
makes the report self-correct, so it acts as a forcing function instead of another
hand-maintained map.
Serves: R-TEST-01..06. Gate: MERGE GATE: PASS.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
R-STATE-02 was the last safety-critical requirement with no verifying test,
exempted via DOCUMENTED_GAPS. Author the L3 fault-injection test that proves the
guarantee and remove the exemption.
tests/l3-fault-injection/node-loss/chainsaw-test.yaml:
GIVEN a 3-replica Longhorn volume with written data attached on node N,
WHEN N is removed from service (cordon + evict the attached workload, forcing
the volume to detach from N),
THEN a fresh workload on a SURVIVING node re-attaches the same volume and reads
the canary intact — volume stays available, cluster reconverges.
Safe-by-construction for the 3-node embedded-etcd dev cluster:
- Disruption scoped to EXACTLY ONE node, at the scheduling layer only (cordon +
evict our own pod). kubelet/etcd stay up — etcd quorum is never touched.
- Never `kubectl drain`; only our own writer pod is deleted, so no unrelated or
system pod is disturbed.
- Victim node labelled conformance.home.k3s/cordoned-by=<test-ns>; `finally`
ALWAYS uncordons every node with THAT label and removes it — restored even on
failure/timeout, and never touches an operator-cordoned node.
- Self-skips green until a longhorn StorageClass AND >= 3 Ready nodes exist.
Bookkeeping:
- run.sh: remove R-STATE-02 from DOCUMENTED_GAPS; add node-loss to CONF_HEAVY_TESTS
(serial pass, matches the other Longhorn/KubeVirt L3 tests).
- incluster/rbac.yaml: enable the scoped nodes get/list/patch rule the test needs
(cordon/uncordon/label); no eviction rights (no drain).
- README.md / coverage.py docstring: refresh the documented-gap example.
Coverage: 36/51 -> 37/51 (70.6% -> 72.5%); safety-critical gaps 1 -> 0. Only
R-STATE-02 changed ([] -> node-loss); no other requirement or golden touched.
NOTE: LIVE execution is DEFERRED to an operator-chosen maintenance window. This
change AUTHORS + statically validates the test (renders, chainsaw-lints, harness
selects it, coverage updates). The static gate runs L2/L3 as SKIP without a
cluster (expected). The node-disruption test was NOT run live.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Phase 0 foundation shell (read-only spec at ../home-network-k3s). Top-level
tree + rules; workstream agents fill their subtrees on feature branches.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
jakman85andClaude Opus 5