home-network-k3s-impl

772 Commits 140 Branches 0 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
jakman85andClaude Opus 5	e9a4b669d4	feat(conformance): committed run ledger — the only thing that can move PROVEN off zero status.py has always reported PROVEN: 0 for all 51 requirements, and named the fix in its own header: "a committed, machine-readable run ledger". There was none — _artifacts/ is gitignored, the root .gitignore kills .junit.xml, and a stray JUnit was surfaced only as a quarantined mtime advisory. Built BEFORE the first real L3 execution so that run is captured rather than reconstructed from memory. THE TRAP THIS EXISTS TO AVOID. These tests exit 0 and Chainsaw records `passed` on a SELF-SKIP exactly as on a real run, and Chainsaw's JUnit carries no stdout — only properties. So committing the JUnit as proof would let DR_RESTORE_SKIPPED_OK be recorded as proof of R-STATE-03: authoritative-looking and wrong, which is worse than no ledger. The sentinel token is the ONLY thing distinguishing proven from skipped, and it lives solely in the run log. The ledger therefore records the LOG alongside the JUnit and derives proof from the sentinel, never from the JUnit result. Sentinel tokens are DERIVED, not hand-listed: record_run.py parses every contains($stdout,'TOKEN') out of each test's own chainsaw-test.yaml, so the list cannot drift from the tests. 23 of 35 tests declare no sentinel at all. decide() is the single place `proven` is emitted, ordered WORST-FIRST: JUnit failure/error -> failed JUnit <skipped/> -> junit-skipped ANY declared skip token in the log -> skipped (beats a green JUnit AND beats a pass token present in the same log) no sentinel declared -> proven ONLY for L0/L1 (whose JUnit is truthful on its own, since run_l1_offline.py maps exit 3 to <skipped/>); written as an explicit exempt-set so an unknown/blank layer fails CLOSED EVERY declared pass token present -> proven otherwise -> unknown-sentinel-absent Log attribution is a state machine over Chainsaw's per-test headers; a log with no recognisable header falls back to whole-log scanning restricted to GLOBALLY-UNIQUE tokens, so DNS_OK (owned by two tests) can never prove either. record_run.py exits 3 if the JUnit or the log is missing — nothing recorded, never green. STALENESS: a proven row is honoured only while `git merge-base --is-ancestor` holds between the requirement's last implementing/test change and the row's git_sha; otherwise it degrades to stale-proof. A -dirty sha, an unknown rev, or an uncommitted test path all fail ancestry the same way. A requirement reads proven only when EVERY verifying test does (worst-of), and the latest row per test wins — not best-ever. REFUSAL POSTURE PRESERVED: with an empty ledger the report is unchanged in intent — every requirement not-derivable, headline "PROVEN: 0 — NOT DERIVABLE". This replaces "not derivable" with real evidence only where evidence exists, never with optimism. --record is explicit, never implicit (a run that silently records itself is how a bad run becomes permanent false proof), is rejected for --status/--coverage-only/--lint, and re-execs under tee returning ${PIPESTATUS[0]} so recording can never soften a gate. It rm -f's a stale _artifacts/report.junit.xml first — a hazard hit live during development, where a stale JUnit got paired with a fresh log. Selftests: record_run.py 23 checks (incl. end-to-end Chainsaw-passed + NODE_LOSS_SKIPPED_OK -> proof "skipped"), status.py +16 against a throwaway git repo (self-skip -> skipped; proven-at-older-sha -> stale-proof; dirty/unresolvable -> stale-proof; worst-of; latest-row-wins; malformed lines dropped). No regressions: --coverage-only, --status, --lint, --strict-coverage and the merge-gate shape all rc=0 and unchanged; nothing in ci/ references status.py. Root .gitignore needed no change — conformance/runs/ is not itself excluded, so the deeper negations override the root's global .junit.xml. Verified with git check-ignore. Serves: R-TEST-01, R-TEST-02, R-TEST-03, R-TEST-04. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>	2026-09-01 18:49:01 -05:00
jakman85andClaude Opus 5	1252823d80	feat(conformance): repo-derived spec status report (conformance/run.sh --status) Operator ruling: "I care about the state of the repository and what the status of each spec is based on that state" — not a hand-maintained task board. This is that report. conformance/lib/status.py (stdlib-only, --selftest) sits beside lib/coverage.py and IMPORTS it rather than forking the registry/test parsers. Driven by a new `conformance/run.sh --status`; artifacts land in _artifacts/status/ alongside coverage and are git-ignored by the existing convention. Exit codes follow ci/lib.sh (0/1/3); --strict-coverage also applies the status hygiene gate. Signals derived per requirement: is there an implementing artifact; is it RECONCILED (walk resources: from clusters/dev, expand each Flux Kustomization's spec.path, recurse — so a commented-out resource correctly does not count); is there a tagged test and is its body executable or a TODO scaffold (TODO markers OUTRANK an executable: "true" annotation — ambiguity never upgrades a row). DELIBERATELY NOT DERIVED, and reported as such rather than defaulting to green: proof-of-pass (nothing tracked in git records a run result — _artifacts/ and results/ are git-ignored) and cluster liveness. "Reconciled" means Flux WOULD apply it, not that it is healthy. PROVEN is reported as "not derivable" for all 51. WHAT IT SHOWS THAT coverage.md DOES NOT. Coverage reports 37/51 covered, 0 safety-critical gaps. Three of those claims do not survive repo state: 1. 5 of the 37 "covered" requirements have only a TODO-body scaffold (R-STATE-03, R-OBS-02, R-OBS-04, R-GITOPS-03, R-GITOPS-02). Coverage counts a tagged file; it never looks inside it. 2. R-STATE-03 is SAFETY-CRITICAL and its L3 fault test asserts nothing. Coverage reports "0 safety-critical gaps" only because dr-restore/chainsaw-test.yaml exists and carries an L3 label. --strict now fails on this. 3. R-UPD-01/02/03 are NOT reconciled: platform/image-automation/ is commented out of platform/kustomization.yaml. Coverage shows them covered with executable L3 tests, which reads as delivered; automated updates are actually 0/3 wired+testable. Headline: 51 requirements \| reconciled 20 (+12 partial) \| executable test 32 \| wired+testable 22 (+8 partial) \| scaffold-only 5 \| PROVEN 0 (not derivable). Also surfaces 18 requirements with no reconciled manifest citing their R-ID — listed as a traceability gap (R-TEST-04) rather than guessed either way. Annotating those manifests makes the report self-correct, so it acts as a forcing function instead of another hand-maintained map. Serves: R-TEST-01..06. Gate: MERGE GATE: PASS. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>	2026-08-31 22:44:10 -05:00
jakman85andClaude Opus 4.8	9d8e03892f	T-0.4 Chainsaw conformance harness + coverage + behavior scaffolds (R-TEST-01..06) Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>	2026-08-16 00:08:32 -05:00

Author

SHA1

Message

Date

jakman85andClaude Opus 5

e9a4b669d4

feat(conformance): committed run ledger — the only thing that can move PROVEN off zero

status.py has always reported PROVEN: 0 for all 51 requirements, and named the fix in its
own header: "a committed, machine-readable run ledger". There was none — _artifacts/ is
gitignored, the root .gitignore kills *.junit.xml, and a stray JUnit was surfaced only as
a quarantined mtime advisory. Built BEFORE the first real L3 execution so that run is
captured rather than reconstructed from memory.

THE TRAP THIS EXISTS TO AVOID. These tests exit 0 and Chainsaw records `passed` on a
SELF-SKIP exactly as on a real run, and Chainsaw's JUnit carries no stdout — only
properties. So committing the JUnit as proof would let DR_RESTORE_SKIPPED_OK be recorded
as proof of R-STATE-03: authoritative-looking and wrong, which is worse than no ledger.
The sentinel token is the ONLY thing distinguishing proven from skipped, and it lives
solely in the run log. The ledger therefore records the LOG alongside the JUnit and
derives proof from the sentinel, never from the JUnit result.

Sentinel tokens are DERIVED, not hand-listed: record_run.py parses every
contains($stdout,'TOKEN') out of each test's own chainsaw-test.yaml, so the list cannot
drift from the tests. 23 of 35 tests declare no sentinel at all.

decide() is the single place `proven` is emitted, ordered WORST-FIRST:
  JUnit failure/error -> failed
  JUnit <skipped/>    -> junit-skipped
  ANY declared skip token in the log -> skipped   (beats a green JUnit AND beats a pass
                                                   token present in the same log)
  no sentinel declared -> proven ONLY for L0/L1 (whose JUnit is truthful on its own, since
                          run_l1_offline.py maps exit 3 to <skipped/>); written as an
                          explicit exempt-set so an unknown/blank layer fails CLOSED
  EVERY declared pass token present -> proven
  otherwise -> unknown-sentinel-absent
Log attribution is a state machine over Chainsaw's per-test headers; a log with no
recognisable header falls back to whole-log scanning restricted to GLOBALLY-UNIQUE tokens,
so DNS_OK (owned by two tests) can never prove either. record_run.py exits 3 if the JUnit
or the log is missing — nothing recorded, never green.

STALENESS: a proven row is honoured only while `git merge-base --is-ancestor` holds
between the requirement's last implementing/test change and the row's git_sha; otherwise
it degrades to stale-proof. A -dirty sha, an unknown rev, or an uncommitted test path all
fail ancestry the same way. A requirement reads proven only when EVERY verifying test
does (worst-of), and the latest row per test wins — not best-ever.

REFUSAL POSTURE PRESERVED: with an empty ledger the report is unchanged in intent — every
requirement not-derivable, headline "PROVEN: 0 — NOT DERIVABLE". This replaces "not
derivable" with real evidence only where evidence exists, never with optimism.

--record is explicit, never implicit (a run that silently records itself is how a bad run
becomes permanent false proof), is rejected for --status/--coverage-only/--lint, and
re-execs under tee returning ${PIPESTATUS[0]} so recording can never soften a gate. It
rm -f's a stale _artifacts/report.junit.xml first — a hazard hit live during development,
where a stale JUnit got paired with a fresh log.

Selftests: record_run.py 23 checks (incl. end-to-end Chainsaw-passed + NODE_LOSS_SKIPPED_OK
-> proof "skipped"), status.py +16 against a throwaway git repo (self-skip -> skipped;
proven-at-older-sha -> stale-proof; dirty/unresolvable -> stale-proof; worst-of;
latest-row-wins; malformed lines dropped). No regressions: --coverage-only, --status,
--lint, --strict-coverage and the merge-gate shape all rc=0 and unchanged; nothing in ci/
references status.py.

Root .gitignore needed no change — conformance/runs/ is not itself excluded, so the deeper
negations override the root's global *.junit.xml. Verified with git check-ignore.

Serves: R-TEST-01, R-TEST-02, R-TEST-03, R-TEST-04.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

2026-09-01 18:49:01 -05:00

jakman85andClaude Opus 5

1252823d80

feat(conformance): repo-derived spec status report (conformance/run.sh --status)

Operator ruling: "I care about the state of the repository and what the status of each
spec is based on that state" — not a hand-maintained task board. This is that report.

conformance/lib/status.py (stdlib-only, --selftest) sits beside lib/coverage.py and
IMPORTS it rather than forking the registry/test parsers. Driven by a new
`conformance/run.sh --status`; artifacts land in _artifacts/status/ alongside coverage
and are git-ignored by the existing convention. Exit codes follow ci/lib.sh (0/1/3);
--strict-coverage also applies the status hygiene gate.

Signals derived per requirement: is there an implementing artifact; is it RECONCILED
(walk resources: from clusters/dev, expand each Flux Kustomization's spec.path, recurse
— so a commented-out resource correctly does not count); is there a tagged test and is
its body executable or a TODO scaffold (TODO markers OUTRANK an executable: "true"
annotation — ambiguity never upgrades a row).

DELIBERATELY NOT DERIVED, and reported as such rather than defaulting to green:
proof-of-pass (nothing tracked in git records a run result — _artifacts/ and results/
are git-ignored) and cluster liveness. "Reconciled" means Flux WOULD apply it, not that
it is healthy. PROVEN is reported as "not derivable" for all 51.

WHAT IT SHOWS THAT coverage.md DOES NOT. Coverage reports 37/51 covered, 0
safety-critical gaps. Three of those claims do not survive repo state:

1. 5 of the 37 "covered" requirements have only a TODO-body scaffold (R-STATE-03,
   R-OBS-02, R-OBS-04, R-GITOPS-03, R-GITOPS-02). Coverage counts a tagged file; it
   never looks inside it.
2. R-STATE-03 is SAFETY-CRITICAL and its L3 fault test asserts nothing. Coverage reports
   "0 safety-critical gaps" only because dr-restore/chainsaw-test.yaml exists and carries
   an L3 label. --strict now fails on this.
3. R-UPD-01/02/03 are NOT reconciled: platform/image-automation/ is commented out of
   platform/kustomization.yaml. Coverage shows them covered with executable L3 tests,
   which reads as delivered; automated updates are actually 0/3 wired+testable.

Headline: 51 requirements | reconciled 20 (+12 partial) | executable test 32 |
wired+testable 22 (+8 partial) | scaffold-only 5 | PROVEN 0 (not derivable).

Also surfaces 18 requirements with no reconciled manifest citing their R-ID — listed as
a traceability gap (R-TEST-04) rather than guessed either way. Annotating those manifests
makes the report self-correct, so it acts as a forcing function instead of another
hand-maintained map.

Serves: R-TEST-01..06. Gate: MERGE GATE: PASS.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

2026-08-31 22:44:10 -05:00

jakman85andClaude Opus 4.8

9d8e03892f

T-0.4 Chainsaw conformance harness + coverage + behavior scaffolds (R-TEST-01..06)

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

2026-08-16 00:08:32 -05:00