fix(conformance): the ledger counted as evidence FOR the requirements it proves

Committing the first PROVEN row immediately degraded it to `stale-proof`. Cause: a
self-defeating loop.

`IMPL_EXCLUDE_PREFIXES` excluded `conformance/tests/` and `ci/tests/` but NOT
`conformance/runs/`. The ledger and the preserved run logs CONTAIN the requirement IDs they
are evidence FOR — `ledger.jsonl` literally holds `"requirements":["R-STATE-02"]` — so they
were scanned as IMPLEMENTING evidence for R-STATE-02. That made `last_change` for the
requirement move to the ledger commit itself, which the row's `git_sha` necessarily predates,
so the freshness check failed.

The consequence was total, not marginal: **recording a proof destroyed it, so NO ledger row
could ever have been non-stale.** The ledger would have reported `PROVEN: 0 (stale N)` forever
while looking like it was working — the same shape of silent-uselessness as the COMMAND-echo
bug in the previous commit, and found the same way: by actually running the thing and
disbelieving a green-looking result.

Fix: exclude `conformance/runs/`. The ledger is the proof RECORD; it must not participate in
the freshness computation about itself.

Verified: `PROVEN: 1 (stale 0, self-skipped 0, failed 0, unknown 0) from 1 ledger row(s)`;
R-STATE-02 reads **proven**. status.py --selftest PASS. The staleness machinery is otherwise
untouched — a genuinely stale row (impl changed after the run) still degrades, which the
selftest continues to assert.

Serves: R-TEST-01..04.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed 2026-09-01 19:08:38 -05:00
1 parent 4cb5e7a8d4
commit fdc3265585
1 file changed
+8 -1
+8 -1
View File
@@ -108,7 +108,14 @@ FLUX_API = "kustomize.toolkit.fluxcd.io"
# conformance/tests — those are the TESTS; they are the VERIFICATION axis, counted there.
# ci/tests — gate unit-test fixtures embed example R-IDs as DATA.
# registry.yaml — the requirement mirror itself, not an implementation of anything.
IMPL_EXCLUDE_PREFIXES = ("conformance/tests/", "ci/tests/")
# `conformance/runs/` is excluded for a subtler reason than the test dirs: the ledger and the
# preserved run logs CONTAIN the requirement IDs they are evidence FOR. Counting them as
# implementing evidence makes `last_change` move every time proof is recorded, so a row's
# git_sha always predates it and EVERY proof degrades to stale-proof the instant it is
# committed — recording the proof destroys it. The ledger is the proof record; it must not
# participate in the freshness computation about itself. (Found live 2026-09-01 on the first
# PROVEN row.)
IMPL_EXCLUDE_PREFIXES = ("conformance/tests/", "ci/tests/", "conformance/runs/")
IMPL_EXCLUDE_PATHS = ("conformance/requirements/registry.yaml",)