home-network-k3s-impl

1,078 Commits 138 Branches 0 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
jakman85andClaude Opus 5	755c08ab40	fix(conformance): the meta-check was layer-agnostic and reported failures silently Declaring whole-repo scope on the L1 tests made whole-repo-invariants-wired (R-TEST-03) fail. It was right to change behaviour and wrong about why, and it could not tell anyone either way. Two independent defects. 1. LAYER-AGNOSTIC WHERE THE DESIGN IS LAYER-BOUNDED The "every scope-declared test is selected despite a requirement filter matching none" assertion compared EVERY scope-declaring dir against an `--layer l0` selection. That silently assumed only L0 tests would ever declare the scope — while ci/l1.sh was written from the opposite premise, that a whole-repo invariant could appear at any layer. So the first L1 declaration made the check blame the union for correctly excluding another layer. Now partitioned by layer, and the L1 half of the property is asserted too: every scope-declared L1 test must be selected by `--layer l1 --include-scope`, and that selection must not leak non-L1 tests. That is what makes the L1 declaration load-bearing rather than decorative — previously nothing proved the union reached L1 at all, because nothing at L1 declared the scope. 2. EVERY ASSERTION FAILED SILENTLY The idiom throughout was a bare test on one line and `chk $?` on the next, under `set -euo pipefail`. A false test aborts the script BEFORE chk can print, so the diagnostic — which is the entire value of this check — was never reached. The real failure above emitted 26 `ok` lines, rc=1, and no message at all; it read as a tooling glitch rather than a finding. All 12 assertion sites now capture the status (`chk $arc`) so the check reports and continues. Proven by forcing MIN_DECLARED=999 against the real check: the failure prints its full explanation and the run proceeds to the remaining assertions, where before it died mute. A meta-check that cannot say why it failed is worse than no meta-check. Verified: 35 assertions, rc=0; full offline l0,l1,meta 25 tests / 0 failures. Co-Authored-By: Claude Opus 5 <[email protected]>	2026-09-18 22:43:17 -05:00
jakman85andClaude Opus 5	a76ae3bdfa	fix(ci): the content scan could not match R-C2-01/02 — a digit in the workstream segment Checked as part of closing the C2 coverage gaps, and it was live on main. The content scan's pattern was `R-[A-Z]+-[0-9]+`, which CANNOT match an ID whose workstream segment carries a digit: `[A-Z]+` consumes the `C` of `R-C2-01` and then demands a `-` where the `2` is. `R-C2-01` and `R-C2-02` are the only two such IDs in conformance/requirements/registry.yaml, and the scan was blind to BOTH. REPRODUCED before the fix, on a throwaway repo whose only change is a manifest declaring both IDs on real (non-comment) lines, in a path the map keys to nothing: $ REPO_ROOT=$T BASE_REF=main ci/touched-requirements.sh R-NET-06 # the all-letter control ID was the ONLY output It failed by UNDER-attributing, which is the fail-OPEN direction: a change that retyped a `conformance.home.k3s/R-C2-01` tag, or a manifest self-declaring R-C2-02, contributed those IDs to NOBODY, so the C2 tests were filtered out of the merge gate. Same defect class as the map's `) : ;;` arm — a miss that reads as "nothing touched". Every other consumer in the repo already allowed the digit, which is why this was the lone dissenter: test_requirements_for_path() twenty lines above in this same script, conformance/lib/coverage.py, conformance/lib/status.py and conformance/lib/ run_l1_offline.py all use `R-[A-Z0-9]+-[0-9]+`. Aligned to that form, with the reason recorded at the line. SCOPE OF THE HOLE. path_requirements() has no arm for platform/homepage/ or platform/headlamp/* either, so before this the ONLY route to R-C2-01/02 was detection rule 3 (editing the C2 conformance test itself). No path-map arm is added here: c2-root-surface declares `scope: whole-repo`, so ci/l0.sh Stage 3b's --include-scope union selects it on EVERY gate invocation regardless of the touched set — which is the right mechanism for an invariant whose input surface is the whole tree, and strictly stronger than a two-component path arm would be. REGRESSION-TESTED, in ci/tests/touched-requirements.test.sh (the suite `make -C ci test` runs): a fixture declaring R-C2-01 / R-C2-02 / R-OBS-04 from `notes/` — a path outside path_requirements(), so the IDs can only come from the content scan — asserts both digit IDs ARE attributed, that the all-letter form still is (control), and that a purely numeric workstream is not invented (the pattern must not widen the gate on noise). PROVEN TO DETECT BREAKAGE: with the pattern reverted to `R-[A-Z]+-[0-9]+` the suite reports `FAIL (2 failed, 14 passed)` naming both C2 IDs as missing; restored, all three ci/ suites pass (16 assertions in this one). ALSO: conformance/tests/l0-static/whole-repo-invariants-wired/check.sh cites this pattern in the comment explaining why its BOGUS_REQ fixture is lowercase. Citation updated so the reasoning still reads true; the fixture is unaffected (grep -E is case-sensitive, and the check still passes 26/26). NOTE: another branch may be editing this same script concurrently. This is a one-line change to one statement plus its comment — expect a trivial conflict and keep the `[A-Z0-9]` form. Co-Authored-By: Claude Opus 5 <[email protected]>	2026-09-13 16:28:21 -05:00
jakman85andClaude Opus 5	5fa278f61c	fix(ci): declare whole-repo scope instead of hand-writing a gate stage per invariant R-TEST-03 runs the conformance subset a change TOUCHES. That has one blind spot, and it is the worst-shaped kind: an invariant that quantifies over the whole tracked tree — or over subtrees ci/touched-requirements.sh's coarse path map keys to NOTHING (apps/, infra/, most of platform/) — cannot be selected BY the requirements a change touches, because the change most likely to break it attributes no requirement at all. Requirement-scoped selection then filters the check out of exactly that change. The repo's answer had been a hand-written "UNCONDITIONAL" stage per such check in ci/l0.sh: 3c flux-postbuild-substitution, 3d keycloak-client-secret-reconciler, 3e bulk-tier-distinct-from-replicated. Three near-identical 12-line rationales, three hardcoded check.sh paths, three copies of the same rc 0/3/* case block — and worst, a mechanism that failed SILENTLY, because whoever added the NEXT whole-repo invariant had to REMEMBER to write a stage, and if they forgot the check ran only when some unrelated requirement happened to be touched. A test now DECLARES the scope of its invariant and the runner enforces it: metadata: annotations: conformance.home.k3s/scope: "whole-repo" * conformance/lib/common.sh conf_dirs_for_scope(s) resolves the annotation. Carries the general rationale, once, instead of per stage. * conformance/run.sh --include-scope SCOPE unions declaring tests into select_dirs, still bounded by --layer. New --print-selection resolves the selection and exits, running nothing. * lib/run_l1_offline.py --unwired-scope: a SKIP of a declaring test exits 3, not 0. --summary-out: one step-summary line per executed check. * ci/l0.sh Stage 3b gains --include-scope whole-repo and an rc-3 arm. Stages 3c/3d/3e DELETED (-150 lines). * ci/l1.sh, ci/conformance.sh pass --include-scope too, so the next whole-repo invariant at ANY layer needs only the annotation. rc 3 SEMANTICS PRESERVED EXACTLY. A whole-repo check reporting "nothing to audit" is reporting that the tree it audits has VANISHED. The deleted stages called that "not a pass" and set the gate's rc=3; the runner now returns 3 and ci/l0.sh maps it to rc=3. A missing check file still degrades to not-green, never to silence. THE REASONING WAS RELOCATED, NOT DELETED. Each stage's incident narrative moved into the header of the test it describes, beside its scope annotation: the Cilium k8sServiceHost empty-substitution CNI outage (2026-09-01), 12 unusable Keycloak clients behind a green cluster (2026-09-06), and a multi-TB bulk volume landing on the 3-replica tier. AUDITED ALL 13 EXISTING l0-static TESTS, individually. 10 declare whole-repo scope; 3 are DELIBERATELY left requirement-scoped and say so in their own headers: * placeholder-guard-coupling — inputs are bootstrap/ + clusters/, both already mapped to a requirement it declares. Scoping cannot hide a breaking change. * dev-nodes-provisioned-as-code — inputs are infra/ and nothing else. * spec-immutability-enforced — inputs are ci/ and the read-only spec repo. The latter two were nonetheless under-gated because the PATH MAP was incomplete, which is the right place to fix a path-scoped invariant: infra/* -> R-ENV-01 added, and R-TEST-06 added to .gitea/\|ci/ (it is no longer an uncovered ID — spec-immutability-enforced carries the label, so attributing it no longer false-blocks the coverage gate). NEW conformance/tests/l0-static/whole-repo-invariants-wired (R-TEST-03) keeps the declarative mechanism honest for ALL declaring tests at once, replacing the three tests' individual "ci/l0.sh greps my own check.sh path" self-assertions. It proves the union EMPIRICALLY: with a requirement filter matching no test the L0 selection is empty, and with --include-scope it is every declaring test; a scope-declared synthetic SKIP exits 3 while an undeclared one exits 0; and a ci/.sh that hardcodes a check.sh path must belong to a test that also declares the scope, so the bespoke pattern cannot creep back. TASK 2 — a changed test attributes the requirements it CERTIFIES. ci/touched-requirements.sh gains detection rule 3: a change anywhere under conformance/tests/<test>/ attributes that test's own conformance.home.k3s/R- labels, read from its chainsaw-test.yaml. The content scan could not close this — it reads only added/removed lines, so it sees a change to the LABEL BLOCK but not to the test BODY, which is exactly the case that matters. Labels are read from HEAD or, failing that, from the BASE, so DELETING a test still attributes what it used to certify; otherwise removing the only verifier of a requirement is the cheapest way to make the gate stop checking it. Whole-line `#` comments are dropped first and *.md is untouched, honouring the existing "reference, not declaration" rules. Both changes are fail-closed: every one of them can only ADD tests or requirements to a run, never remove one. TESTS: new ci/tests/touched-requirements-test-attribution.test.sh (15 assertions) covers rule 3 — body-only edit, nested-fixture walk-up, test deletion, header-comment R-IDs not attributed, non-test paths unaffected, and both new path-map entries. The two existing suites still pass (39 assertions). `make -C ci test` runs all three. Co-Authored-By: Claude Opus 5 <[email protected]>	2026-09-12 23:45:06 -05:00

Author

SHA1

Message

Date

jakman85andClaude Opus 5

755c08ab40

fix(conformance): the meta-check was layer-agnostic and reported failures silently

Declaring whole-repo scope on the L1 tests made whole-repo-invariants-wired
(R-TEST-03) fail. It was right to change behaviour and wrong about why, and it
could not tell anyone either way. Two independent defects.

1. LAYER-AGNOSTIC WHERE THE DESIGN IS LAYER-BOUNDED

The "every scope-declared test is selected despite a requirement filter matching
none" assertion compared EVERY scope-declaring dir against an `--layer l0`
selection. That silently assumed only L0 tests would ever declare the scope —
while ci/l1.sh was written from the opposite premise, that a whole-repo invariant
could appear at any layer. So the first L1 declaration made the check blame the
union for correctly excluding another layer.

Now partitioned by layer, and the L1 half of the property is asserted too:
every scope-declared L1 test must be selected by `--layer l1 --include-scope`,
and that selection must not leak non-L1 tests. That is what makes the L1
declaration load-bearing rather than decorative — previously nothing proved the
union reached L1 at all, because nothing at L1 declared the scope.

2. EVERY ASSERTION FAILED SILENTLY

The idiom throughout was a bare test on one line and `chk $?` on the next, under
`set -euo pipefail`. A false test aborts the script BEFORE chk can print, so the
diagnostic — which is the entire value of this check — was never reached. The
real failure above emitted 26 `ok` lines, rc=1, and no message at all; it read
as a tooling glitch rather than a finding.

All 12 assertion sites now capture the status (`chk $arc`) so the check reports
and continues. Proven by forcing MIN_DECLARED=999 against the real check: the
failure prints its full explanation and the run proceeds to the remaining
assertions, where before it died mute.

A meta-check that cannot say why it failed is worse than no meta-check.

Verified: 35 assertions, rc=0; full offline l0,l1,meta 25 tests / 0 failures.

Co-Authored-By: Claude Opus 5 <[email protected]>

2026-09-18 22:43:17 -05:00

jakman85andClaude Opus 5

a76ae3bdfa

fix(ci): the content scan could not match R-C2-01/02 — a digit in the workstream segment

Checked as part of closing the C2 coverage gaps, and it was live on main. The content
scan's pattern was `R-[A-Z]+-[0-9]+`, which CANNOT match an ID whose workstream segment
carries a digit: `[A-Z]+` consumes the `C` of `R-C2-01` and then demands a `-` where the
`2` is. `R-C2-01` and `R-C2-02` are the only two such IDs in
conformance/requirements/registry.yaml, and the scan was blind to BOTH.

REPRODUCED before the fix, on a throwaway repo whose only change is a manifest declaring
both IDs on real (non-comment) lines, in a path the map keys to nothing:

    $ REPO_ROOT=$T BASE_REF=main ci/touched-requirements.sh
    R-NET-06                     # the all-letter control ID was the ONLY output

It failed by UNDER-attributing, which is the fail-OPEN direction: a change that retyped a
`conformance.home.k3s/R-C2-01` tag, or a manifest self-declaring R-C2-02, contributed
those IDs to NOBODY, so the C2 tests were filtered out of the merge gate. Same defect
class as the map's `*) : ;;` arm — a miss that reads as "nothing touched".

Every other consumer in the repo already allowed the digit, which is why this was the lone
dissenter: test_requirements_for_path() twenty lines above in this same script,
conformance/lib/coverage.py, conformance/lib/status.py and conformance/lib/
run_l1_offline.py all use `R-[A-Z0-9]+-[0-9]+`. Aligned to that form, with the reason
recorded at the line.

SCOPE OF THE HOLE. path_requirements() has no arm for platform/homepage/* or
platform/headlamp/* either, so before this the ONLY route to R-C2-01/02 was detection rule
3 (editing the C2 conformance test itself). No path-map arm is added here: c2-root-surface
declares `scope: whole-repo`, so ci/l0.sh Stage 3b's --include-scope union selects it on
EVERY gate invocation regardless of the touched set — which is the right mechanism for an
invariant whose input surface is the whole tree, and strictly stronger than a two-component
path arm would be.

REGRESSION-TESTED, in ci/tests/touched-requirements.test.sh (the suite `make -C ci test`
runs): a fixture declaring R-C2-01 / R-C2-02 / R-OBS-04 from `notes/` — a path outside
path_requirements(), so the IDs can only come from the content scan — asserts both digit
IDs ARE attributed, that the all-letter form still is (control), and that a purely numeric
workstream is not invented (the pattern must not widen the gate on noise).

PROVEN TO DETECT BREAKAGE: with the pattern reverted to `R-[A-Z]+-[0-9]+` the suite reports
`FAIL (2 failed, 14 passed)` naming both C2 IDs as missing; restored, all three ci/ suites
pass (16 assertions in this one).

ALSO: conformance/tests/l0-static/whole-repo-invariants-wired/check.sh cites this pattern
in the comment explaining why its BOGUS_REQ fixture is lowercase. Citation updated so the
reasoning still reads true; the fixture is unaffected (grep -E is case-sensitive, and the
check still passes 26/26).

NOTE: another branch may be editing this same script concurrently. This is a one-line
change to one statement plus its comment — expect a trivial conflict and keep the
`[A-Z0-9]` form.

Co-Authored-By: Claude Opus 5 <[email protected]>

2026-09-13 16:28:21 -05:00

jakman85andClaude Opus 5

5fa278f61c

fix(ci): declare whole-repo scope instead of hand-writing a gate stage per invariant

R-TEST-03 runs the conformance subset a change TOUCHES. That has one blind spot, and it
is the worst-shaped kind: an invariant that quantifies over the whole tracked tree — or
over subtrees ci/touched-requirements.sh's coarse path map keys to NOTHING (apps/, infra/,
most of platform/) — cannot be selected BY the requirements a change touches, because the
change most likely to break it attributes no requirement at all. Requirement-scoped
selection then filters the check out of exactly that change.

The repo's answer had been a hand-written "UNCONDITIONAL" stage per such check in
ci/l0.sh: 3c flux-postbuild-substitution, 3d keycloak-client-secret-reconciler, 3e
bulk-tier-distinct-from-replicated. Three near-identical 12-line rationales, three
hardcoded check.sh paths, three copies of the same rc 0/3/* case block — and worst, a
mechanism that failed SILENTLY, because whoever added the NEXT whole-repo invariant had
to REMEMBER to write a stage, and if they forgot the check ran only when some unrelated
requirement happened to be touched.

A test now DECLARES the scope of its invariant and the runner enforces it:

    metadata:
      annotations:
        conformance.home.k3s/scope: "whole-repo"

* conformance/lib/common.sh  conf_dirs_for_scope(s) resolves the annotation. Carries the
                             general rationale, once, instead of per stage.
* conformance/run.sh         --include-scope SCOPE unions declaring tests into select_dirs,
                             still bounded by --layer. New --print-selection resolves the
                             selection and exits, running nothing.
* lib/run_l1_offline.py      --unwired-scope: a SKIP of a declaring test exits 3, not 0.
                             --summary-out: one step-summary line per executed check.
* ci/l0.sh                   Stage 3b gains --include-scope whole-repo and an rc-3 arm.
                             Stages 3c/3d/3e DELETED (-150 lines).
* ci/l1.sh, ci/conformance.sh  pass --include-scope too, so the next whole-repo invariant
                             at ANY layer needs only the annotation.

rc 3 SEMANTICS PRESERVED EXACTLY. A whole-repo check reporting "nothing to audit" is
reporting that the tree it audits has VANISHED. The deleted stages called that "not a
pass" and set the gate's rc=3; the runner now returns 3 and ci/l0.sh maps it to rc=3.
A missing check file still degrades to not-green, never to silence.

THE REASONING WAS RELOCATED, NOT DELETED. Each stage's incident narrative moved into the
header of the test it describes, beside its scope annotation: the Cilium k8sServiceHost
empty-substitution CNI outage (2026-09-01), 12 unusable Keycloak clients behind a green
cluster (2026-09-06), and a multi-TB bulk volume landing on the 3-replica tier.

AUDITED ALL 13 EXISTING l0-static TESTS, individually. 10 declare whole-repo scope; 3 are
DELIBERATELY left requirement-scoped and say so in their own headers:
  * placeholder-guard-coupling — inputs are bootstrap/ + clusters/, both already mapped to
    a requirement it declares. Scoping cannot hide a breaking change.
  * dev-nodes-provisioned-as-code — inputs are infra/ and nothing else.
  * spec-immutability-enforced — inputs are ci/ and the read-only spec repo.
The latter two were nonetheless under-gated because the PATH MAP was incomplete, which is
the right place to fix a path-scoped invariant: infra/* -> R-ENV-01 added, and R-TEST-06
added to .gitea/*|ci/* (it is no longer an uncovered ID — spec-immutability-enforced
carries the label, so attributing it no longer false-blocks the coverage gate).

NEW conformance/tests/l0-static/whole-repo-invariants-wired (R-TEST-03) keeps the
declarative mechanism honest for ALL declaring tests at once, replacing the three tests'
individual "ci/l0.sh greps my own check.sh path" self-assertions. It proves the union
EMPIRICALLY: with a requirement filter matching no test the L0 selection is empty, and
with --include-scope it is every declaring test; a scope-declared synthetic SKIP exits 3
while an undeclared one exits 0; and a ci/*.sh that hardcodes a check.sh path must belong
to a test that also declares the scope, so the bespoke pattern cannot creep back.

TASK 2 — a changed test attributes the requirements it CERTIFIES.
ci/touched-requirements.sh gains detection rule 3: a change anywhere under
conformance/tests/<test>/ attributes that test's own conformance.home.k3s/R-* labels, read
from its chainsaw-test.yaml. The content scan could not close this — it reads only
added/removed lines, so it sees a change to the LABEL BLOCK but not to the test BODY,
which is exactly the case that matters. Labels are read from HEAD or, failing that, from
the BASE, so DELETING a test still attributes what it used to certify; otherwise removing
the only verifier of a requirement is the cheapest way to make the gate stop checking it.
Whole-line `#` comments are dropped first and *.md is untouched, honouring the existing
"reference, not declaration" rules.

Both changes are fail-closed: every one of them can only ADD tests or requirements to a
run, never remove one.

TESTS: new ci/tests/touched-requirements-test-attribution.test.sh (15 assertions) covers
rule 3 — body-only edit, nested-fixture walk-up, test deletion, header-comment R-IDs not
attributed, non-test paths unaffected, and both new path-map entries. The two existing
suites still pass (39 assertions). `make -C ci test` runs all three.

Co-Authored-By: Claude Opus 5 <[email protected]>

2026-09-12 23:45:06 -05:00