ci/touched-requirements.sh's coarse path map had no arm for apps/*. A change adding
or editing ANY workload on the cluster — 49 tracked manifests across apps/wave-0,
-1, -2 and apps/vms — attributed zero requirements, so the merge gate selected only
the whole-repo-scoped checks and none of the requirement-scoped L2 behaviour tests.
The tests that enforce default-deny, egress class and auth mode did not run on
exactly the changes that can violate them. Kyverno admission and the L0 egress-path
check are real backstops, but admission is post-merge and the gate exists to catch
it first.
platform/homepage and platform/headlamp were unmapped too — the C2 UI. That gap was
already written down in changed_diff()'s rule-3 comment ("the ONLY route to
R-C2-01/02 was rule 3", i.e. editing the C2 test itself) and had sat there since.
Twelve arms added: apps/vms/* (ordered BEFORE apps/*, because `case` takes the first
match), apps/*, platform/homepage|headlamp, platform/kubevirt, platform/gateway,
platform/coredns, platform/crafty, platform/external-snapshotter|backup, runbooks/*,
platform-config/*, platform/kustomization.yaml and .sops.yaml.
platform/kustomization.yaml earns its own arm: it is the assembly list, so one
deleted line disables an entire platform component while every file it points at
stays perfectly valid on disk. That is the most consequential single-line edit in
the repo and it attributed nothing.
Measured effect for an apps/ change, via --print-selection:
before: 0 requirement-scoped tests
after: 13 (3 L1, 8 L2, 2 L3)
Coverage stays 51/51, so the gate's "attributed IDs must be covered" assertion does
not false-block.
Ten new assertions in ci/tests/touched-requirements.test.sh (26 total, all passing),
including the ORDER property — apps/vms/* must win over apps/* — because arm order
is a real failure mode of `case`, not merely a question of both arms existing. Each
fixture cites no R-ID of its own, so any ID in the output came from the PATH and not
the content scan.
A correction on numbers I reported earlier in this session: I claimed "192 unmapped
files", then "182". Both were inflated by my own audit script rather than the repo.
Its regex used `[^)]*`, which matches across a newline, so the first arm parsed as
`case "$1" in\n catalog/*` and never matched anything; I also mis-parsed
alternation arms like `.gitea/*|ci/*`, which made ci/ look unmapped when it is
mapped. True remainder after this change is 18 files, all prose or captured
artifacts (docs/plans/, root READMEs, .standup-dns/*.txt).
Gated: offline l0,l1,meta --include-scope whole-repo, exit 0.
Co-Authored-By: Claude Opus 5 <[email protected]>
433 lines
27 KiB
Bash
Executable File
433 lines
27 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# ci/touched-requirements.sh — compute the requirement subset a change "touches".
|
|
#
|
|
# R-TEST-03 gates a merge on the conformance subset tagged with the requirement
|
|
# IDs the change touches. This script derives that subset from the diff so the
|
|
# conformance runner can be filtered to it (conformance.sh --requirements ...).
|
|
#
|
|
# Output (stdout): newline-separated, sorted, unique R-* IDs. Nothing else goes
|
|
# to stdout, so callers can `mapfile`/`$(...)` it directly.
|
|
#
|
|
# SOURCE OF TRUTH (reconciled, PM 2026-08-16): the conformance requirement TAGS are
|
|
# authoritative. Each Chainsaw test declares the requirements it verifies via
|
|
# `conformance.home.k3s/<R-ID>` labels; the coverage report (conformance/_artifacts/
|
|
# coverage/coverage.json) is generated from those tags. This script's job is only to
|
|
# select which tagged subset a change TOUCHES, then hand those IDs to
|
|
# `conformance.sh --requirements ...` which runs exactly the tests carrying those tags.
|
|
#
|
|
# Detection = union of:
|
|
# 1. R-* tokens on the ADDED/REMOVED LINES of the diff, in non-Markdown files —
|
|
# including the authoritative `conformance.home.k3s/R-*` tags on changed tests, and
|
|
# the R-IDs that policies/ServiceDefinitions self-declare (docs/14 + ADR-0011).
|
|
# Untouched lines and prose are NOT scanned; see changed_diff() for why.
|
|
# 2. A coarse path->requirement fallback for changes that don't self-declare IDs.
|
|
# 3. For a change anywhere under `conformance/tests/<test>/`: THAT TEST'S OWN requirement
|
|
# labels, read from its chainsaw-test.yaml. See test_requirements_for_path().
|
|
# The coarse map (2) NEVER overrides an explicit tag; it only widens coverage for
|
|
# untagged changes so nothing silently escapes the gate. (3) is the same principle applied
|
|
# to the tests themselves: editing or weakening a test must be gated by the requirements
|
|
# that test CERTIFIES, not merely by R-TEST-0x.
|
|
set -euo pipefail
|
|
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
# shellcheck disable=SC1091
|
|
source "${CI_DIR}/lib.sh"
|
|
|
|
# ---------------------------------------------------------------------------------
|
|
# BASE-REF RESOLUTION — and why it now REFUSES rather than guesses.
|
|
#
|
|
# BUG THIS FIXES (local/CI divergence, 2026-08-31): every failure to establish a
|
|
# reliable diff base used to degrade SILENTLY to an EMPTY touched set, which main()
|
|
# reported as "no changed files vs base" and exit 0. An empty set means the coverage
|
|
# `--require` gate is never applied, so the merge gate printed "MERGE GATE: PASS"
|
|
# having checked NOTHING — while CI, with a correct base, computed 17 touched
|
|
# requirements and blocked. A false green is the worst outcome this project can
|
|
# produce, so every one of those degradations is now a LOUD exit 3:
|
|
#
|
|
# * HEAD~1 last resort REMOVED. It silently narrowed the diff to the last commit
|
|
# (and on a one-commit repo failed into an empty set).
|
|
# * two-dot fallback REMOVED. It masked a failing three-dot diff as "no changes".
|
|
# * unresolvable base exit 3 (was: HEAD~1 or empty).
|
|
# * diff command fails exit 3 (was: empty set).
|
|
# * HEAD not ahead of exit 3 (was: exit 0 with an empty set — this is the case
|
|
# the base that produced the false green: running the gate while
|
|
# sitting on `main`, or on a branch already merged).
|
|
#
|
|
# Exit 3 is the ci/README.md "SKIPPED / not-run" code: NOT a pass, NOT a fail. Callers
|
|
# (ci/gate.sh) must surface it as an UNDECIDED merge decision. An honest, noisy skip
|
|
# beats a quiet false pass.
|
|
#
|
|
# Order of preference: BASE_REF (explicit) > GITHUB_BASE_REF (Actions PR runs, target
|
|
# branch) > origin/main > main. Each candidate is VERIFIED to resolve to a commit
|
|
# before it is used, so a shallow checkout that lacks origin/main is reported, not
|
|
# papered over.
|
|
: "${BASE_REF:=}"
|
|
BASE="" # resolved base ref
|
|
BASE_SOURCE="" # where it came from (for the log line)
|
|
BASE_PROBLEM="" # why resolution failed, when it did
|
|
|
|
rev() { git -C "$REPO_ROOT" rev-parse --verify -q "$1^{commit}" 2>/dev/null; }
|
|
|
|
# Loud, structured refusal. Prints to stderr only; stdout stays empty so a caller that
|
|
# captures stdout gets NO requirement list rather than a silently-empty one.
|
|
#
|
|
# $1 is a machine-readable REASON CODE, also written to $TOUCHED_REASON_FILE when that
|
|
# env var names a path. ci/gate.sh uses it to tell the two cases apart:
|
|
# on-base HEAD is the base branch (or already merged) — nothing to gate. Legitimate
|
|
# (a push-to-main CI run lands here), so the gate reports NOT APPLICABLE
|
|
# rather than PASS, and does not fail the job.
|
|
# no-base no reliable base ref exists in this checkout — the ENVIRONMENT is broken,
|
|
# diff-failed or git could not produce the diff. A merge decision is impossible, so the
|
|
# gate BLOCKS rather than guessing.
|
|
undecidable() {
|
|
local code="$1"; shift
|
|
if [ -n "${TOUCHED_REASON_FILE:-}" ]; then printf '%s\n' "$code" > "$TOUCHED_REASON_FILE"; fi
|
|
err "TOUCHED-REQUIREMENT SET NOT DETERMINABLE (${code})"
|
|
err " reason: $*"
|
|
err " consequence: the R-TEST-03 coverage gate CANNOT be applied to this checkout."
|
|
err " This is exit 3 (SKIPPED / not-run), never a pass — see ci/README.md."
|
|
err " Fix one of:"
|
|
err " * check out the feature branch you mean to gate (not the base branch), or"
|
|
err " * git fetch origin main (so origin/main exists and is current), or"
|
|
err " * BASE_REF=<ref> make -C ci gate to name the base explicitly."
|
|
exit 3
|
|
}
|
|
|
|
resolve_base() {
|
|
local cand
|
|
if [ -n "$BASE_REF" ]; then
|
|
if rev "$BASE_REF" >/dev/null; then BASE="$BASE_REF"; BASE_SOURCE="BASE_REF"; return 0; fi
|
|
BASE_PROBLEM="BASE_REF='${BASE_REF}' does not resolve to a commit in ${REPO_ROOT}"
|
|
return 1
|
|
fi
|
|
if [ -n "${GITHUB_BASE_REF:-}" ]; then
|
|
for cand in "origin/${GITHUB_BASE_REF}" "${GITHUB_BASE_REF}"; do
|
|
if rev "$cand" >/dev/null; then BASE="$cand"; BASE_SOURCE="GITHUB_BASE_REF"; return 0; fi
|
|
done
|
|
BASE_PROBLEM="GITHUB_BASE_REF='${GITHUB_BASE_REF}' is set, but neither 'origin/${GITHUB_BASE_REF}' nor '${GITHUB_BASE_REF}' exists here (shallow checkout? the workflow needs fetch-depth: 0)"
|
|
return 1
|
|
fi
|
|
for cand in origin/main main; do
|
|
if rev "$cand" >/dev/null; then BASE="$cand"; BASE_SOURCE="default"; return 0; fi
|
|
done
|
|
BASE_PROBLEM="no base ref resolves (tried origin/main, main) in ${REPO_ROOT}"
|
|
return 1
|
|
}
|
|
|
|
# Both diff helpers assume BASE is already resolved and validated by main(). They return
|
|
# git's exit status so the CALLER can distinguish "no changes" from "the diff failed" —
|
|
# conflating those two is exactly what produced the false green.
|
|
changed_files() {
|
|
# Three-dot: changes on this branch since it diverged from base.
|
|
git -C "$REPO_ROOT" diff --name-only "${BASE}...HEAD"
|
|
}
|
|
|
|
# The ADDED/REMOVED lines of the change, for the R-ID content scan — excluding
|
|
# documentation.
|
|
#
|
|
# THREE scoping rules, all learned from the gate blocking merges it should not have:
|
|
#
|
|
# 1. Scan the DIFF, not whole file contents. Grepping whole files meant a change
|
|
# inherited every requirement those files had EVER mentioned, on lines the author
|
|
# never touched. Since some of those are legitimately uncovered, `--require-all`
|
|
# then failed and BLOCKED the merge over unrelated prose.
|
|
#
|
|
# 2. Skip `*.md`. A requirement ID in prose is a REFERENCE, not a declaration —
|
|
# explaining a requirement is not the same as changing behaviour against it.
|
|
# Without this, the decision log cannot record which requirements a change
|
|
# concerned without thereby claiming to have touched them, and the CI-fix commit
|
|
# that explains a mis-attribution re-triggers the very mis-attribution it fixes.
|
|
# Nothing escapes: authoritative declarations (`conformance.home.k3s/R-*` tags,
|
|
# Kyverno policies, ServiceDefinitions) all live in YAML, and the doc subtrees
|
|
# that matter (conformance/, schemas/, catalog/, clusters/, ci/) are covered by
|
|
# the path->requirement fallback regardless of file type.
|
|
#
|
|
# 3. Skip COMMENT-ONLY lines in non-Markdown files — rule 2's logic, applied to the
|
|
# prose that lives INSIDE code files. A requirement ID cited in a YAML/shell `#`
|
|
# comment is documentation, not a declaration, exactly like an ID in a `*.md` file.
|
|
# Rule 2 drops whole Markdown files, but a `.gitea/workflows/*.yml` or `ci/*.sh`
|
|
# HEADER that explains a requirement would otherwise re-attribute it. Concretely:
|
|
# gate.yml's header cites R-TEST-06 (enforced by the spec-immutability STAGE, not by
|
|
# any Chainsaw test, so a known-uncovered ID); before this rule, editing that comment
|
|
# re-attributed R-TEST-06 and the coverage `--require` check false-blocked the merge —
|
|
# the same self-referential trap rule 2 fixed for `*.md`. Applied in main()'s content
|
|
# scan (see there): after the diff marker is stripped, a line whose first non-blank
|
|
# character is `#` is dropped BEFORE the R-ID grep. Deliberately conservative — only
|
|
# WHOLE-LINE comments are dropped; an R-ID on a real code/spec line is still detected
|
|
# (authoritative declarations — test tags, policies, ServiceDefinitions — are never a
|
|
# bare comment), so this only narrows PROSE, never a declaration.
|
|
#
|
|
# Removed lines count: deleting a test's requirement tag, or a policy's self-declared
|
|
# ID, touches that requirement just as much as adding one.
|
|
# `:(exclude)*.md` drops Markdown from the content scan (rule 2 above).
|
|
# `:(exclude)ci/tests/` drops this dir's OWN unit-test fixtures: those files embed example
|
|
# R-IDs as DATA (asserting the scan's behaviour), never as declarations — the same
|
|
# "reference, not declaration" logic as rule 2. Their gate coverage is unaffected: any
|
|
# ci/ change (ci/tests/ included) still attributes R-GITOPS-01/R-TEST-02/R-TEST-03 via the
|
|
# path->requirement map below. NOTE: this excludes ci/tests/ only — conformance/tests/
|
|
# carry AUTHORITATIVE `conformance.home.k3s/R-*` labels and are deliberately still scanned.
|
|
# Three-dot to match changed_files(): changes on this branch since it diverged from base.
|
|
changed_diff() {
|
|
local -a excl=(':(exclude)*.md' ':(exclude)ci/tests/')
|
|
git -C "$REPO_ROOT" diff -U0 "${BASE}...HEAD" -- . "${excl[@]}"
|
|
}
|
|
|
|
# Coarse path -> requirement map. Deliberately conservative; extend as subtrees
|
|
# fill in. Left side is a path prefix (relative to repo root).
|
|
path_requirements() {
|
|
case "$1" in
|
|
catalog/*) echo "R-GITOPS-02" ;;
|
|
schemas/*) echo "R-GITOPS-02" ;;
|
|
# apps/* was UNMAPPED until 2026-09-19, and it is the largest such hole in the map: 49 tracked
|
|
# manifests (apps/wave-0, -1, -2, apps/vms) that describe every workload on the cluster. A change
|
|
# adding or editing an app attributed NOTHING, so the merge gate ran only the whole-repo-scoped
|
|
# checks and none of the requirement-scoped L2 behaviour tests — default-deny enforcement, egress
|
|
# class enforcement, auth-mode enforcement — which are exactly what a new or edited app can
|
|
# violate. Kyverno admission and the L0 egress-path check are real backstops, but "admission will
|
|
# catch it at apply time" is post-merge, and the gate exists to catch it before.
|
|
#
|
|
# ORDER MATTERS: `case` takes the FIRST match, so apps/vms/* must precede apps/*.
|
|
# apps/vms/ are KubeVirt VMs (R-PLAT-03 lifecycle) with root/data volumes (R-STATE-01), still
|
|
# subject to default-deny (R-NET-01).
|
|
apps/vms/*) echo "R-PLAT-03 R-NET-01 R-STATE-01" ;;
|
|
# Everything else under apps/ is a ServiceDefinition rendered by catalog/render.py: a container
|
|
# workload (R-PLAT-02) rendered from the catalog (R-GITOPS-02), carrying the egress-class
|
|
# (R-NET-05/06/07) and auth-mode (R-IDP-02/R-IDP-06) labels that admission requires, under
|
|
# default-deny (R-NET-01).
|
|
apps/*) echo "R-PLAT-02 R-GITOPS-02 R-NET-01 R-NET-05 R-NET-06 R-NET-07 R-IDP-02 R-IDP-06" ;;
|
|
platform/kyverno/*) echo "R-TEST-01 R-NET-01 R-SEC-01" ;;
|
|
platform/cilium/*) echo "R-NET-01 R-NET-02 R-NET-05" ;;
|
|
platform/longhorn/*) echo "R-STATE-01 R-STATE-02" ;;
|
|
platform/cert-manager/*) echo "R-TLS-01 R-TLS-02" ;;
|
|
platform/externaldns/*) echo "R-DNS-02" ;;
|
|
platform/velero/*) echo "R-STATE-03" ;;
|
|
platform/monitoring/*) echo "R-OBS-01 R-OBS-02" ;;
|
|
# platform/cnpg was UNMAPPED until 2026-09-06. It is the PostgreSQL DR path for keycloak-db /
|
|
# nextcloud-db / semaphore-db (barmanObjectStore WAL archiving + ScheduledBackup), so a change
|
|
# there is an R-IDP-05 / R-STATE-03 change; it now also carries the platform-health alert rules
|
|
# for that path (R-OBS-04). Without this line an edit under platform/cnpg/ contributed ZERO
|
|
# requirements unless it happened to retype an R-ID on a non-comment line, and escaped the gate.
|
|
platform/cnpg/*) echo "R-IDP-05 R-STATE-03 R-OBS-04" ;;
|
|
# vpn-egress was UNMAPPED until 2026-09-06. platform/vpn-egress/egress-steering.yaml IS the
|
|
# R-NET-04 datapath (the rule that selects between the two egress paths) and the R-NET-05/06/07
|
|
# steering policy — it even self-declares those IDs in its `homelab.keithjackson.dev/requirements`
|
|
# annotation. But the content scan only reads ADDED/REMOVED lines, so any edit that does not
|
|
# happen to retype an R-ID (a selector change, an excludedCIDRs change, a comment) contributed
|
|
# ZERO requirements and escaped the gate entirely. Mapped explicitly; the annotation still wins
|
|
# when a diff does carry the tags.
|
|
platform/vpn-egress/*) echo "R-NET-04 R-NET-05 R-NET-06 R-NET-07" ;;
|
|
# Keycloak is the whole SSO surface: the realm import (declarative identity config),
|
|
# the SOPS client secrets, and the reconciler that heals them. Added 2026-09-06 after
|
|
# the client-secret incident, in which editing platform/keycloak/ attributed NO
|
|
# requirement, so the L2 keycloak tests were filtered out of the merge gate's touched
|
|
# subset — the change that could break every OIDC login was gated by nothing.
|
|
# R-IDP-05 is in scope because platform/keycloak/ also carries the CNPG DB Cluster and
|
|
# its off-cluster backup credential, not only the realm.
|
|
platform/keycloak/*) echo "R-IDP-01 R-IDP-02 R-IDP-03 R-IDP-04 R-IDP-05 R-IDP-06" ;;
|
|
platform/oauth2-proxy/*) echo "R-IDP-02 R-IDP-06" ;;
|
|
platform/image-automation/*) echo "R-UPD-01 R-UPD-02" ;;
|
|
# The C2 UI was UNMAPPED until 2026-09-19. This is the gap named explicitly in changed_diff()'s
|
|
# rule-3 comment above: "path_requirements() has no arm for platform/homepage/* or
|
|
# platform/headlamp/* either, so the ONLY route to R-C2-01/02 was rule 3 (editing the C2 test
|
|
# itself)". homepage + headlamp ARE the command-centre surface (registry: R-C2-01/02 ->
|
|
# docs/12-command-center-ui.md), so editing them must select the C2 tests.
|
|
platform/homepage/*|platform/headlamp/*) echo "R-C2-01 R-C2-02" ;;
|
|
# KubeVirt is the VM platform itself (R-PLAT-03); its virt-* components are also scrape targets.
|
|
platform/kubevirt/*) echo "R-PLAT-03 R-OBS-01" ;;
|
|
# The shared ingress Gateway: the L2-announced VIP every HTTP service answers on (R-DNS-01), the
|
|
# TLS listeners and their certificates (R-TLS-01/02), under default-deny (R-NET-01).
|
|
platform/gateway/*) echo "R-DNS-01 R-TLS-01 R-TLS-02 R-NET-01" ;;
|
|
# CoreDNS is cluster DNS resolution (R-DNS-01).
|
|
platform/coredns/*) echo "R-DNS-01" ;;
|
|
# crafty is a first-class app living under platform/: a container workload with a Longhorn PVC and
|
|
# its OWN L4 LoadBalancer VIP (Minecraft 25565 is not HTTP, so it does not use the Gateway).
|
|
platform/crafty/*) echo "R-PLAT-02 R-NET-01 R-DNS-01 R-STATE-01" ;;
|
|
# Snapshot machinery is the storage/DR path: CSI snapshots (R-STATE-01) feeding backup (R-STATE-03).
|
|
platform/external-snapshotter/*|platform/backup/*) echo "R-STATE-01 R-STATE-03" ;;
|
|
# runbooks/ IS the R-OBS-04 surface: alert-runbooks-resolvable asserts every alert's runbook_url
|
|
# resolves to a served page, and runbooks/serving/ is the Deployment that serves them. That check
|
|
# is whole-repo-scoped so it runs regardless, but the attribution should still be honest — a
|
|
# runbook change is an observability change.
|
|
runbooks/*) echo "R-OBS-04" ;;
|
|
# platform-config/ is the env-specific configuration Flux substitutes into and applies.
|
|
platform-config/*|platform-config-cdi/*) echo "R-ENV-02 R-GITOPS-01" ;;
|
|
# platform/kustomization.yaml is the ASSEMBLY: the list of platform components Flux applies.
|
|
# Deleting one line there disables an entire component (cilium, longhorn, velero...) while every
|
|
# file it points at stays perfectly valid on disk, so nothing else in the map would attribute the
|
|
# change. That is the most consequential single-line edit in the repo and it attributed nothing.
|
|
platform/kustomization.yaml) echo "R-GITOPS-01" ;;
|
|
# .sops.yaml holds the encryption rules (which paths must be encrypted, with which keys). A change
|
|
# here can silently stop a new secret path from being encrypted, which is R-SEC-01's whole point.
|
|
.sops.yaml) echo "R-SEC-01 R-GITOPS-01" ;;
|
|
conformance/*) echo "R-TEST-01 R-TEST-02 R-TEST-03 R-TEST-04" ;;
|
|
# infra/ was UNMAPPED until 2026-09-12 — the OpenTofu + Ansible that provision the dev
|
|
# nodes. conformance/tests/l0-static/dev-nodes-provisioned-as-code reads NOTHING but
|
|
# infra/, yet an infra/ edit attributed ZERO requirements, so the one check that asserts
|
|
# the nodes are provisioned as code was filtered out of every change that could break it.
|
|
# That test's invariant IS path-scoped (it is deliberately NOT whole-repo-scoped; see its
|
|
# chainsaw-test.yaml), and the correct fix for a path-scoped invariant is a path-map entry.
|
|
infra/*) echo "R-ENV-01" ;;
|
|
clusters/*) echo "R-GITOPS-01 R-ENV-02" ;;
|
|
# bootstrap/ was UNMAPPED until 2026-09-06. The entry STAYS, but its original rationale
|
|
# was written against a file that is not here, and that is corrected on 2026-09-13:
|
|
# it claimed "bootstrap/standup.sh carries the placeholder guards that decide whether
|
|
# `velero` may be enabled against a real backup endpoint". bootstrap/standup.sh HAS NEVER
|
|
# BEEN ON `main` — it was not deleted (no deletion commit exists anywhere), it landed only
|
|
# on unmerged prod-standup branches. `git ls-tree main bootstrap/` lists README.md, flux/,
|
|
# gitea/, sops/. So R-STATE-03 here is ANTICIPATORY, not descriptive.
|
|
#
|
|
# WHY IT IS STILL RIGHT. conformance/tests/l0-static/placeholder-guard-coupling DISCOVERS
|
|
# its guard (any tracked *.sh defining `<name>_is_placeholder()`) instead of hardcoding a
|
|
# path, so the moment a standup guard lands under bootstrap/ that test's coupling half
|
|
# activates — and this entry is what selects the test for the change that lands it.
|
|
# Deleting the entry would mean the change introducing a placeholder guard is gated by
|
|
# nothing, which is the 2026-09-05 class (a prefix glob that stopped matching when the
|
|
# value it guards changed, in a different file) left to reintroduce itself. R-GITOPS-01 is
|
|
# descriptive today: bootstrap/flux/, gitea/ and sops/ ARE the GitOps bootstrap path.
|
|
# Locked down by ci/tests/touched-requirements.test.sh's path-map section.
|
|
bootstrap/*) echo "R-STATE-03 R-GITOPS-01" ;;
|
|
# R-TEST-06 added 2026-09-12. The comment in changed_diff()'s rule 3 above records why it
|
|
# was ABSENT: R-TEST-06 used to be enforced by the spec-immutability STAGE and carried by no
|
|
# Chainsaw test, so attributing it made the coverage `--require` check false-block. It is now
|
|
# carried by conformance/tests/l0-static/spec-immutability-enforced — whose entire input
|
|
# surface is ci/ — so a ci/ change must attribute it, or the test that proves the
|
|
# spec-immutability mechanism is still load-bearing runs on nothing that can break it.
|
|
.gitea/*|ci/*) echo "R-GITOPS-01 R-TEST-02 R-TEST-03 R-TEST-06" ;;
|
|
*) : ;;
|
|
esac
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------------
|
|
# A CHANGED TEST ATTRIBUTES THE REQUIREMENTS IT CERTIFIES (detection rule 3).
|
|
#
|
|
# THE GAP THIS CLOSES (2026-09-12). The path map keys `conformance/*` to
|
|
# R-TEST-01..04 — the requirements about the HARNESS. It says nothing about what a given
|
|
# test PROVES. So editing `conformance/tests/l2-behavioral/tenant-isolation/check.sh` to
|
|
# weaken or delete an assertion attributed R-TEST-01..04 and nothing else: the merge gate
|
|
# then ran the harness's self-tests and NOT the weakened test's own requirement subset.
|
|
# Weakening a test is the one change that most needs the requirement it certifies to be
|
|
# re-verified, and it was the change least likely to select it.
|
|
#
|
|
# The `conformance.home.k3s/R-*: "true"` labels on a test ARE the authoritative declaration of
|
|
# what it verifies (lib/coverage.py reads the same labels); this reads them directly. Note the
|
|
# content scan in main() only sees ADDED/REMOVED lines, so it catches a change to the LABEL
|
|
# BLOCK but not a change to the test BODY — which is exactly the case that matters here.
|
|
#
|
|
# WHY IT READS HEAD *OR* THE BASE: a change that DELETES a test, or deletes its
|
|
# chainsaw-test.yaml, must still attribute what that test used to certify — otherwise removing
|
|
# the only verifier of a requirement is the cheapest way to make the gate stop checking it.
|
|
# HEAD first (the post-change truth), base as the fallback (the deleted case). Fail-closed:
|
|
# this function can only ADD requirements, never remove one.
|
|
: "${CONF_TEST_LABEL_PREFIX:=conformance.home.k3s}" # == CONF_LABEL_PREFIX, conformance/lib/common.sh
|
|
|
|
file_text_head_or_base() {
|
|
local path="$1"
|
|
git -C "$REPO_ROOT" show "HEAD:${path}" 2>/dev/null && return 0
|
|
git -C "$REPO_ROOT" show "${BASE}:${path}" 2>/dev/null && return 0
|
|
return 0
|
|
}
|
|
|
|
# test_requirements_for_path <changed-path> -> space-separated R-IDs (possibly empty).
|
|
# Walks UP from the changed file to the nearest ancestor directory carrying a
|
|
# chainsaw-test.yaml, so it works for any file in a test dir (check.sh, mutations.py,
|
|
# fixtures, the chainsaw-test.yaml itself) and for nested test layouts.
|
|
test_requirements_for_path() {
|
|
local f="$1" dir yaml text
|
|
case "$f" in conformance/tests/*/*) ;; *) return 0 ;; esac
|
|
dir="${f%/*}"
|
|
while :; do
|
|
case "$dir" in
|
|
conformance/tests|conformance|.|"") return 0 ;;
|
|
*/*) ;;
|
|
*) return 0 ;;
|
|
esac
|
|
yaml="${dir}/chainsaw-test.yaml"
|
|
text="$(file_text_head_or_base "$yaml")"
|
|
if [ -n "$text" ]; then
|
|
# Whole-line `#` comments are dropped first — same "reference, not declaration" rule the
|
|
# content scan applies (changed_diff() rule 3): a label shown inside a header example is
|
|
# documentation. Only a real YAML mapping line counts.
|
|
printf '%s\n' "$text" \
|
|
| grep -vE '^[[:space:]]*#' \
|
|
| sed -nE "s#^[[:space:]]*${CONF_TEST_LABEL_PREFIX}/(R-[A-Z0-9]+-[0-9]+):[[:space:]]*[\"']?true.*#\1#p" \
|
|
| tr '\n' ' '
|
|
return 0
|
|
fi
|
|
dir="${dir%/*}"
|
|
done
|
|
}
|
|
|
|
main() {
|
|
local files reqs="" rc=0 base_sha head_sha local_main merge_base
|
|
|
|
resolve_base || undecidable no-base "$BASE_PROBLEM"
|
|
base_sha="$(rev "$BASE")" || undecidable no-base "resolved base '${BASE}' vanished"
|
|
head_sha="$(rev HEAD)" || undecidable no-base "HEAD does not resolve to a commit (empty repo?)"
|
|
log "diff base: ${BASE} (${BASE_SOURCE}) @ ${base_sha:0:12} ... HEAD @ ${head_sha:0:12}"
|
|
|
|
# Advisory: a never-fetched origin/main silently WIDENS the set (over-broad, never a
|
|
# false green) — surfaced so an unexpectedly large touched set has an explanation.
|
|
if [ "$BASE" = "origin/main" ] && local_main="$(rev main)" && [ -n "$local_main" ] \
|
|
&& [ "$local_main" != "$base_sha" ]; then
|
|
warn "local 'main' (${local_main:0:12}) != origin/main (${base_sha:0:12}) — if origin/main"
|
|
warn " was not fetched recently the touched set may be stale: git fetch origin main"
|
|
fi
|
|
|
|
merge_base="$(git -C "$REPO_ROOT" merge-base "$BASE" HEAD 2>/dev/null)" \
|
|
|| undecidable no-base "no merge base between '${BASE}' and HEAD (unrelated histories, or a shallow clone)"
|
|
if [ "$merge_base" = "$head_sha" ]; then
|
|
undecidable on-base "HEAD is not ahead of '${BASE}' (merge-base == HEAD): this checkout contains no branch commits to gate. The empty touched set would be empty BY DEFAULT, not by verification."
|
|
fi
|
|
|
|
files="$(changed_files)" || rc=$?
|
|
[ "$rc" -eq 0 ] || undecidable diff-failed "git diff --name-only ${BASE}...HEAD failed (rc=${rc})"
|
|
if [ -z "$files" ]; then
|
|
warn "HEAD is ahead of ${BASE} but the three-dot diff is empty (self-cancelling commits?)"
|
|
warn " -> genuinely empty touched-requirement set; base WAS established."
|
|
exit 0
|
|
fi
|
|
|
|
while IFS= read -r f; do
|
|
[ -n "$f" ] || continue
|
|
reqs+=" $(path_requirements "$f")"
|
|
# Detection rule 3: a change under conformance/tests/<test>/ attributes THAT TEST'S own
|
|
# requirement labels, so weakening a test is gated by what the test certifies.
|
|
reqs+=" $(test_requirements_for_path "$f")"
|
|
done <<< "$files"
|
|
|
|
# Self-declared R-IDs on the ADDED/REMOVED lines of the change (see changed_diff).
|
|
#
|
|
# THE ID PATTERN IS `R-[A-Z0-9]+-[0-9]+`, NOT `R-[A-Z]+-[0-9]+`. The WORKSTREAM segment may
|
|
# contain a DIGIT: `R-C2-01` and `R-C2-02` (command & control UI) are the only two such IDs
|
|
# in conformance/requirements/registry.yaml today, and the stricter pattern silently failed
|
|
# to match BOTH of them — `[A-Z]+` consumes the `C` and then demands a `-` where the `2` is.
|
|
# Every other consumer in the repo already allows the digit: test_requirements_for_path()
|
|
# twenty lines above uses `R-[A-Z0-9]+-[0-9]+`, as do conformance/lib/coverage.py,
|
|
# conformance/lib/status.py and conformance/lib/run_l1_offline.py. This grep was the lone
|
|
# dissenter, and it dissented by UNDER-attributing — the fail-OPEN direction. A change that
|
|
# retyped a `conformance.home.k3s/R-C2-01` tag, or a manifest self-declaring R-C2-02 from a
|
|
# path the map keys elsewhere, contributed those IDs to NOBODY, so the C2 tests were
|
|
# filtered out of the merge gate. Same defect class as the map's `*) : ;;` arm: a miss that
|
|
# reads as "nothing touched". path_requirements() has no arm for platform/homepage/* or
|
|
# platform/headlamp/* either, so before this fix the ONLY route to R-C2-01/02 was rule 3
|
|
# (editing the C2 test itself). Regression-tested in ci/tests/touched-requirements.test.sh.
|
|
# `^[+-]` keeps hunk headers (`@@`) and the `+++`/`---` file headers out of scope.
|
|
# `sed 's/^[+-]//'` then strips the single-char diff marker (safe: -U0 emits one), so
|
|
# what remains is the SOURCE line, and `grep -vE '^[[:space:]]*#'` drops whole-line
|
|
# YAML/shell comments (rule 3 in changed_diff's header) BEFORE the R-ID grep — a
|
|
# requirement cited only in a `#` comment is prose, not a declaration.
|
|
local diff_txt
|
|
diff_txt="$(changed_diff)" || rc=$?
|
|
[ "$rc" -eq 0 ] || undecidable diff-failed "git diff -U0 ${BASE}...HEAD failed (rc=${rc})"
|
|
reqs+=" $(printf '%s\n' "$diff_txt" \
|
|
| grep -E '^[+-]' \
|
|
| grep -vE '^(\+\+\+|---)' \
|
|
| sed -E 's/^[+-]//' \
|
|
| grep -vE '^[[:space:]]*#' \
|
|
| grep -hoE 'R-[A-Z0-9]+-[0-9]+' \
|
|
| tr '\n' ' ' || true)"
|
|
|
|
# Normalize: split, unique, sort. Emit to stdout.
|
|
printf '%s\n' $reqs | sed '/^$/d' | sort -u
|
|
}
|
|
main "$@"
|