`ci/touched-requirements.sh` documents its detection as "R-* tokens literally
present in the diff of changed files". It did not do that — it grepped the ENTIRE
CONTENT of every changed file. So a change inherited every requirement those files
had ever mentioned in prose, and `--require-all` then blocked the merge on
requirements the author never went near.
On this branch that meant editing the decision log and the tofu docs dragged in
five unrelated requirements, all on untouched lines, all legitimately uncovered:
[coverage] REQUIRE: FAILED — required requirement(s) uncovered: [5 IDs]
The practical effect was that the PM could not append to the decision log without
failing CI — which is a fair part of why the log went stale in the first place.
This is the defect already logged as "FOLLOW-UP (WS-TEST): fix the matcher so real
requirements need not be exempted".
Two scoping rules, both learned from the gate blocking merges it should not have:
1. Scan the ADDED/REMOVED lines of the diff, as the header always claimed.
Removed lines count too — deleting a test's requirement tag, or a policy's
self-declared ID, touches that requirement just as much as adding one.
2. Skip `*.md`. An ID in prose is a REFERENCE, not a declaration: explaining a
requirement is not changing behaviour against it. Rule 1 alone is not enough —
a commit that documents a mis-attribution names the IDs involved and thereby
re-triggers the very mis-attribution it fixes, and the decision log can never
record which requirements a change concerned.
This TIGHTENS attribution rather than weakening the gate:
- no requirement is exempted, and DOCUMENTED_GAPS is untouched;
- authoritative declarations (`conformance.home.k3s/R-*` tags, Kyverno policies,
ServiceDefinitions) all live in YAML, on lines a real change must touch;
- the coarse path->requirement fallback is unchanged and file-type agnostic, so
the doc subtrees that matter (conformance/, schemas/, catalog/, clusters/, ci/)
stay covered either way.
Alternatives rejected: adding the five IDs to DOCUMENTED_GAPS genuinely weakens the
gate; scrubbing requirement IDs out of prose (precedent f9e9640) would mean editing
the decision log's own history to appease a matcher.
Gate semantics change — flagged in DECISIONS-LOG.md for review.
Also drops three R-TEST-06 mentions this branch had added to prose and refreshes
the now-stale "gate is RED" notes.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
122 lines
5.9 KiB
Bash
Executable File
122 lines
5.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# ci/touched-requirements.sh — compute the requirement subset a change "touches".
|
|
#
|
|
# R-TEST-03 gates a merge on the conformance subset tagged with the requirement
|
|
# IDs the change touches. This script derives that subset from the diff so the
|
|
# conformance runner can be filtered to it (conformance.sh --requirements ...).
|
|
#
|
|
# Output (stdout): newline-separated, sorted, unique R-* IDs. Nothing else goes
|
|
# to stdout, so callers can `mapfile`/`$(...)` it directly.
|
|
#
|
|
# SOURCE OF TRUTH (reconciled, PM 2026-08-16): the conformance requirement TAGS are
|
|
# authoritative. Each Chainsaw test declares the requirements it verifies via
|
|
# `conformance.home.k3s/<R-ID>` labels; the coverage report (conformance/_artifacts/
|
|
# coverage/coverage.json) is generated from those tags. This script's job is only to
|
|
# select which tagged subset a change TOUCHES, then hand those IDs to
|
|
# `conformance.sh --requirements ...` which runs exactly the tests carrying those tags.
|
|
#
|
|
# Detection = union of:
|
|
# 1. R-* tokens on the ADDED/REMOVED LINES of the diff, in non-Markdown files —
|
|
# including the authoritative `conformance.home.k3s/R-*` tags on changed tests, and
|
|
# the R-IDs that policies/ServiceDefinitions self-declare (docs/14 + ADR-0011).
|
|
# Untouched lines and prose are NOT scanned; see changed_diff() for why.
|
|
# 2. A coarse path->requirement fallback for changes that don't self-declare IDs.
|
|
# The coarse map (2) NEVER overrides an explicit tag; it only widens coverage for
|
|
# untagged changes so nothing silently escapes the gate.
|
|
set -euo pipefail
|
|
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
# shellcheck disable=SC1091
|
|
source "${CI_DIR}/lib.sh"
|
|
|
|
# Base ref to diff against. In Gitea/GitHub Actions PR runs this is the target
|
|
# branch (main). Locally, default to main; override with BASE_REF.
|
|
: "${BASE_REF:=}"
|
|
resolve_base() {
|
|
if [ -n "$BASE_REF" ]; then echo "$BASE_REF"; return; fi
|
|
if [ -n "${GITHUB_BASE_REF:-}" ]; then echo "origin/${GITHUB_BASE_REF}"; return; fi
|
|
if git -C "$REPO_ROOT" rev-parse --verify -q origin/main >/dev/null; then echo "origin/main"; return; fi
|
|
if git -C "$REPO_ROOT" rev-parse --verify -q main >/dev/null; then echo "main"; return; fi
|
|
echo "HEAD~1" # last resort
|
|
}
|
|
|
|
changed_files() {
|
|
local base; base="$(resolve_base)"
|
|
# Three-dot: changes on this branch since it diverged from base.
|
|
if git -C "$REPO_ROOT" diff --name-only "${base}...HEAD" 2>/dev/null; then return; fi
|
|
git -C "$REPO_ROOT" diff --name-only "${base}" 2>/dev/null || true
|
|
}
|
|
|
|
# The ADDED/REMOVED lines of the change, for the R-ID content scan — excluding
|
|
# documentation.
|
|
#
|
|
# TWO scoping rules, both learned from the gate blocking merges it should not have:
|
|
#
|
|
# 1. Scan the DIFF, not whole file contents. Grepping whole files meant a change
|
|
# inherited every requirement those files had EVER mentioned, on lines the author
|
|
# never touched. Since some of those are legitimately uncovered, `--require-all`
|
|
# then failed and BLOCKED the merge over unrelated prose.
|
|
#
|
|
# 2. Skip `*.md`. A requirement ID in prose is a REFERENCE, not a declaration —
|
|
# explaining a requirement is not the same as changing behaviour against it.
|
|
# Without this, the decision log cannot record which requirements a change
|
|
# concerned without thereby claiming to have touched them, and the CI-fix commit
|
|
# that explains a mis-attribution re-triggers the very mis-attribution it fixes.
|
|
# Nothing escapes: authoritative declarations (`conformance.home.k3s/R-*` tags,
|
|
# Kyverno policies, ServiceDefinitions) all live in YAML, and the doc subtrees
|
|
# that matter (conformance/, schemas/, catalog/, clusters/, ci/) are covered by
|
|
# the path->requirement fallback regardless of file type.
|
|
#
|
|
# Removed lines count: deleting a test's requirement tag, or a policy's self-declared
|
|
# ID, touches that requirement just as much as adding one.
|
|
# `:(exclude)*.md` drops Markdown from the content scan (rule 2 above). Three-dot to
|
|
# match changed_files(): changes on this branch since it diverged from base.
|
|
changed_diff() {
|
|
local base; base="$(resolve_base)"
|
|
if git -C "$REPO_ROOT" diff -U0 "${base}...HEAD" -- . ':(exclude)*.md' 2>/dev/null; then return; fi
|
|
git -C "$REPO_ROOT" diff -U0 "${base}" -- . ':(exclude)*.md' 2>/dev/null || true
|
|
}
|
|
|
|
# Coarse path -> requirement map. Deliberately conservative; extend as subtrees
|
|
# fill in. Left side is a path prefix (relative to repo root).
|
|
path_requirements() {
|
|
case "$1" in
|
|
catalog/*) echo "R-GITOPS-02" ;;
|
|
schemas/*) echo "R-GITOPS-02" ;;
|
|
platform/kyverno/*) echo "R-TEST-01 R-NET-01 R-SEC-01" ;;
|
|
platform/cilium/*) echo "R-NET-01 R-NET-02 R-NET-05" ;;
|
|
platform/longhorn/*) echo "R-STATE-01 R-STATE-02" ;;
|
|
platform/cert-manager/*) echo "R-TLS-01 R-TLS-02" ;;
|
|
platform/externaldns/*) echo "R-DNS-02" ;;
|
|
platform/velero/*) echo "R-STATE-03" ;;
|
|
platform/monitoring/*) echo "R-OBS-01 R-OBS-02" ;;
|
|
platform/image-automation/*) echo "R-UPD-01 R-UPD-02" ;;
|
|
conformance/*) echo "R-TEST-01 R-TEST-02 R-TEST-03 R-TEST-04" ;;
|
|
clusters/*) echo "R-GITOPS-01 R-ENV-02" ;;
|
|
.gitea/*|ci/*) echo "R-GITOPS-01 R-TEST-02 R-TEST-03" ;;
|
|
*) : ;;
|
|
esac
|
|
}
|
|
|
|
main() {
|
|
local files reqs=""
|
|
files="$(changed_files)"
|
|
[ -n "$files" ] || { log "no changed files vs base; empty touched-requirement set"; exit 0; }
|
|
|
|
while IFS= read -r f; do
|
|
[ -n "$f" ] || continue
|
|
reqs+=" $(path_requirements "$f")"
|
|
done <<< "$files"
|
|
|
|
# Self-declared R-IDs on the ADDED/REMOVED lines of the change (see changed_diff).
|
|
# `^[+-]` keeps hunk headers (`@@`) and the `+++`/`---` file headers out of scope.
|
|
reqs+=" $(changed_diff \
|
|
| grep -E '^[+-]' \
|
|
| grep -vE '^(\+\+\+|---)' \
|
|
| grep -hoE 'R-[A-Z]+-[0-9]+' \
|
|
| tr '\n' ' ' || true)"
|
|
|
|
# Normalize: split, unique, sort. Emit to stdout.
|
|
printf '%s\n' $reqs | sed '/^$/d' | sort -u
|
|
}
|
|
main "$@"
|