ci: exclude comment-only prose from touched-requirements scan; gate.yml -> make -C ci gate
HAND-OFF 1 (make -C ci gate parity). touched-requirements.sh gained a third scoping rule: after stripping the diff marker, whole-line YAML/shell `#` comments are dropped BEFORE the R-ID grep. A requirement cited only in a code-file comment is prose (a REFERENCE), not a declaration -- the same logic rule 2 already applies to `*.md`. This unblocks the self-referential trap where editing gate.yml's header (which cites R-TEST-06, a known-uncovered ID enforced by the spec-immutability STAGE, not a Chainsaw test) re-attributed R-TEST-06 so the coverage `--require` check false-blocked the merge. Conservative: only whole-line comments are dropped; an R-ID on a real code/spec line is still detected, and authoritative declarations never live in a bare comment. gate.yml now invokes the aggregate `make -C ci gate` (ci/gate.sh) instead of the individual stage scripts, so CI runs byte-for-byte the same gate a human runs in the devcontainer (R-TEST-02) and applies the R-TEST-03 decision. A BLOCKED gate exits non-zero and fails the job. The cluster-gated L2/L3 stage SKIPS honestly (no CLUSTER_AVAILABLE). spec-immutability is retained as an explicit early UI step. Adds ci/tests/touched-requirements.test.sh (+ `make -C ci test`) asserting rule 3: comment-only R-IDs (added or removed) are not attributed while real-line R-IDs are. ci/lib.sh REPO_ROOT is now env-overridable so tests can run the real script against a throwaway git repo (default unchanged). Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 authored and gitea-admin committed
1 parent
b2752aec48
commit
27bc02471c
5 files changed
+181
-32
No files matched your search
+7
-3
@@ -1,8 +1,9 @@
|
||||
# ci/Makefile — portable, single-command local equivalent of the Gitea Actions
|
||||
# merge gate (R-TEST-02: the gates are runnable TODAY, without Gitea).
|
||||
#
|
||||
# The Gitea Actions workflow (.gitea/workflows/merge-gate.yaml) calls these exact
|
||||
# scripts, so `make -C ci gate` locally == the CI merge gate. Run from anywhere:
|
||||
# The LIVE Gitea Actions workflow (.gitea/workflows/gate.yml) invokes `make -C ci gate`
|
||||
# directly, so `make -C ci gate` locally IS the CI merge gate — byte-for-byte. Run from
|
||||
# anywhere:
|
||||
# make -C ci gate
|
||||
#
|
||||
# Every stage is a thin wrapper over a ci/*.sh script; see ci/README.md.
|
||||
@@ -11,7 +12,7 @@ CI_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
|
||||
|
||||
.DEFAULT_GOAL := help
|
||||
|
||||
.PHONY: help spec-immutability lint schema l0 render-golden l1 conformance gate all touched versions tools
|
||||
.PHONY: help spec-immutability lint schema l0 render-golden l1 conformance gate all touched versions tools test
|
||||
|
||||
help: ## Show this help
|
||||
@awk 'BEGIN{FS=":.*##"} /^[a-zA-Z0-9_-]+:.*##/{printf " \033[36m%-14s\033[0m %s\n",$$1,$$2}' $(MAKEFILE_LIST)
|
||||
@@ -42,6 +43,9 @@ all: gate ## Alias for `gate`
|
||||
touched: ## Print the touched-requirement subset for the current branch vs main
|
||||
@"$(CI_DIR)touched-requirements.sh"
|
||||
|
||||
test: ## Run ci/ script unit tests (no cluster; e.g. touched-requirements content scan)
|
||||
@for t in "$(CI_DIR)"tests/*.test.sh; do echo "== $$t"; bash "$$t"; done
|
||||
|
||||
versions: ## Print pinned tool versions (R-TEST-02)
|
||||
@cat "$(CI_DIR)versions.env"
|
||||
|
||||
|
||||
@@ -10,9 +10,12 @@
|
||||
set -euo pipefail
|
||||
|
||||
# --- Paths -------------------------------------------------------------------
|
||||
# CI_DIR = this ci/ directory; REPO_ROOT = its parent (impl repo root).
|
||||
# CI_DIR = this ci/ directory (always the real script location). REPO_ROOT = its
|
||||
# parent (impl repo root) by default, but OVERRIDABLE via the environment so the
|
||||
# scripts can be pointed at a throwaway git repo under test (see ci/tests/) without
|
||||
# copying them out of the tree. CI and normal runs leave it unset -> unchanged.
|
||||
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd "${CI_DIR}/.." && pwd)"
|
||||
: "${REPO_ROOT:=$(cd "${CI_DIR}/.." && pwd)}"
|
||||
# shellcheck disable=SC1091
|
||||
source "${CI_DIR}/versions.env"
|
||||
|
||||
|
||||
Executable
+98
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env bash
|
||||
# ci/tests/touched-requirements.test.sh — regression tests for the content scan in
|
||||
# ci/touched-requirements.sh, focused on rule 3: R-IDs cited only in a whole-line
|
||||
# YAML/shell `#` comment are PROSE (a reference), not a declaration, and must NOT be
|
||||
# attributed — while R-IDs on real code/spec lines still ARE.
|
||||
#
|
||||
# The tests point the real script at a THROWAWAY git repo via the REPO_ROOT override
|
||||
# (ci/lib.sh honours a pre-set REPO_ROOT); the script itself is run from its true
|
||||
# location so it still sources the real lib.sh. No network, no cluster.
|
||||
#
|
||||
# Run: ci/tests/touched-requirements.test.sh (or `make -C ci test`)
|
||||
set -euo pipefail
|
||||
|
||||
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
SCRIPT="${CI_DIR}/touched-requirements.sh"
|
||||
|
||||
PASS=0 FAIL=0
|
||||
ok() { printf ' ok %s\n' "$*"; PASS=$((PASS+1)); }
|
||||
bad() { printf ' FAIL %s\n' "$*"; FAIL=$((FAIL+1)); }
|
||||
|
||||
# assert_absent <needle> <haystack> <msg>
|
||||
assert_absent() {
|
||||
if grep -qxF "$1" <<<"$2"; then bad "$3 (found unexpected '$1' in: $(tr '\n' ' ' <<<"$2"))";
|
||||
else ok "$3"; fi
|
||||
}
|
||||
# assert_present <needle> <haystack> <msg>
|
||||
assert_present() {
|
||||
if grep -qxF "$1" <<<"$2"; then ok "$3";
|
||||
else bad "$3 (missing '$1' from: $(tr '\n' ' ' <<<"$2"))"; fi
|
||||
}
|
||||
|
||||
TMP="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMP"' EXIT
|
||||
|
||||
git -C "$TMP" init -q -b main
|
||||
git -C "$TMP" config user.email [email protected]
|
||||
git -C "$TMP" config user.name t
|
||||
git -C "$TMP" config commit.gpgsign false # throwaway repo: never sign (ignore host gpg config)
|
||||
mkdir -p "$TMP/notes" # a path OUTSIDE path_requirements() so we isolate the content scan
|
||||
echo "baseline" > "$TMP/notes/base.txt"
|
||||
git -C "$TMP" add -A && git -C "$TMP" commit -qm baseline
|
||||
|
||||
git -C "$TMP" checkout -q -b feature
|
||||
|
||||
# A YAML file: an R-ID in a whole-line comment (must be IGNORED) and an R-ID on a real
|
||||
# value line (must be DETECTED). Also an indented comment, to prove leading space is fine.
|
||||
cat > "$TMP/notes/example.yaml" <<'YAML'
|
||||
# This header explains R-DOC-01 and must NOT be attributed.
|
||||
kind: Thing
|
||||
metadata:
|
||||
name: implements-R-REAL-01
|
||||
spec:
|
||||
note: value # trailing inline comment mentioning R-INLINE-01 (conservatively still seen)
|
||||
# deeply indented comment about R-DOC-02 — also NOT attributed
|
||||
YAML
|
||||
|
||||
# A shell file: a `#` comment R-ID (IGNORED) plus a real command line R-ID (DETECTED).
|
||||
cat > "$TMP/notes/example.sh" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
# doc: this script relates to R-SHDOC-01 (comment only — ignored)
|
||||
echo "runs check for R-SHREAL-01"
|
||||
SH
|
||||
|
||||
git -C "$TMP" add -A && git -C "$TMP" commit -qm feature
|
||||
|
||||
OUT="$(REPO_ROOT="$TMP" BASE_REF=main "$SCRIPT" 2>/dev/null)"
|
||||
printf 'touched set:\n%s\n---\n' "$OUT"
|
||||
|
||||
# Comment-only citations are dropped (rule 3):
|
||||
assert_absent "R-DOC-01" "$OUT" "whole-line YAML comment R-ID is NOT attributed"
|
||||
assert_absent "R-DOC-02" "$OUT" "indented YAML comment R-ID is NOT attributed"
|
||||
assert_absent "R-SHDOC-01" "$OUT" "shell '#' comment R-ID is NOT attributed"
|
||||
# Real code/spec lines still count:
|
||||
assert_present "R-REAL-01" "$OUT" "R-ID on a real YAML value line IS attributed"
|
||||
assert_present "R-SHREAL-01" "$OUT" "R-ID on a real shell command line IS attributed"
|
||||
# Conservative: a trailing inline comment on a REAL line is still seen (only whole-line
|
||||
# comments are dropped) — documents the intentional boundary of the fix.
|
||||
assert_present "R-INLINE-01" "$OUT" "inline comment on a real line is (conservatively) still seen"
|
||||
|
||||
# Removed comment lines are also excluded, while removed real lines still count. The file
|
||||
# must EXIST at the base, so we diff the removal against the commit that added it.
|
||||
git -C "$TMP" checkout -q -b feature2 main
|
||||
cat > "$TMP/notes/todelete.yaml" <<'YAML'
|
||||
# will delete a comment citing R-DELDOC-01 and a real line citing R-DELREAL-01
|
||||
kind: Thing
|
||||
name: implements-R-DELREAL-01
|
||||
YAML
|
||||
git -C "$TMP" add -A && git -C "$TMP" commit -qm add
|
||||
ADD_SHA="$(git -C "$TMP" rev-parse HEAD)"
|
||||
git -C "$TMP" rm -q "$TMP/notes/todelete.yaml" && git -C "$TMP" commit -qm remove
|
||||
OUT2="$(REPO_ROOT="$TMP" BASE_REF="$ADD_SHA" "$SCRIPT" 2>/dev/null)"
|
||||
printf 'touched set (removal):\n%s\n---\n' "$OUT2"
|
||||
assert_absent "R-DELDOC-01" "$OUT2" "REMOVED whole-line comment R-ID is NOT attributed"
|
||||
assert_present "R-DELREAL-01" "$OUT2" "REMOVED real-line R-ID IS attributed"
|
||||
|
||||
echo
|
||||
if [ "$FAIL" -eq 0 ]; then echo "touched-requirements tests: PASS ($PASS assertions)"; exit 0;
|
||||
else echo "touched-requirements tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi
|
||||
@@ -49,7 +49,7 @@ changed_files() {
|
||||
# The ADDED/REMOVED lines of the change, for the R-ID content scan — excluding
|
||||
# documentation.
|
||||
#
|
||||
# TWO scoping rules, both learned from the gate blocking merges it should not have:
|
||||
# THREE scoping rules, all learned from the gate blocking merges it should not have:
|
||||
#
|
||||
# 1. Scan the DIFF, not whole file contents. Grepping whole files meant a change
|
||||
# inherited every requirement those files had EVER mentioned, on lines the author
|
||||
@@ -66,14 +66,36 @@ changed_files() {
|
||||
# that matter (conformance/, schemas/, catalog/, clusters/, ci/) are covered by
|
||||
# the path->requirement fallback regardless of file type.
|
||||
#
|
||||
# 3. Skip COMMENT-ONLY lines in non-Markdown files — rule 2's logic, applied to the
|
||||
# prose that lives INSIDE code files. A requirement ID cited in a YAML/shell `#`
|
||||
# comment is documentation, not a declaration, exactly like an ID in a `*.md` file.
|
||||
# Rule 2 drops whole Markdown files, but a `.gitea/workflows/*.yml` or `ci/*.sh`
|
||||
# HEADER that explains a requirement would otherwise re-attribute it. Concretely:
|
||||
# gate.yml's header cites R-TEST-06 (enforced by the spec-immutability STAGE, not by
|
||||
# any Chainsaw test, so a known-uncovered ID); before this rule, editing that comment
|
||||
# re-attributed R-TEST-06 and the coverage `--require` check false-blocked the merge —
|
||||
# the same self-referential trap rule 2 fixed for `*.md`. Applied in main()'s content
|
||||
# scan (see there): after the diff marker is stripped, a line whose first non-blank
|
||||
# character is `#` is dropped BEFORE the R-ID grep. Deliberately conservative — only
|
||||
# WHOLE-LINE comments are dropped; an R-ID on a real code/spec line is still detected
|
||||
# (authoritative declarations — test tags, policies, ServiceDefinitions — are never a
|
||||
# bare comment), so this only narrows PROSE, never a declaration.
|
||||
#
|
||||
# Removed lines count: deleting a test's requirement tag, or a policy's self-declared
|
||||
# ID, touches that requirement just as much as adding one.
|
||||
# `:(exclude)*.md` drops Markdown from the content scan (rule 2 above). Three-dot to
|
||||
# match changed_files(): changes on this branch since it diverged from base.
|
||||
# `:(exclude)*.md` drops Markdown from the content scan (rule 2 above).
|
||||
# `:(exclude)ci/tests/` drops this dir's OWN unit-test fixtures: those files embed example
|
||||
# R-IDs as DATA (asserting the scan's behaviour), never as declarations — the same
|
||||
# "reference, not declaration" logic as rule 2. Their gate coverage is unaffected: any
|
||||
# ci/ change (ci/tests/ included) still attributes R-GITOPS-01/R-TEST-02/R-TEST-03 via the
|
||||
# path->requirement map below. NOTE: this excludes ci/tests/ only — conformance/tests/
|
||||
# carry AUTHORITATIVE `conformance.home.k3s/R-*` labels and are deliberately still scanned.
|
||||
# Three-dot to match changed_files(): changes on this branch since it diverged from base.
|
||||
changed_diff() {
|
||||
local base; base="$(resolve_base)"
|
||||
if git -C "$REPO_ROOT" diff -U0 "${base}...HEAD" -- . ':(exclude)*.md' 2>/dev/null; then return; fi
|
||||
git -C "$REPO_ROOT" diff -U0 "${base}" -- . ':(exclude)*.md' 2>/dev/null || true
|
||||
local -a excl=(':(exclude)*.md' ':(exclude)ci/tests/')
|
||||
if git -C "$REPO_ROOT" diff -U0 "${base}...HEAD" -- . "${excl[@]}" 2>/dev/null; then return; fi
|
||||
git -C "$REPO_ROOT" diff -U0 "${base}" -- . "${excl[@]}" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Coarse path -> requirement map. Deliberately conservative; extend as subtrees
|
||||
@@ -109,9 +131,15 @@ main() {
|
||||
|
||||
# Self-declared R-IDs on the ADDED/REMOVED lines of the change (see changed_diff).
|
||||
# `^[+-]` keeps hunk headers (`@@`) and the `+++`/`---` file headers out of scope.
|
||||
# `sed 's/^[+-]//'` then strips the single-char diff marker (safe: -U0 emits one), so
|
||||
# what remains is the SOURCE line, and `grep -vE '^[[:space:]]*#'` drops whole-line
|
||||
# YAML/shell comments (rule 3 in changed_diff's header) BEFORE the R-ID grep — a
|
||||
# requirement cited only in a `#` comment is prose, not a declaration.
|
||||
reqs+=" $(changed_diff \
|
||||
| grep -E '^[+-]' \
|
||||
| grep -vE '^(\+\+\+|---)' \
|
||||
| sed -E 's/^[+-]//' \
|
||||
| grep -vE '^[[:space:]]*#' \
|
||||
| grep -hoE 'R-[A-Z]+-[0-9]+' \
|
||||
| tr '\n' ' ' || true)"
|
||||
|
||||
|
||||
Reference in new issue
Block a user