Home-Network/home-network-k3s-impl
2
0 Fork 0
Code Issues 3 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

fix(ci): kill the false-green base resolution; cover R-SEC-01/02 + R-TEST-06 (L0)

Browse Source 
Unblocks the enforced Gitea gate, which correctly BLOCKED this branch on
['R-SEC-01','R-SEC-02','R-TEST-06'] being touched but uncovered. Tests were authored;
the coverage rule was NOT narrowed.

=== THE FALSE GREEN (root cause) ===
`make -C ci gate` locally reported PASS having checked NOTHING. resolve_base() validated
nothing, and EVERY way of failing to find a base degraded silently to an empty touched
set + exit 0: an `HEAD~1` last resort, a two-dot diff fallback masking a failing
three-dot diff, and no check that HEAD is even AHEAD of the base. Run while sitting on
`main` (main == origin/main), `origin/main...HEAD` is empty -> "no changed files vs
base" -> the empty set means run.sh is invoked with NO --requirements -> coverage.py's
--require gate never runs at all. Green by vacuum.

Now: candidates are BASE_REF > GITHUB_BASE_REF > origin/main > main, each verified to
resolve. HEAD~1 and the two-dot fallback are gone. Unresolvable base / no merge base /
failed diff / HEAD-not-ahead -> loud banner, EMPTY stdout, exit 3, and a machine-readable
reason (on-base | no-base | diff-failed). gate.sh turns exit 3 into an UNDECIDED
decision: `on-base` -> "NOT APPLICABLE — no branch under gate" (exit 0 so push-to-main
runs don't break, but it never reads as PASS); `no-base`/`diff-failed` -> BLOCKED. It
also now prints the diff base it used. A stale local main warns (widens, never falsely
greens). Also fixed `make -C ci test`, which was SWALLOWING failing suites.

Regression: ci/tests/touched-requirements-base.test.sh — 31 assertions over 8
throwaway-repo scenarios including the exact false-green case, an already-merged branch,
no base ref, a single-commit repo, bogus BASE_REF, a shallow GITHUB_BASE_REF, and
origin/main-preferred-over-stale-main.

=== NEW L0 STATIC LAYER (conformance/tests/l0-static/) ===
Where docs/14 actually puts these invariants. Each check.sh really executes offline via
`run.sh --layer l0` and is wired into ci/l0.sh, so it runs on every PR. No scaffolds.

R-SEC-01 no-plaintext-secrets (165 assertions / 445 tracked files): .sops.yaml holds only
PUBLIC age recipients; every committed Secret is SOPS-wrapped with a mac and ENC[...]
values; ansible vault.yml files are $ANSIBLE_VAULT; no secret-valued ansible key carries
a literal outside a vault; no PEM key / age secret key / kubeconfig client-key-data is
tracked; Flux declares sops decryption; the disallow-plaintext-secrets policy Enforces
and has a proving `result: fail` row. Exemptions are SELF-POLICING, not path allowlists —
a .example must be referenced by no kustomization and hold only placeholders, and the
Kyverno fixture is exempt ONLY while a test asserts the policy rejects it. Proven against
7 injected violations, including deleting the negative row and weakening Enforce->Audit.

R-SEC-02 least-privilege-rbac (41 assertions): no wildcard apiGroup/resource/verb and no
escalate|bind|impersonate in any in-repo Role/ClusterRole; no ServiceAccount bound to
cluster-admin or a wildcard role; the conformance runner's own test identity is scoped
and holds no rights over RBAC objects; the default-deny/host-namespace/egress-class
policies Enforce and each carry a negative row. Proven against 6 injected violations.

R-TEST-06 spec-immutability-enforced (16 assertions): drives the REAL
ci/spec-immutability.sh against HERMETIC throwaway spec repos and asserts its exit code
in every state — clean-at-pin 0, edits under docs/ or decisions/ 1, untracked addition 1,
unadopted drift 1, explicit pin bump 0, ABSENT spec repo 3 (honest skip, never green),
unpinned SPEC_PIN 1 — plus that gate.sh sets BLOCK=1 on its failure, so the mechanism is
load-bearing. The real spec repo is never written to.

Each check documents what it does NOT cover (runtime decryption, git history, runtime
least-privilege, branch protection) rather than overclaiming. status.py now reads
check.sh too, so a placeholder L0 body still outranks an `executable: true` annotation.

Coverage 37/51 -> 40/51 (11 gaps); executable tests 32 -> 35; scaffold-only stays 5 —
no sixth scaffold was added.

Serves: R-SEC-01, R-SEC-02, R-TEST-01..06.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed 2026-08-31 23:26:49 -05:00
1 parent 9f59c36000
commit 6e77150ae1
21 files changed
+1450 -97
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

conformance/tests/README.md
+35 -2
View File
@@ -4,6 +4,7 @@ Every test is a **Chainsaw `Test`** in its own directory:
```
tests/
l0-static/ L0 — whole-repo invariants over the tracked tree (no cluster)
l1-rendering/ L1 — ServiceDefinition -> rendered resources match golden
l2-behavioral/ L2 — real runtime behavior (probes against a cluster)
l3-fault-injection/ L3 — break something; assert fail-closed / no-data-loss
@@ -14,6 +15,35 @@ tests/
The layer a test belongs to is its subtree **and** its `.../layer` annotation; the
coverage generator cross-checks the two.
### L0 static (`l0-static/`) — what belongs here, and what does not
docs/14's L0 row is *"Pre-apply, in CI · manifests/ServiceDefinitions obey invariants"*.
Kyverno (`platform/kyverno/`) covers the invariants expressible over a **single
resource**. This subtree covers the L0 invariants that are **properties of the repository
as a whole** and that no per-resource policy can state — e.g. *no plaintext secret exists
anywhere in the tracked tree*, *no tracked RoleBinding grants a workload identity a
privileged role*, *the spec-immutability gate actually blocks a modified spec*.
Each `l0-static/<name>/` dir carries a **`check.sh`**: a self-contained offline assertion
script (git + Python stdlib/PyYAML; no cluster, no network) that exits 0/1. It is executed:
- by `conformance/run.sh --layer l0` (and as the offline leg of a full run), via
`lib/run_l1_offline.py`, producing a REAL pass/fail JUnit entry — never a skip; and
- by the merge gate's **`ci/l0.sh`** stage, so these assertions run on every PR; and
- by the wrapping Chainsaw `Test`, which runs the SAME `check.sh`, so an in-cluster run
executes byte-identical assertions.
Two authoring rules specific to this layer:
1. **State the gap.** A statically-provable half of a requirement is still only half.
Every `check.sh` header ends with an explicit *"what this does NOT prove"* list naming
the cluster-layer remainder. Covering a requirement at L0 does not finish it.
2. **Exemptions must be self-policing.** Where a check must tolerate something (a
deliberately-plaintext policy fixture, an `.example` template), the exemption is
conditioned on a property that would break if the exemption were abused — e.g. the
plaintext Secret fixture is exempt *only while* a `kyverno-test.yaml` asserts the
policy rejects it. Never a bare path allowlist.
## Requirement-tagging convention (authoritative)
A test declares the requirement(s) it verifies **three** ways; the label is the source
@@ -28,7 +58,7 @@ metadata:
annotations:
# Human-readable canonical list — MUST equal the label set.
conformance.home.k3s/requirements: "R-NET-01"
# Declared layer (L1|L2|L3|META) — must match the subtree.
# Declared layer (L0|L1|L2|L3|META) — must match the subtree.
conformance.home.k3s/layer: "L2"
# true => R-TEST-05 requires this to be (or have) an L3 fault-injection test.
conformance.home.k3s/safety-critical: "false"
@@ -65,4 +95,7 @@ and coverage extraction exact and robust.
the PM — do not relax the assertion or the requirement.
Scaffolds in this tree carry `TODO(T-x.y)` markers where a fixture/assertion lands as a
later roadmap phase delivers the capability under test.
later roadmap phase delivers the capability under test. `lib/status.py` treats such a
marker — in the `chainsaw-test.yaml`, in `render-check.sh`, or in `check.sh` — as
outranking any `conformance.home.k3s/executable: "true"` annotation, so a placeholder body
can never be reported as an executable test.
conformance/tests/l0-static/least-privilege-rbac/chainsaw-test.yaml
+44
View File
@@ -0,0 +1,44 @@
# =============================================================================
# L0 STATIC — least privilege for workload identities and network reach (R-SEC-02)
# Catalog: docs/14 L0 row, "Manifests/ServiceDefinitions obey invariants" — runs in
# CI WITHOUT a cluster.
#
# GIVEN the repository as git tracks it,
# WHEN every tracked RBAC object and every least-network-reach policy is inspected,
# THEN no in-repo role grants a wildcard or a privilege-escalation verb, no workload
# ServiceAccount is bound to a privileged role, the conformance runner's own
# privileged test identity stays scoped to its authored ClusterRole, and each
# least-network-reach policy both Enforces and is proven by an L0 negative row
# (or a generated-resource row) to actually close reach.
#
# STATUS: EXECUTABLE — and DELIBERATELY PARTIAL. `check.sh` is a pure offline assertion
# script (git ls-files + PyYAML); conformance/run.sh --layer l0 and ci/l0.sh run it
# directly, and this Chainsaw Test wraps the SAME script. It proves the STATIC half of
# both clauses. The RUNTIME half — that the API server actually denies an unrequested
# verb, and that pod-to-pod traffic is actually dropped — needs a cluster and is NOT
# asserted under this requirement; see the gap list in check.sh's header. R-SEC-02 is
# therefore covered, not finished.
# =============================================================================
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: least-privilege-rbac
labels:
conformance.home.k3s/R-SEC-02: "true"
annotations:
conformance.home.k3s/requirements: "R-SEC-02"
conformance.home.k3s/layer: "L0"
conformance.home.k3s/safety-critical: "false"
conformance.home.k3s/executable: "true"
conformance.home.k3s/intent: >-
Workload identities carry no wildcard, no escalation verb and no privileged
binding; the least-network-reach policies are enforced and proven to reject.
spec:
steps:
- name: scan-rbac-and-least-reach-policies
description: >-
Assert the authored least-privilege posture of every tracked RBAC object and
the enforced, test-proven status of the least-network-reach policies.
try:
- script:
content: bash ./check.sh
conformance/tests/l0-static/least-privilege-rbac/check.sh
Executable
+269
View File
@@ -0,0 +1,269 @@
#!/usr/bin/env bash
# =============================================================================
# L0 static check (offline, no cluster) — least-privilege-rbac
# Serves: R-SEC-02 ("The platform MUST follow least-privilege for workload
# identities and network reach." — spec docs/00-vision-and-scope.md;
# detail in docs/07 "least privilege ... least network reach").
#
# The requirement has TWO halves and this check covers the STATIC half of each,
# at the layer docs/14 puts them ("L0 Static/policy | Pre-apply, in CI").
#
# GIVEN the repository as git tracks it,
# WHEN every tracked RBAC object and every least-network-reach policy is inspected,
# THEN — workload identity —
# (1) no in-repo Role/ClusterRole grants a wildcard apiGroup, resource or verb,
# and none grants escalate/bind/impersonate (the privilege-escalation verbs);
# (2) no (Cluster)RoleBinding binds a ServiceAccount — a WORKLOAD identity — to
# `cluster-admin` or to any in-repo role holding wildcard rules. Human/Group
# subjects are out of scope: R-SEC-02 constrains workload identities, and the
# operator's own admin path is an OIDC group binding, deliberately;
# (3) every ServiceAccount subject is namespaced (no cluster-wide ambiguity);
# (4) the conformance runner's own SA — this workstream's PRIVILEGED TEST
# IDENTITY, whose least-privilege authoring docs/14 explicitly delegates to
# WS-TEST — is bound only to its own scoped ClusterRole, carries no wildcard
# and no privilege-escalation verb, and grants nothing over RBAC objects
# themselves (no rbac.authorization.k8s.io rules);
# — network reach —
# (5) the policies that implement least network reach exist, and each is Enforce
# (or carries the Flux token that substitutes to Enforce): the default-deny
# baseline generator, the baseline SHAPE validator, the host-namespace/
# privilege restriction, and the egress-class label requirement;
# (6) each of those policies is actually EXERCISED by the L0 Kyverno suite with a
# NEGATIVE row (result: fail) — a policy nobody proves rejects anything is a
# declaration, not a gate.
#
# WHAT THIS DOES *NOT* PROVE (honest gap, cluster layer — R-SEC-02 is NOT fully covered):
# - RUNTIME least privilege of workload identities: that the API server actually
# denies the conformance SA (or any workload SA) the verbs it was not granted.
# That needs a live cluster (L2) and no test asserts it today.
# - RUNTIME least network reach: that pod-to-pod traffic is actually denied. The
# behaviour is exercised by tests/l2-behavioral/{default-deny,deny-undeclared},
# but those are tagged R-NET-01/R-NET-02 and are cluster-gated (skipped in Phase 0);
# nothing proves it under an R-SEC-02 tag.
# - Helm-chart-generated RBAC (e.g. what a HelmRelease's chart creates in-cluster) is
# not in the repo, so it is outside this static scan entirely.
# =============================================================================
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
PYTHON="${PYTHON:-python3}"
cd "${ROOT}"
"${PYTHON}" - "${ROOT}" <<'PY'
import os, re, subprocess, sys
from pathlib import Path
try:
import yaml
except ImportError: # pragma: no cover
print("check FAIL: PyYAML is required (same dependency as catalog/render.py)",
file=sys.stderr)
raise SystemExit(1)
ROOT = Path(sys.argv[1])
FAILURES: list[str] = []
CHECKS = 0
def check(cond, msg):
global CHECKS
CHECKS += 1
if not cond:
FAILURES.append(msg)
def tracked() -> list[str]:
out = subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z"],
capture_output=True, text=True, check=True).stdout
return [p for p in out.split("\0") if p]
def docs_of(rel: str):
try:
return [d for d in yaml.safe_load_all((ROOT / rel).read_text(errors="replace"))
if isinstance(d, dict)]
except Exception:
return []
FILES = tracked()
YAML_FILES = [f for f in FILES if re.search(r"\.(ya?ml)(\.example)?$", f)]
check(len(YAML_FILES) > 50, f"only {len(YAML_FILES)} YAML files seen — not a real checkout")
ESCALATION_VERBS = {"escalate", "bind", "impersonate"}
PRIVILEGED_ROLES = {"cluster-admin", "admin", "edit"}
roles: dict[tuple[str, str], dict] = {} # (kind, name) -> {"file":…, "wildcard":bool, …}
bindings: list[dict] = []
for rel in YAML_FILES:
for d in docs_of(rel):
kind = d.get("kind")
name = (d.get("metadata") or {}).get("name")
if kind in ("Role", "ClusterRole"):
wildcard, escalating, rbac_over_rbac = False, set(), False
for r in d.get("rules") or []:
if not isinstance(r, dict):
continue
for field in ("apiGroups", "resources", "verbs"):
if "*" in (r.get(field) or []):
wildcard = True
escalating |= set(v for v in (r.get("verbs") or [])) & ESCALATION_VERBS
if "rbac.authorization.k8s.io" in (r.get("apiGroups") or []):
rbac_over_rbac = True
roles[(kind, name)] = {"file": rel, "wildcard": wildcard,
"escalating": escalating, "rbac": rbac_over_rbac,
"doc": d}
# (1) no wildcards, no privilege-escalation verbs.
check(not wildcard,
f"{rel}: {kind}/{name} grants a wildcard apiGroup/resource/verb — that is "
"not least-privilege (R-SEC-02)")
check(not escalating,
f"{rel}: {kind}/{name} grants privilege-escalation verb(s) "
f"{sorted(escalating)} (R-SEC-02)")
elif kind in ("RoleBinding", "ClusterRoleBinding"):
bindings.append({"file": rel, "name": name, "doc": d})
check(len(roles) > 0, "no Role/ClusterRole found in the repo — the RBAC scan is not looking at it")
check(len(bindings) > 0, "no RoleBinding/ClusterRoleBinding found — the RBAC scan is not looking at it")
for b in bindings:
d, rel, name = b["doc"], b["file"], b["name"]
ref = d.get("roleRef") or {}
ref_kind, ref_name = ref.get("kind"), ref.get("name")
sa_subjects = [s for s in (d.get("subjects") or [])
if isinstance(s, dict) and s.get("kind") == "ServiceAccount"]
local = roles.get((ref_kind, ref_name))
ref_is_privileged = (ref_name in PRIVILEGED_ROLES) or bool(local and local["wildcard"])
# (2) a WORKLOAD identity must never hold a privileged role.
check(not (sa_subjects and ref_is_privileged),
f"{rel}: {d.get('kind')}/{name} binds ServiceAccount(s) "
f"{[s.get('name') for s in sa_subjects]} to the privileged role {ref_name!r} — "
"workload identities must be least-privilege (R-SEC-02)")
# (3) SA subjects must name their namespace (a ClusterRoleBinding without one is
# ambiguous and, in a RoleBinding, silently defaults to the binding's namespace).
for s in sa_subjects:
check(bool(s.get("namespace")) or d.get("kind") == "RoleBinding",
f"{rel}: {d.get('kind')}/{name} has a ServiceAccount subject "
f"{s.get('name')!r} with no namespace (R-SEC-02)")
# --------------------------------- (4) our own privileged test identity, in detail
RBAC_FILE = "conformance/incluster/rbac.yaml"
check(RBAC_FILE in FILES, f"{RBAC_FILE} is missing — the conformance runner has no authored RBAC")
runner_role = roles.get(("ClusterRole", "conformance-runner"))
check(runner_role is not None,
f"{RBAC_FILE}: no ClusterRole/conformance-runner — the test identity's grants are undefined")
if runner_role:
check(runner_role["file"] == RBAC_FILE,
f"ClusterRole/conformance-runner is defined in {runner_role['file']}, not {RBAC_FILE}")
check(not runner_role["wildcard"], "ClusterRole/conformance-runner uses a wildcard (R-SEC-02)")
check(not runner_role["escalating"],
"ClusterRole/conformance-runner grants a privilege-escalation verb (R-SEC-02)")
check(not runner_role["rbac"],
"ClusterRole/conformance-runner grants rights over RBAC objects themselves — a test "
"identity that can rewrite RBAC is not least-privilege (R-SEC-02)")
runner_bindings = [b for b in bindings
if any(s.get("kind") == "ServiceAccount" and s.get("name") == "conformance-runner"
for s in (b["doc"].get("subjects") or []))]
check(len(runner_bindings) >= 1, "the conformance-runner ServiceAccount has no binding at all")
for b in runner_bindings:
ref = b["doc"].get("roleRef") or {}
check(ref.get("name") == "conformance-runner",
f"{b['file']}: conformance-runner is bound to {ref.get('name')!r} instead of its own "
"scoped ClusterRole (R-SEC-02)")
for s in b["doc"].get("subjects") or []:
if s.get("kind") == "ServiceAccount":
check(s.get("namespace") == "conformance",
f"{b['file']}: conformance-runner subject is in namespace "
f"{s.get('namespace')!r}, expected 'conformance' (R-SEC-02)")
# --------------------- (5)/(6) least NETWORK REACH: the policies exist, Enforce, and
# are proven to reject something at L0.
# Validating policies: must Enforce AND be proven to REJECT something at L0.
REQUIRED_POLICIES = {
"validate-default-deny-shape": "a baseline that leaves a direction open is rejected",
"restrict-host-namespaces": "host namespaces / privileged containers are rejected",
"require-egress-class-label": "workloads must declare their egress class",
}
# Generating policies have no validationFailureAction and no negative row: the proof
# that they close network reach is that the L0 suite asserts the resource they GENERATE.
REQUIRED_GENERATE_POLICIES = {
"require-default-deny-baseline": "every namespace gets a default-deny baseline",
}
policies: dict[str, dict] = {}
for rel in YAML_FILES:
for d in docs_of(rel):
if d.get("kind") in ("ClusterPolicy", "Policy"):
policies[(d.get("metadata") or {}).get("name")] = {"file": rel, "doc": d}
for pname, why in REQUIRED_POLICIES.items():
p = policies.get(pname)
check(p is not None, f"policy {pname!r} is missing — nothing enforces: {why} (R-SEC-02)")
if not p:
continue
action = str((p["doc"].get("spec") or {}).get("validationFailureAction", ""))
check(action == "Enforce" or action.startswith("${"),
f"{p['file']}: policy {pname!r} validationFailureAction is {action!r} — a least-reach "
"policy in Audit mode blocks nothing (R-SEC-02)")
negatives: dict[str, int] = {}
suites = [f for f in FILES if os.path.basename(f) == "kyverno-test.yaml"]
check(len(suites) > 0, "no kyverno-test.yaml — the L0 policy gate is not exercised at all")
for suite in suites:
for d in docs_of(suite):
for r in d.get("results") or []:
if isinstance(r, dict) and r.get("result") == "fail":
negatives[r.get("policy")] = negatives.get(r.get("policy"), 0) + 1
for pname, why in REQUIRED_POLICIES.items():
check(negatives.get(pname, 0) > 0,
f"no L0 test row asserts that policy {pname!r} REJECTS a violating resource "
f"(result: fail) — unproven gate for: {why} (R-SEC-02)")
# The generating half: the policy must exist, and the L0 suite must assert the resource
# it generates (a `generatedResource` row that passes) — otherwise nothing proves a new
# namespace actually arrives with its reach closed.
for pname, why in REQUIRED_GENERATE_POLICIES.items():
p = policies.get(pname)
check(p is not None, f"policy {pname!r} is missing — nothing enforces: {why} (R-SEC-02)")
generated_rows = []
for suite in suites:
for d in docs_of(suite):
for r in d.get("results") or []:
if isinstance(r, dict) and r.get("policy") == pname \
and r.get("generatedResource") and r.get("result") == "pass":
generated_rows.append((suite, r.get("generatedResource")))
check(len(generated_rows) > 0,
f"no L0 test row asserts the resource generated by {pname!r} — unproven gate for: "
f"{why} (R-SEC-02)")
for suite, gen in generated_rows:
gp = os.path.normpath(os.path.join(os.path.dirname(suite), str(gen)))
check(gp in FILES, f"{suite}: generatedResource {gen!r} does not resolve to a tracked file")
golden = "\n".join(l for l in (ROOT / gp).read_text(errors="replace").splitlines()
if not l.lstrip().startswith("#")) if gp in FILES else ""
check("NetworkPolicy" in golden and re.search(r"policyTypes:", golden),
f"{gp}: the asserted default-deny baseline is not a shaped NetworkPolicy (R-SEC-02)")
# The host-namespace policy must prove BOTH of its rules reject, not just one.
host_rules = set()
for suite in suites:
for d in docs_of(suite):
for r in d.get("results") or []:
if isinstance(r, dict) and r.get("policy") == "restrict-host-namespaces" \
and r.get("result") == "fail":
host_rules.add(r.get("rule"))
check({"disallow-host-namespaces", "disallow-privileged-containers"} <= host_rules,
"the L0 suite does not prove BOTH restrict-host-namespaces rules reject a violating pod "
f"(negative rows seen for: {sorted(host_rules)}) — R-SEC-02")
# --------------------------------------------------------------------------- verdict
if FAILURES:
print(f"CHECK FAIL (least-privilege-rbac): {len(FAILURES)} of {CHECKS} assertions failed",
file=sys.stderr)
for f in FAILURES:
print(f" - {f}", file=sys.stderr)
raise SystemExit(1)
print(f"check OK: least-privilege-rbac — {CHECKS} assertions over {len(roles)} role(s), "
f"{len(bindings)} binding(s), {len(REQUIRED_POLICIES)} least-reach policies")
PY
conformance/tests/l0-static/no-plaintext-secrets/chainsaw-test.yaml
+43
View File
@@ -0,0 +1,43 @@
# =============================================================================
# L0 STATIC — no plaintext secrets in git (R-SEC-01)
# Catalog: docs/14 L0 row, "Manifests/ServiceDefinitions obey invariants ...
# no plaintext secrets" — runs in CI WITHOUT a cluster.
#
# GIVEN the repository as git tracks it,
# WHEN every tracked file is inspected,
# THEN every committed Secret is SOPS ciphertext, every ansible group_vars vault is
# ansible-vault encrypted, no private key material is tracked, decryption is
# delegated to an in-cluster SOPS key, and the L0 policy that rejects plaintext
# Secrets exists, Enforces, and is proven by a negative test row to reject one.
#
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git ls-files +
# PyYAML, no cluster, no network); conformance/run.sh --layer l0 and ci/l0.sh run it
# directly, and this Chainsaw Test wraps the SAME script so an in-cluster run executes
# the identical assertions. Encryption at rest is necessary but NOT sufficient: the
# runtime half of "decrypted only in-cluster", and a scan of git HISTORY, are named as
# explicit gaps in check.sh's header and are NOT claimed here.
# =============================================================================
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: no-plaintext-secrets
labels:
conformance.home.k3s/R-SEC-01: "true"
annotations:
conformance.home.k3s/requirements: "R-SEC-01"
conformance.home.k3s/layer: "L0"
conformance.home.k3s/safety-critical: "false"
conformance.home.k3s/executable: "true"
conformance.home.k3s/intent: >-
No secret is stored in plaintext in git: committed Secrets are SOPS ciphertext,
ansible vaults are encrypted, no private key material is tracked, and the
no-plaintext-secrets policy gate is enforced and proven to reject.
spec:
steps:
- name: scan-tracked-tree-for-plaintext-secrets
description: >-
Assert over the whole tracked tree that no secret material is stored in
plaintext and that decryption is delegated to the in-cluster SOPS key.
try:
- script:
content: bash ./check.sh
conformance/tests/l0-static/no-plaintext-secrets/check.sh
Executable
+287
View File
@@ -0,0 +1,287 @@
#!/usr/bin/env bash
# =============================================================================
# L0 static check (offline, no cluster) — no-plaintext-secrets
# Serves: R-SEC-01 ("Secrets MUST NOT be stored in plaintext in git; they MUST be
# encrypted at rest in the repo and decrypted only in-cluster."
# — spec docs/00-vision-and-scope.md; detail in docs/07.)
#
# docs/14 puts exactly this invariant at L0 ("Static/policy | Pre-apply, in CI |
# ... no plaintext secrets"), so it is asserted here over the WHOLE tracked tree,
# with no cluster. Run by `conformance/run.sh --layer l0` and by ci/l0.sh.
#
# GIVEN the repository as git tracks it,
# WHEN every tracked file is inspected,
# THEN (1) SOPS is configured at the repo root with a PUBLIC age recipient;
# (2) every committed Kubernetes Secret is SOPS-encrypted — a top-level
# `sops:` block with a mac AND every data/stringData value is ENC[...];
# (3) the only two exempt classes are narrow and self-policing:
# a. `*.example` templates, which must NOT be referenced by any
# kustomization and whose secret-named values must be placeholders;
# b. the Kyverno L0 negative fixture, which is exempt ONLY while it is
# actually wired into a kyverno-test.yaml that asserts
# `disallow-plaintext-secrets` REJECTS it (result: fail);
# (4) every ansible-vault group_vars file is `$ANSIBLE_VAULT;`-encrypted, and
# no secret-valued ansible key carries a literal outside a vault;
# (5) no private key material of any kind is tracked (PEM private keys, an
# age SECRET key, kubeconfig client-key-data);
# (6) decryption is delegated to the cluster: a Flux Kustomization declares
# `decryption.provider: sops` with a secretRef, and the private half of
# that key pair is nowhere in the repo (5).
#
# WHAT THIS DOES *NOT* PROVE (honest gap, cluster layer):
# - that the decrypted Secret only ever materialises in-cluster (the runtime half
# of "decrypted only in-cluster") — needs a live cluster (L2), untested today.
# - that no plaintext secret exists in git HISTORY. This walks the WORKING TREE at
# HEAD only; docs/07's history-scan acceptance gate needs a secret scanner over
# all revisions and is NOT implemented here.
# - value-level entropy: a placeholder-looking string that is in fact a real
# credential is not detectable statically.
# =============================================================================
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
PYTHON="${PYTHON:-python3}"
cd "${ROOT}"
"${PYTHON}" - "${ROOT}" <<'PY'
import os, re, subprocess, sys
from pathlib import Path
try:
import yaml
except ImportError: # pragma: no cover
print("check FAIL: PyYAML is required (same dependency as catalog/render.py)",
file=sys.stderr)
raise SystemExit(1)
ROOT = Path(sys.argv[1])
FAILURES: list[str] = []
CHECKS = 0
def check(cond, msg):
global CHECKS
CHECKS += 1
if not cond:
FAILURES.append(msg)
def tracked() -> list[str]:
out = subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z"],
capture_output=True, text=True, check=True).stdout
return [p for p in out.split("\0") if p]
FILES = tracked()
check(len(FILES) > 50, f"git ls-files returned only {len(FILES)} paths — not a real checkout")
def docs_of(rel: str):
"""Parsed YAML documents of a tracked file, or [] when it is not parseable YAML."""
p = ROOT / rel
try:
return [d for d in yaml.safe_load_all(p.read_text(errors="replace")) if isinstance(d, dict)]
except Exception:
return []
YAMLISH = re.compile(r"\.(ya?ml)(\.example)?$")
YAML_FILES = [f for f in FILES if YAMLISH.search(f)]
# ---------------------------------------------------------------- (1) SOPS config
sops_cfg = ROOT / ".sops.yaml"
check(sops_cfg.is_file(), ".sops.yaml is missing from the repository root (R-SEC-01)")
if sops_cfg.is_file():
cfg = yaml.safe_load(sops_cfg.read_text()) or {}
rules = cfg.get("creation_rules") or []
check(len(rules) > 0, ".sops.yaml declares no creation_rules — new secrets would be committed unencrypted")
for r in rules:
pr = r.get("path_regex", "<none>")
recips = str(r.get("age", ""))
check(bool(recips), f".sops.yaml creation_rule {pr!r} has no age recipient")
check(all(x.startswith("age1") for x in re.split(r"[,\s]+", recips) if x),
f".sops.yaml creation_rule {pr!r} recipient is not a PUBLIC age1... key")
check("AGE-SECRET-KEY" not in recips.upper(),
f".sops.yaml creation_rule {pr!r} contains an age PRIVATE key")
# --------------------------------------------- Kyverno negative-fixture exemption
# A plaintext Secret is tolerated ONLY where it is the negative case that proves the
# L0 policy rejects plaintext. We do not trust the path: the file must be listed in a
# kyverno-test.yaml whose results assert disallow-plaintext-secrets returns `fail`.
POLICY_NAME = "disallow-plaintext-secrets"
negative_fixtures: set[str] = set()
kyverno_suites = [f for f in FILES if os.path.basename(f) == "kyverno-test.yaml"]
check(len(kyverno_suites) > 0, "no kyverno-test.yaml found — the L0 policy gate is not exercised at all")
for suite in kyverno_suites:
docs = docs_of(suite)
if not docs:
continue
d = docs[0]
results = d.get("results") or []
rejects_plaintext = any(r.get("policy") == POLICY_NAME and r.get("result") == "fail"
for r in results if isinstance(r, dict))
if not rejects_plaintext:
continue
base = os.path.dirname(suite)
for res in d.get("resources") or []:
negative_fixtures.add(os.path.normpath(os.path.join(base, str(res))))
check(len(negative_fixtures) > 0,
f"no kyverno-test.yaml asserts that {POLICY_NAME} REJECTS a plaintext Secret "
"(result: fail) — the plaintext-secret gate is unproven, so nothing may be exempt")
# --------------------------------------------------- (2)/(3) committed Secret scan
SECRET_KEY_RE = re.compile(r"(pass|secret|token|key|credential|cookie|cert)", re.I)
PLACEHOLDER_RE = re.compile(r"^\s*(|<[^>]*>|replace[-_ ]?me.*|change[-_ ]?me.*|placeholder.*|x{3,})\s*$", re.I)
secrets_seen = 0
for rel in YAML_FILES:
for d in docs_of(rel):
if d.get("kind") != "Secret":
continue
secrets_seen += 1
values = []
for field in ("data", "stringData"):
for k, v in (d.get(field) or {}).items():
values.append((k, "" if v is None else str(v)))
is_example = rel.endswith(".example")
is_negative_fixture = os.path.normpath(rel) in negative_fixtures
if is_example:
# Template: never applied, and its secret-named values must be placeholders.
refs = [k for k in YAML_FILES
if os.path.basename(k) == "kustomization.yaml"
and os.path.basename(rel) in (ROOT / k).read_text(errors="replace")]
check(not refs,
f"{rel}: an unencrypted .example template is referenced by a kustomization "
f"({refs}) — it would be applied as a plaintext Secret")
for k, v in values:
if SECRET_KEY_RE.search(k):
check(bool(PLACEHOLDER_RE.match(v)),
f"{rel}: template key {k!r} holds {v[:24]!r}, not a placeholder — "
"a real credential may have been committed (R-SEC-01)")
continue
if is_negative_fixture:
# Exempt, but only as the proven negative case; assert it really is plaintext
# (an encrypted 'bad' fixture would silently stop testing anything).
continue
# Everything else: must be SOPS-encrypted at rest.
sops_block = d.get("sops")
check(isinstance(sops_block, dict) and bool(sops_block.get("mac")),
f"{rel}: committed Secret {d.get('metadata', {}).get('name')!r} has no top-level "
"`sops:` block with a mac — it is stored in plaintext (R-SEC-01)")
for k, v in values:
check(v.startswith("ENC["),
f"{rel}: Secret value {k!r} is not SOPS ciphertext (expected ENC[...]) — "
"plaintext secret material in git (R-SEC-01)")
check(secrets_seen > 0, "no Kubernetes Secret manifests found at all — the scan is not looking at the repo")
# ------------------------------------------------------------ (4) ansible vaults
VAULT_HEADER = "$ANSIBLE_VAULT;"
vault_files = [f for f in FILES
if os.path.basename(f) in ("vault.yml", "vault.yaml")]
check(len(vault_files) > 0, "no ansible vault.yml files found — the vault scan is not looking at the repo")
for rel in vault_files:
head = (ROOT / rel).read_text(errors="replace")[:len(VAULT_HEADER)]
check(head == VAULT_HEADER,
f"{rel}: an ansible group_vars vault file is NOT ansible-vault encrypted "
"(must start with the $ANSIBLE_VAULT header) — R-SEC-01")
SECRET_ANSIBLE_KEYS = ("ansible_become_password", "ansible_become_pass", "ansible_password",
"ansible_ssh_pass", "ansible_sudo_pass", "ansible_ssh_private_key_file")
ASSIGN_RE = re.compile(
r"^\s*(" + "|".join(SECRET_ANSIBLE_KEYS) + r")\s*:\s*(.*?)\s*$")
for rel in FILES:
p = ROOT / rel
if not p.is_file() or p.is_symlink():
continue
try:
text = p.read_text(errors="replace")
except Exception:
continue
if text.startswith(VAULT_HEADER):
continue # encrypted at rest: opaque, by design
for lineno, line in enumerate(text.splitlines(), 1):
if line.lstrip().startswith("#"):
continue
m = ASSIGN_RE.match(line)
if not m:
continue
key, val = m.group(1), m.group(2).strip().strip("'\"")
ok = (val.startswith("{{") or val.startswith("$")
or bool(PLACEHOLDER_RE.match(val)) or val == "")
check(ok, f"{rel}:{lineno}: {key} is assigned the literal {val[:24]!r} outside an "
"ansible-vault file — a plaintext secret in git (R-SEC-01)")
# ------------------------------------------------------ (5) no private key material
# NOTE: each marker is assembled from fragments so that THIS file never contains the
# literal string it hunts for — otherwise the scanner would flag itself the moment it
# is tracked, and the only way to silence it would be to exempt the scanner (which is
# exactly the kind of self-exemption this suite exists to prevent).
_DASH5 = "-" * 5
PRIVKEY_PATTERNS = [
(re.compile(_DASH5 + r"BEGIN (?:[A-Z ]+ )?PRIVATE" + r" KEY" + _DASH5),
"a PEM private key block"),
(re.compile("AGE" + "-SECRET" + "-KEY-1"), "an age SECRET (decryption) key"),
(re.compile(r"^\s*client" + r"-key-data\s*:\s*\S"), "kubeconfig client-key-data"),
]
for rel in FILES:
p = ROOT / rel
if not p.is_file() or p.is_symlink():
continue
try:
text = p.read_text(errors="replace")
except Exception:
continue
if text.startswith(VAULT_HEADER):
continue
for rx, what in PRIVKEY_PATTERNS:
for lineno, line in enumerate(text.splitlines(), 1):
if line.lstrip().startswith("#"):
continue
if rx.search(line):
check(False, f"{rel}:{lineno}: tracked file contains {what} (R-SEC-01)")
# ----------------------------------------- (6) decryption is delegated to the cluster
sops_decryption = []
for rel in YAML_FILES:
for d in docs_of(rel):
if d.get("kind") != "Kustomization":
continue
dec = d.get("spec", {}).get("decryption") if isinstance(d.get("spec"), dict) else None
if isinstance(dec, dict) and dec.get("provider") == "sops" and dec.get("secretRef"):
sops_decryption.append(rel)
# Flux overlay patches carry the same block inside a patch body.
for rel in YAML_FILES:
text = (ROOT / rel).read_text(errors="replace")
if re.search(r"^\s*decryption:\s*$", text, re.M) and re.search(r"^\s*provider:\s*sops\s*$", text, re.M) \
and re.search(r"^\s*secretRef:\s*$", text, re.M):
sops_decryption.append(rel)
check(len(sops_decryption) > 0,
"no Flux Kustomization declares `decryption: {provider: sops, secretRef: ...}` — nothing "
"proves decryption happens in-cluster rather than from committed key material (R-SEC-01)")
# ------------------------------------------- (7) the L0 policy gate is real and strict
policy_files = [f for f in FILES if f.endswith(f"{POLICY_NAME}.yaml")]
check(len(policy_files) == 1, f"expected exactly one {POLICY_NAME} ClusterPolicy, found {policy_files}")
for rel in policy_files:
d = (docs_of(rel) or [{}])[0]
action = str(d.get("spec", {}).get("validationFailureAction", ""))
check(action == "Enforce" or action.startswith("${"),
f"{rel}: validationFailureAction is {action!r} — the no-plaintext-secrets gate must "
"Enforce (or carry the Flux token that substitutes to Enforce)")
# --------------------------------------------------------------------------- verdict
if FAILURES:
print(f"CHECK FAIL (no-plaintext-secrets): {len(FAILURES)} of {CHECKS} assertions failed",
file=sys.stderr)
for f in FAILURES:
print(f" - {f}", file=sys.stderr)
raise SystemExit(1)
print(f"check OK: no-plaintext-secrets — {CHECKS} assertions over {len(FILES)} tracked files, "
f"{secrets_seen} committed Secret document(s), {len(vault_files)} ansible vault file(s)")
PY
conformance/tests/l0-static/spec-immutability-enforced/chainsaw-test.yaml
+46
View File
@@ -0,0 +1,46 @@
# =============================================================================
# L0 STATIC — the specification is immutable to agents (R-TEST-06)
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
#
# A governance rule is made testable by testing its ENFORCEMENT MECHANISM, not by
# observing that no agent misbehaved today.
#
# GIVEN the enforcement mechanism (ci/spec-immutability.sh + ci/SPEC_PIN),
# WHEN it is driven against hermetic THROWAWAY spec repositories in each state,
# THEN a clean spec at the pinned SHA passes; an uncommitted edit under docs/ or
# decisions/, an untracked spec addition, and an unadopted committed drift each
# exit 1 and block the merge; an absent spec repo exits 3 (honest skip, never
# green); an unpinned SPEC_PIN fails closed; and the merge gate runs the stage
# and blocks on its failure.
#
# THE REAL SPEC REPO IS NEVER TOUCHED: every case runs against a git repo created under
# mktemp with SPEC_REPO pointed at it and a copy of ci/ carrying a fixture SPEC_PIN.
#
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script; conformance/run.sh
# --layer l0 and ci/l0.sh run it directly, and this Chainsaw Test wraps the SAME script.
# NOT proven (named in check.sh's header): that the Gitea server REQUIRES this status
# check before merge — branch protection is operator configuration outside the repo.
# =============================================================================
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
name: spec-immutability-enforced
labels:
conformance.home.k3s/R-TEST-06: "true"
annotations:
conformance.home.k3s/requirements: "R-TEST-06"
conformance.home.k3s/layer: "L0"
conformance.home.k3s/safety-critical: "false"
conformance.home.k3s/executable: "true"
conformance.home.k3s/intent: >-
A spec modification is detected and blocks the merge; an unverifiable spec skips
honestly rather than passing; and the gate wires that stage as blocking.
spec:
steps:
- name: drive-the-spec-immutability-gate
description: >-
Run the real ci/spec-immutability.sh against throwaway spec repos in every
state and assert its exit code, then assert the gate wires it as blocking.
try:
- script:
content: bash ./check.sh
conformance/tests/l0-static/spec-immutability-enforced/check.sh
Executable
+165
View File
@@ -0,0 +1,165 @@
#!/usr/bin/env bash
# =============================================================================
# L0 static check (offline, no cluster) — spec-immutability-enforced
# Serves: R-TEST-06 ("AI agents MUST NOT modify the specifications (this docs/ tree,
# the requirements, or accepted ADRs). Specs are changed only by the human
# operator/architect." — spec docs/00-vision-and-scope.md.)
#
# HOW A GOVERNANCE RULE IS MADE TESTABLE
# -------------------------------------
# "An agent must not do X" cannot be proven by observing that no agent did X today.
# What CAN be proven — and is what actually protects the spec — is that the MECHANISM
# which detects and blocks a spec modification works, and is wired into the merge gate
# as a blocking stage. So this test drives the real `ci/spec-immutability.sh` against
# HERMETIC THROWAWAY spec repositories and asserts its exit-code behaviour.
#
# THE REAL SPEC REPO IS NEVER TOUCHED. Every case runs against a fixture git repo under
# mktemp, with SPEC_REPO pointed at it and a COPY of ci/ (so the fixture's SPEC_PIN can
# be set without editing the tracked one). Nothing here writes outside its temp dir —
# which is itself the R-TEST-06 discipline this test exists to defend.
#
# GIVEN the enforcement mechanism (ci/spec-immutability.sh + ci/SPEC_PIN),
# WHEN it is run against a spec repo in each state,
# THEN (1) clean and at the pinned SHA -> exit 0 (pass)
# (2) an UNCOMMITTED edit under docs/ -> exit 1 (blocks the merge)
# (3) an uncommitted edit under decisions/ -> exit 1 (accepted ADRs are spec too)
# (4) a NEW COMMIT (HEAD drifted from pin) -> exit 1 (an "adopted" change must be
# an explicit reviewed SPEC_PIN bump)
# (5) an untracked file added to the spec -> exit 1 (porcelain catches additions)
# (6) the spec repo absent -> exit 3 (honest SKIP, never green)
# (7) an empty/comment-only SPEC_PIN -> exit 1 (an unpinned gate is no gate)
# AND (8) the tracked ci/SPEC_PIN really is a 40-hex commit SHA, and
# (9) the gate wires spec-immutability as a stage whose FAIL sets the merge block —
# i.e. the mechanism above is load-bearing, not dead code.
#
# WHAT THIS DOES *NOT* PROVE (honest gap):
# - That the Gitea server actually REQUIRES this status check before merge. Branch
# protection is operator configuration outside the repo; nothing in-repo can assert it.
# - Intent: an agent could still open a spec PR. The rule is enforced by the gate
# blocking it, which is exactly what (1)-(9) assert; it is not enforced by prediction.
# =============================================================================
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
CI_SRC="${ROOT}/ci"
PASS=0
FAILED=0
note() { printf ' ok %s\n' "$*"; PASS=$((PASS + 1)); }
bad() { printf ' FAIL %s\n' "$*" >&2; FAILED=$((FAILED + 1)); }
WORK="$(mktemp -d)"
trap 'rm -rf "${WORK}"' EXIT
# --- a hermetic stand-in for the spec repo -----------------------------------------
# Mirrors the real spec's shape: a docs/ tree and a decisions/ tree, one commit.
make_spec_repo() {
local d="$1"
mkdir -p "${d}/docs" "${d}/decisions"
# Fixture prose deliberately carries NO R-* token: ci/touched-requirements.sh scans the
# non-comment lines of changed files, so an example ID here would be attributed to the
# change as a touched requirement that does not exist in the registry.
printf 'normative spec text (fixture)\n' > "${d}/docs/00-vision-and-scope.md"
printf 'accepted decision\n' > "${d}/decisions/adr-0001-example.md"
git -C "$d" init -q -b main
git -C "$d" config user.email [email protected]
git -C "$d" config user.name spec
git -C "$d" config commit.gpgsign false # throwaway: never sign, ignore host gpg config
git -C "$d" add -A
git -C "$d" commit -qm "spec baseline"
}
# A COPY of ci/ whose SPEC_PIN we may set freely. The script resolves CI_DIR from its own
# location and sources lib.sh + versions.env from there, so a copy is self-contained.
make_ci_copy() {
local dst="$1" pin="$2"
mkdir -p "$dst"
cp "${CI_SRC}/spec-immutability.sh" "${CI_SRC}/lib.sh" "${CI_SRC}/versions.env" "$dst/"
printf '%s\n' "$pin" > "${dst}/SPEC_PIN"
chmod +x "${dst}/spec-immutability.sh"
}
# run_case <expected-rc> <description> <spec-repo|-> <pin>
run_case() {
local want="$1" desc="$2" spec="$3" pin="$4" rc=0
local cidir; cidir="$(mktemp -d -p "$WORK" ci.XXXXXX)"
make_ci_copy "$cidir" "$pin"
SPEC_REPO="$spec" REPO_ROOT="$WORK" "${cidir}/spec-immutability.sh" >/dev/null 2>&1 || rc=$?
if [ "$rc" = "$want" ]; then note "${desc} -> exit ${rc}"; else
bad "${desc}: expected exit ${want}, got ${rc}"; fi
}
for f in spec-immutability.sh lib.sh versions.env SPEC_PIN; do
if [ -f "${CI_SRC}/${f}" ]; then note "ci/${f} present"; else bad "ci/${f} is missing"; fi
done
SPEC="${WORK}/spec"
make_spec_repo "$SPEC"
PINNED="$(git -C "$SPEC" rev-parse HEAD)"
# (1) clean + at the pin -> pass.
run_case 0 "clean spec at the pinned SHA passes" "$SPEC" "$PINNED"
# (2) an uncommitted edit under docs/ -> block.
printf 'normative spec text REWRITTEN BY AN AGENT\n' > "${SPEC}/docs/00-vision-and-scope.md"
run_case 1 "uncommitted edit under docs/ BLOCKS" "$SPEC" "$PINNED"
git -C "$SPEC" checkout -q -- docs/00-vision-and-scope.md
# (3) an uncommitted edit to an accepted ADR -> block (decisions/ is spec too).
printf 'reversed decision\n' > "${SPEC}/decisions/adr-0001-example.md"
run_case 1 "uncommitted edit under decisions/ BLOCKS" "$SPEC" "$PINNED"
git -C "$SPEC" checkout -q -- decisions/adr-0001-example.md
# (5) an untracked addition to the spec tree -> block.
printf 'a spec doc an agent invented\n' > "${SPEC}/docs/99-invented.md"
run_case 1 "untracked file added to the spec BLOCKS" "$SPEC" "$PINNED"
rm -f "${SPEC}/docs/99-invented.md"
# clean again -> still passes (proves cases 2/3/5 failed for the edit, not permanently).
run_case 0 "spec restored to clean passes again" "$SPEC" "$PINNED"
# (4) a committed spec change that the pin has not adopted -> block.
printf 'an extra normative line added by an agent\n' >> "${SPEC}/docs/00-vision-and-scope.md"
git -C "$SPEC" add -A
git -C "$SPEC" commit -qm "agent edits the spec"
DRIFTED="$(git -C "$SPEC" rev-parse HEAD)"
run_case 1 "committed spec drift from the pinned SHA BLOCKS" "$SPEC" "$PINNED"
# ...and adopting it EXPLICITLY (a reviewed SPEC_PIN bump) is the only way to green.
run_case 0 "an explicit SPEC_PIN bump adopts the advance" "$SPEC" "$DRIFTED"
# (6) spec repo unavailable -> honest SKIP (exit 3), never a pass.
run_case 3 "absent spec repo SKIPS (exit 3), never green" "${WORK}/no-such-spec" "$PINNED"
# (7) an unpinned gate is not a gate.
run_case 1 "empty SPEC_PIN FAILS closed" "$SPEC" "# only a comment, no SHA"
# (8) the tracked pin is a real commit SHA.
REAL_PIN="$(grep -vE '^\s*#' "${CI_SRC}/SPEC_PIN" | tr -d '[:space:]')"
if printf '%s' "$REAL_PIN" | grep -Eq '^[0-9a-f]{40}$'; then
note "ci/SPEC_PIN is a 40-hex commit SHA"
else
bad "ci/SPEC_PIN is not a 40-hex commit SHA: '${REAL_PIN}'"
fi
# (9) the mechanism is load-bearing: the merge gate runs it as a stage AND a FAIL there
# sets the block. Asserted against the gate source because running the whole gate
# from inside a conformance check would recurse into this very check.
GATE="${CI_SRC}/gate.sh"
if grep -Eq '^[[:space:]]*run_stage[[:space:]]+spec-immutability' "$GATE"; then
note "ci/gate.sh runs spec-immutability as a gate stage"
else
bad "ci/gate.sh does not run spec-immutability as a stage — the gate is not wired"
fi
if awk '/OUTCOME\[spec-immutability\]/,/^esac/' "$GATE" | grep -Eq '^[[:space:]]*fail\).*BLOCK=1'; then
note "ci/gate.sh blocks the merge when spec-immutability FAILS"
else
bad "ci/gate.sh does not set the merge block on a spec-immutability failure"
fi
echo
if [ "$FAILED" -eq 0 ]; then
echo "check OK: spec-immutability-enforced — ${PASS} assertions (R-TEST-06)"
exit 0
fi
echo "CHECK FAIL (spec-immutability-enforced): ${FAILED} failed, ${PASS} passed" >&2
exit 1