Unblocks the enforced Gitea gate, which correctly BLOCKED this branch on ['R-SEC-01','R-SEC-02','R-TEST-06'] being touched but uncovered. Tests were authored; the coverage rule was NOT narrowed. === THE FALSE GREEN (root cause) === `make -C ci gate` locally reported PASS having checked NOTHING. resolve_base() validated nothing, and EVERY way of failing to find a base degraded silently to an empty touched set + exit 0: an `HEAD~1` last resort, a two-dot diff fallback masking a failing three-dot diff, and no check that HEAD is even AHEAD of the base. Run while sitting on `main` (main == origin/main), `origin/main...HEAD` is empty -> "no changed files vs base" -> the empty set means run.sh is invoked with NO --requirements -> coverage.py's --require gate never runs at all. Green by vacuum. Now: candidates are BASE_REF > GITHUB_BASE_REF > origin/main > main, each verified to resolve. HEAD~1 and the two-dot fallback are gone. Unresolvable base / no merge base / failed diff / HEAD-not-ahead -> loud banner, EMPTY stdout, exit 3, and a machine-readable reason (on-base | no-base | diff-failed). gate.sh turns exit 3 into an UNDECIDED decision: `on-base` -> "NOT APPLICABLE — no branch under gate" (exit 0 so push-to-main runs don't break, but it never reads as PASS); `no-base`/`diff-failed` -> BLOCKED. It also now prints the diff base it used. A stale local main warns (widens, never falsely greens). Also fixed `make -C ci test`, which was SWALLOWING failing suites. Regression: ci/tests/touched-requirements-base.test.sh — 31 assertions over 8 throwaway-repo scenarios including the exact false-green case, an already-merged branch, no base ref, a single-commit repo, bogus BASE_REF, a shallow GITHUB_BASE_REF, and origin/main-preferred-over-stale-main. === NEW L0 STATIC LAYER (conformance/tests/l0-static/) === Where docs/14 actually puts these invariants. Each check.sh really executes offline via `run.sh --layer l0` and is wired into ci/l0.sh, so it runs on every PR. No scaffolds. R-SEC-01 no-plaintext-secrets (165 assertions / 445 tracked files): .sops.yaml holds only PUBLIC age recipients; every committed Secret is SOPS-wrapped with a mac and ENC[...] values; ansible vault.yml files are $ANSIBLE_VAULT; no secret-valued ansible key carries a literal outside a vault; no PEM key / age secret key / kubeconfig client-key-data is tracked; Flux declares sops decryption; the disallow-plaintext-secrets policy Enforces and has a proving `result: fail` row. Exemptions are SELF-POLICING, not path allowlists — a .example must be referenced by no kustomization and hold only placeholders, and the Kyverno fixture is exempt ONLY while a test asserts the policy rejects it. Proven against 7 injected violations, including deleting the negative row and weakening Enforce->Audit. R-SEC-02 least-privilege-rbac (41 assertions): no wildcard apiGroup/resource/verb and no escalate|bind|impersonate in any in-repo Role/ClusterRole; no ServiceAccount bound to cluster-admin or a wildcard role; the conformance runner's own test identity is scoped and holds no rights over RBAC objects; the default-deny/host-namespace/egress-class policies Enforce and each carry a negative row. Proven against 6 injected violations. R-TEST-06 spec-immutability-enforced (16 assertions): drives the REAL ci/spec-immutability.sh against HERMETIC throwaway spec repos and asserts its exit code in every state — clean-at-pin 0, edits under docs/ or decisions/ 1, untracked addition 1, unadopted drift 1, explicit pin bump 0, ABSENT spec repo 3 (honest skip, never green), unpinned SPEC_PIN 1 — plus that gate.sh sets BLOCK=1 on its failure, so the mechanism is load-bearing. The real spec repo is never written to. Each check documents what it does NOT cover (runtime decryption, git history, runtime least-privilege, branch protection) rather than overclaiming. status.py now reads check.sh too, so a placeholder L0 body still outranks an `executable: true` annotation. Coverage 37/51 -> 40/51 (11 gaps); executable tests 32 -> 35; scaffold-only stays 5 — no sixth scaffold was added. Serves: R-SEC-01, R-SEC-02, R-TEST-01..06. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
109 lines
5.5 KiB
Bash
Executable File
109 lines
5.5 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# ci/l0.sh — Stage 2+3: ServiceDefinition schema validation + Kyverno L0 policy gate.
|
|
# Both run in CI WITHOUT a cluster (docs/14 §"Where each layer runs") — live now.
|
|
#
|
|
# Stage 2 schema : catalog/*.yaml validated against the ServiceDefinition
|
|
# schema (owner: T-0.3, schemas/).
|
|
# Stage 3 L0 : `kyverno test` over platform/kyverno/ — default-deny baseline
|
|
# present, no plaintext secrets, required labels/egress-class,
|
|
# ServiceDefinition schema conformance (owner: T-0.5, ADR-0011).
|
|
# Stage 3b L0 : conformance/run.sh --layer l0 — the WHOLE-REPO L0 invariants that
|
|
# a per-resource policy cannot express (secret hygiene across every
|
|
# tracked file, least-privilege RBAC, spec-immutability enforcement).
|
|
#
|
|
# Integration seams are env-overridable (see ci/lib.sh + README). When an owning
|
|
# subtree is still a placeholder, this reports SKIPPED (exit 3) for that sub-stage
|
|
# rather than a false pass — the gate then knows the seam is unwired, not green.
|
|
set -euo pipefail
|
|
CI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
# shellcheck disable=SC1091
|
|
source "${CI_DIR}/lib.sh"
|
|
|
|
rc=0
|
|
|
|
# ---- Stage 2: ServiceDefinition schema validation ---------------------------
|
|
log "Stage: ServiceDefinition schema validation (catalog/ against schemas/)"
|
|
if ! entrypoint_ready "$SCHEMA_VALIDATE_CMD"; then
|
|
skip "schema validator not present at '${SCHEMA_VALIDATE_CMD}' (T-0.3 / schemas/ not delivered yet) — UNWIRED, not green"
|
|
summary "- schema: **SKIPPED** (validator entrypoint \`${SCHEMA_VALIDATE_CMD}\` missing — T-0.3)"
|
|
else
|
|
if [ -d "$CATALOG_DIR" ] && compgen -G "${CATALOG_DIR}/*.y*ml" >/dev/null 2>&1; then
|
|
if "$SCHEMA_VALIDATE_CMD" "${CATALOG_DIR}"/*.y*ml; then
|
|
ok "schema validation passed"
|
|
summary "- schema: **passed**"
|
|
else
|
|
err "schema validation failed"; summary "- schema: **FAILED**"; rc=1
|
|
fi
|
|
else
|
|
log "no ServiceDefinitions in ${CATALOG_DIR} yet — validating validator is callable only"
|
|
"$SCHEMA_VALIDATE_CMD" --version >/dev/null 2>&1 || true
|
|
ok "schema validator present; no catalog entries to validate"
|
|
summary "- schema: **passed** (no catalog entries yet)"
|
|
fi
|
|
fi
|
|
|
|
# ---- Stage 3: Kyverno L0 policy gate ----------------------------------------
|
|
log "Stage: Kyverno L0 policy gate (kyverno test ${KYVERNO_POLICY_DIR})"
|
|
# `kyverno test <dir>` discovers kyverno-test.yaml files recursively.
|
|
if ! have kyverno; then
|
|
skip "kyverno CLI not installed (pin v${KYVERNO_VERSION}) — cannot run L0 gate"
|
|
summary "- kyverno-l0: **SKIPPED** (kyverno CLI not installed)"
|
|
# Missing tool is an environment gap, not a code pass -> exit 3 unless already failing.
|
|
[ "$rc" -eq 0 ] && rc=3
|
|
elif [ ! -d "$KYVERNO_POLICY_DIR" ] || ! compgen -G "${KYVERNO_POLICY_DIR}/**/kyverno-test.yaml" >/dev/null 2>&1 \
|
|
&& ! find "$KYVERNO_POLICY_DIR" -name 'kyverno-test.yaml' 2>/dev/null | grep -q .; then
|
|
skip "no kyverno-test.yaml under '${KYVERNO_POLICY_DIR}' (T-0.5 / platform/kyverno not delivered yet) — UNWIRED, not green"
|
|
summary "- kyverno-l0: **SKIPPED** (no policy tests yet — T-0.5)"
|
|
else
|
|
# The ClusterPolicies carry a Flux postBuild token in validationFailureAction
|
|
# (${KYVERNO_VALIDATION_ACTION}, substituted in-cluster from
|
|
# clusters/base/cluster-vars.yaml). The offline `kyverno test` CLI does NOT run
|
|
# Flux substitution, so the literal token is an invalid enum value and those
|
|
# policies fail to load (no result rows -> "Not found"). Render a throwaway copy
|
|
# with the base/prod value (Enforce — the strict posture the gate must prove)
|
|
# before testing. This does not change what the policies enforce in-cluster.
|
|
: "${KYVERNO_VALIDATION_ACTION:=Enforce}"
|
|
l0_render="$(mktemp -d)"
|
|
cp -a "${KYVERNO_POLICY_DIR}/." "$l0_render/"
|
|
find "$l0_render" -type f -name '*.yaml' -print0 \
|
|
| xargs -0 sed -i "s/\${KYVERNO_VALIDATION_ACTION}/${KYVERNO_VALIDATION_ACTION}/g"
|
|
if kyverno test "$l0_render"; then
|
|
ok "Kyverno L0 policy tests passed"
|
|
summary "- kyverno-l0: **passed**"
|
|
else
|
|
err "Kyverno L0 policy tests failed"; summary "- kyverno-l0: **FAILED**"; rc=1
|
|
fi
|
|
rm -rf "$l0_render"
|
|
fi
|
|
|
|
# ---- Stage 3b: L0 STATIC conformance checks ---------------------------------
|
|
# docs/14's L0 row is "manifests/ServiceDefinitions obey invariants", which is wider
|
|
# than what Kyverno alone can express over single resources: whole-repo invariants
|
|
# (no plaintext secrets anywhere in git, least-privilege RBAC across every tracked
|
|
# binding, the spec-immutability gate's own behaviour) live in the conformance
|
|
# harness's tests/l0-static subtree and run with no cluster. This stage executes them
|
|
# through the same single-command runner CI and a laptop both use (R-TEST-02).
|
|
log "Stage: L0 static conformance checks (${CONFORMANCE_CMD} --layer l0)"
|
|
if ! entrypoint_ready "$CONFORMANCE_CMD"; then
|
|
skip "conformance runner not present at '${CONFORMANCE_CMD}' — UNWIRED, not green"
|
|
summary "- l0-static: **SKIPPED** (conformance runner missing)"
|
|
if [ "$rc" -eq 0 ]; then rc=3; fi
|
|
else
|
|
load_touched_requirements
|
|
if "$CONFORMANCE_CMD" --layer l0 ${REQ_ARGS[@]+"${REQ_ARGS[@]}"}; then
|
|
ok "L0 static conformance checks passed"
|
|
summary "- l0-static: **passed**"
|
|
else
|
|
err "L0 static conformance checks failed"
|
|
summary "- l0-static: **FAILED**"
|
|
rc=1
|
|
fi
|
|
fi
|
|
|
|
case "$rc" in
|
|
0) ok "L0 stage complete" ;;
|
|
3) skip "L0 stage incomplete (unwired seam / missing tool) — not a pass" ;;
|
|
*) err "L0 stage failed" ;;
|
|
esac
|
|
exit "$rc"
|