jakman85andClaude Opus 5
6e77150ae1
Unblocks the enforced Gitea gate, which correctly BLOCKED this branch on ['R-SEC-01','R-SEC-02','R-TEST-06'] being touched but uncovered. Tests were authored; the coverage rule was NOT narrowed. === THE FALSE GREEN (root cause) === `make -C ci gate` locally reported PASS having checked NOTHING. resolve_base() validated nothing, and EVERY way of failing to find a base degraded silently to an empty touched set + exit 0: an `HEAD~1` last resort, a two-dot diff fallback masking a failing three-dot diff, and no check that HEAD is even AHEAD of the base. Run while sitting on `main` (main == origin/main), `origin/main...HEAD` is empty -> "no changed files vs base" -> the empty set means run.sh is invoked with NO --requirements -> coverage.py's --require gate never runs at all. Green by vacuum. Now: candidates are BASE_REF > GITHUB_BASE_REF > origin/main > main, each verified to resolve. HEAD~1 and the two-dot fallback are gone. Unresolvable base / no merge base / failed diff / HEAD-not-ahead -> loud banner, EMPTY stdout, exit 3, and a machine-readable reason (on-base | no-base | diff-failed). gate.sh turns exit 3 into an UNDECIDED decision: `on-base` -> "NOT APPLICABLE — no branch under gate" (exit 0 so push-to-main runs don't break, but it never reads as PASS); `no-base`/`diff-failed` -> BLOCKED. It also now prints the diff base it used. A stale local main warns (widens, never falsely greens). Also fixed `make -C ci test`, which was SWALLOWING failing suites. Regression: ci/tests/touched-requirements-base.test.sh — 31 assertions over 8 throwaway-repo scenarios including the exact false-green case, an already-merged branch, no base ref, a single-commit repo, bogus BASE_REF, a shallow GITHUB_BASE_REF, and origin/main-preferred-over-stale-main. === NEW L0 STATIC LAYER (conformance/tests/l0-static/) === Where docs/14 actually puts these invariants. Each check.sh really executes offline via `run.sh --layer l0` and is wired into ci/l0.sh, so it runs on every PR. No scaffolds. R-SEC-01 no-plaintext-secrets (165 assertions / 445 tracked files): .sops.yaml holds only PUBLIC age recipients; every committed Secret is SOPS-wrapped with a mac and ENC[...] values; ansible vault.yml files are $ANSIBLE_VAULT; no secret-valued ansible key carries a literal outside a vault; no PEM key / age secret key / kubeconfig client-key-data is tracked; Flux declares sops decryption; the disallow-plaintext-secrets policy Enforces and has a proving `result: fail` row. Exemptions are SELF-POLICING, not path allowlists — a .example must be referenced by no kustomization and hold only placeholders, and the Kyverno fixture is exempt ONLY while a test asserts the policy rejects it. Proven against 7 injected violations, including deleting the negative row and weakening Enforce->Audit. R-SEC-02 least-privilege-rbac (41 assertions): no wildcard apiGroup/resource/verb and no escalate|bind|impersonate in any in-repo Role/ClusterRole; no ServiceAccount bound to cluster-admin or a wildcard role; the conformance runner's own test identity is scoped and holds no rights over RBAC objects; the default-deny/host-namespace/egress-class policies Enforce and each carry a negative row. Proven against 6 injected violations. R-TEST-06 spec-immutability-enforced (16 assertions): drives the REAL ci/spec-immutability.sh against HERMETIC throwaway spec repos and asserts its exit code in every state — clean-at-pin 0, edits under docs/ or decisions/ 1, untracked addition 1, unadopted drift 1, explicit pin bump 0, ABSENT spec repo 3 (honest skip, never green), unpinned SPEC_PIN 1 — plus that gate.sh sets BLOCK=1 on its failure, so the mechanism is load-bearing. The real spec repo is never written to. Each check documents what it does NOT cover (runtime decryption, git history, runtime least-privilege, branch protection) rather than overclaiming. status.py now reads check.sh too, so a placeholder L0 body still outranks an `executable: true` annotation. Coverage 37/51 -> 40/51 (11 gaps); executable tests 32 -> 35; scaffold-only stays 5 — no sixth scaffold was added. Serves: R-SEC-01, R-SEC-02, R-TEST-01..06. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
125 lines
7.3 KiB
Markdown
125 lines
7.3 KiB
Markdown
# `ci/` — portable merge gate (runnable today, without Gitea)
|
|
|
|
**Task T-0.2 · R-TEST-02 (reproducible, single-command) · Owner WS-GITOPS (CI).**
|
|
|
|
The same merge gate the Gitea Actions workflow runs
|
|
([`.gitea/workflows/`](../.gitea/workflows/README.md)), as plain scripts + a Makefile so the gates are
|
|
runnable **now**, before any Gitea host or cluster exists. CI and a laptop call the *identical*
|
|
scripts, so `make -C ci gate` == the CI merge gate.
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
make -C ci gate # the single command: run every stage + the R-TEST-03 decision
|
|
make -C ci help # list targets
|
|
make -C ci tools # which pinned tools are installed locally
|
|
make -C ci touched # print the touched-requirement subset for this branch vs main
|
|
```
|
|
|
|
Individual stages:
|
|
|
|
```bash
|
|
make -C ci lint # yaml/lint (no cluster; live now)
|
|
make -C ci l0 # schema + Kyverno + L0 static conformance (no cluster; live now)
|
|
make -C ci l1 # L1 rendering checks (no cluster; live now)
|
|
make -C ci conformance # L2/L3 in-cluster Chainsaw Job (SKIPS unless CLUSTER_AVAILABLE=true)
|
|
make -C ci test # unit tests for these scripts themselves (ci/tests/*.test.sh)
|
|
```
|
|
|
|
The `l0` stage has three sub-stages: ServiceDefinition schema validation, the Kyverno policy gate
|
|
(`kyverno test`), and **`conformance/run.sh --layer l0`** — the whole-repo L0 invariants a
|
|
per-resource policy cannot express (secret hygiene across every tracked file, least-privilege RBAC,
|
|
spec-immutability enforcement). See `conformance/tests/README.md`.
|
|
|
|
## Exit-code contract
|
|
|
|
Every stage script uses: **`0` = pass · `1` = fail (blocks merge) · `3` = SKIPPED** (cluster-gated
|
|
with no cluster, or an unwired seam / missing tool). A `3` is **never** counted as green — that is how
|
|
the gate stays honest in Phase 0 (no fake green).
|
|
|
|
### The touched-requirement subset can also be *undeterminable* (exit 3)
|
|
|
|
`touched-requirements.sh` is the input to the R-TEST-03 decision. It resolves a diff base in this
|
|
order, **verifying each candidate exists** before using it:
|
|
|
|
```
|
|
BASE_REF (explicit) > GITHUB_BASE_REF (Actions PR runs) > origin/main > main
|
|
```
|
|
|
|
If no candidate resolves, if there is no merge base, if the diff command fails, or if **HEAD is not
|
|
ahead of the base** (you are sitting on `main`, or the branch is already merged), it prints a loud
|
|
banner, writes **nothing** to stdout, and exits **3** — plus a machine-readable reason code
|
|
(`on-base` / `no-base` / `diff-failed`) to `$TOUCHED_REASON_FILE` when that variable names a path.
|
|
|
|
**Why this is not cosmetic.** Every one of those conditions used to degrade *silently* to an empty
|
|
touched set and exit 0. An empty set means `conformance/run.sh` is invoked with no `--requirements`,
|
|
which means its coverage `--require` gate never runs — so on 2026-08-31 the local gate printed
|
|
`MERGE GATE: PASS` having checked nothing, while CI (correct base, 17 touched requirements) BLOCKED
|
|
on real coverage debt. There is no silent narrowing left: the `HEAD~1` last resort and the two-dot
|
|
diff fallback are both gone. Locked down by `ci/tests/touched-requirements-base.test.sh`.
|
|
|
|
`gate.sh` turns exit 3 into an **UNDECIDED** merge decision, never a pass:
|
|
|
|
| reason | headline | exit |
|
|
|---|---|---|
|
|
| `on-base` | `MERGE GATE: NOT APPLICABLE — no branch under gate` | 0 (a push-to-`main` CI run legitimately lands here; it just must not read as "my branch is green") |
|
|
| `no-base`, `diff-failed` | `MERGE GATE: BLOCKED` | 1 (the environment cannot support a merge decision; refusing to decide must not look like deciding yes) |
|
|
|
|
A stage that needs the subset (`l0.sh`, `l1.sh`) still runs its FULL layer as a superset sanity net
|
|
in that case — nothing goes untested — and says plainly that the *scoped* check did not happen.
|
|
|
|
## Reproducibility (R-TEST-02)
|
|
|
|
- **Version-pinned:** all tool versions live in [`versions.env`](versions.env) — the single source of
|
|
truth shared by the local scripts and the CI installer. Bumping a version is a normal
|
|
feature-branch → main change.
|
|
- **Single command, environment-agnostic:** `make -C ci gate` on any machine with the pinned tools.
|
|
- **Deterministic:** no cluster state is assumed; L2/L3 SKIP unless a cluster is explicitly declared.
|
|
|
|
## Files
|
|
|
|
```
|
|
ci/
|
|
Makefile targets: lint · l0 · l1 · conformance · gate · touched · test · tools · versions
|
|
versions.env PINNED tool versions (R-TEST-02) — single source of truth
|
|
lib.sh shared helpers: logging, tool checks, cluster detection, artifact paths
|
|
yamllint.yaml lint config for the yaml/lint stage
|
|
lint.sh Stage: yaml/lint
|
|
l0.sh Stage: schema validation + Kyverno L0 gate + L0 static conformance
|
|
l1.sh Stage: L1 rendering checks (Chainsaw L1 via the conformance runner)
|
|
conformance.sh Stage: L2/L3 in-cluster Chainsaw Job (trigger + collect)
|
|
touched-requirements.sh compute the R-* subset a change touches (drives the R-TEST-03 filter);
|
|
exits 3 LOUDLY rather than guessing when the diff base is unreliable
|
|
gate.sh run all stages + the merge decision (the single command)
|
|
tests/*.test.sh unit tests for these scripts (`make -C ci test`), run against
|
|
throwaway git repos via the REPO_ROOT override — no network, no cluster
|
|
README.md this file
|
|
```
|
|
|
|
## Integration seams — entrypoints owned by OTHER agents
|
|
|
|
This gate is an **orchestrator**. It does not implement schema validation, Kyverno policies, or the
|
|
conformance runner — those are other workstreams' deliverables. Each seam is **env-overridable** so
|
|
the owning agent / PM can reconcile the real entrypoint **without editing these scripts**. Defaults
|
|
below are this workstream's **assumptions** — flagged for the PM to verify at integration.
|
|
|
|
| Seam | Owner / task | Env override | Assumed default | Assumed interface |
|
|
|---|---|---|---|---|
|
|
| ServiceDefinition schema validator | WS-GITOPS · **T-0.3** (`schemas/`) | `SCHEMA_VALIDATE_CMD` | `schemas/validate.sh` | `validate.sh catalog/*.yaml` → exit 0/1; `--version` supported |
|
|
| Kyverno L0 policies + tests | WS-TEST · **T-0.5** (`platform/kyverno/`) | `KYVERNO_POLICY_DIR` | `platform/kyverno` | contains `kyverno-test.yaml` files discoverable by `kyverno test <dir>` |
|
|
| Conformance runner (Chainsaw) | WS-TEST · **T-0.4** (`conformance/`) | `CONFORMANCE_CMD` | `conformance/run.sh` | `run.sh --layer <l0\|l1\|l2,l3> [--requirements R-A,R-B]`; artifacts land in `conformance/_artifacts/` |
|
|
| Catalog (ServiceDefinitions) | WS-APPS | `CATALOG_DIR` | `catalog` | `*.yaml` ServiceDefinition files |
|
|
|
|
Artifact locations (git-ignored): JUnit → `results/junit/*.xml`; coverage → `reports/coverage/**`.
|
|
Override with `JUNIT_DIR` / `COVERAGE_DIR` if the runner writes elsewhere.
|
|
|
|
**A seam that is missing or not executable reports SKIPPED (exit 3), not a false pass** — so during
|
|
parallel Phase 0 development the gate is green-honest, and each seam goes live automatically the moment
|
|
its owner delivers a runnable entrypoint.
|
|
|
|
## Cluster gating
|
|
|
|
`CLUSTER_AVAILABLE=true` (env locally, `vars.CLUSTER_AVAILABLE` in Gitea) makes the L2/L3 stage run and
|
|
be **required**; anything else makes it **SKIP** (Phase 0). `REQUIRE_CLUSTER_TESTS=true` (Phase 1+)
|
|
turns any unverified cluster-needing touched requirement into a hard block.
|