Merge main into feature/vulnerability-visibility

Keeps both ADR index rows (0026 from #3 and 0027 from this branch).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5.5 committed 2026-11-09 12:52:25 -06:00
commit 46996d1f8a
13 files changed
+76 -41

No files matched your search

+11 -6
View File
@@ -17,7 +17,8 @@ Related: [ADR-0010](../decisions/adr-0010-conformance-testing.md).
environment-agnostic. Any conforming cluster + a kubeconfig ⇒ the same pass/fail. No flakiness
budget; eventual-consistency is handled with bounded polling, not sleeps-and-hope.
3. **Conformance gates delivery (R-TEST-03).** The suite blocks merges (for touched requirements)
and blocks phase exits (full phase suite). Green suite is the definition of progress.
and blocks phase exits (full phase suite). Green suite is the definition of progress. Fault
injection is operator-run and does not gate individual merges (ADR-0026).
4. **Traceable (R-TEST-04).** Each test declares its `R-*` IDs; a generated coverage report maps
every requirement to its verifying test(s). Uncovered requirements are visible gaps.
5. **Fault-injection for safety-critical behavior (R-TEST-05).** Default-deny, VPN kill-switch,
@@ -52,7 +53,7 @@ Related: [ADR-0010](../decisions/adr-0010-conformance-testing.md).
| **L0 Static/policy** | Pre-apply, in CI | Manifests/ServiceDefinitions obey invariants | Every namespace has a default-deny baseline; no plaintext secrets; schema-valid ServiceDefinition |
| **L1 Rendering** | CI, no cluster | ServiceDefinition → rendered resources match golden expectations | `egress: vpn` renders the kill-switched egress path |
| **L2 Behavioral/integration** | Against the live cluster | Real runtime behavior | Default-deny probe, dependency allow/deny, egress-IP, TLS validity, DNS, PVC rebuild |
| **L3 Fault-injection** | Against the live cluster | Fail-closed / no-data-loss under failure | VPN tunnel forced down ⇒ no leak; node killed ⇒ data survives; bad update ⇒ auto-rollback; DR restore |
| **L3 Fault-injection** | Against the live cluster, operator-run only | Fail-closed / no-data-loss under failure | VPN tunnel forced down ⇒ no leak; node killed ⇒ data survives; bad update ⇒ auto-rollback; DR restore |
| **L4 Conformance suite** | CI + on-demand | Aggregate, requirement-tagged gate | The union of the above, reported per `R-*` |
## Behavior test catalog (normative — implementers author these as automated tests)
@@ -113,10 +114,13 @@ Each maps to requirement IDs; phrased as Given/When/Then intent.
## Gating rules (R-TEST-03)
- **Per-change (merge gate):** the conformance subset tagged with the requirement IDs the change
touches MUST pass before merge.
touches MUST pass before merge, across the automatable layers (L0–L2).
- **Per-phase (exit gate):** the full conformance suite for the phase's requirements MUST pass
before the phase's exit criteria are considered met ([`../roadmap/phases.md`](../roadmap/phases.md)).
- **Regression:** the growing suite runs continuously against the live cluster; a previously-green
- **Fault injection (L3) is operator-run** ([ADR-0026](../decisions/adr-0026-fault-injection-operator-run.md)):
no pipeline or schedule ever starts it. The operator runs it deliberately and records each result
in the run ledger.
- **Regression:** the automatable layers run continuously; a previously-green
requirement going red blocks further delivery until fixed.
- **Coverage:** a requirement with no passing test is **not** satisfied, regardless of implementation
state.
@@ -176,7 +180,8 @@ pinned for determinism.
- **L0 (Kyverno CLI) and L1 (rendering assertions)** run **in CI without a cluster** — pure manifest/
ServiceDefinition validation; fast merge-gate feedback.
- **L2 behavioral and L3 fault-injection run as an in-cluster Chainsaw Job** on the live cluster.
CI **triggers** the Job (and waits for it); the Job — not CI — holds cluster access, so CI never
CI **triggers** the L2 Job (and waits for it), and only the operator starts an L3 run
(ADR-0026). The Job — not CI — holds cluster access, so CI never
needs a broad kubeconfig. CI holds only a narrow trigger credential, whose reach is bounded by a
fail-closed admission policy on the Job's shape. This keeps credentials in-cluster and
least-privileged (R-SEC-02).
@@ -198,7 +203,7 @@ pinned for determinism.
## Open questions
- Tool mix is **resolved**: Kyverno (L0) + Chainsaw (L1/L2/L3), [ADR-0011](../decisions/adr-0011-test-tooling-kyverno-chainsaw.md).
- Harness location is **resolved**: L2/L3 run as an **in-cluster Chainsaw Job** triggered by CI; CI
- Harness location is **resolved**: L2/L3 run as an **in-cluster Chainsaw Job**, L2 triggered by CI and L3 by the operator; CI
holds no broad cluster credential, only the admission-bounded trigger.
- Kyverno-as-admission-enforcer is **resolved**: yes — the L0 policies also enforce at admission
in-cluster (defense in depth), in addition to the CI gate.