jakman85andClaude Opus 5
cb1916f092
Coverage 45/51 -> 48/51, gaps 6 -> 3 (R-C2-01, R-C2-02, R-STATE-04), 0 safety-critical. --strict-coverage exit 0; run.sh --layer l0 11/11 PASS. R-PLAT-01 — split L0 + L2, because "3 nodes with production-grade HA control plane" is half authored and half observable. l0-static/ha-control-plane-as-code (92 assertions): environments DERIVED from the R-ENV-02 seam (never a hardcoded group name); k3s installed as `server` with no --datastore-endpoint/--disable-etcd anywhere; exactly ONE --cluster-init founder per environment; odd, quorum-capable server count with tolerance >= 1; tls-sans carry the VIP and every server IP; kube-vip leader-elected with cp_enable/svc_enable parsed from the template's OWN env pairs; each VIP outside its environment's LB pool and not a node IP. l2-behavioral/ha-control-plane: control-plane AND etcd role on every node, every node a REGISTERED etcd member, exactly 3 when the cluster says CLUSTER_ENV=prod, and an in-cluster Job curling /readyz through the VIP AND every server's own address — which proves the VIP fronts the API and that every server runs an apiserver, not one founder plus two joiners. An earlier draft probed those endpoints FROM THE RUNNER: that passes from prod's LAN and fails against dev (nodes behind the pfSense NAT), making the test a property of where it was launched. Hence the in-cluster Job. R-NET-04 — what it adds over R-NET-05/07 is the PATH SET and the totality/disjointness of the selecting rules, not another per-service check. l0-static/egress-paths-selected-by-rules (68 assertions): exactly two paths, the same closed set in schema + Kyverno policy + renderer; EXACTLY ONE CiliumEgressGatewayPolicy with a single selector matching only egress-class=vpn; nothing selects `internet` (absence of a rule IS the path); RFC1918 containment computed with `ipaddress` plus every address each overlay assigns itself; the tunnel default route and blackhole confined to table 51820 behind an ip rule that must carry a from/fwmark/iif selector; and the nft chain `policy accept` with every drop qualified — the internet-fails-OPEN direction that R-NET-06's L3 test does not assert. R-SEC-03 — refuses the tautology "a test exists, therefore it is auditable". l0-static/auditability-proofs-producible (78 assertions) drives the real machinery: each of docs/07's five proofs must have an EXECUTABLE test at the layer the spec's wording demands (fault injection and DR drill => L3), taken from status.py's own verdict; run.sh --requirements must resolve and pass the gate's --require assertion; each proof declares a sentinel record_run.py can attribute, computed with record_run.parse_sentinels itself; and TWO FORGERIES are mounted against the real tools — a green JUnit whose log holds only a SKIP token, and a ledger row claiming `proven` at the root commit — which must come back not-proven and stale-proof. The five proofs are parsed from docs/07 when the spec is mounted, not hand-listed. *** A REAL DEFECT FOUND AND FIXED, not asserted around. *** l2-behavioral's default-deny, trusted-tls and gateway-ingress-reachable declared NO pass sentinel, so record_run.py could never attribute a pass to them — the isolation and TLS audit proofs were STRUCTURALLY UNRECORDABLE (landing as unknown-no-sentinel-declared, and status.py requires all of a requirement's tests to be proven). Each now ends with an additive terminal sentinel step reached only when every prior step passed. Verified: all three carry it. TWO HOLLOW DRAFT ASSERTIONS CAUGHT BY THE AGENT ITSELF, both the failure mode this repo keeps hitting: (a) `svc_enable=false` matched a COMMENT in the kube-vip role defaults, so flipping the template to true still passed — now parses the DaemonSet template's env pairs; (b) the nft scoping check searched for the VALUE of a Jinja variable while the template carries its NAME, matching nothing and passing vacuously — templates are now rendered against role defaults first. 30 perturbations across the three checks, all caught, plus 16 synthetic L2 perturbations via a fake-kubectl shim. R-C2-01/02 judged IMPLEMENTATION gaps, so no test was written: Homepage's widgets.yaml carries only datetime and search, kubernetes.yaml is empty, automountServiceAccountToken is false, and its NetworkPolicy allows DNS egress only — no health source is even reachable. It has already drifted (crafty authors a route with no tile). A test today would go red on both; it must ship WITH the config fix. VERIFIED: full CI gate PASS. Neither cluster mutated. Refs R-PLAT-01, R-NET-04, R-SEC-03, R-TEST-01..05. Co-Authored-By: Claude Opus 5 <[email protected]>
113 lines
6.1 KiB
YAML
113 lines
6.1 KiB
YAML
# =============================================================================
|
|
# L2 BEHAVIORAL — Trusted TLS (R-TLS-01, R-TLS-03) [RUNNABLE]
|
|
# Catalog: docs/14 "Trusted TLS (R-TLS-01/02)"
|
|
# Spec: docs/09 (Gateway API + DNS-01 wildcard), ADR-0006 (LE issuers)
|
|
#
|
|
# ENVIRONMENT-AGNOSTIC (R-ENV-02): the hostnames are built from the ${BASE_DOMAIN} of the CLUSTER
|
|
# UNDER TEST, read from flux-system/cluster-vars (the ConfigMap Flux substitutes from) by a
|
|
# test-level chainsaw binding. The same authored test therefore verifies dev's wildcard on dev and
|
|
# prod's on prod, with nothing to pass at run time.
|
|
#
|
|
# GIVEN the shared Gateway terminating TLS with the cert-manager wildcard cert,
|
|
# WHEN an in-cluster probe dials it by hostname (SNI *.${BASE_DOMAIN}) and
|
|
# verifies the chain against the SYSTEM trust store,
|
|
# THEN the served cert is non-self-signed, unexpired, covers the wildcard SAN, and
|
|
# chains to a PUBLICLY-TRUSTED CA root (R-TLS-01). The wildcard SAN also covers
|
|
# internal-only hostnames, so internal services get the same trusted cert (R-TLS-03).
|
|
#
|
|
# TRUST MODEL: the cluster issues from Let's Encrypt **PRODUCTION** (ACME_CLUSTER_ISSUER
|
|
# = letsencrypt-prod), whose root (ISRG Root X1) is in the system trust store — so the
|
|
# probe verifies against the container's DEFAULT CA bundle (no --cacert). A clean verify
|
|
# is exactly what real in-cluster OIDC clients (oauth2-proxy, Grafana) depend on. The
|
|
# probe resolves the hostname via in-cluster split-horizon DNS (coredns-custom → Gateway
|
|
# VIP, R-DNS-01) — the same path real clients use — so no --connect-to hack is needed.
|
|
# Replaced the earlier LE-staging + embedded-root check when dev switched to LE-prod so
|
|
# in-cluster SSO clients trust the cert (DECISIONS-LOG Session 7).
|
|
# =============================================================================
|
|
apiVersion: chainsaw.kyverno.io/v1alpha1
|
|
kind: Test
|
|
metadata:
|
|
name: trusted-tls
|
|
labels:
|
|
conformance.home.k3s/R-TLS-01: "true"
|
|
conformance.home.k3s/R-TLS-03: "true"
|
|
annotations:
|
|
conformance.home.k3s/requirements: "R-TLS-01, R-TLS-03"
|
|
conformance.home.k3s/layer: "L2"
|
|
conformance.home.k3s/safety-critical: "false"
|
|
conformance.home.k3s/executable: "true"
|
|
conformance.home.k3s/intent: >-
|
|
The shared Gateway serves a valid, unexpired, non-self-signed wildcard cert
|
|
that chains to a publicly-trusted CA root (LE production); the wildcard SAN
|
|
covers internal-only hostnames too.
|
|
spec:
|
|
# -- R-ENV-02: the hostname suffix is READ FROM THE CLUSTER UNDER TEST (flux-system/cluster-vars
|
|
# — the ConfigMap Flux substitutes ${BASE_DOMAIN} from), never authored per environment.
|
|
bindings:
|
|
- name: baseDomain
|
|
value: (x_k8s_get($client, 'v1', 'ConfigMap', 'flux-system', 'cluster-vars').data.BASE_DOMAIN)
|
|
steps:
|
|
- name: then-gateway-cert-chains-to-trusted-root
|
|
try:
|
|
- apply:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata:
|
|
name: tls-probe
|
|
spec:
|
|
backoffLimit: 4
|
|
activeDeadlineSeconds: 180
|
|
template:
|
|
spec:
|
|
restartPolicy: Never
|
|
containers:
|
|
- name: probe
|
|
image: curlimages/curl:8.11.1
|
|
env:
|
|
- name: BASE_DOMAIN
|
|
value: ($baseDomain)
|
|
command:
|
|
- sh
|
|
- -c
|
|
- |
|
|
set -eu
|
|
# Probe TWO wildcard hostnames — one "public", one "internal-only". Both
|
|
# match *.${BASE_DOMAIN}, so a clean verify against the SYSTEM CA
|
|
# bundle for BOTH proves R-TLS-01 (chains to a publicly-trusted root) AND
|
|
# R-TLS-03 (the same trusted wildcard covers internal-only names). Names
|
|
# resolve via in-cluster split-horizon DNS to the Gateway VIP. curl (no
|
|
# --cacert, no -k) validates against the default CA bundle and exits 0 on
|
|
# the gateway's 404 once TLS+hostname verify; it exits 60 if the chain or
|
|
# hostname fails. TLS terminates at the gateway before HTTP routing, so no
|
|
# backend is needed.
|
|
for host in "conformance.${BASE_DOMAIN}" "internal-only.${BASE_DOMAIN}"; do
|
|
curl -sS --retry 5 --retry-connrefused --retry-delay 3 --max-time 20 \
|
|
-o /dev/null "https://${host}/"
|
|
echo "TLS_TRUSTED ${host}"
|
|
done
|
|
echo ALL_TRUSTED
|
|
- assert:
|
|
resource:
|
|
apiVersion: batch/v1
|
|
kind: Job
|
|
metadata: { name: tls-probe }
|
|
status: { succeeded: 1 }
|
|
# -----------------------------------------------------------------------------------
|
|
# PROOF-OF-PASS SENTINEL (R-SEC-03). Every step above is an assert/Job-status gate, so
|
|
# chainsaw already fails the test before reaching this step if any of them fails — but a
|
|
# green Chainsaw JUnit carries NO stdout, and the L2/L3 tests in this suite can self-skip
|
|
# GREEN, so lib/record_run.py refuses to treat a bare `passed` as evidence and records
|
|
# `unknown-no-sentinel-declared` instead. Without a declared token this test could pass
|
|
# forever and NEVER become a recorded proof of R-TLS-01/03 — the TLS half of docs/07's audit proof list
|
|
# — which is precisely what docs/07 asks for ("These proofs are part of the platform
|
|
# acceptance suite, not one-time checks"). This step exists only to emit that token, and it
|
|
# is reached only when everything above has passed. Asserted by
|
|
# tests/l0-static/auditability-proofs-producible.
|
|
- name: then-emit-the-proof-of-pass-sentinel
|
|
try:
|
|
- script:
|
|
content: echo TLS_TRUSTED_PROVEN_OK
|
|
check:
|
|
(contains($stdout, 'TLS_TRUSTED_PROVEN_OK')): true
|