jakman85andClaude Opus 5
fe45d0f5c5
DECISIONS-LOG records Phase 7 being opened knowingly while the uncovered set included R-ENV-01/02/03 — the very requirements prod provisioning serves. Two of those three are now covered, plus R-TLS-02. tls-lifecycle-automated R-TLS-02 L0 (in the merge gate) cert-lifecycle-automated R-TLS-02 L2 (authored, NOT executed) dev-nodes-provisioned-as-code R-ENV-01 L0 prod-baremetal-not-dev-surface R-ENV-03 L0 Coverage 42/51 (82.4%) -> 45/51 (88.2%), 6 gaps, 0 safety-critical. Remaining: R-C2-01, R-C2-02, R-NET-04, R-PLAT-01, R-SEC-03, R-STATE-04. Verified through the real runner: --layer l0 7/7 PASS, the three new ones 3/3 PASS, --strict-coverage exit 0 with 0 label/annotation mismatches. *** R-TLS-02 "REVOKED" IS NOT SATISFIED — A REAL GAP, NOT A TEST DEFECT. *** The spec requires certificates be "issued, renewed, AND REVOKED without manual steps", and docs/08 reads revocation as "removing/rotating a certificate is a declarative change". That needs enableCertificateOwnerRef: true; platform/cert-manager/helmrelease.yaml does not set it (chart default false). Verified read-only on BOTH live clusters: gateway/wildcard-home-tls has ZERO ownerReferences, so deleting the Certificate would leave the private key serving indefinitely. The L2 test asserts the requirement AS WRITTEN and will fail until this is decided. It was deliberately NOT softened — weakening a test to match the implementation is how a suite stops meaning anything. NOT FLIPPED UNATTENDED, and the reason is specific rather than caution: the wildcard Certificate lives under ./platform, which runs prune: true. With owner refs ON, any transient prune of that Certificate — exactly what happened to the CDI StorageProfiles hours ago — would delete the SERVING TLS Secret for every host, and reissuance is bounded by Let's Encrypt's duplicate-certificate limit (5/week per identical name set), which could leave a cluster with no trusted cert for a week. Security gap vs a plausible week-long TLS outage is an operator call. NON-VACUITY DEMONSTRATED, which is the part that makes these worth having: 7/7 perturbations fail for R-TLS-02, 8/8 for R-ENV-01, 13/13 for R-ENV-03, and each restored copy passes. Perturbations were applied to a throwaway copy; neither this repo nor the read-only spec repo was modified. TWO CHECKS INITIALLY PASSED WHEN PERTURBED and were rewritten — the exact degrade-to-no-op failure this repo keeps hitting. The first draft asked "is this file's DIRECTORY referenced by a kustomization", which a directory reference always satisfies, and keyed Secrets by name alone, so external-dns's copy of `cloudflare-api-token` satisfied cert-manager's reference. Now a recursive kustomize resource-graph walk from each Flux path, and Secrets keyed by (namespace, name). Both bite. SECURITY NOTE, recorded because this project has already leaked a password once: `ansible-inventory --list` PRINTS DECRYPTED VAULT VALUES — it emitted the prod become-password to stdout during exploration, since the vault passphrase file is mounted into the dev container. Both new checks use only --syntax-check and --list-hosts, and say so in a comment. Confirmed no plaintext value reached disk: vault.yml is still AES256-encrypted and the tree has no assigned value. WHAT IS NOT PROVEN, per requirement: R-TLS-02 "revoked" (above), and issuance + observed renewal await the unexecuted L2. R-ENV-01 cannot prove the running VMs came from this module (needs OpenTofu state) — it proves the four lifecycle stages exist as code and cover every dev node, and that the chain RESOLVES under real ansible-playbook. R-ENV-03 cannot prove the HARDWARE (the K8s API exposes no "am I virtual" fact; mini-PC/bare-metal is operator-attested) — it proves prod is AUTHORED as 3 directly-reached nodes with no IaC creating them, sharing dev's playbooks and roles, with no prod-only manifest, branch or role anywhere. VERIFIED BY ME: coverage delta and the 3/3 L0 pass through the real runner; the zero-ownerReferences finding on both clusters; that the new checks avoid ansible-inventory; that no credential material reached disk. Full CI gate PASS. Refs R-TLS-02, R-ENV-01, R-ENV-03, R-TEST-01..05. Co-Authored-By: Claude Opus 5 <[email protected]>
70 lines
4.0 KiB
YAML
70 lines
4.0 KiB
YAML
# =============================================================================
|
|
# L0 STATIC — certificate lifecycle needs no manual step (R-TLS-02) [EXECUTABLE]
|
|
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
|
# Spec: docs/00 R-TLS-02 ("Certificates MUST be automatically issued, renewed, and
|
|
# revoked without manual steps"); mechanism in docs/08 + ADR-0006.
|
|
#
|
|
# R-TLS-02 is one adverbial clause plus three runtime verbs. This test asserts the
|
|
# clause — "without manual steps" — which in a GitOps platform is a property of the
|
|
# REPOSITORY: a human step in the certificate path leaves evidence in git (committed
|
|
# cert material, a Gateway serving a Secret nothing declares, an issuer with no
|
|
# automated solver, a credential never delivered, auto-approval switched off, or CRs
|
|
# nothing reconciles). The three verbs are observed by the companion L2 test.
|
|
#
|
|
# ENVIRONMENT-AGNOSTIC (R-ENV-02): the issuer a Certificate names is `${ACME_CLUSTER_ISSUER}`,
|
|
# so the check resolves it against the BASE cluster-vars default AND every overlay's value and
|
|
# requires each resolved issuer to exist. dev's issuer and prod's issuer are both verified from
|
|
# one authored assertion, with no hostname, address or environment name written down.
|
|
#
|
|
# GIVEN the repository as git tracks it,
|
|
# WHEN every serving-TLS reference, Certificate, Issuer and the cert-manager release
|
|
# is inspected offline,
|
|
# THEN every Gateway listener that terminates TLS serves a Secret a tracked Certificate
|
|
# produces; no certificate material is committed anywhere; every Certificate's
|
|
# issuer resolves — in EVERY environment — to a tracked ACME issuer with an
|
|
# automated dns01/http01 solver whose credential is SOPS-delivered from git;
|
|
# cert-manager's automatic request approval is not disabled; and every one of
|
|
# those objects is rendered by a Flux Kustomization, so nothing in the chain
|
|
# depends on a human running kubectl.
|
|
#
|
|
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git + PyYAML, no
|
|
# cluster, no network); conformance/run.sh --layer l0 and ci/l0.sh run it directly, and
|
|
# this Chainsaw Test wraps the SAME script.
|
|
#
|
|
# KNOWN, DELIBERATE GAPS (named in check.sh's header too — this test claims NONE of them):
|
|
# * ISSUED / RENEWED as runtime events: L2 (l2-behavioral/cert-lifecycle-automated).
|
|
# * REVOKED: not proven at any layer today. cert-manager performs no ACME revocation,
|
|
# and the spec's own reading (docs/08: "removing/rotating a certificate is a
|
|
# declarative change") needs `enableCertificateOwnerRef`, which the tracked
|
|
# cert-manager HelmRelease does not set — so a removed Certificate leaves its
|
|
# serving Secret behind. The L2 test asserts the withdrawal behaviour and is
|
|
# EXPECTED TO FAIL until that knob is set; escalated, not softened (R-TEST-06).
|
|
# * That the live cluster is serving the tracked Certificate (cluster state).
|
|
# =============================================================================
|
|
apiVersion: chainsaw.kyverno.io/v1alpha1
|
|
kind: Test
|
|
metadata:
|
|
name: tls-lifecycle-automated
|
|
labels:
|
|
conformance.home.k3s/R-TLS-02: "true"
|
|
annotations:
|
|
conformance.home.k3s/requirements: "R-TLS-02"
|
|
conformance.home.k3s/layer: "L0"
|
|
conformance.home.k3s/safety-critical: "false"
|
|
conformance.home.k3s/executable: "true"
|
|
conformance.home.k3s/intent: >-
|
|
Given the tracked tree, when the certificate path is inspected offline, then no
|
|
manual step exists anywhere in it: serving Secrets are cert-manager-produced, no
|
|
cert material is committed, every environment's issuer is a tracked ACME issuer
|
|
with an automated solver and a git-delivered credential, auto-approval is on, and
|
|
the whole chain is Flux-reconciled.
|
|
spec:
|
|
steps:
|
|
- name: then-no-manual-step-exists-in-the-certificate-path
|
|
description: >-
|
|
Run the offline assertion script over the tracked tree (same script ci/l0.sh
|
|
and conformance/run.sh --layer l0 execute).
|
|
try:
|
|
- script:
|
|
content: bash ./check.sh
|