jakman85andClaude Opus 5
fe45d0f5c5
DECISIONS-LOG records Phase 7 being opened knowingly while the uncovered set included R-ENV-01/02/03 — the very requirements prod provisioning serves. Two of those three are now covered, plus R-TLS-02. tls-lifecycle-automated R-TLS-02 L0 (in the merge gate) cert-lifecycle-automated R-TLS-02 L2 (authored, NOT executed) dev-nodes-provisioned-as-code R-ENV-01 L0 prod-baremetal-not-dev-surface R-ENV-03 L0 Coverage 42/51 (82.4%) -> 45/51 (88.2%), 6 gaps, 0 safety-critical. Remaining: R-C2-01, R-C2-02, R-NET-04, R-PLAT-01, R-SEC-03, R-STATE-04. Verified through the real runner: --layer l0 7/7 PASS, the three new ones 3/3 PASS, --strict-coverage exit 0 with 0 label/annotation mismatches. *** R-TLS-02 "REVOKED" IS NOT SATISFIED — A REAL GAP, NOT A TEST DEFECT. *** The spec requires certificates be "issued, renewed, AND REVOKED without manual steps", and docs/08 reads revocation as "removing/rotating a certificate is a declarative change". That needs enableCertificateOwnerRef: true; platform/cert-manager/helmrelease.yaml does not set it (chart default false). Verified read-only on BOTH live clusters: gateway/wildcard-home-tls has ZERO ownerReferences, so deleting the Certificate would leave the private key serving indefinitely. The L2 test asserts the requirement AS WRITTEN and will fail until this is decided. It was deliberately NOT softened — weakening a test to match the implementation is how a suite stops meaning anything. NOT FLIPPED UNATTENDED, and the reason is specific rather than caution: the wildcard Certificate lives under ./platform, which runs prune: true. With owner refs ON, any transient prune of that Certificate — exactly what happened to the CDI StorageProfiles hours ago — would delete the SERVING TLS Secret for every host, and reissuance is bounded by Let's Encrypt's duplicate-certificate limit (5/week per identical name set), which could leave a cluster with no trusted cert for a week. Security gap vs a plausible week-long TLS outage is an operator call. NON-VACUITY DEMONSTRATED, which is the part that makes these worth having: 7/7 perturbations fail for R-TLS-02, 8/8 for R-ENV-01, 13/13 for R-ENV-03, and each restored copy passes. Perturbations were applied to a throwaway copy; neither this repo nor the read-only spec repo was modified. TWO CHECKS INITIALLY PASSED WHEN PERTURBED and were rewritten — the exact degrade-to-no-op failure this repo keeps hitting. The first draft asked "is this file's DIRECTORY referenced by a kustomization", which a directory reference always satisfies, and keyed Secrets by name alone, so external-dns's copy of `cloudflare-api-token` satisfied cert-manager's reference. Now a recursive kustomize resource-graph walk from each Flux path, and Secrets keyed by (namespace, name). Both bite. SECURITY NOTE, recorded because this project has already leaked a password once: `ansible-inventory --list` PRINTS DECRYPTED VAULT VALUES — it emitted the prod become-password to stdout during exploration, since the vault passphrase file is mounted into the dev container. Both new checks use only --syntax-check and --list-hosts, and say so in a comment. Confirmed no plaintext value reached disk: vault.yml is still AES256-encrypted and the tree has no assigned value. WHAT IS NOT PROVEN, per requirement: R-TLS-02 "revoked" (above), and issuance + observed renewal await the unexecuted L2. R-ENV-01 cannot prove the running VMs came from this module (needs OpenTofu state) — it proves the four lifecycle stages exist as code and cover every dev node, and that the chain RESOLVES under real ansible-playbook. R-ENV-03 cannot prove the HARDWARE (the K8s API exposes no "am I virtual" fact; mini-PC/bare-metal is operator-attested) — it proves prod is AUTHORED as 3 directly-reached nodes with no IaC creating them, sharing dev's playbooks and roles, with no prod-only manifest, branch or role anywhere. VERIFIED BY ME: coverage delta and the 3/3 L0 pass through the real runner; the zero-ownerReferences finding on both clusters; that the new checks avoid ansible-inventory; that no credential material reached disk. Full CI gate PASS. Refs R-TLS-02, R-ENV-01, R-ENV-03, R-TEST-01..05. Co-Authored-By: Claude Opus 5 <[email protected]>
76 lines
4.5 KiB
YAML
76 lines
4.5 KiB
YAML
# =============================================================================
|
|
# L0 STATIC — dev/test nodes are provisioned as code, end to end (R-ENV-01) [EXECUTABLE]
|
|
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
|
# Spec: docs/00 R-ENV-01 ("dev/test MUST be provisioned as local KVMs via IaC:
|
|
# created, baselined, cluster-joined, and managed as code"); the four stages
|
|
# are enumerated in docs/02 §"Node lifecycle stages".
|
|
#
|
|
# WHY L0: R-ENV-01 is a requirement about HOW the dev nodes came to exist, and a live
|
|
# Node object cannot answer it — a node built by OpenTofu and one built by hand in a
|
|
# hypervisor UI are indistinguishable through the Kubernetes API. What is provable, with
|
|
# no cluster, is that a COMPLETE and EXECUTABLE provisioning path exists for all four
|
|
# stages and covers EVERY dev node. For the Ansible half that is proven by DRIVING the
|
|
# real toolchain offline (`--syntax-check`, `--list-hosts` — neither opens a connection
|
|
# to any node), not by reading files: a chain that does not resolve fails the check.
|
|
#
|
|
# ENVIRONMENT-AGNOSTIC: no hostname, address, prefix or node count appears in the test.
|
|
# The dev group names are read from the playbooks' own default host patterns, the node
|
|
# names are generated from the IaC module's prefix/count variables, and the two sets are
|
|
# compared against each other — so the assertion survives a rename or a resize.
|
|
#
|
|
# GIVEN the repository as git tracks it,
|
|
# WHEN the provisioning chain is parsed and (where offline execution is possible) driven,
|
|
# THEN the IaC module creates the dev nodes on a LOCAL KVM hypervisor (allowlisted
|
|
# provider — a cloud provider fails), from a for_each map rather than a count index,
|
|
# each with CPU/RAM/OS disk/dedicated Longhorn data disk/NIC/cloud-init seed; the
|
|
# node set it creates is EXACTLY the set Ansible manages; the baseline stage applies
|
|
# the package floor, kernel modules, sysctls, data-disk mount, swap-off and iscsid
|
|
# declaratively; one k3s role founds the cluster with --cluster-init and joins the
|
|
# rest with --server while disabling flannel/kube-proxy/traefik/servicelb at
|
|
# install; and site.yml's chain (baseline -> k3s -> Cilium -> verify -> Flux)
|
|
# resolves and targets every dev node at every stage.
|
|
#
|
|
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git, PyYAML, and —
|
|
# where present — ansible-playbook in no-connection modes; no cluster, no network);
|
|
# conformance/run.sh --layer l0 and ci/l0.sh run it directly, and this Chainsaw Test
|
|
# wraps the SAME script.
|
|
#
|
|
# KNOWN, DELIBERATE GAPS (enumerated in check.sh's header; this test claims none of them):
|
|
# * That the RUNNING dev VMs were created by this module — needs the OpenTofu state or
|
|
# hypervisor access, neither available offline.
|
|
# * `tofu validate` — needs `tofu init` (provider registry), i.e. network. The module is
|
|
# parsed, not evaluated; a network-allowed CI stage would close this.
|
|
# * "Never hand-edited on the box" (docs/02 stage 4's second half) — a live drift check.
|
|
# * REBUILD/re-converge (docs/02's own proof line) — exercised at L3 by
|
|
# tests/l3-fault-injection/node-loss (R-STATE-02).
|
|
# * docs/02 prose puts the baseline in cloud-init; the implementation moved it into the
|
|
# Ansible node_baseline role. R-ENV-01 requires the node to be "baselined ... as code",
|
|
# not that cloud-init be the tool, so the check asserts the STAGE and RECORDS the
|
|
# divergence rather than hiding it.
|
|
# =============================================================================
|
|
apiVersion: chainsaw.kyverno.io/v1alpha1
|
|
kind: Test
|
|
metadata:
|
|
name: dev-nodes-provisioned-as-code
|
|
labels:
|
|
conformance.home.k3s/R-ENV-01: "true"
|
|
annotations:
|
|
conformance.home.k3s/requirements: "R-ENV-01"
|
|
conformance.home.k3s/layer: "L0"
|
|
conformance.home.k3s/safety-critical: "false"
|
|
conformance.home.k3s/executable: "true"
|
|
conformance.home.k3s/intent: >-
|
|
Given the tracked tree, when the dev provisioning chain is parsed and driven
|
|
offline, then every dev node is created by local-KVM IaC, baselined, cluster-joined
|
|
and handed to Flux by one code path that covers all of them — with no node outside
|
|
it and no stage missing.
|
|
spec:
|
|
steps:
|
|
- name: then-every-dev-node-is-created-baselined-joined-and-managed-as-code
|
|
description: >-
|
|
Run the offline assertion script over the tracked tree (same script ci/l0.sh and
|
|
conformance/run.sh --layer l0 execute).
|
|
try:
|
|
- script:
|
|
content: bash ./check.sh
|