jakman85andClaude Opus 5
8a2be824f6
Adds conformance/tests/l0-static/flux-postbuild-substitution/ — an offline, stdlib-only
L0 check tagged R-ENV-02 + R-GITOPS-01 (deliberately NOT R-TEST-01, which would
overclaim). It reuses lib/status.py's `resources:` walk rather than forking it, so "what a
path renders" has exactly one definition. It asserts: every ${...} on a substituted path
is defined-or-escaped (comments included); no default-operator form on an undefined name;
no unparseable expansion; the rendering path actually substitutes cluster-vars; every
overlay-patched /data/KEY has a base default; and that the gate itself is load-bearing.
WIRED AS AN UNCONDITIONAL STAGE (3c) in ci/l0.sh, and that matters: stage 3b scopes to
touched requirements, and ci/touched-requirements.sh maps only clusters/* to R-ENV-02 —
so an edit to a substituted manifest under apps/ or platform/ would have filtered this
check out of exactly the change that introduces the bug. l0 fail -> BLOCK=1 in gate.sh.
Mutation-tested all five assertion classes (undefined token, indexed expansion, :- form,
missing substituteFrom, un-wired gate); each fails as intended, baseline green.
STRICT MODE IS PROVEN SAFE TO ENABLE — measured, not inspected. For all 11 substituted
paths, `kubectl kustomize --load-restrictor LoadRestrictionsNone <path> | flux envsubst
--strict` exits 0 against cluster-vars base defaults. Negative control confirms the
incident: with K3S_API_HOST unset, non-strict renders k8sServiceHost empty while strict
exits 1 with `variable not set (strict mode)`. The feature gate itself is NOT flipped
here — that is a live-cluster change and the operator's call.
Escapes the six comment hazards that stood in the way (apps/vms/unifi x4,
platform/longhorn x2). All were prose ABOUT the escaping convention; since the repo's own
contract is that $${...} emits a literal ${...}, escaping them is both precise and
self-consistent. kustomize strips these particular comments today, but comment survival is
an implementation detail — a body comment carrying $ WAS observed surviving into built
apps/wave-1 — and a survivor would kill the whole Kustomization.
DOCTRINE CORRECTED (measured against flux 2.4.0): clusters/base/apps-vms.yaml claimed an
indexed expansion is fatal "escaped or not". Escaping is TOTAL — $${ARR[0]} emits the
literal text safely; only the UNESCAPED indexed form is fatal. Corrected in place.
Also empties DOCUMENTED_GAPS in conformance/run.sh with rationale: R-ENV-02 and
R-GITOPS-01 now have a real verifying test, which run.sh's own comment named as the
trigger for removing them.
Advisory output (not failures): 6 dead cluster-vars knobs — BASE_DOMAIN, CLUSTER_ENV,
CLUSTER_POD_CIDR, PIA_REGION, PLATFORM_NODE_SELECTOR_{KEY,VALUE}. The last four are
documented reference-only; BASE_DOMAIN and CLUSTER_ENV look like genuine rot (hostnames
are hardcoded elsewhere) and are worth a follow-up.
Gate: MERGE GATE: PASS. 102 assertions, 0 hazards.
Serves: R-ENV-02, R-GITOPS-01.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
61 lines
3.4 KiB
YAML
61 lines
3.4 KiB
YAML
# =============================================================================
|
|
# L0 STATIC — Flux postBuild substitution is complete and fail-closed-safe
|
|
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
|
#
|
|
# R-ENV-02 ("authored once, promoted dev -> prod") is implemented by ONE mechanism:
|
|
# `postBuild.substituteFrom cluster-vars` injecting `${TOKEN}`s into the manifests each
|
|
# Flux Kustomization renders. That mechanism FAILS OPEN — an undefined variable becomes
|
|
# the empty string while the Kustomization still reports Ready — which emptied Cilium's
|
|
# k8sServiceHost and broke the CNI on 2/3 dev nodes (DECISIONS-LOG 2026-09-01).
|
|
#
|
|
# GIVEN every Flux Kustomization under clusters/ that declares postBuild.substituteFrom,
|
|
# WHEN each `spec.path` is resolved to the exact file set it renders (following
|
|
# kustomization.yaml `resources:` transitively, not globbing the repo),
|
|
# THEN every `${...}` in those files is either DEFINED in clusters/base/cluster-vars.yaml
|
|
# or deliberately ESCAPED `$${...}`; no unescaped expansion carries a default
|
|
# operator on an undefined name (the silent-empty class strict mode does NOT catch);
|
|
# no unescaped expansion is unparseable by envsubst (which fails the WHOLE
|
|
# Kustomization); every path rendering a knob actually substitutes cluster-vars;
|
|
# every overlay-patched knob has a base default; and the check is wired into the
|
|
# merge gate as a blocking, unconditionally-run stage.
|
|
#
|
|
# This is the prerequisite proof for enabling kustomize-controller's
|
|
# StrictPostBuildSubstitutions feature gate, and the permanent guard that keeps it safe.
|
|
#
|
|
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (standard library
|
|
# only); conformance/run.sh --layer l0 and ci/l0.sh run it directly, and this Chainsaw
|
|
# Test wraps the SAME script.
|
|
#
|
|
# NOT proven (named in check.sh's header): this is a STATIC scan of the SOURCE manifests
|
|
# a path includes — a deliberate superset of the post-`kustomize build` stream. It does
|
|
# not render, does not prove the feature gate is enabled on the live kustomize-controller,
|
|
# and does not validate knob VALUES. R-GITOPS-01 is tagged for the narrow static slice
|
|
# only: what git says a Kustomization renders is fully determined, with no variable
|
|
# silently emptied on the way in. The live-state half is L2, untested today.
|
|
# =============================================================================
|
|
apiVersion: chainsaw.kyverno.io/v1alpha1
|
|
kind: Test
|
|
metadata:
|
|
name: flux-postbuild-substitution
|
|
labels:
|
|
conformance.home.k3s/R-ENV-02: "true"
|
|
conformance.home.k3s/R-GITOPS-01: "true"
|
|
annotations:
|
|
conformance.home.k3s/requirements: "R-ENV-02, R-GITOPS-01"
|
|
conformance.home.k3s/layer: "L0"
|
|
conformance.home.k3s/safety-critical: "false"
|
|
conformance.home.k3s/executable: "true"
|
|
conformance.home.k3s/intent: >-
|
|
Every variable a substituted Flux path renders is defined in cluster-vars or
|
|
escaped, so post-build substitution cannot silently empty a value and enabling
|
|
StrictPostBuildSubstitutions cannot fail a legitimate manifest.
|
|
spec:
|
|
steps:
|
|
- name: audit-every-substituted-flux-path
|
|
description: >-
|
|
Resolve each postBuild.substituteFrom Kustomization to the files it renders and
|
|
assert every ${...} is defined or escaped, then assert the gate wires it blocking.
|
|
try:
|
|
- script:
|
|
content: bash ./check.sh
|