feat(ci): substitution-safety gate + escape 6 comment hazards; strict mode is now SAFE
Adds conformance/tests/l0-static/flux-postbuild-substitution/ — an offline, stdlib-only
L0 check tagged R-ENV-02 + R-GITOPS-01 (deliberately NOT R-TEST-01, which would
overclaim). It reuses lib/status.py's `resources:` walk rather than forking it, so "what a
path renders" has exactly one definition. It asserts: every ${...} on a substituted path
is defined-or-escaped (comments included); no default-operator form on an undefined name;
no unparseable expansion; the rendering path actually substitutes cluster-vars; every
overlay-patched /data/KEY has a base default; and that the gate itself is load-bearing.
WIRED AS AN UNCONDITIONAL STAGE (3c) in ci/l0.sh, and that matters: stage 3b scopes to
touched requirements, and ci/touched-requirements.sh maps only clusters/* to R-ENV-02 —
so an edit to a substituted manifest under apps/ or platform/ would have filtered this
check out of exactly the change that introduces the bug. l0 fail -> BLOCK=1 in gate.sh.
Mutation-tested all five assertion classes (undefined token, indexed expansion, :- form,
missing substituteFrom, un-wired gate); each fails as intended, baseline green.
STRICT MODE IS PROVEN SAFE TO ENABLE — measured, not inspected. For all 11 substituted
paths, `kubectl kustomize --load-restrictor LoadRestrictionsNone <path> | flux envsubst
--strict` exits 0 against cluster-vars base defaults. Negative control confirms the
incident: with K3S_API_HOST unset, non-strict renders k8sServiceHost empty while strict
exits 1 with `variable not set (strict mode)`. The feature gate itself is NOT flipped
here — that is a live-cluster change and the operator's call.
Escapes the six comment hazards that stood in the way (apps/vms/unifi x4,
platform/longhorn x2). All were prose ABOUT the escaping convention; since the repo's own
contract is that $${...} emits a literal ${...}, escaping them is both precise and
self-consistent. kustomize strips these particular comments today, but comment survival is
an implementation detail — a body comment carrying $ WAS observed surviving into built
apps/wave-1 — and a survivor would kill the whole Kustomization.
DOCTRINE CORRECTED (measured against flux 2.4.0): clusters/base/apps-vms.yaml claimed an
indexed expansion is fatal "escaped or not". Escaping is TOTAL — $${ARR[0]} emits the
literal text safely; only the UNESCAPED indexed form is fatal. Corrected in place.
Also empties DOCUMENTED_GAPS in conformance/run.sh with rationale: R-ENV-02 and
R-GITOPS-01 now have a real verifying test, which run.sh's own comment named as the
trigger for removing them.
Advisory output (not failures): 6 dead cluster-vars knobs — BASE_DOMAIN, CLUSTER_ENV,
CLUSTER_POD_CIDR, PIA_REGION, PLATFORM_NODE_SELECTOR_{KEY,VALUE}. The last four are
documented reference-only; BASE_DOMAIN and CLUSTER_ENV look like genuine rot (hostnames
are hardcoded elsewhere) and are worth a follow-up.
Gate: MERGE GATE: PASS. 102 assertions, 0 hazards.
Serves: R-ENV-02, R-GITOPS-01.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed
1 parent
3484955903
commit
8a2be824f6
8 files changed
+491
-20
No files matched your search
@@ -92,10 +92,13 @@ spec:
|
||||
# NOTE: apps-vms NOW carries postBuild.substituteFrom cluster-vars (needed for
|
||||
# ${UNIFI_L4_VIP} in 50-service-l4.yaml), so Flux envsubst DOES run over this userData.
|
||||
# Every shell brace-expansion below is therefore escaped as $${...}, which envsubst emits
|
||||
# as a literal ${...} — verified with `flux envsubst`. Bare $VAR and $(...) are left alone
|
||||
# by envsubst and need no escaping. NEVER write an unescaped ${...} here, and never write
|
||||
# ${ARR[0]} even escaped-adjacent: envsubst hard-fails on `[` ("missing closing brace")
|
||||
# and would break the whole apps-vms apply.
|
||||
# as a literal $${...} — verified with `flux envsubst`. Bare $VAR and $(...) are left alone
|
||||
# by envsubst and need no escaping. NEVER write an unescaped $${...} here, and never write
|
||||
# an INDEXED expansion unescaped: envsubst hard-fails on `[` ("missing closing brace")
|
||||
# and would break the whole apps-vms apply. CORRECTION (2026-09-01, measured against
|
||||
# flux 2.4.0 `envsubst --strict`): escaping IS total — $${ARR[0]} emits the literal
|
||||
# text safely. Only the UNESCAPED indexed form is fatal. The earlier "escaped or not"
|
||||
# wording in clusters/base/apps-vms.yaml overstated it.
|
||||
userData: |-
|
||||
#cloud-config
|
||||
hostname: unifi
|
||||
|
||||
@@ -27,7 +27,7 @@
|
||||
# VIP: ${UNIFI_L4_VIP} (cluster-vars, R-ENV-02; dev 192.168.1.54, confirmed free, inside
|
||||
# default-pool and reserved in tofu — the Gateway already holds the pool's first IP, dev .53).
|
||||
# Substituted by the `apps-vms` Flux Kustomization, which carries postBuild.substituteFrom for
|
||||
# exactly this token; every shell ${...} in 30-virtualmachine.yaml's cloud-init is escaped as $${...}
|
||||
# exactly this token; every shell $${...} in 30-virtualmachine.yaml's cloud-init is escaped as $${...}
|
||||
# accordingly. A new environment MUST set UNIFI_L4_VIP to a free address in ITS pool.
|
||||
#
|
||||
# REUSABLE PATTERN (the deferred `minecraft` :25565/L4 problem is the same shape): expose an
|
||||
|
||||
@@ -100,6 +100,43 @@ else
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---- Stage 3c: Flux post-build substitution safety (UNCONDITIONAL) ----------
|
||||
# R-ENV-02's promote-once mechanism is `postBuild.substituteFrom cluster-vars`, and it
|
||||
# FAILS OPEN: an undefined ${TOKEN} becomes the empty string and the Kustomization still
|
||||
# reports Ready. That emptied Cilium's k8sServiceHost and broke the CNI on 2/3 dev nodes
|
||||
# (DECISIONS-LOG 2026-09-01, "substitution race that fails OPEN"). The check asserts every
|
||||
# ${...} in every substituted path is DEFINED in cluster-vars or deliberately ESCAPED
|
||||
# `$${...}` — i.e. that enabling the StrictPostBuildSubstitutions feature gate cannot fail
|
||||
# a legitimate manifest, and that no expansion silently empties today.
|
||||
#
|
||||
# WHY THIS IS ITS OWN STAGE and not left to Stage 3b: 3b scopes the conformance run to the
|
||||
# TOUCHED requirements (R-TEST-03), and ci/touched-requirements.sh's coarse path map keys
|
||||
# only clusters/* to R-ENV-02 — an edit to a SUBSTITUTED manifest under apps/ or platform/
|
||||
# attributes no requirement, so the requirement-scoped selection would filter this check
|
||||
# out of exactly the change that introduces the bug. The invariant is whole-repo (same
|
||||
# reasoning as spec-immutability's own gate stage), so it runs on EVERY gate invocation.
|
||||
# When 3b's touched set does include R-ENV-02/R-GITOPS-01 the check runs twice; it is a
|
||||
# sub-second offline scan with no side effects, so the duplication is not worth avoiding.
|
||||
SUBST_CHECK="${SUBST_CHECK:-${REPO_ROOT}/conformance/tests/l0-static/flux-postbuild-substitution/check.sh}"
|
||||
log "Stage: Flux postBuild substitution safety (${SUBST_CHECK})"
|
||||
if [ ! -f "$SUBST_CHECK" ]; then
|
||||
skip "substitution check not present at '${SUBST_CHECK}' — UNWIRED, not green"
|
||||
summary "- subst-safety: **SKIPPED** (check missing)"
|
||||
if [ "$rc" -eq 0 ]; then rc=3; fi
|
||||
else
|
||||
subst_rc=0
|
||||
bash "$SUBST_CHECK" || subst_rc=$?
|
||||
case "$subst_rc" in
|
||||
0) ok "Flux postBuild substitution safety passed"
|
||||
summary "- subst-safety: **passed**" ;;
|
||||
3) skip "substitution check SKIPPED (clusters/ absent) — not a pass"
|
||||
summary "- subst-safety: **SKIPPED** (nothing to audit)"
|
||||
if [ "$rc" -eq 0 ]; then rc=3; fi ;;
|
||||
*) err "Flux postBuild substitution safety FAILED"
|
||||
summary "- subst-safety: **FAILED**"; rc=1 ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
case "$rc" in
|
||||
0) ok "L0 stage complete" ;;
|
||||
3) skip "L0 stage incomplete (unwired seam / missing tool) — not a pass" ;;
|
||||
|
||||
+15
-13
@@ -57,17 +57,15 @@ FORCE_NO_CLUSTER=0
|
||||
# Documented, out-of-scope coverage gaps exempt from the completeness gate: requirements
|
||||
# owned by another workstream that WS-TEST does not yet scaffold. Listed here so the
|
||||
# exemption is explicit and reviewable, not silent.
|
||||
# R-ENV-02 (dev->prod promote) — owned by WS-INFRA/WS-GITOPS; unscaffolded (a GAP in
|
||||
# coverage.md). The coarse path->requirement map keys
|
||||
# clusters/* AND ci/* to it and the R-ENV-02 doc token
|
||||
# appears in platform/kustomization.yaml's header, so ANY
|
||||
# infra/overlay/CI edit (e.g. enabling a component, bumping
|
||||
# SPEC_PIN) would otherwise false-block on it.
|
||||
# R-GITOPS-01 (state in git/Flux) — same: owned by WS-GITOPS, unscaffolded, and the ci/*
|
||||
# path-map keys to it, so editing any ci/ file (SPEC_PIN
|
||||
# adoption included) would false-block.
|
||||
# These are visible GAPs, not hidden — removing them from this list is the right move the
|
||||
# moment WS-TEST adds their verifying tests.
|
||||
#
|
||||
# EMPTY as of 2026-09-01. R-ENV-02 and R-GITOPS-01 were the last two entries — exempted
|
||||
# because they were unscaffolded while the coarse path->requirement map keys clusters/*
|
||||
# and ci/* to them, so any overlay/CI edit would have false-blocked. They now have a real
|
||||
# verifying L0 test (tests/l0-static/flux-postbuild-substitution), which asserts the
|
||||
# substitution mechanism that IS the "authored once, promoted" contract, so they are
|
||||
# covered rather than exempt and the completeness gate can hold them to it. That test
|
||||
# names its own limits in its header: it proves the STATIC property (no manifest renders
|
||||
# an undefined variable), not the live-cluster half of either requirement.
|
||||
#
|
||||
# FOLLOW-UP (WS-TEST) — HALF DONE. The debt was "make the matcher stop keying on doc-header
|
||||
# tokens so real requirements need not be exempted". The CONTENT-SCAN half is now fixed in
|
||||
@@ -83,7 +81,7 @@ FORCE_NO_CLUSTER=0
|
||||
# R-STATE-02 (node-loss) WAS listed here (owned by WS-STORAGE, "delivered with Longhorn/infra")
|
||||
# but now has a real verifying L3 test — tests/l3-fault-injection/node-loss — so it is no
|
||||
# longer a gap and has been removed from this list; the coverage report counts it as covered.
|
||||
DOCUMENTED_GAPS="R-ENV-02 R-GITOPS-01"
|
||||
DOCUMENTED_GAPS=""
|
||||
|
||||
usage() { sed -n '2,40p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; }
|
||||
|
||||
@@ -108,7 +106,11 @@ log() { printf '\033[1m[run.sh]\033[0m %s\n' "$*"; }
|
||||
|
||||
# --- Coverage report is always produced (cluster-independent, R-TEST-04) --------------
|
||||
gen_coverage() {
|
||||
local flags=(--allow-gap "${DOCUMENTED_GAPS}")
|
||||
# Pass --allow-gap only when there IS an exemption: an empty DOCUMENTED_GAPS means no
|
||||
# requirement is exempt, and handing the flag an empty string would be a silent no-op
|
||||
# that reads like an exemption in the CLI log.
|
||||
local flags=()
|
||||
[ -n "${DOCUMENTED_GAPS}" ] && flags+=(--allow-gap "${DOCUMENTED_GAPS}")
|
||||
[ "${STRICT_COVERAGE}" = 1 ] && flags+=(--strict)
|
||||
# A tagged subset run is also the merge gate: the touched requirements MUST be covered.
|
||||
[ -n "${REQS}" ] && flags+=(--require "${REQS}")
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
# =============================================================================
|
||||
# L0 STATIC — Flux postBuild substitution is complete and fail-closed-safe
|
||||
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
||||
#
|
||||
# R-ENV-02 ("authored once, promoted dev -> prod") is implemented by ONE mechanism:
|
||||
# `postBuild.substituteFrom cluster-vars` injecting `${TOKEN}`s into the manifests each
|
||||
# Flux Kustomization renders. That mechanism FAILS OPEN — an undefined variable becomes
|
||||
# the empty string while the Kustomization still reports Ready — which emptied Cilium's
|
||||
# k8sServiceHost and broke the CNI on 2/3 dev nodes (DECISIONS-LOG 2026-09-01).
|
||||
#
|
||||
# GIVEN every Flux Kustomization under clusters/ that declares postBuild.substituteFrom,
|
||||
# WHEN each `spec.path` is resolved to the exact file set it renders (following
|
||||
# kustomization.yaml `resources:` transitively, not globbing the repo),
|
||||
# THEN every `${...}` in those files is either DEFINED in clusters/base/cluster-vars.yaml
|
||||
# or deliberately ESCAPED `$${...}`; no unescaped expansion carries a default
|
||||
# operator on an undefined name (the silent-empty class strict mode does NOT catch);
|
||||
# no unescaped expansion is unparseable by envsubst (which fails the WHOLE
|
||||
# Kustomization); every path rendering a knob actually substitutes cluster-vars;
|
||||
# every overlay-patched knob has a base default; and the check is wired into the
|
||||
# merge gate as a blocking, unconditionally-run stage.
|
||||
#
|
||||
# This is the prerequisite proof for enabling kustomize-controller's
|
||||
# StrictPostBuildSubstitutions feature gate, and the permanent guard that keeps it safe.
|
||||
#
|
||||
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (standard library
|
||||
# only); conformance/run.sh --layer l0 and ci/l0.sh run it directly, and this Chainsaw
|
||||
# Test wraps the SAME script.
|
||||
#
|
||||
# NOT proven (named in check.sh's header): this is a STATIC scan of the SOURCE manifests
|
||||
# a path includes — a deliberate superset of the post-`kustomize build` stream. It does
|
||||
# not render, does not prove the feature gate is enabled on the live kustomize-controller,
|
||||
# and does not validate knob VALUES. R-GITOPS-01 is tagged for the narrow static slice
|
||||
# only: what git says a Kustomization renders is fully determined, with no variable
|
||||
# silently emptied on the way in. The live-state half is L2, untested today.
|
||||
# =============================================================================
|
||||
apiVersion: chainsaw.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
metadata:
|
||||
name: flux-postbuild-substitution
|
||||
labels:
|
||||
conformance.home.k3s/R-ENV-02: "true"
|
||||
conformance.home.k3s/R-GITOPS-01: "true"
|
||||
annotations:
|
||||
conformance.home.k3s/requirements: "R-ENV-02, R-GITOPS-01"
|
||||
conformance.home.k3s/layer: "L0"
|
||||
conformance.home.k3s/safety-critical: "false"
|
||||
conformance.home.k3s/executable: "true"
|
||||
conformance.home.k3s/intent: >-
|
||||
Every variable a substituted Flux path renders is defined in cluster-vars or
|
||||
escaped, so post-build substitution cannot silently empty a value and enabling
|
||||
StrictPostBuildSubstitutions cannot fail a legitimate manifest.
|
||||
spec:
|
||||
steps:
|
||||
- name: audit-every-substituted-flux-path
|
||||
description: >-
|
||||
Resolve each postBuild.substituteFrom Kustomization to the files it renders and
|
||||
assert every ${...} is defined or escaped, then assert the gate wires it blocking.
|
||||
try:
|
||||
- script:
|
||||
content: bash ./check.sh
|
||||
@@ -0,0 +1,369 @@
|
||||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# L0 static check (offline, no cluster) — flux-postbuild-substitution
|
||||
# Serves: R-ENV-02 ("Same GitOps config promotes dev/test -> prod (authored once)."
|
||||
# — spec docs/02-environments-and-provisioning.md)
|
||||
# R-GITOPS-01 ("All desired state in git, reconciled by Flux."
|
||||
# — spec docs/05-gitops-and-delivery.md)
|
||||
#
|
||||
# WHY THIS EXISTS — Flux post-build substitution FAILS OPEN
|
||||
# --------------------------------------------------------
|
||||
# R-ENV-02 is implemented by exactly one mechanism: `clusters/base/cluster-vars.yaml`
|
||||
# is injected into every `${TOKEN}` of the manifests each Flux Kustomization renders,
|
||||
# via `postBuild.substituteFrom`. That mechanism is silently lossy by default: an
|
||||
# UNDEFINED variable becomes the EMPTY STRING and the Kustomization still reports
|
||||
# Ready. On 2026-09-01 that emptied `k8sServiceHost` in the Cilium HelmRelease and put
|
||||
# cilium into Init:Error on 2 of 3 dev nodes (DECISIONS-LOG "substitution race that
|
||||
# fails OPEN"). "Authored once, promoted" is therefore only true while every token a
|
||||
# substituted path renders is either DEFINED or deliberately ESCAPED — and nothing in
|
||||
# the repo asserted that. This check does.
|
||||
#
|
||||
# ENVSUBST BEHAVIOUR ASSERTED HERE (empirically verified against the pinned toolchain,
|
||||
# `flux envsubst`, flux 2.4.0 — this is measured behaviour, not assumption):
|
||||
# ${DEFINED} -> substituted with the cluster-vars value.
|
||||
# ${UNDEFINED} -> EMPTY STRING, silently. `--strict` / the cluster-wide
|
||||
# StrictPostBuildSubstitutions feature gate turns this into a
|
||||
# loud Kustomization failure ("variable not set (strict mode)").
|
||||
# ${UNDEFINED:-d} -> the DEFAULT is applied AT RECONCILE TIME. Strict mode does
|
||||
# NOT catch this (it exempts default-operator forms by design),
|
||||
# so a shell guard written `[ -z "${X:-}" ]` inside a rendered
|
||||
# manifest is pre-resolved to `[ -z "" ]` and its else-branch
|
||||
# becomes unreachable in the container. Silent, and permanent.
|
||||
# $${VAR}, $${V:-d}, $${ARR[0]} -> emitted as the LITERAL ${...} for a downstream
|
||||
# shell/cloud-init consumer. Escaping is total: it neutralises
|
||||
# even the indexed form.
|
||||
# $VAR, $(cmd) -> left alone by envsubst; they need no escaping.
|
||||
# ${...}, ${a b}, ${ARR[0]} (UNESCAPED) -> HARD PARSE ERROR ("unable to parse
|
||||
# variable name" / "missing closing brace") which fails the
|
||||
# WHOLE Kustomization — every resource in the path, not just
|
||||
# the offending one.
|
||||
# The escape contract this enforces is the one already documented in
|
||||
# `clusters/base/apps-vms.yaml`; this check is that comment made executable.
|
||||
#
|
||||
# GIVEN the repository as authored,
|
||||
# WHEN every Flux Kustomization under clusters/ that declares postBuild.substituteFrom
|
||||
# is resolved to the exact file set its `spec.path` renders (by following
|
||||
# kustomization.yaml `resources:` transitively — a path, not a glob),
|
||||
# THEN (1) every `${...}` in those files is DEFINED in cluster-vars, or ESCAPED `$${...}`;
|
||||
# (2) no UNESCAPED expansion carries a default/length/substitution operator on an
|
||||
# undefined name (the silent-empty class strict mode does not catch);
|
||||
# (3) no UNESCAPED expansion is unparseable by envsubst (`${...}`, `${ARR[0]}`) —
|
||||
# that fails the entire Kustomization, so it is fatal wherever it appears;
|
||||
# (4) every Kustomization that renders a cluster-vars token actually declares
|
||||
# `substituteFrom` a ConfigMap named cluster-vars (else the literal token
|
||||
# ships into the cluster);
|
||||
# (5) every knob an overlay patches (`/data/KEY`) exists in the BASE ConfigMap —
|
||||
# the "add the knob with a base default BEFORE referencing it" rule that the
|
||||
# incident's interim discipline depends on;
|
||||
# AND (6) this check is LOAD-BEARING: ci/l0.sh runs it on every gate run and
|
||||
# ci/gate.sh sets the merge block when the l0 stage fails.
|
||||
# Assertion (1)-(3) are applied to COMMENTS as well as content: `kustomize build`
|
||||
# preserves body comments into the stream envsubst reads (verified: a `$`-carrying
|
||||
# comment survives into the built apps/wave-1 output), so a braced expression in a
|
||||
# comment is a live hazard, not prose.
|
||||
#
|
||||
# WHAT THIS DOES *NOT* PROVE (honest gaps — this is a STATIC check):
|
||||
# - It does NOT render the manifests. It scans the SOURCE files a path includes,
|
||||
# which is a deliberate SUPERSET of the post-`kustomize build` stream (kustomize
|
||||
# drops some comments). A finding in a comment may not reach envsubst TODAY; it is
|
||||
# still reported, because whether a given comment survives the build is a kustomize
|
||||
# implementation detail that changes with the pinned version, and the failure it
|
||||
# causes is total (assertion 3). The end-to-end proof — `kustomize build <path> |
|
||||
# flux envsubst --strict` for every substituted path — needs kustomize + flux,
|
||||
# which the L0 CI toolchain does not install (.gitea/workflows/scripts/install-tools.sh
|
||||
# l0), so it cannot run here without permanently SKIPPING. It was run by hand and
|
||||
# recorded in DECISIONS-LOG; this check is the reproducible static half.
|
||||
# - It does NOT prove the cluster's live state matches git, nor that the
|
||||
# StrictPostBuildSubstitutions feature gate is actually enabled on
|
||||
# kustomize-controller. Both are cluster facts (L2), untested here. Tagging
|
||||
# R-GITOPS-01 claims ONLY the static slice: what git says a Kustomization renders
|
||||
# is fully determined, with no variable silently emptied on the way in.
|
||||
# - It does NOT validate cluster-vars VALUES (that GATEWAY_VIP is inside the LB pool,
|
||||
# that an overlay's LAN block is self-consistent). Presence, not correctness.
|
||||
# - A cluster-vars key nothing references is reported as an ADVISORY, never a failure:
|
||||
# several are deliberately reference-only and say so in cluster-vars.yaml
|
||||
# (PIA_REGION "retained, not substituted"; CLUSTER_POD_CIDR "Documents ..."), so
|
||||
# failing on them would be a false block on documented intent.
|
||||
# =============================================================================
|
||||
set -euo pipefail
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
|
||||
PYTHON="${PYTHON:-python3}"
|
||||
|
||||
# Honest SKIP (never green) if the thing under test is not present at all.
|
||||
if [ ! -d "${ROOT}/clusters" ] || [ ! -f "${ROOT}/clusters/base/cluster-vars.yaml" ]; then
|
||||
echo "check SKIPPED: clusters/ or clusters/base/cluster-vars.yaml absent — nothing to audit" >&2
|
||||
exit 3
|
||||
fi
|
||||
|
||||
cd "${ROOT}"
|
||||
"${PYTHON}" - "${ROOT}" <<'PY'
|
||||
"""Static audit of Flux postBuild substitution safety. Pure standard library.
|
||||
|
||||
No PyYAML: the file shapes parsed here are line-constrained and the parse contracts are
|
||||
documented at each parser, matching lib/coverage.py + lib/status.py. The kustomization
|
||||
`resources:` parser is IMPORTED from lib/status.py rather than forked, so there is one
|
||||
definition of "what a Flux path actually renders" in the harness.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(sys.argv[1])
|
||||
sys.dont_write_bytecode = True # keep __pycache__ out of the tracked tree
|
||||
sys.path.insert(0, str(ROOT / "conformance" / "lib"))
|
||||
import status as st # noqa: E402 — reuse the walk, don't fork it
|
||||
|
||||
FAILURES: list[str] = []
|
||||
ADVISORIES: list[str] = []
|
||||
CHECKS = 0
|
||||
|
||||
|
||||
def check(cond, msg):
|
||||
global CHECKS
|
||||
CHECKS += 1
|
||||
if not cond:
|
||||
FAILURES.append(msg)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ cluster-vars
|
||||
# Parse contract: `data:` block of a single ConfigMap, one ` KEY: "value"` per line
|
||||
# (every entry in clusters/base/cluster-vars.yaml is a quoted scalar; comments and
|
||||
# continuation-comment lines are indented further and never match).
|
||||
_DATA_KEY_RE = re.compile(r'^ ([A-Za-z_][A-Za-z0-9_]*):\s*(?:"|\')')
|
||||
|
||||
|
||||
def cluster_vars_keys(path: Path) -> set[str]:
|
||||
keys: set[str] = set()
|
||||
in_data = False
|
||||
for raw in path.read_text().splitlines():
|
||||
if re.match(r"^data:\s*$", raw):
|
||||
in_data = True
|
||||
continue
|
||||
if not in_data:
|
||||
continue
|
||||
if st._TOP_KEY_RE.match(raw): # a new top-level key ends the data block
|
||||
break
|
||||
m = _DATA_KEY_RE.match(raw)
|
||||
if m:
|
||||
keys.add(m.group(1))
|
||||
return keys
|
||||
|
||||
|
||||
VARS_FILE = ROOT / "clusters" / "base" / "cluster-vars.yaml"
|
||||
KEYS = cluster_vars_keys(VARS_FILE)
|
||||
check(len(KEYS) > 0, f"no data keys parsed from {VARS_FILE} — the parser or the file shape changed")
|
||||
|
||||
# ------------------------------------------- Flux Kustomizations with substituteFrom
|
||||
# Parse contract mirrors lib/status.py's flux_paths(): docs split on a lone `---`, a doc
|
||||
# is a Flux entrypoint only if it carries the kustomize.toolkit apiVersion AND
|
||||
# `kind: Kustomization` AND a spec-level `path:`. Inline `patch: |-` bodies in the
|
||||
# overlays carry Flux Kustomizations WITHOUT a path and are correctly skipped.
|
||||
_SUBFROM_RE = re.compile(r"^(\s*)substituteFrom:\s*$", re.M)
|
||||
_SUBITEM_KIND_RE = re.compile(r"^\s*-\s*kind:\s*(\S+)")
|
||||
_SUBITEM_NAME_RE = re.compile(r"^\s*(?:-\s*)?name:\s*([^\s#]+)")
|
||||
|
||||
|
||||
def substitute_sources(doc: str) -> list[tuple[str, str]]:
|
||||
"""[(kind, name)] declared under postBuild.substituteFrom in this doc."""
|
||||
m = _SUBFROM_RE.search(doc)
|
||||
if not m:
|
||||
return []
|
||||
indent = len(m.group(1))
|
||||
out: list[tuple[str, str]] = []
|
||||
kind = ""
|
||||
for raw in doc[m.end():].splitlines():
|
||||
if raw.strip() == "" or raw.lstrip().startswith("#"):
|
||||
continue
|
||||
if len(raw) - len(raw.lstrip()) <= indent and not raw.lstrip().startswith("-"):
|
||||
break # dedented out of the list
|
||||
km = _SUBITEM_KIND_RE.match(raw)
|
||||
if km:
|
||||
kind = km.group(1)
|
||||
continue
|
||||
nm = _SUBITEM_NAME_RE.match(raw)
|
||||
if nm and kind:
|
||||
out.append((kind, nm.group(1)))
|
||||
kind = ""
|
||||
return out
|
||||
|
||||
|
||||
kustomizations: list[dict] = []
|
||||
for yfile in sorted((ROOT / "clusters").rglob("*.yaml")):
|
||||
text = yfile.read_text()
|
||||
if st.FLUX_API not in text:
|
||||
continue
|
||||
for doc in st._DOC_SPLIT_RE.split(text):
|
||||
if st.FLUX_API not in doc or not re.search(r"^kind:\s*Kustomization\s*$", doc, re.M):
|
||||
continue
|
||||
pm = st._PATH_RE.search(doc)
|
||||
if not pm:
|
||||
continue
|
||||
nm = st._NAME_RE.search(doc)
|
||||
srcs = substitute_sources(doc)
|
||||
if not srcs:
|
||||
continue # no substitution => nothing to audit
|
||||
kustomizations.append({
|
||||
"file": str(yfile.relative_to(ROOT)),
|
||||
"name": nm.group(1) if nm else "?",
|
||||
"path": pm.group(1),
|
||||
"sources": srcs,
|
||||
})
|
||||
|
||||
check(len(kustomizations) > 0,
|
||||
"no Flux Kustomization with postBuild.substituteFrom found under clusters/ — "
|
||||
"the parser or the repo layout changed; this check would silently audit nothing")
|
||||
|
||||
# ------------------------------------------------------------------ token scanning
|
||||
# One `$`-run followed by a braced expression. `[^{}]*` is exhaustive for this tree:
|
||||
# a repo-wide sweep found no nested/unclosed brace expression in any manifest (only
|
||||
# Gitea Actions `${{ ... }}` in .gitea/, which no Flux Kustomization renders).
|
||||
TOKEN_RE = re.compile(r"(\$+)\{([^{}]*)\}")
|
||||
NAME_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
|
||||
# An expansion whose name is followed by a POSIX parameter operator. envsubst EVALUATES
|
||||
# these, so an undefined name yields the default/empty AT RECONCILE TIME.
|
||||
OP_RE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)(:[-=?+].*|:[0-9].*|/.*|#.*|%.*|\^.*|,.*)$")
|
||||
LEN_RE = re.compile(r"^#([A-Za-z_][A-Za-z0-9_]*)$")
|
||||
|
||||
|
||||
def classify(dollars: int, inner: str) -> tuple[str, str] | None:
|
||||
"""Return (severity-tag, remediation) for a hazardous expansion, else None."""
|
||||
if dollars >= 2:
|
||||
return None # escaped: emitted as a literal
|
||||
if NAME_RE.match(inner):
|
||||
if inner in KEYS:
|
||||
return None # defined knob — the happy path
|
||||
return ("UNDEFINED",
|
||||
f"strict mode fails the Kustomization; today it renders EMPTY. Either add "
|
||||
f"'{inner}' to clusters/base/cluster-vars.yaml with a base default, or "
|
||||
f"escape it as $${{{inner}}} if it is meant for a shell/cloud-init consumer")
|
||||
m = OP_RE.match(inner) or LEN_RE.match(inner)
|
||||
if m:
|
||||
base = m.group(1)
|
||||
if base in KEYS:
|
||||
return None # defined knob with an operator
|
||||
return ("SILENT-EMPTY",
|
||||
f"envsubst resolves this AT RECONCILE TIME because '{base}' is not a "
|
||||
f"cluster-var, so the rendered manifest carries the default, not the "
|
||||
f"expression. StrictPostBuildSubstitutions does NOT catch this form. "
|
||||
f"Escape it as $${{{inner}}} if the CONTAINER's shell is meant to expand it")
|
||||
return ("FATAL-PARSE",
|
||||
"envsubst cannot parse this and fails the ENTIRE Kustomization (every resource "
|
||||
"in the path). Escape it as $${...} or rewrite the prose without braces")
|
||||
|
||||
|
||||
def rendered_files(path_spec: str) -> tuple[list[str], bool]:
|
||||
"""Repo-relative files the Flux `spec.path` renders, resolving `resources:` transitively."""
|
||||
rel = os.path.normpath(path_spec.lstrip("./")) if path_spec not in (".", "./") else "."
|
||||
if not (ROOT / rel).is_dir():
|
||||
return [], False
|
||||
applied, _ = st.reconciled_files(ROOT, rel)
|
||||
return sorted(applied), True
|
||||
|
||||
|
||||
findings: list[str] = []
|
||||
referenced: dict[str, set[str]] = {}
|
||||
inventory: list[str] = []
|
||||
|
||||
for k in kustomizations:
|
||||
files, exists = rendered_files(k["path"])
|
||||
check(exists, f"{k['file']}: Kustomization '{k['name']}' renders path '{k['path']}' "
|
||||
f"which is not a directory in this repo")
|
||||
has_cluster_vars = ("ConfigMap", "cluster-vars") in k["sources"]
|
||||
n_def = n_esc = n_bad = 0
|
||||
for rel in files:
|
||||
f = ROOT / rel
|
||||
try:
|
||||
text = f.read_text()
|
||||
except (OSError, UnicodeDecodeError):
|
||||
continue
|
||||
for lineno, line in enumerate(text.splitlines(), 1):
|
||||
for m in TOKEN_RE.finditer(line):
|
||||
dollars, inner = len(m.group(1)), m.group(2)
|
||||
verdict = classify(dollars, inner)
|
||||
if verdict is None:
|
||||
if dollars >= 2:
|
||||
n_esc += 1
|
||||
else:
|
||||
n_def += 1
|
||||
referenced.setdefault(re.split(r"[:/#%^,]", inner)[0] or inner,
|
||||
set()).add(k["name"])
|
||||
# (4) a DEFINED knob is only substituted if this Kustomization
|
||||
# actually pulls cluster-vars in; otherwise the literal ships.
|
||||
check(has_cluster_vars,
|
||||
f"{rel}:{lineno}: renders cluster-var ${{{inner}}} but "
|
||||
f"Kustomization '{k['name']}' ({k['file']}) does not "
|
||||
f"substituteFrom ConfigMap/cluster-vars — the literal token "
|
||||
f"would ship into the cluster")
|
||||
continue
|
||||
n_bad += 1
|
||||
tag, fix = verdict
|
||||
findings.append(f" [{tag}] {rel}:{lineno} {m.group(0)}\n"
|
||||
f" rendered by Flux Kustomization '{k['name']}' "
|
||||
f"({k['file']}, path {k['path']})\n"
|
||||
f" {fix}\n"
|
||||
f" | {line.strip()[:120]}")
|
||||
inventory.append(f" {k['name']:<16} path={k['path']:<22} files={len(files):<3} "
|
||||
f"defined={n_def:<3} escaped={n_esc:<3} hazards={n_bad}")
|
||||
|
||||
check(not findings,
|
||||
f"{len(findings)} unescaped, undefined or unparseable expansion(s) in substituted paths")
|
||||
|
||||
# --------------------------------------------- (5) overlay knobs exist in the base
|
||||
# Parse contract: overlays patch cluster-vars with JSON6902 ops (`- op: replace` /
|
||||
# `path: /data/KEY`) inside an inline `patch: |-` block. The repo rule (cluster-vars.yaml
|
||||
# header) is that a knob is ADDED TO THE BASE with a default BEFORE any overlay or
|
||||
# manifest references it — that ordering is what closes the define-then-use race.
|
||||
_OP_PATH_RE = re.compile(r"^\s*(?:-\s*)?path:\s*/data/([A-Za-z_][A-Za-z0-9_]*)\s*$", re.M)
|
||||
for overlay in sorted((ROOT / "clusters").glob("*/kustomization.yaml")):
|
||||
for key in set(_OP_PATH_RE.findall(overlay.read_text())):
|
||||
check(key in KEYS,
|
||||
f"{overlay.relative_to(ROOT)}: patches /data/{key} but '{key}' has no BASE "
|
||||
f"default in clusters/base/cluster-vars.yaml — a second overlay would render "
|
||||
f"${{{key}}} empty (R-ENV-02: author once, supply per env)")
|
||||
|
||||
# ------------------------------------------------------- (6) the check is load-bearing
|
||||
L0 = (ROOT / "ci" / "l0.sh").read_text() if (ROOT / "ci" / "l0.sh").is_file() else ""
|
||||
GATE = (ROOT / "ci" / "gate.sh").read_text() if (ROOT / "ci" / "gate.sh").is_file() else ""
|
||||
check("flux-postbuild-substitution/check.sh" in L0,
|
||||
"ci/l0.sh does not run flux-postbuild-substitution/check.sh — the requirement-scoped "
|
||||
"conformance selection can filter this check out of a gate run, so l0.sh must invoke "
|
||||
"it directly and unconditionally or the gate is not actually enforcing it")
|
||||
check(re.search(r"^for s in .*\bl0\b.*; do", GATE, re.M) and re.search(r"fail\).*BLOCK=1", GATE),
|
||||
"ci/gate.sh does not set the merge block when the l0 stage fails")
|
||||
|
||||
# ------------------------------------------------------------- advisory: dead knobs
|
||||
for key in sorted(KEYS - set(referenced)):
|
||||
ADVISORIES.append(f"cluster-vars knob '{key}' is referenced by NO reconciled manifest "
|
||||
f"(dead knob, or documentation-only — advisory, not a failure)")
|
||||
|
||||
# ------------------------------------------------------------------------- report
|
||||
print("Flux postBuild substitution inventory (per Kustomization):")
|
||||
for line in inventory:
|
||||
print(line)
|
||||
print(f"\ncluster-vars knobs defined: {len(KEYS)} referenced: {len(referenced)}")
|
||||
|
||||
if ADVISORIES:
|
||||
print("\nADVISORIES (not failures):")
|
||||
for a in ADVISORIES:
|
||||
print(f" - {a}")
|
||||
|
||||
if findings:
|
||||
print("\nHAZARDS — unescaped expansions that are not cluster-vars knobs:", file=sys.stderr)
|
||||
for f in findings:
|
||||
print(f, file=sys.stderr)
|
||||
|
||||
print()
|
||||
if not FAILURES:
|
||||
print(f"check OK: flux-postbuild-substitution — {CHECKS} assertions "
|
||||
f"(R-ENV-02, R-GITOPS-01)")
|
||||
raise SystemExit(0)
|
||||
print(f"CHECK FAIL (flux-postbuild-substitution): {len(FAILURES)} failed, "
|
||||
f"{CHECKS - len(FAILURES)} passed", file=sys.stderr)
|
||||
for f in FAILURES:
|
||||
print(f" FAIL {f}", file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
PY
|
||||
@@ -2,7 +2,7 @@
|
||||
# Longhorn — distributed replicated block storage across the 3 nodes (ADR-0003).
|
||||
# Serves R-STATE-01 (state survives destroy/rebuild) and R-STATE-02 (3-way replication).
|
||||
#
|
||||
# Dev-appropriate values. Environment differences enter ONLY via ${...} substitution from
|
||||
# Dev-appropriate values. Environment differences enter ONLY via $${...} substitution from
|
||||
# cluster-vars (R-ENV-02); everything else is identical base config for dev and prod.
|
||||
apiVersion: helm.toolkit.fluxcd.io/v2
|
||||
kind: HelmRelease
|
||||
|
||||
@@ -20,7 +20,7 @@ parameters:
|
||||
# (replicaSoftAntiAffinity=false) guarantees these land on distinct nodes, so any single node
|
||||
# loss keeps the volume available.
|
||||
# HARDCODED, not ${LONGHORN_DEFAULT_REPLICA_COUNT}: StorageClass.parameters is map[string]string,
|
||||
# but kustomize strips the quotes around a bare "${VAR}" placeholder, so Flux postBuild substitution
|
||||
# but kustomize strips the quotes around a bare "$${VAR}" placeholder, so Flux postBuild substitution
|
||||
# yields an unquoted numeric (3) that server-side apply rejects (int, not string). A quoted numeric
|
||||
# literal like "3"/"30" IS preserved by kustomize. The replica knob still drives the GLOBAL default
|
||||
# via defaultSettings.defaultReplicaCount in helmrelease.yaml (a Helm int value, unaffected).
|
||||
|
||||
Reference in new issue
Block a user