fix(conformance): R-ENV-03 shared-path rule tests the harm, not the filename
The blanket regex around RENDERED_DIRS failed any tracked file whose name
carries a delimited `prod` token. That caught the four per-environment backup
credential halves (BACKUP_CREDENTIALS_SUFFIX) even though BOTH halves are
applied to BOTH clusters as base resources and the overlay selects only which
one the shared consumer REFERENCES — i.e. exactly the R-ENV-02 contract the
regex exists to protect. Renaming the files to dodge the regex was rejected as
gaming the guard, so the rule now encodes the property instead.
The fix is not an allowlist of those four paths. A prod-token file in a shared
path is excused only when the tree proves all of:
(A) it declares nothing but Secrets whose names end in the environment suffix
(1) its own directory's kustomization applies it, and EVERY environment value
of the selector resolves to a Secret that some resource of that same
kustomization declares, in the same namespace
(2) nothing names the suffixed Secret — or the suffixed file — literally;
every reference is `<base-name>${VAR}`
(3) that VAR is defined for the base and for every overlay, and prod's value
is the one that selects this file
References are read from parsed YAML values, never raw text: the mechanism is
spelled out by name in comments beside the resources, and a comment references
nothing.
No other assertion is weakened; the check still runs 96 assertions, and the
pass output now notes each excused file and why.
This commit is contained in:
jakman85 committed
1 parent
863af01062
commit
730d12ddc4
1 file changed
+166
-6
@@ -437,12 +437,172 @@ tracked = [f for f in subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z"],
|
||||
capture_output=True, text=True, check=True
|
||||
).stdout.split("\0") if f]
|
||||
RENDERED_DIRS = ("platform/", "platform-config", "apps/", "catalog/", "clusters/base/")
|
||||
for f in tracked:
|
||||
if f.startswith(RENDERED_DIRS) and re.search(rf"(^|[/_.-]){re.escape(prod_token)}([/_.-]|$)",
|
||||
f, re.I):
|
||||
check(False,
|
||||
f"{f} is a prod-specific file inside the shared configuration — every environment "
|
||||
"renders these paths, so prod-only content here is development against prod")
|
||||
|
||||
# THE ONE STRUCTURAL EXCEPTION: AN OVERLAY-SELECTED ENVIRONMENT HALF.
|
||||
# ------------------------------------------------------------------
|
||||
# A filename carrying an environment token is a HEURISTIC for the harm R-ENV-03 names
|
||||
# (content only prod runs, in a path every environment renders). It is not the harm
|
||||
# itself. There is exactly one shape where a `-prod` name is the OPPOSITE of a prod-only
|
||||
# fork: a credential that is TWO Secret objects, BOTH applied to BOTH clusters as base
|
||||
# resources, where a cluster-vars knob decides only which one the shared consumer
|
||||
# REFERENCES (clusters/base/cluster-vars.yaml, BACKUP_CREDENTIALS_SUFFIX: dev backs up to
|
||||
# MinIO, prod to Cloudflare R2, and one shared Secret meant either environment's rotation
|
||||
# broke the other). Nothing is forked: every environment renders the identical manifests
|
||||
# and the identical object set, which is precisely the R-ENV-02 contract.
|
||||
#
|
||||
# So the exception is not a path allowlist — it is the PROPERTY that makes that shape
|
||||
# safe, asserted from the tree, and a file is excused ONLY when all of it holds:
|
||||
#
|
||||
# (A) it declares nothing but Secrets whose names end in the environment suffix — the
|
||||
# only kind of object this repo has a selection mechanism for;
|
||||
# (1) it is listed in its own directory's kustomization.yaml, so EVERY environment
|
||||
# applies it, and every environment's value of the selector resolves to an object
|
||||
# that some resource of that same kustomization declares (same namespace) — no
|
||||
# environment is left selecting a Secret that does not exist;
|
||||
# (2) nothing references the suffixed name literally: every reference is a substituted
|
||||
# expression `<base-name>${VAR}`, and the suffixed FILE is named only by the
|
||||
# kustomization that applies it — so no shared manifest hardcodes prod's half;
|
||||
# (3) that VAR is defined for the base AND for every overlay (StrictPostBuildSubstitutions
|
||||
# fails on an UNDEFINED variable, so an overlay that forgot it would break that
|
||||
# cluster's reconciliation), and prod's value is the one that selects THIS file.
|
||||
#
|
||||
# References are read from PARSED YAML VALUES, never from raw text: the mechanism is
|
||||
# documented by name in comments beside the resources (cluster-vars says so explicitly),
|
||||
# and a name in a comment references nothing. Anything that fails any condition still
|
||||
# fails, naming the condition it broke.
|
||||
ENV_SUFFIX = f"-{prod_token}"
|
||||
# Every environment that renders the shared configuration.
|
||||
ENVS = ["base"] + sorted(p.parent.name
|
||||
for p in (ROOT / "clusters").glob("*/kustomization.yaml")
|
||||
if p.parent.name != "base")
|
||||
|
||||
|
||||
def _docs(rel: str):
|
||||
p = ROOT / rel
|
||||
if not p.is_file():
|
||||
return None
|
||||
try:
|
||||
return [d for d in yaml.safe_load_all(p.read_text(errors="replace"))
|
||||
if isinstance(d, dict)]
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _strings(node):
|
||||
if isinstance(node, str):
|
||||
yield node
|
||||
elif isinstance(node, dict):
|
||||
for k, v in node.items():
|
||||
yield from _strings(k)
|
||||
yield from _strings(v)
|
||||
elif isinstance(node, (list, tuple)):
|
||||
for v in node:
|
||||
yield from _strings(v)
|
||||
|
||||
|
||||
RENDERED_YAML = [f for f in tracked
|
||||
if f.startswith(RENDERED_DIRS) and f.endswith((".yaml", ".yml"))]
|
||||
VALUE_STRINGS = {f: list(_strings(_docs(f) or [])) for f in RENDERED_YAML}
|
||||
|
||||
|
||||
def overlay_selected(f: str) -> list[str]:
|
||||
"""[] if f is an overlay-SELECTED environment half; else the conditions it breaks."""
|
||||
docs = _docs(f)
|
||||
objs = [((d.get("kind")), (d.get("metadata") or {}).get("name"),
|
||||
(d.get("metadata") or {}).get("namespace")) for d in (docs or [])]
|
||||
if not objs or any(k != "Secret" or not isinstance(n, str) or not n.endswith(ENV_SUFFIX)
|
||||
for k, n, _ in objs):
|
||||
return [f"(A) it does not consist solely of Secret objects whose names end in "
|
||||
f"{ENV_SUFFIX!r} — an overlay can select which SECRET a shared manifest "
|
||||
f"references, and nothing else, so any other prod-suffixed object in a "
|
||||
f"shared path is prod-only content"]
|
||||
|
||||
reasons: list[str] = []
|
||||
d = str(Path(f).parent)
|
||||
kfile = f"{d}/kustomization.yaml"
|
||||
kdocs = _docs(kfile) or []
|
||||
res = [str(r) for r in ((kdocs[0].get("resources") if kdocs else None) or [])]
|
||||
if Path(f).name not in res:
|
||||
reasons.append(f"(1) {kfile} does not list it in `resources`, so it is not applied "
|
||||
f"in every environment — one environment would carry an object the "
|
||||
f"other does not")
|
||||
|
||||
# Every Secret the applying kustomization declares: name -> namespace.
|
||||
declared: dict[str, object] = {}
|
||||
for r in res:
|
||||
for dd in (_docs(f"{d}/{r}") or []):
|
||||
if dd.get("kind") == "Secret":
|
||||
md = dd.get("metadata") or {}
|
||||
declared[str(md.get("name"))] = md.get("namespace")
|
||||
|
||||
for _, name, ns in objs:
|
||||
base_name = name[:-len(ENV_SUFFIX)]
|
||||
pat = re.escape(base_name) + r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}"
|
||||
sel = sorted({m for ss in VALUE_STRINGS.values() for s in ss
|
||||
for m in re.findall(pat, s)})
|
||||
# (2) literal references to the suffixed OBJECT — anywhere but this file, except
|
||||
# the applying kustomization naming this FILE (which is how it gets applied).
|
||||
lit = sorted({f2 for f2, ss in VALUE_STRINGS.items() for s in ss
|
||||
if name in s and f2 != f
|
||||
and not (f2 == kfile and s == Path(f).name)})
|
||||
if lit:
|
||||
reasons.append(f"(2) {', '.join(lit)} names the Secret {name!r} literally — a "
|
||||
f"shared manifest that hardcodes prod's half makes every "
|
||||
f"environment use prod's credential; reference "
|
||||
f"{base_name}${{...}} instead")
|
||||
# (2) literal references to the suffixed FILE from anywhere but its kustomization.
|
||||
stray = sorted({f2 for f2, ss in VALUE_STRINGS.items() for s in ss
|
||||
if Path(f).name in s and f2 not in (f, kfile)})
|
||||
if stray:
|
||||
reasons.append(f"(2) {', '.join(stray)} names the file {Path(f).name!r} "
|
||||
f"literally — only {kfile} may apply it")
|
||||
if not sel:
|
||||
reasons.append(f"(2) no manifest references {base_name}${{...}}: nothing selects "
|
||||
f"between the halves, so this file is prod-only content that "
|
||||
f"only prod could ever consume")
|
||||
continue
|
||||
if len(sel) > 1:
|
||||
reasons.append(f"(2) {base_name} is selected by more than one variable {sel} — "
|
||||
f"they can drift apart and leave one consumer on the other "
|
||||
f"environment's credential")
|
||||
var = sel[0]
|
||||
vals = env_values(var)
|
||||
missing = [e for e in ENVS if e not in vals]
|
||||
if missing:
|
||||
reasons.append(f"(3) ${{{var}}} is not defined for {missing} (defined for "
|
||||
f"{sorted(vals)}) — StrictPostBuildSubstitutions fails on an "
|
||||
f"undefined variable, so that environment would stop "
|
||||
f"reconciling, or silently inherit the other's credential")
|
||||
for e, v in sorted(vals.items()):
|
||||
want = base_name + v
|
||||
if want not in declared:
|
||||
reasons.append(f"(1) environment {e!r} selects Secret {want!r} via "
|
||||
f"${{{var}}}={v!r}, which no resource of {kfile} declares — "
|
||||
f"that environment renders a dangling reference")
|
||||
elif declared[want] != ns:
|
||||
reasons.append(f"(1) environment {e!r} selects Secret {want!r}, declared in "
|
||||
f"namespace {declared[want]!r}, but this half is in {ns!r} — "
|
||||
f"the halves are not interchangeable")
|
||||
if prod_overlay and prod_overlay in vals and base_name + vals[prod_overlay] != name:
|
||||
reasons.append(f"(3) the prod overlay's ${{{var}}}={vals[prod_overlay]!r} selects "
|
||||
f"{base_name + vals[prod_overlay]!r}, not this file's {name!r} — "
|
||||
f"prod's half is applied but no environment selects it")
|
||||
return reasons
|
||||
|
||||
|
||||
flagged = [f for f in tracked
|
||||
if f.startswith(RENDERED_DIRS)
|
||||
and re.search(rf"(^|[/_.-]){re.escape(prod_token)}([/_.-]|$)", f, re.I)]
|
||||
for f in flagged:
|
||||
why = overlay_selected(f)
|
||||
check(not why,
|
||||
f"{f} is a prod-specific file inside the shared configuration — every environment "
|
||||
f"renders these paths, so prod-only content here is development against prod. It is "
|
||||
f"not an overlay-selected environment half either: " + " AND ".join(why))
|
||||
if not why:
|
||||
note(f"{f} carries the {prod_token!r} token but is an OVERLAY-SELECTED half: both "
|
||||
f"halves are applied in every environment and only the cluster-vars selector "
|
||||
f"differs (R-ENV-02), so it is not prod-only content")
|
||||
check(len(tracked) > 50, f"git ls-files returned only {len(tracked)} paths — not a real checkout")
|
||||
|
||||
# --------------------------- (6) prod work exists only with a declaration of readiness
|
||||
|
||||
Reference in new issue
Block a user