#!/usr/bin/env bash # ============================================================================= # L0 static check (offline, no cluster) — prod-baremetal-not-dev-surface # Serves: R-ENV-03 ("prod is a 3× mini-PC bare-metal k3s cluster with local storage; it # MUST NOT be an active development surface until the roadmap declares it ready." # — spec docs/00-vision-and-scope.md. Detail in docs/02: prod substrate "3× mini-PC # bare metal", "3 physical nodes", storage "local SSD/NVMe presented to Longhorn", # "For prod bare-metal, the *same* Ansible bootstrap runs; only the machine-creation # step differs", and "**Never develop against prod.** Prod hardware is a target, not # a sandbox (R-ENV-03)". The roadmap gate is roadmap/phases.md Phase 7: entry # "operator declares prod hardware ready"; note "Do not begin building on prod # hardware before this phase is explicitly opened".) # # HOW A "MUST NOT" IS MADE TESTABLE # -------------------------------- # The second clause is a governance rule, and the same discipline applies as to R-TEST-06 # (tests/l0-static/spec-immutability-enforced): "nobody developed against prod today" # cannot be observed, but the PROPERTY THAT MAKES DEVELOPING AGAINST PROD IMPOSSIBLE can # be. In this repo that property is structural — prod receives the base configuration plus # environment-scoped overrides and NOTHING ELSE, so there is no place to put prod-only work # even if someone wanted to. This check asserts that, together with the first clause's # authored facts (three nodes, no VM IaC behind them, node-local storage, the same # bootstrap code as dev). # # The first clause's HARDWARE facts (that the three machines really are mini-PCs, really # are bare metal, really hold local NVMe) are operator-attested, not machine-checkable — # see the gap list. What IS checkable is that the repository describes prod that way and # that no IaC creates prod nodes as virtual machines. # # GIVEN the repository as git tracks it (plus the read-only spec repo when mounted), # WHEN the prod inventory, the prod overlay, the IaC module and the roadmap record are # inspected — and, where offline execution is possible, the provisioning chain is # actually retargeted at prod with `--list-hosts`, # THEN (1) prod resolves to EXACTLY THREE nodes, reached directly on their own sshd # (no NAT indirection, i.e. real machines on the LAN); # (2) NO IaC resource creates any prod node: the VM module's generated node names are # disjoint from the prod host set and no prod hostname appears anywhere under # infra/tofu — prod is imaged by the operator (docs/02 stage 1), not instantiated; # (3) prod's storage is NODE-LOCAL: the Longhorn data path for prod is an absolute # local path, and the replica count equals the node count (one replica per node's # own disk). A distinct bulk/media tier is explicitly permitted (R-STATE-04) and # is not counted against this; # (4) prod runs THE SAME bootstrap code as dev: the same playbooks/roles, retargeted # by variables only — every prod group_vars key is an existing role/global # variable (an invented prod-only knob would be a fork), and no prod-only role # or playbook exists; # (5) prod is NOT a development surface: clusters/prod carries only the base plus # sync.yaml, every patch it applies targets an ENVIRONMENT-SCOPED object # (cluster-vars, a Flux Kustomization's sops decryption, or the Git source), # prod's Git ref is the SAME ref dev tracks (no prod-only branch to develop on), # and no prod-only manifest exists anywhere in the rendered configuration — a file # whose NAME carries the environment token is excused ONLY when the tree proves it # is an OVERLAY-SELECTED half that every environment applies (see the four # conditions at "THE ONE STRUCTURAL EXCEPTION" below); anything else still fails; # (6) prod artifacts exist only WITH a declaration of readiness: the spec roadmap # carries the phase that opens prod build (checked when the spec repo is mounted), # and the repository records that the operator opened it. # # WHAT THIS DOES *NOT* PROVE (honest gaps — do not read this test as claiming them): # - The HARDWARE. That the three nodes are mini-PCs, are bare metal, and present local # SSD/NVMe is operator-attested in infra/PROD-STANDUP.md / PROD-PROVISIONING.md; the # node-health and BIOS surveys that establish it write to gitignored artifacts, so # nothing tracked can be asserted against. Cluster-side, the Kubernetes API exposes no # "am I virtual" fact either — a bare-metal check would need a privileged host-path # probe of DMI data, which is a heavier and more invasive test than the claim warrants. # - The LIVE prod cluster's shape (three Ready nodes, all control-plane+etcd, Longhorn # disks at the local path). That is cluster state and belongs to R-PLAT-01's # (currently uncovered) HA test; asserting it here would also make this test # environment-specific, since the same suite must pass against dev. # - That prod's HISTORY contains no development. This is the working tree at HEAD. # - Whether the roadmap SHOULD have opened prod (the entry-gate judgement). Only that # prod work is accompanied by the operator's recorded declaration. # ============================================================================= set -euo pipefail DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root PYTHON="${PYTHON:-python3}" : "${SPEC_REPO:=${ROOT}/../home-network-k3s}" export SPEC_REPO cd "${ROOT}" "${PYTHON}" - "${ROOT}" <<'PY' import os import re import shutil import subprocess import sys from pathlib import Path try: import yaml except ImportError: # pragma: no cover print("check FAIL: PyYAML is required (same dependency as catalog/render.py)", file=sys.stderr) raise SystemExit(1) ROOT = Path(sys.argv[1]) FAILURES: list[str] = [] NOTES: list[str] = [] CHECKS = 0 def check(cond, msg): global CHECKS CHECKS += 1 if not cond: FAILURES.append(msg) def note(msg): NOTES.append(msg) # A LOCAL DEPENDENCY the repo deliberately does not carry — the vault passphrase (kept # outside the repo, R-SEC-01) or the Galaxy collections (gitignored; installed from # collections/requirements.yml). Their absence is an ENVIRONMENT gap, not a repo defect, # and is the ONLY tolerated reason the driven sub-assertions may not run. Deliberately # narrow: every other non-zero exit is a real failure. The parse assertions always run. ENV_ONLY_FAILURE = re.compile( r"vault password|vault-id|Attempting to decrypt" r"|No module named 'ansible_collections|couldn't resolve module/action", re.I) ANS = ROOT / "infra" / "ansible" INV = ANS / "inventory" / "hosts.ini" GV = ANS / "inventory" / "group_vars" TOFU = ROOT / "infra" / "tofu" check(INV.is_file(), "infra/ansible/inventory/hosts.ini is missing — prod is not described anywhere") if not INV.is_file(): print("CHECK FAIL (prod-baremetal-not-dev-surface): no inventory", file=sys.stderr) raise SystemExit(1) # --------------------------------------------------------------- inventory parsing ini_groups: dict[str, list[dict]] = {} children: dict[str, list[str]] = {} cur = None for line in INV.read_text().splitlines(): s = line.strip() if not s or s.startswith("#"): continue m = re.match(r"^\[([^\]]+)\]$", s) if m: cur = m.group(1) continue if cur is None or cur.endswith(":vars"): continue if cur.endswith(":children"): children.setdefault(cur.split(":")[0], []).append(s.split()[0]) continue parts = s.split() host = {"name": parts[0]} for kv in parts[1:]: if "=" in kv: k, v = kv.split("=", 1) host[k] = v ini_groups.setdefault(cur, []).append(host) def hosts_in(group: str, seen=None) -> list[dict]: seen = seen or set() if group in seen: return [] seen.add(group) out = list(ini_groups.get(group, [])) for child in children.get(group, []): out += hosts_in(child, seen) return out def load_gv(group: str) -> dict: d: dict = {} for p in [GV / f"{group}.yml", GV / group / "vars.yml"]: if p.is_file(): try: d.update(yaml.safe_load(p.read_text()) or {}) except Exception: pass return d # WHICH GROUP IS PROD? Derived, never named here: it is the group whose group_vars # RETARGETS the k3s roles (k3s_primary_group / k3s_servers_group) away from the global # dev default. That is exactly the seam docs/02 + infra/ansible/README describe for prod. ALL_VARS = load_gv("all") dev_primary_group = str(ALL_VARS.get("k3s_primary_group", "")) candidates = [] for entry in sorted(GV.iterdir()) if GV.is_dir() else []: g = entry.name[:-4] if entry.name.endswith(".yml") else entry.name if g == "all": continue v = load_gv(g) if v.get("k3s_primary_group") and str(v["k3s_primary_group"]) != dev_primary_group: candidates.append((g, v)) check(len(candidates) == 1, f"expected exactly ONE environment whose group_vars retargets the k3s roles (prod); found " f"{[c[0] for c in candidates]} — R-ENV-03 describes a single prod estate reached through " "the same roles as dev") if not candidates: print("CHECK FAIL (prod-baremetal-not-dev-surface): no prod group_vars found", file=sys.stderr) for f in FAILURES: print(f" - {f}", file=sys.stderr) raise SystemExit(1) PROD_GROUP, PROD_VARS = candidates[0] PROD_PRIMARY = str(PROD_VARS["k3s_primary_group"]) prod_hosts = hosts_in(PROD_GROUP) prod_names = sorted(h["name"] for h in prod_hosts) # ------------------------------------------------- (1) exactly three, directly reachable check(len(prod_names) == 3, f"prod group {PROD_GROUP!r} resolves to {len(prod_names)} host(s) {prod_names} — R-ENV-03 " "says prod is a 3× mini-PC cluster") dev_hosts = hosts_in(str(ALL_VARS.get("k3s_servers_group", "k3s_servers"))) for h in prod_hosts: # Real machines on the LAN answer their own sshd. Dev's hosts are reached through a # NAT port-forward (ansible_port 225x -> a VM's :22), which is precisely the # indirection a hypervisor-hosted estate needs and bare metal does not. check(str(h.get("ansible_port", "22")) == "22", f"prod node {h['name']} is reached on port {h.get('ansible_port')} — a forwarded port " "implies an indirection (NAT to a virtual estate), not a machine on the LAN") check(h.get("k3s_node_ip") == h.get("ansible_host"), f"prod node {h['name']} has ansible_host={h.get('ansible_host')} but " f"k3s_node_ip={h.get('k3s_node_ip')} — a split address is the dev NAT pattern; a " "bare-metal node's SSH address IS its cluster address") # ------------------------------------------------------ (2) nothing creates prod as a VM TF = "\n".join(p.read_text(errors="replace") for p in sorted(TOFU.glob("*.tf"))) \ if TOFU.is_dir() else "" check(len(TF) > 500, "infra/tofu/*.tf is empty — cannot show that it does NOT create prod") def tf_default(var: str): m = re.search(r'variable\s+"' + re.escape(var) + r'"\s*\{(.*?)\n\}', TF, re.S) if not m: return None d = re.search(r"default\s*=\s*(.+)", m.group(1)) return d.group(1).strip() if d else None prefix = (tf_default("node_name_prefix") or "").strip('"') try: node_count = int((tf_default("node_count") or "0").strip()) except ValueError: node_count = 0 iac_names = {f"{prefix}{i + 1}" for i in range(node_count)} check(len(iac_names) > 0, "the IaC module generates no node names — cannot compare with prod") check(iac_names.isdisjoint(set(prod_names)), f"the IaC VM module would create {sorted(iac_names & set(prod_names))}, which are PROD nodes " "— prod is bare metal that the operator images (docs/02 stage 1), never instantiated by the " "hypervisor module") for name in prod_names: check(name not in TF, f"prod node {name} is named inside infra/tofu — prod hardware must not be described by " "the VM module at all") # The prod host addresses must not appear as IaC node addresses either (a renamed VM is # still a VM). tf_ips = set(re.findall(r'"(\d+\.\d+\.\d+\.\d+)"', tf_default("node_ips") or "")) prod_ips = {str(h.get("ansible_host")) for h in prod_hosts} check(tf_ips.isdisjoint(prod_ips), f"the IaC module addresses {sorted(tf_ips & prod_ips)}, which are prod node addresses") # --------------------------------------------------------------- (3) node-local storage # Every environment's value for a cluster-vars knob (base default + each overlay patch). def env_values(key: str) -> dict[str, str]: vals: dict[str, str] = {} base = ROOT / "clusters" / "base" / "cluster-vars.yaml" if base.is_file(): for d in yaml.safe_load_all(base.read_text()): if isinstance(d, dict) and d.get("kind") == "ConfigMap" \ and (d.get("metadata") or {}).get("name") == "cluster-vars": v = (d.get("data") or {}).get(key) if v is not None: vals["base"] = str(v) for overlay in sorted((ROOT / "clusters").glob("*/kustomization.yaml")): env = overlay.parent.name if env == "base": continue try: k = yaml.safe_load(overlay.read_text()) or {} except Exception: continue for p in (k.get("patches") or []): tgt = p.get("target") or {} if tgt.get("kind") != "ConfigMap" or tgt.get("name") != "cluster-vars": continue try: ops = yaml.safe_load(p.get("patch") or "") or [] except Exception: continue for op in ops if isinstance(ops, list) else []: if isinstance(op, dict) and str(op.get("path", "")) == f"/data/{key}": vals[env] = str(op.get("value")) return vals # The prod OVERLAY is the one whose CLUSTER_ENV is not the base default. cluster_envs = env_values("CLUSTER_ENV") prod_overlays = [e for e, v in cluster_envs.items() if e != "base" and v != cluster_envs.get("base")] check(len(prod_overlays) >= 1, "no clusters/ overlay sets its own CLUSTER_ENV — prod has no overlay of its own") prod_overlay = None for e in prod_overlays: if PROD_GROUP.replace("k3s_", "") in cluster_envs[e] or cluster_envs[e] in PROD_GROUP: prod_overlay = e check(prod_overlay is not None, f"no overlay's CLUSTER_ENV corresponds to the prod inventory group {PROD_GROUP!r} — the " "cluster config and the machine inventory disagree about which estate is prod") data_paths = env_values("LONGHORN_DATA_PATH") replicas = env_values("LONGHORN_DEFAULT_REPLICA_COUNT") if prod_overlay: p = data_paths.get(prod_overlay, data_paths.get("base", "")) check(p.startswith("/"), f"prod's LONGHORN_DATA_PATH is {p!r} — R-ENV-03's 'local storage' requires an absolute " "node-local filesystem path") check(not re.match(r"^/(net|nfs|mnt/nfs)|:", p), f"prod's LONGHORN_DATA_PATH {p!r} looks like a network mount, not local SSD/NVMe") r = replicas.get(prod_overlay, replicas.get("base", "")) check(r.isdigit() and int(r) == len(prod_names), f"prod's LONGHORN_DEFAULT_REPLICA_COUNT is {r!r} for {len(prod_names)} node(s) — " "'local storage' on a 3-node estate means one replica per node's own disk") # The DEFAULT storage class must be the local (Longhorn) driver. A separate bulk/media tier # on network storage is explicitly a different requirement (R-STATE-04, docs/04) and is not # counted here. default_scs = [] for f in subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z", "*.yaml", "*.yml"], capture_output=True, text=True, check=True).stdout.split("\0"): if not f: continue try: for d in yaml.safe_load_all((ROOT / f).read_text(errors="replace")): if isinstance(d, dict) and d.get("kind") == "StorageClass" and \ str(((d.get("metadata") or {}).get("annotations") or {}) .get("storageclass.kubernetes.io/is-default-class", "")) == "true": default_scs.append((f, str(d.get("provisioner", "")))) except Exception: continue check(len(default_scs) == 1, f"expected exactly one default StorageClass, found {default_scs} — the tier prod's workloads " "land on by default must be unambiguous") for f, prov in default_scs: check(prov == "driver.longhorn.io", f"{f}: the default StorageClass provisioner is {prov!r}, not Longhorn — prod's default " "tier would not be its own local disks") # ------------------------------------- (4) prod runs the SAME bootstrap code, retargeted PLAYBOOKS = ANS / "playbooks" ROLES = ANS / "roles" prod_token = PROD_GROUP.replace("k3s_", "") for d, what in ((ROLES, "role"), (PLAYBOOKS, "playbook")): for entry in sorted(d.iterdir()) if d.is_dir() else []: check(prod_token not in entry.name.lower(), f"infra/ansible/{d.name}/{entry.name} is a prod-specific {what} — docs/02 requires " "the SAME Ansible bootstrap for prod, differing only in the machine-creation step") # Every prod group_vars key must be a variable the shared code already knows: a role default, # a global, or an ansible built-in. An invented key is a fork of the role by another name. known: set[str] = set(ALL_VARS) for rd in sorted(ROLES.iterdir()) if ROLES.is_dir() else []: for p in (rd / "defaults").glob("*.yml") if (rd / "defaults").is_dir() else []: try: known |= set((yaml.safe_load(p.read_text()) or {}).keys()) except Exception: pass check(len(known) > 20, f"only {len(known)} shared variables found — (4) would prove nothing") for k in sorted(PROD_VARS): check(k.startswith("ansible_") or k in known, f"prod group_vars sets {k!r}, which is not a role default or a global — prod-only " "configuration that the dev path never exercises is developing against prod") # ...and it must retarget, not re-implement: the group it names must exist in the inventory. check(PROD_PRIMARY in ini_groups or PROD_PRIMARY in children, f"prod group_vars names primary group {PROD_PRIMARY!r}, which the inventory does not define") # ---------------------------------------- (5) prod is not a development surface (overlay) if prod_overlay: odir = ROOT / "clusters" / prod_overlay files = sorted(p.name for p in odir.iterdir() if p.is_file()) check(files == ["kustomization.yaml", "sync.yaml"], f"clusters/{prod_overlay}/ contains {files} — a prod overlay may hold only its " "kustomization and its own root sync; anything else is configuration authored FOR PROD") k = yaml.safe_load((odir / "kustomization.yaml").read_text()) or {} check(sorted(str(r) for r in (k.get("resources") or [])) == ["../base", "sync.yaml"], f"clusters/{prod_overlay} resources are {k.get('resources')} — prod must render the base " "plus its own sync only (docs/02: 'apply the prod overlay (no base edits)')") # Every patch must be an ENVIRONMENT-SCOPED override, never a change to what runs. ALLOWED = {("ConfigMap", "cluster-vars"), ("GitRepository", None)} for p in (k.get("patches") or []): tgt = p.get("target") or {} body = p.get("patch") or "" kind = tgt.get("kind") if kind is None: # inline patch: kind comes from the body try: doc = yaml.safe_load(body) or {} except Exception: doc = {} kind = doc.get("kind") name = (doc.get("metadata") or {}).get("name") if kind == "Kustomization": # Flux Kustomization patches are allowed ONLY to wire sops decryption — # anything else (path, interval, images, values) changes what prod RUNS. keys = set((doc.get("spec") or {}).keys()) check(keys <= {"decryption"}, f"clusters/{prod_overlay}: patch of Kustomization/{name} touches {sorted(keys)} " "— only `decryption` (the per-cluster sops key) is an environment-scoped " "override; changing anything else is developing against prod") continue check((kind, None) in ALLOWED or (kind, name) in ALLOWED, f"clusters/{prod_overlay}: patch targets {kind}/{name}, which is not an " "environment-scoped object — prod must not carry configuration of its own") else: check((kind, tgt.get("name")) in ALLOWED or (kind, None) in ALLOWED, f"clusters/{prod_overlay}: patch targets {kind}/{tgt.get('name')}, which is not " "an environment-scoped object — prod must not carry configuration of its own") # Same Git ref as the other environment(s): a prod-only branch is a development line # that only prod runs, which is exactly what R-ENV-03 forbids. def git_ref(overlay: str): kk = yaml.safe_load((ROOT / "clusters" / overlay / "kustomization.yaml").read_text()) or {} for p in (kk.get("patches") or []): body = p.get("patch") or "" try: doc = yaml.safe_load(body) or {} except Exception: continue if doc.get("kind") == "GitRepository": return (doc.get("spec") or {}).get("ref") return None others = [o.name for o in (ROOT / "clusters").iterdir() if o.is_dir() and o.name not in ("base", prod_overlay)] prod_ref = git_ref(prod_overlay) check(prod_ref is not None, f"clusters/{prod_overlay} does not pin a Git ref — prod would follow whatever the base " "points at, with no record of what it runs") for o in others: check(git_ref(o) == prod_ref, f"clusters/{o} tracks {git_ref(o)} but prod tracks {prod_ref} — a prod-only ref is a " "development line only prod runs (R-ENV-03: prod is a target, not a sandbox)") # No prod-only manifest anywhere in the rendered configuration. tracked = [f for f in subprocess.run(["git", "-C", str(ROOT), "ls-files", "-z"], capture_output=True, text=True, check=True ).stdout.split("\0") if f] RENDERED_DIRS = ("platform/", "platform-config", "apps/", "catalog/", "clusters/base/") # THE ONE STRUCTURAL EXCEPTION: AN OVERLAY-SELECTED ENVIRONMENT HALF. # ------------------------------------------------------------------ # A filename carrying an environment token is a HEURISTIC for the harm R-ENV-03 names # (content only prod runs, in a path every environment renders). It is not the harm # itself. There is exactly one shape where a `-prod` name is the OPPOSITE of a prod-only # fork: a credential that is TWO Secret objects, BOTH applied to BOTH clusters as base # resources, where a cluster-vars knob decides only which one the shared consumer # REFERENCES (clusters/base/cluster-vars.yaml, BACKUP_CREDENTIALS_SUFFIX: dev backs up to # MinIO, prod to Cloudflare R2, and one shared Secret meant either environment's rotation # broke the other). Nothing is forked: every environment renders the identical manifests # and the identical object set, which is precisely the R-ENV-02 contract. # # So the exception is not a path allowlist — it is the PROPERTY that makes that shape # safe, asserted from the tree, and a file is excused ONLY when all of it holds: # # (A) it declares nothing but Secrets whose names end in the environment suffix — the # only kind of object this repo has a selection mechanism for; # (1) it is listed in its own directory's kustomization.yaml, so EVERY environment # applies it, and every environment's value of the selector resolves to an object # that some resource of that same kustomization declares (same namespace) — no # environment is left selecting a Secret that does not exist; # (2) nothing references the suffixed name literally: every reference is a substituted # expression `${VAR}`, and the suffixed FILE is named only by the # kustomization that applies it — so no shared manifest hardcodes prod's half; # (3) that VAR is defined for the base AND for every overlay (StrictPostBuildSubstitutions # fails on an UNDEFINED variable, so an overlay that forgot it would break that # cluster's reconciliation), and prod's value is the one that selects THIS file. # # References are read from PARSED YAML VALUES, never from raw text: the mechanism is # documented by name in comments beside the resources (cluster-vars says so explicitly), # and a name in a comment references nothing. Anything that fails any condition still # fails, naming the condition it broke. ENV_SUFFIX = f"-{prod_token}" # Every environment that renders the shared configuration. ENVS = ["base"] + sorted(p.parent.name for p in (ROOT / "clusters").glob("*/kustomization.yaml") if p.parent.name != "base") def _docs(rel: str): p = ROOT / rel if not p.is_file(): return None try: return [d for d in yaml.safe_load_all(p.read_text(errors="replace")) if isinstance(d, dict)] except Exception: return None def _strings(node): if isinstance(node, str): yield node elif isinstance(node, dict): for k, v in node.items(): yield from _strings(k) yield from _strings(v) elif isinstance(node, (list, tuple)): for v in node: yield from _strings(v) RENDERED_YAML = [f for f in tracked if f.startswith(RENDERED_DIRS) and f.endswith((".yaml", ".yml"))] VALUE_STRINGS = {f: list(_strings(_docs(f) or [])) for f in RENDERED_YAML} def overlay_selected(f: str) -> list[str]: """[] if f is an overlay-SELECTED environment half; else the conditions it breaks.""" docs = _docs(f) objs = [((d.get("kind")), (d.get("metadata") or {}).get("name"), (d.get("metadata") or {}).get("namespace")) for d in (docs or [])] if not objs or any(k != "Secret" or not isinstance(n, str) or not n.endswith(ENV_SUFFIX) for k, n, _ in objs): return [f"(A) it does not consist solely of Secret objects whose names end in " f"{ENV_SUFFIX!r} — an overlay can select which SECRET a shared manifest " f"references, and nothing else, so any other prod-suffixed object in a " f"shared path is prod-only content"] reasons: list[str] = [] d = str(Path(f).parent) kfile = f"{d}/kustomization.yaml" kdocs = _docs(kfile) or [] res = [str(r) for r in ((kdocs[0].get("resources") if kdocs else None) or [])] if Path(f).name not in res: reasons.append(f"(1) {kfile} does not list it in `resources`, so it is not applied " f"in every environment — one environment would carry an object the " f"other does not") # Every Secret the applying kustomization declares: name -> namespace. declared: dict[str, object] = {} for r in res: for dd in (_docs(f"{d}/{r}") or []): if dd.get("kind") == "Secret": md = dd.get("metadata") or {} declared[str(md.get("name"))] = md.get("namespace") for _, name, ns in objs: base_name = name[:-len(ENV_SUFFIX)] pat = re.escape(base_name) + r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}" sel = sorted({m for ss in VALUE_STRINGS.values() for s in ss for m in re.findall(pat, s)}) # (2) literal references to the suffixed OBJECT — anywhere but this file, except # the applying kustomization naming this FILE (which is how it gets applied). lit = sorted({f2 for f2, ss in VALUE_STRINGS.items() for s in ss if name in s and f2 != f and not (f2 == kfile and s == Path(f).name)}) if lit: reasons.append(f"(2) {', '.join(lit)} names the Secret {name!r} literally — a " f"shared manifest that hardcodes prod's half makes every " f"environment use prod's credential; reference " f"{base_name}${{...}} instead") # (2) literal references to the suffixed FILE from anywhere but its kustomization. stray = sorted({f2 for f2, ss in VALUE_STRINGS.items() for s in ss if Path(f).name in s and f2 not in (f, kfile)}) if stray: reasons.append(f"(2) {', '.join(stray)} names the file {Path(f).name!r} " f"literally — only {kfile} may apply it") if not sel: reasons.append(f"(2) no manifest references {base_name}${{...}}: nothing selects " f"between the halves, so this file is prod-only content that " f"only prod could ever consume") continue if len(sel) > 1: reasons.append(f"(2) {base_name} is selected by more than one variable {sel} — " f"they can drift apart and leave one consumer on the other " f"environment's credential") var = sel[0] vals = env_values(var) missing = [e for e in ENVS if e not in vals] if missing: reasons.append(f"(3) ${{{var}}} is not defined for {missing} (defined for " f"{sorted(vals)}) — StrictPostBuildSubstitutions fails on an " f"undefined variable, so that environment would stop " f"reconciling, or silently inherit the other's credential") for e, v in sorted(vals.items()): want = base_name + v if want not in declared: reasons.append(f"(1) environment {e!r} selects Secret {want!r} via " f"${{{var}}}={v!r}, which no resource of {kfile} declares — " f"that environment renders a dangling reference") elif declared[want] != ns: reasons.append(f"(1) environment {e!r} selects Secret {want!r}, declared in " f"namespace {declared[want]!r}, but this half is in {ns!r} — " f"the halves are not interchangeable") if prod_overlay and prod_overlay in vals and base_name + vals[prod_overlay] != name: reasons.append(f"(3) the prod overlay's ${{{var}}}={vals[prod_overlay]!r} selects " f"{base_name + vals[prod_overlay]!r}, not this file's {name!r} — " f"prod's half is applied but no environment selects it") return reasons flagged = [f for f in tracked if f.startswith(RENDERED_DIRS) and re.search(rf"(^|[/_.-]){re.escape(prod_token)}([/_.-]|$)", f, re.I)] for f in flagged: why = overlay_selected(f) check(not why, f"{f} is a prod-specific file inside the shared configuration — every environment " f"renders these paths, so prod-only content here is development against prod. It is " f"not an overlay-selected environment half either: " + " AND ".join(why)) if not why: note(f"{f} carries the {prod_token!r} token but is an OVERLAY-SELECTED half: both " f"halves are applied in every environment and only the cluster-vars selector " f"differs (R-ENV-02), so it is not prod-only content") check(len(tracked) > 50, f"git ls-files returned only {len(tracked)} paths — not a real checkout") # --------------------------- (6) prod work exists only with a declaration of readiness # In-repo: the operator's recorded decision to open the prod phase. log = ROOT / "DECISIONS-LOG.md" check(log.is_file() and re.search(r"PHASE\s*7\b.*OPEN", log.read_text(errors="replace"), re.I), "DECISIONS-LOG.md carries no record of the operator OPENING the prod phase, yet prod " "inventory/overlay exist — R-ENV-03 allows prod build only once the roadmap declares it ready") # In the spec (read-only, may not be mounted): the roadmap phase that opens prod. spec = Path(os.environ.get("SPEC_REPO", "")) phases = spec / "roadmap" / "phases.md" if phases.is_file(): txt = phases.read_text(errors="replace") check(re.search(r"##\s*Phase\s*7\b", txt) is not None and re.search(r"prod", txt, re.I) is not None, "the spec roadmap has no Phase 7 opening prod build — the declaration R-ENV-03 defers " "to does not exist") check(re.search(r"explicitly opened|declares prod hardware ready|operator declares", txt, re.I) is not None, "the spec roadmap's prod phase states no explicit entry declaration — the gate R-ENV-03 " "names would be unenforceable") else: note(f"spec repo not mounted at {spec} — the roadmap half of (6) was NOT checked " "(the in-repo declaration still was)") # --------------------------- (4b) DRIVE the retarget offline: same chain, prod hosts if shutil.which("ansible-playbook") is None: note("ansible-playbook is not installed here (e.g. the in-cluster chainsaw image), so the " "prod retarget was PARSED but not DRIVEN; the merge gate's container has it") else: sec_groups = [g for g in children.get(PROD_GROUP, []) if g != PROD_PRIMARY] args = ["--list-hosts", "playbooks/site.yml", "-e", f"k3s_hosts={PROD_GROUP}", "-e", f"k3s_primary_hosts={PROD_PRIMARY}"] if sec_groups: args += ["-e", f"k3s_secondary_hosts={sec_groups[0]}"] lh = subprocess.run(["ansible-playbook", *args], cwd=ANS, capture_output=True, text=True) out = (lh.stderr or "") + (lh.stdout or "") if lh.returncode != 0 and ENV_ONLY_FAILURE.search(out): # See the same narrow, self-policing skip in dev-nodes-provisioned-as-code. note("ansible-playbook lacks a LOCAL dependency kept outside the repo by design (the vault " "passphrase and/or the gitignored Galaxy collections), so the prod retarget was not " "driven; every parse assertion above still ran") else: check(lh.returncode == 0, f"retargeting the SHARED chain at prod failed: {out.strip()[:400]} — docs/02 requires " "the same Ansible bootstrap to run for prod") seen: set[str] = set() in_hosts = False for line in lh.stdout.splitlines(): if re.match(r"\s*hosts \(\d+\):", line): in_hosts = True continue if re.search(r"play #\d+", line): in_hosts = False continue if in_hosts and line.strip(): seen.add(line.strip()) check(seen == set(prod_names), f"the shared chain retargeted at prod resolves to {sorted(seen)}, not the three prod " f"nodes {prod_names} — prod would be bootstrapped by something other than dev's path") # --------------------------------------------------------------------------- verdict check(CHECKS >= 25, f"only {CHECKS} assertions ran — the check degenerated to a near-no-op and its pass " "would be meaningless") if FAILURES: print(f"CHECK FAIL (prod-baremetal-not-dev-surface): {len(FAILURES)} of {CHECKS} assertions failed", file=sys.stderr) for f in FAILURES: print(f" - {f}", file=sys.stderr) raise SystemExit(1) for n in NOTES: print(f" note: {n}") print(f"check OK: prod-baremetal-not-dev-surface — {CHECKS} assertions; prod group {PROD_GROUP!r} = " f"{prod_names} (3 nodes, none created by IaC), overlay clusters/{prod_overlay} renders the base " f"plus environment-scoped patches only, local Longhorn path " f"{data_paths.get(prod_overlay, data_paths.get('base'))!r} with " f"{replicas.get(prod_overlay, replicas.get('base'))} replica(s). R-ENV-03's AUTHORED clauses " "only — the hardware facts (mini-PC, bare metal, local NVMe) are operator-attested and the " "live cluster's shape is NOT asserted here (see the header's gap list).") PY