merge: coverage debt paid — 51/51, and the root really did hide a service
R-C2-01 and R-C2-02 were the last two uncovered requirements (49/51). Now
100% with 0 gaps, via an offline L0 check (49 assertions) that compares two
repository facts: every home-zone hostname the repo routes over HTTP must be
linked from the apex dashboard, and every link the dashboard offers must
resolve to something the repo actually serves at L7 or L4.
'A service the root hid' was still a live defect: crafty.${BASE_DOMAIN} was
served and reconciled while nothing on the root linked to it. Committed RED
first so the history shows the check catching it, then fixed.
Also fixes a regex that could never have worked: ci/touched-requirements.sh
matched R-[A-Z]+-[0-9]+, where [A-Z]+ eats the C of R-C2-01 and then demands
a dash where the 2 is — so a manifest declaring either ID attributed nothing.
Every other consumer already used [A-Z0-9]+.
16 mutations proven caught. R-TLS-02's drafted amendment is recovered into
DECISIONS-LOG: revocation is not implementable with cert-manager, and the
choice between amending the requirement and a reviewed DOCUMENTED_GAPS entry
is the owner's.
# Conflicts:
# DECISIONS-LOG.md
This commit is contained in:
jakman85 committed
commit
088ff13d15
8 files changed
+921
-4
No files matched your search
@@ -3163,3 +3163,84 @@ scaffold containing 8 `TODO` echoes and asserts none of the behaviours it names
|
||||
bound, default-deny, egress class, DNS/TLS, metrics, update policy). This is a concrete instance of
|
||||
the fail-open the `feat/coverage-debt-paydown` and `fix/touched-requirements-fail-closed` branches are
|
||||
about: a stage that reports green while verifying nothing.
|
||||
## R-TLS-02 — the "revoked" clause still needs the OWNER's decision (recovered 2026-09-13)
|
||||
|
||||
**This is a decision only the repo owner/architect can make. It has now been escalated twice and
|
||||
lost twice, which is why it is written into this log rather than into another branch's commit
|
||||
message.** The analysis was first drafted on `feat/coverage-debt-paydown` (`b37defe`, 2026-09-07),
|
||||
a branch whose tree is ~107 commits behind main and which must never be merged; the drafted
|
||||
amendment lived only in that branch's copy of this file, so it disappeared from view the moment the
|
||||
branch went stale. Recovered here, re-checked against today's main, and restated unchanged in
|
||||
substance.
|
||||
|
||||
**R-TLS-02, verbatim (spec `docs/00-vision-and-scope.md`):** *"Certificates MUST be automatically
|
||||
**issued, renewed, and revoked** without manual steps."*
|
||||
|
||||
Three verbs. Two hold; the third does not, and cannot with the stack ADR-0006 selects:
|
||||
|
||||
- **Issued — satisfied.** Both ClusterIssuers are declarative (ACME DNS-01, Cloudflare token
|
||||
referenced by name from SOPS), the wildcard Certificate plus its apex SAN make onboarding
|
||||
certificate-free, and the active issuer is chosen per environment by `${ACME_CLUSTER_ISSUER}`.
|
||||
- **Renewed — satisfied, and unhobbled.** cert-manager renews at ~2/3 of lifetime with no operator
|
||||
action; nothing in the repo sets a `duration`/`renewBefore` that would defeat it, and
|
||||
`CertificateExpiringSoon` / `CertificateExpiryImminent` / `CertificateNotReady` fire if renewal
|
||||
silently stops.
|
||||
- **Revoked — NOT implemented, and not implementable here.** cert-manager has no automatic ACME
|
||||
revocation: no revoke-on-delete, no `Certificate` field that requests revocation, no controller
|
||||
that calls ACME `revokeCert`. Deleting a `Certificate` stops renewal; it does not revoke, and the
|
||||
issued certificate stays valid at the CA until it expires. Neither ADR-0006 nor `docs/08`
|
||||
specifies a mechanism — `docs/08`'s own "Renewal & revocation" section describes replacement
|
||||
issuance, which is rotation.
|
||||
|
||||
**`enableCertificateOwnerRef` is NOT the remedy, and is still not set.** That cert-manager
|
||||
controller flag makes a `Certificate` own its `Secret`, so deleting the Certificate
|
||||
garbage-collects the key material. That is *cleanup*, not revocation. It does remove one real
|
||||
manual step (today a deleted Certificate leaves an orphaned TLS Secret and private key to be
|
||||
deleted by hand), so it is defensible on its own merits — but it is a GLOBAL controller flag, and
|
||||
any future rename or prune that deletes and recreates a `Certificate` would take the Gateway's
|
||||
`wildcard-home-tls` Secret with it, dropping TLS on every service until reissuance, against Let's
|
||||
Encrypt rate limits, on two clusters. It deserves its own risk assessment, not a side effect of a
|
||||
coverage or test task.
|
||||
|
||||
**WHAT HAS CHANGED SINCE THE FIRST ESCALATION.** R-TLS-02 is no longer a COVERAGE gap — main
|
||||
carries `l0-static/tls-lifecycle-automated` and `l2-behavioral/cert-lifecycle-automated`, and the
|
||||
coverage report counts it covered. The honesty was kept where it matters: the L0 check's own verdict
|
||||
line ends *"R-TLS-02 'without manual steps' clause only — issued/renewed/revoked are RUNTIME verbs
|
||||
(see l2-behavioral/cert-lifecycle-automated; 'revoked' is NOT proven at any layer)"*. But the
|
||||
decision is now also LOAD-BEARING on the gate in a second way: the L2 test deletes a Certificate and
|
||||
requires its Secret to disappear, and on failure prints *"set enableCertificateOwnerRef: true in
|
||||
platform/cert-manager values (owner decision — WS-TEST does not weaken the assertion instead)"*. So
|
||||
that test is written to FAIL on any live cluster run until the owner decides. Nobody should weaken
|
||||
it to make a run green; that is the decision below, arriving as a red test instead of as a question.
|
||||
|
||||
**THE TWO OPTIONS, in the drafted preference order. The spec repo is READ-ONLY to every agent
|
||||
(R-TEST-06), so neither has been applied — this is a request for a human decision.**
|
||||
|
||||
1. **PREFERRED — amend R-TLS-02 to the property that is achievable and actually protective**, and
|
||||
move revocation to an explicitly manual, documented break-glass procedure:
|
||||
|
||||
> **R-TLS-02** — Certificates MUST be automatically **issued and renewed** without manual steps,
|
||||
> and decommissioning a service MUST NOT leave its key material behind. Certificate
|
||||
> **revocation** is an exceptional, operator-initiated action (suspected key compromise); it is
|
||||
> NOT automated, and MUST have a documented procedure. Rationale: the ACME / Let's Encrypt model
|
||||
> relies on SHORT certificate lifetimes plus automated rotation rather than on revocation — Let's
|
||||
> Encrypt retired OCSP in 2025 and cert-manager implements no automatic revocation — so an
|
||||
> "auto-revoked" requirement cannot be met by the stack ADR-0006 selects, and could only be met
|
||||
> by bespoke revocation automation the threat model does not justify.
|
||||
|
||||
Under this wording R-TLS-02 becomes fully testable: issuance and renewal are already asserted
|
||||
statically, and "no key material left behind" is exactly what `enableCertificateOwnerRef`
|
||||
implements — which makes that flag a *consequence of the requirement*, decided on its own risk
|
||||
assessment, rather than a fudge to turn a gate green.
|
||||
|
||||
2. **Keep R-TLS-02 as written and accept it as permanently partially satisfied**, with a reviewed
|
||||
`DOCUMENTED_GAPS` entry in `conformance/run.sh` naming the revocation clause, the reason (no ACME
|
||||
revocation exists in cert-manager) and the removal condition (cert-manager ships revocation, or
|
||||
the requirement is amended). Cheaper, and records a requirement the system does not meet.
|
||||
|
||||
**Until the owner picks one, the state is: R-TLS-02 counted as covered, its third clause
|
||||
unimplemented and labelled as such in the test's own output, and the L2 withdrawal assertion
|
||||
standing as a deliberate red.** Writing a test tagged R-TLS-02 that silently verifies only issuance
|
||||
and renewal would report the requirement as met while a third of its normative text is
|
||||
unimplemented — which is the one thing this harness exists to prevent. The choice is the
|
||||
architect's, not an agent's.
|
||||
@@ -136,6 +136,39 @@ printf 'touched set (clusters/ path map):\n%s\n---\n' "$OUT4"
|
||||
assert_present "R-ENV-02" "$OUT4" "clusters/* attributes R-ENV-02 (selects placeholder-guard-coupling HALF A)"
|
||||
assert_present "R-GITOPS-01" "$OUT4" "clusters/* attributes R-GITOPS-01"
|
||||
|
||||
# ---------------------------------------------------------------------------------------
|
||||
# THE ID PATTERN MUST ALLOW A DIGIT IN THE WORKSTREAM SEGMENT (added 2026-09-13)
|
||||
#
|
||||
# The content scan's pattern was `R-[A-Z]+-[0-9]+`, which CANNOT match `R-C2-01` or
|
||||
# `R-C2-02` — `[A-Z]+` consumes the `C` and then demands a `-` where the `2` is. Those are
|
||||
# the only two registry IDs whose workstream segment carries a digit, and the scan was blind
|
||||
# to BOTH, in the fail-OPEN direction: a change self-declaring a C2 requirement contributed
|
||||
# it to nobody, so the C2 conformance tests were filtered out of the merge gate. Found while
|
||||
# wiring conformance/tests/l0-static/c2-root-surface, which closes those two coverage gaps.
|
||||
#
|
||||
# `notes/` is outside path_requirements() on purpose, so the IDs below can only have come
|
||||
# from the CONTENT SCAN. A neighbouring all-letter ID is asserted too, so a regex that
|
||||
# somehow matched only the digit form would still fail here.
|
||||
git -C "$TMP" checkout -q -b feature5 main
|
||||
cat > "$TMP/notes/c2.yaml" <<'YAML'
|
||||
kind: Test
|
||||
metadata:
|
||||
labels:
|
||||
conformance.home.k3s/R-C2-01: "true"
|
||||
conformance.home.k3s/R-C2-02: "true"
|
||||
conformance.home.k3s/R-OBS-04: "true"
|
||||
YAML
|
||||
git -C "$TMP" add -A && git -C "$TMP" commit -qm "declare R-C2-01 / R-C2-02"
|
||||
OUT5="$(REPO_ROOT="$TMP" BASE_REF=main "$SCRIPT" 2>/dev/null)"
|
||||
printf 'touched set (digit in the workstream segment):\n%s\n---\n' "$OUT5"
|
||||
assert_present "R-C2-01" "$OUT5" "an R-ID with a DIGIT in the workstream segment IS attributed (R-C2-01)"
|
||||
assert_present "R-C2-02" "$OUT5" "the second such ID is attributed too (R-C2-02)"
|
||||
assert_present "R-OBS-04" "$OUT5" "an all-letter workstream segment still works (control)"
|
||||
# The pattern must not become so loose that a non-ID token is read as a requirement: a bare
|
||||
# numeric workstream is not a registry shape, and attributing one would WIDEN the gate on
|
||||
# noise (over-broad selection, never a false green, but noisy and misleading).
|
||||
assert_absent "R-1-01" "$OUT5" "a purely numeric workstream segment is not invented from thin air"
|
||||
|
||||
echo
|
||||
if [ "$FAIL" -eq 0 ]; then echo "touched-requirements tests: PASS ($PASS assertions)"; exit 0;
|
||||
else echo "touched-requirements tests: FAIL ($FAIL failed, $PASS passed)"; exit 1; fi
|
||||
@@ -345,6 +345,21 @@ main() {
|
||||
done <<< "$files"
|
||||
|
||||
# Self-declared R-IDs on the ADDED/REMOVED lines of the change (see changed_diff).
|
||||
#
|
||||
# THE ID PATTERN IS `R-[A-Z0-9]+-[0-9]+`, NOT `R-[A-Z]+-[0-9]+`. The WORKSTREAM segment may
|
||||
# contain a DIGIT: `R-C2-01` and `R-C2-02` (command & control UI) are the only two such IDs
|
||||
# in conformance/requirements/registry.yaml today, and the stricter pattern silently failed
|
||||
# to match BOTH of them — `[A-Z]+` consumes the `C` and then demands a `-` where the `2` is.
|
||||
# Every other consumer in the repo already allows the digit: test_requirements_for_path()
|
||||
# twenty lines above uses `R-[A-Z0-9]+-[0-9]+`, as do conformance/lib/coverage.py,
|
||||
# conformance/lib/status.py and conformance/lib/run_l1_offline.py. This grep was the lone
|
||||
# dissenter, and it dissented by UNDER-attributing — the fail-OPEN direction. A change that
|
||||
# retyped a `conformance.home.k3s/R-C2-01` tag, or a manifest self-declaring R-C2-02 from a
|
||||
# path the map keys elsewhere, contributed those IDs to NOBODY, so the C2 tests were
|
||||
# filtered out of the merge gate. Same defect class as the map's `*) : ;;` arm: a miss that
|
||||
# reads as "nothing touched". path_requirements() has no arm for platform/homepage/* or
|
||||
# platform/headlamp/* either, so before this fix the ONLY route to R-C2-01/02 was rule 3
|
||||
# (editing the C2 test itself). Regression-tested in ci/tests/touched-requirements.test.sh.
|
||||
# `^[+-]` keeps hunk headers (`@@`) and the `+++`/`---` file headers out of scope.
|
||||
# `sed 's/^[+-]//'` then strips the single-char diff marker (safe: -U0 emits one), so
|
||||
# what remains is the SOURCE line, and `grep -vE '^[[:space:]]*#'` drops whole-line
|
||||
@@ -358,7 +373,7 @@ main() {
|
||||
| grep -vE '^(\+\+\+|---)' \
|
||||
| sed -E 's/^[+-]//' \
|
||||
| grep -vE '^[[:space:]]*#' \
|
||||
| grep -hoE 'R-[A-Z]+-[0-9]+' \
|
||||
| grep -hoE 'R-[A-Z0-9]+-[0-9]+' \
|
||||
| tr '\n' ' ' || true)"
|
||||
|
||||
# Normalize: split, unique, sort. Emit to stdout.
|
||||
|
||||
@@ -34,8 +34,12 @@ spec:
|
||||
kind: GitRepository
|
||||
name: flux-system
|
||||
path: ./platform/homepage
|
||||
# Homepage references no ${VARS} today; the substitution is retained for parity with the other
|
||||
# tiers / future overlay knobs.
|
||||
# LOAD-BEARING, not parity decoration (the comment here used to say the opposite). This component
|
||||
# references ${BASE_DOMAIN} in TWO places: httproute.yaml's apex hostname, and EVERY tile href in
|
||||
# configmap.yaml. Without this substitution the token is applied LITERALLY — the apex route binds
|
||||
# the host `${BASE_DOMAIN}`, which matches no Gateway listener, and every link on the operator's
|
||||
# front door points at a host that does not exist — while Flux still reports this Kustomization
|
||||
# Ready. It fails OPEN. Asserted by conformance/tests/l0-static/c2-root-surface (5).
|
||||
postBuild:
|
||||
substituteFrom:
|
||||
- kind: ConfigMap
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
# =============================================================================
|
||||
# L0 STATIC — the C2 root surfaces every service the cluster runs, plus platform
|
||||
# health and controlled actions (R-C2-01, R-C2-02) [EXECUTABLE]
|
||||
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
|
||||
#
|
||||
# THE REQUIREMENTS, VERBATIM (spec docs/00-vision-and-scope.md):
|
||||
# R-C2-01 "The root `home.keithjackson.dev` MUST serve a UI that surfaces the services
|
||||
# running in the cluster and links to them."
|
||||
# R-C2-02 "The C2 UI MUST surface platform health (cluster, storage, GitOps, alerts) and
|
||||
# provide controlled operational actions where safe."
|
||||
# Detail: spec docs/12-command-center-ui.md — Homepage at the root is "the primary R-C2-01
|
||||
# surface"; Headlamp is "the R-C2-02 'controlled operational actions where safe' surface";
|
||||
# Grafana is the dashboards/alerts surface; "never a raw cluster-admin kubeconfig in a browser".
|
||||
#
|
||||
# WHY L0, AND NOT L2
|
||||
# ------------------
|
||||
# The REACHABILITY half of R-C2-01 is already exercised against a live cluster:
|
||||
# l2-behavioral/gateway-ingress-reachable dials the Gateway and l2-behavioral/trusted-tls
|
||||
# verifies the certificate it serves. What no live probe can decide is the half the
|
||||
# requirement is actually about — whether the UI at the root surfaces THE SERVICES RUNNING
|
||||
# IN THE CLUSTER. Deciding that is a comparison of two SETS, and both are repository facts:
|
||||
# what git declares the cluster serves (every in-repo HTTPRoute hostname, every L4 Service
|
||||
# that publishes a hostname, every catalog ServiceDefinition) against what the dashboard's
|
||||
# ConfigMap links. A probe of the root gets HTTP 200 from a dashboard that hides half the
|
||||
# estate and cannot tell the difference.
|
||||
#
|
||||
# It is also the assertion that ROTS, in the direction nothing else notices: onboard a
|
||||
# service, forget the dashboard, and every cluster test stays green while the operator's
|
||||
# front door hides a running service. That is not a styling preference — it is exactly the
|
||||
# R-C2-01 failure mode, and this check found a live instance of it when it was written (see
|
||||
# the commit message: platform/crafty served crafty.${BASE_DOMAIN} and nothing on the root
|
||||
# linked to it).
|
||||
#
|
||||
# GIVEN the repository as git tracks it,
|
||||
# WHEN the apex ingress, the dashboard's declared link set, every in-repo HTTPRoute, every
|
||||
# hostname-publishing L4 Service, the versioned catalog and the platform-health
|
||||
# surfaces are read and compared,
|
||||
# THEN — R-C2-01 —
|
||||
# the bare apex is claimed by exactly one route, that route binds the ONE Gateway
|
||||
# listener whose hostname matches a bare apex, the apex is a SAN on the certificate
|
||||
# that listener terminates with, the dashboard is RECONCILED (not merely authored),
|
||||
# FORWARD every HTTP service the repo routes is linked from the root, FORWARD every
|
||||
# catalog `kind: container` service with a route is linked, and REVERSE every
|
||||
# home-zone link the root offers resolves to something this repo actually serves —
|
||||
# an HTTPRoute (L7) or a hostname-publishing LoadBalancer Service (L4);
|
||||
# — R-C2-02 —
|
||||
# both health surfaces docs/12 names are linked AND reconciled, the health DATA exists
|
||||
# as code (PrometheusRule groups for cluster/nodes, storage, GitOps and cert expiry,
|
||||
# reconciled by the platform-config tier), and actions are CONTROLLED: the console
|
||||
# binds NO ClusterRole to its own ServiceAccount, authenticates per-user by native
|
||||
# OIDC from a named SOPS Secret, and cluster rights arrive through OIDC GROUPS at more
|
||||
# than one privilege level.
|
||||
#
|
||||
# STATUS: EXECUTABLE. `check.sh` is a pure offline assertion script (git ls-files + PyYAML —
|
||||
# the same dependency catalog/render.py already requires). conformance/run.sh --layer l0 and
|
||||
# ci/l0.sh run it directly; this Chainsaw Test wraps the SAME script.
|
||||
#
|
||||
# WHAT IT DOES NOT PROVE — honest gaps, restated in check.sh's own header:
|
||||
# - That the apex answers HTTP 200, or that any tile's status ping is green. That is L2;
|
||||
# the Gateway/TLS half is covered under R-DNS-01 / R-TLS-01.
|
||||
# - That the C2 is access-CONTROLLED in fact. The auth assertions read a DECLARED posture,
|
||||
# because the Gateway ext-authz attachment that would enforce forward-auth is not built
|
||||
# anywhere in this repo (platform/homepage/httproute.yaml says so in its own header, and
|
||||
# every forward-auth app shares the gap). Declared, not enforced — and labelled as such
|
||||
# in the assertion text.
|
||||
# - That an operational action in the console succeeds, or that it is audited. The
|
||||
# assertions prove there is no shared cluster-admin identity to act with and that rights
|
||||
# arrive per-user; the audit trail is a runtime property.
|
||||
# - Whether a tile is USEFUL. Homepage service-widget API keys are a deliberate follow-up
|
||||
# (platform/homepage/configmap.yaml); link presence is what R-C2-01 asks for.
|
||||
# =============================================================================
|
||||
apiVersion: chainsaw.kyverno.io/v1alpha1
|
||||
kind: Test
|
||||
metadata:
|
||||
name: c2-root-surface
|
||||
labels:
|
||||
conformance.home.k3s/R-C2-01: "true"
|
||||
conformance.home.k3s/R-C2-02: "true"
|
||||
annotations:
|
||||
conformance.home.k3s/requirements: "R-C2-01, R-C2-02"
|
||||
conformance.home.k3s/layer: "L0"
|
||||
# WHOLE-REPO SCOPE, and it is the whole point of the forward assertion.
|
||||
#
|
||||
# The invariant is "every service the repo routes is linked from the C2 root". Its input
|
||||
# surface is therefore EVERY tracked manifest that can declare an HTTPRoute — apps/wave-*,
|
||||
# platform/*, runbooks/ — not just the two components that own the C2. ci/touched-
|
||||
# requirements.sh keys `platform/homepage/*` and `platform/headlamp/*` to nothing at all
|
||||
# today (there is no path-map arm for either), and even with one, the change that BREAKS
|
||||
# this invariant is the change that adds a route somewhere else: `apps/wave-1/newthing.yaml`
|
||||
# attributes R-GITOPS-02 / R-NET-* / nothing, never R-C2-01. Requirement-scoped selection
|
||||
# would filter this check out of exactly the change that introduces the bug — the R-TEST-03
|
||||
# blind spot recorded in conformance/lib/common.sh.
|
||||
#
|
||||
# Declaring the scope is the whole fix: conformance/run.sh --include-scope whole-repo
|
||||
# (ci/l0.sh Stage 3b) unions this test into EVERY gate invocation. Do NOT hand-add a stage
|
||||
# to ci/l0.sh — tests/l0-static/whole-repo-invariants-wired fails the gate if anyone does.
|
||||
# A SKIP of a scope-declaring check is rc 3 (UNWIRED), never green.
|
||||
conformance.home.k3s/scope: "whole-repo"
|
||||
conformance.home.k3s/safety-critical: "false"
|
||||
conformance.home.k3s/executable: "true"
|
||||
conformance.home.k3s/intent: >-
|
||||
The bare apex is bound to the landing dashboard over trusted TLS and is reconciled; the
|
||||
dashboard links every HTTP service this repo routes and every link it offers resolves to
|
||||
something the repo serves; and the C2 surfaces platform health backed by real alert rules
|
||||
while granting operational rights per-user through OIDC groups rather than through a
|
||||
shared cluster-admin identity.
|
||||
spec:
|
||||
steps:
|
||||
- name: compare-the-c2-root-against-what-the-repo-actually-serves
|
||||
description: >-
|
||||
Read the apex ingress chain, the dashboard's declared links, every in-repo HTTPRoute and
|
||||
hostname-publishing Service, the catalog, the platform-health alert rules and the
|
||||
console's RBAC posture, and assert the two sets agree in both directions.
|
||||
try:
|
||||
- script:
|
||||
content: bash ./check.sh
|
||||
+656
@@ -0,0 +1,656 @@
|
||||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# L0 static check (offline, no cluster) — c2-root-surface
|
||||
#
|
||||
# Serves (spec docs/00-vision-and-scope.md; detail in docs/12-command-center-ui.md):
|
||||
#
|
||||
# R-C2-01 "The root `home.keithjackson.dev` MUST serve a UI that surfaces the
|
||||
# services running in the cluster and links to them."
|
||||
# R-C2-02 "The C2 UI MUST surface platform health (cluster, storage, GitOps,
|
||||
# alerts) and provide controlled operational actions where safe."
|
||||
#
|
||||
# WHY THIS IS AN L0 STATIC CHECK AND NOT AN L2 PROBE
|
||||
# --------------------------------------------------
|
||||
# The REACHABILITY half of R-C2-01 is already exercised against a live cluster:
|
||||
# l2-behavioral/gateway-ingress-reachable dials the Gateway and l2-behavioral/trusted-tls
|
||||
# verifies the certificate it serves. What no live probe can decide is the half the
|
||||
# requirement is actually about — whether the UI at the root surfaces THE SERVICES RUNNING
|
||||
# IN THE CLUSTER. Deciding that is a comparison of two SETS, and both are repository facts:
|
||||
# what git declares the cluster serves (every in-repo HTTPRoute hostname, every
|
||||
# hostname-publishing L4 Service, every catalog ServiceDefinition) against what the
|
||||
# dashboard's ConfigMap links. A probe of the root gets HTTP 200 from a dashboard that hides
|
||||
# half the estate and cannot tell the difference.
|
||||
#
|
||||
# It is also the assertion that ROTS, in the direction nothing else notices: onboard a
|
||||
# service, forget the dashboard, and every cluster test stays green while the operator's
|
||||
# front door hides a running service. That is not a styling preference — it is exactly the
|
||||
# R-C2-01 failure mode, and it was a LIVE defect on main when this check was written
|
||||
# (platform/crafty served crafty.${BASE_DOMAIN} and nothing on the root linked to it).
|
||||
#
|
||||
# THE RULE THIS IMPLEMENTS, in one sentence: every home-zone hostname the repository routes
|
||||
# over HTTP must be linked from the apex dashboard's configuration, every home-zone link the
|
||||
# apex dashboard offers must resolve to something the repository actually serves, and the two
|
||||
# R-C2-02 surfaces (health data + a per-user-authorized console) must be linked, reconciled
|
||||
# and backed by real alert rules.
|
||||
#
|
||||
# GIVEN the repository as git tracks it,
|
||||
# WHEN the apex ingress chain, the dashboard's declared link set, every in-repo HTTPRoute,
|
||||
# every hostname-publishing L4 Service, the versioned catalog, the platform-health
|
||||
# alert rules and the console's RBAC posture are read and compared,
|
||||
# THEN — R-C2-01 —
|
||||
# (0) the INPUT SURFACE is intact: every Flux-reconciled path lies inside the roots
|
||||
# this check scans, and every manifest in them parses. A root this check does not
|
||||
# scan, or a file it cannot parse, is a service it cannot see — which would
|
||||
# silently weaken every assertion below;
|
||||
# (1) EXACTLY ONE HTTPRoute claims the bare apex (an ambiguous root is not a root),
|
||||
# and it belongs to the landing-dashboard component;
|
||||
# (2) that route pins by sectionName the ONE Gateway listener whose hostname is the
|
||||
# bare apex, and that listener is HTTPS — the wildcard `*.${BASE_DOMAIN}` listener
|
||||
# does NOT intersect the bare apex, so without the sectionName the root binds
|
||||
# nothing and serves nothing (asserted against the wildcard listener's existence,
|
||||
# so the claim is proven rather than asserted from prose);
|
||||
# (3) the apex is a SAN on the Certificate whose Secret that listener terminates with;
|
||||
# (4) the dashboard is RECONCILED, not merely authored: one Flux Kustomization points
|
||||
# at its path and is listed in the base entrypoint set;
|
||||
# (5) that Kustomization carries postBuild.substituteFrom, because the dashboard's
|
||||
# hrefs and its route's hostname are written as `${BASE_DOMAIN}` tokens — without
|
||||
# substitution the apex route binds a literal `${BASE_DOMAIN}` and EVERY tile on
|
||||
# the root links to a nonexistent host, while Flux still reports Ready;
|
||||
# (6) the component's own kustomization ships config + workload + Service + route;
|
||||
# (7) the dashboard is a service CATALOG: grouped entries carrying hrefs;
|
||||
# (8) FORWARD — every home-zone hostname an in-repo HTTPRoute serves, other than the
|
||||
# apex itself, is LINKED from the root. This is the rot-catcher;
|
||||
# (9) FORWARD — every catalog `kind: container` ServiceDefinition whose declared
|
||||
# ingress host is routed is likewise linked, tying the C2 to the VERSIONED
|
||||
# onboarding contract and not only to whatever is rendered today;
|
||||
# (10) REVERSE — every home-zone link the root offers resolves to something this repo
|
||||
# actually serves: an HTTPRoute (L7) or a hostname-publishing LoadBalancer Service
|
||||
# (L4). A dead tile is a link the root offers to nothing;
|
||||
# (11) the off-cluster escape hatch is not a blanket one: every name declared in the
|
||||
# ConfigMap's `off-cluster-links` annotation is a specific home-zone hostname and
|
||||
# none of them shadows a hostname the repo DOES serve;
|
||||
# (12) no href names a LITERAL base-domain host. Every hostname is the `${BASE_DOMAIN}`
|
||||
# token, so prod's root cannot link dev's services — the 2026-09-04 incident class
|
||||
# (two clusters authored to describe the same names), applied to the front door;
|
||||
# — R-C2-02 —
|
||||
# (13) both health surfaces docs/12 names are linked: the Kubernetes console
|
||||
# (cluster/storage/GitOps state) and the dashboards/alerts surface;
|
||||
# (14) both are RECONCILED by Flux, so the links lead to something running;
|
||||
# (15) the health DATA those surfaces render exists as code — PrometheusRule groups for
|
||||
# each axis R-C2-02 enumerates (cluster/nodes, storage, GitOps reconciliation)
|
||||
# plus certificate expiry — and those rule files are reconciled;
|
||||
# (16) actions are CONTROLLED, not cluster-admin-in-a-browser (docs/12: "never a raw
|
||||
# cluster-admin kubeconfig in a browser"): the console chart is explicitly told NOT
|
||||
# to bind its ServiceAccount to a ClusterRole, so there is no shared privileged
|
||||
# identity to act as;
|
||||
# (17) actions are instead PER-USER: the console is configured for native OIDC with
|
||||
# credentials from a named (SOPS) Secret, not inline;
|
||||
# (18) cluster rights arrive through OIDC GROUPS at MORE THAN ONE privilege level (an
|
||||
# admin group and a lesser one) — least-privilege, not one tier for everybody;
|
||||
# (19) the C2 root DECLARES the authenticated-group posture docs/12 requires, on its own
|
||||
# route (declared, NOT enforced — see the gap note below).
|
||||
#
|
||||
# WHAT THIS DOES *NOT* PROVE — honest gaps, the same ones the Test declaration names:
|
||||
# - That the apex answers HTTP 200, or that any tile's status ping is green. That is L2; the
|
||||
# Gateway/TLS half is covered under R-DNS-01 / R-TLS-01.
|
||||
# - That the C2 is access-CONTROLLED in fact. (19) reads a DECLARED posture, because the
|
||||
# Gateway ext-authz attachment that would ENFORCE forward-auth is not built anywhere in
|
||||
# this repo (platform/homepage/httproute.yaml says so in its own header, and every
|
||||
# forward-auth app shares the gap). Declared, not enforced, and labelled as such in the
|
||||
# assertion text itself.
|
||||
# - That an operational action in the console succeeds, or that it is audited. (16)-(18)
|
||||
# prove there is no shared cluster-admin identity to act with and that rights arrive
|
||||
# per-user; the audit trail is a runtime property.
|
||||
# - Whether a tile is USEFUL. Homepage service-widget API keys are a deliberate follow-up
|
||||
# (platform/homepage/configmap.yaml); link presence is what R-C2-01 asks for.
|
||||
# - That an L4-ONLY endpoint is surfaced. A raw TCP/UDP endpoint (a Minecraft game port) has
|
||||
# no URL a browser can follow, so the FORWARD direction quantifies over HTTP routes;
|
||||
# L4-published hostnames are listed as ADVISORIES and are honoured in the REVERSE
|
||||
# direction (that is how the UniFi tile, an L4 LoadBalancer on :11443, resolves).
|
||||
# - It does not read ../home-network-k3s. The spec is read-only (R-TEST-06) and the
|
||||
# requirement text above is quoted, not parsed.
|
||||
#
|
||||
# STATUS: EXECUTABLE. Pure offline: `git ls-files` + PyYAML (the dependency ten neighbouring
|
||||
# l0-static checks already take). No cluster, no network, no spec repo.
|
||||
# =============================================================================
|
||||
set -euo pipefail
|
||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
|
||||
PYTHON="${PYTHON:-python3}"
|
||||
|
||||
# Honest SKIP (never green, and rc 3 is UNWIRED for a scope-declaring test) if the landing
|
||||
# dashboard component is absent entirely — there is then no C2 root to audit, which is a
|
||||
# report about the tree, not a pass.
|
||||
if [ ! -f "${ROOT}/platform/homepage/configmap.yaml" ]; then
|
||||
echo "check SKIPPED: platform/homepage/configmap.yaml absent — no C2 landing dashboard to audit" >&2
|
||||
exit 3
|
||||
fi
|
||||
|
||||
cd "${ROOT}"
|
||||
"${PYTHON}" - "${ROOT}" <<'PY'
|
||||
"""Static audit: the C2 root surfaces every service the repository serves, plus health.
|
||||
|
||||
Every parse contract is documented at its parser. PyYAML rather than regex because the
|
||||
subjects are genuinely structured: multi-document apps/ bundles, and a Homepage config that
|
||||
is nested YAML *inside* a ConfigMap string value.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError: # pragma: no cover
|
||||
print("check FAIL: PyYAML is required (same dependency as catalog/render.py)",
|
||||
file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
|
||||
ROOT = Path(sys.argv[1])
|
||||
sys.dont_write_bytecode = True # keep __pycache__ out of the tracked tree
|
||||
|
||||
FAILURES: list[str] = []
|
||||
ADVISORIES: list[str] = []
|
||||
CHECKS = 0
|
||||
|
||||
|
||||
def check(cond, msg):
|
||||
global CHECKS
|
||||
CHECKS += 1
|
||||
if cond:
|
||||
print(f" ok {msg}")
|
||||
else:
|
||||
print(f" FAIL {msg}", file=sys.stderr)
|
||||
FAILURES.append(msg)
|
||||
|
||||
|
||||
def read(p: Path) -> str:
|
||||
try:
|
||||
return p.read_text()
|
||||
except (OSError, UnicodeDecodeError):
|
||||
return ""
|
||||
|
||||
|
||||
# ============================================================ the hostname vocabulary
|
||||
# BASE_DOMAIN is a cluster-vars knob (R-ENV-02): every public hostname the cluster serves is
|
||||
# `<svc>.${BASE_DOMAIN}`, and dev/prod differ ONLY in that value. So a hostname in this repo
|
||||
# is canonically the TOKEN form, and the literal value below exists only to be FORBIDDEN in
|
||||
# (12): a literal is how prod came to be authored to serve dev's hostnames (2026-09-04,
|
||||
# clusters/base/cluster-vars.yaml's own header).
|
||||
TOKEN = "${BASE_DOMAIN}"
|
||||
_vars = yaml.safe_load(read(ROOT / "clusters" / "base" / "cluster-vars.yaml")) or {}
|
||||
LITERAL_BASE = ((_vars.get("data") or {}).get("BASE_DOMAIN") or "").strip()
|
||||
|
||||
|
||||
def strip_port(host: str) -> str:
|
||||
return host.split(":", 1)[0]
|
||||
|
||||
|
||||
def zone_label(host: str):
|
||||
"""The sub-label of a home-zone hostname, "" for the bare apex, or None if not in zone.
|
||||
|
||||
Accepts only the TOKEN form; a literal-base hostname returns None here and is caught
|
||||
separately by (12), so a literal can never silently satisfy a set comparison.
|
||||
"""
|
||||
h = strip_port(host)
|
||||
if h == TOKEN:
|
||||
return ""
|
||||
if h.endswith("." + TOKEN):
|
||||
return h[: -len("." + TOKEN)]
|
||||
return None
|
||||
|
||||
|
||||
def is_literal_base(host: str) -> bool:
|
||||
h = strip_port(host)
|
||||
return bool(LITERAL_BASE) and (h == LITERAL_BASE or h.endswith("." + LITERAL_BASE))
|
||||
|
||||
|
||||
# ============================================================ (0) the input surface
|
||||
# DELIVERY ROOTS — the subtrees Flux reconciles, plus catalog/ (the versioned source the
|
||||
# apps/ manifests are rendered FROM). conformance/ is deliberately EXCLUDED: its fixtures and
|
||||
# goldens declare synthetic HTTPRoutes (`paas-demo.…`, `gw-declared-conformance.…`) that are
|
||||
# test data, not services the cluster serves, and counting them would make (8) assert that
|
||||
# the operator's dashboard links this harness's fixtures.
|
||||
DELIVERY_ROOTS = ("apps", "platform", "platform-config", "platform-config-cdi",
|
||||
"runbooks", "clusters", "bootstrap", "catalog")
|
||||
|
||||
|
||||
def tracked_yaml() -> list[Path]:
|
||||
out = subprocess.run(["git", "ls-files", "-z", "--", "*.yaml", "*.yml"],
|
||||
cwd=ROOT, capture_output=True, text=True, check=True)
|
||||
return [ROOT / p for p in out.stdout.split("\0") if p]
|
||||
|
||||
|
||||
ALL_TRACKED = tracked_yaml()
|
||||
DELIVERY = [p for p in ALL_TRACKED
|
||||
if p.relative_to(ROOT).parts[0] in DELIVERY_ROOTS]
|
||||
|
||||
parse_failed: list[str] = []
|
||||
DOCS: dict[Path, list[dict]] = {}
|
||||
for p in DELIVERY:
|
||||
try:
|
||||
DOCS[p] = [d for d in yaml.safe_load_all(read(p)) if isinstance(d, dict)]
|
||||
except yaml.YAMLError:
|
||||
DOCS[p] = []
|
||||
parse_failed.append(str(p.relative_to(ROOT)))
|
||||
|
||||
|
||||
def docs_all():
|
||||
for p, ds in DOCS.items():
|
||||
for d in ds:
|
||||
yield p, d
|
||||
|
||||
|
||||
check(len(DELIVERY) >= 100,
|
||||
f"(0) the delivery tree is present and scanned ({len(DELIVERY)} tracked manifest(s) "
|
||||
f"under {'/, '.join(DELIVERY_ROOTS)}/)")
|
||||
check(not parse_failed,
|
||||
f"(0) every manifest in the scanned roots PARSES — an unparseable one is a service "
|
||||
f"this check cannot see, so it would weaken every set comparison below "
|
||||
f"{parse_failed if parse_failed else ''}")
|
||||
|
||||
# Every Flux-reconciled path must lie inside a root this check scans. This is the guard on
|
||||
# the guard: if a future tier reconciles, say, ./services, the forward direction would stop
|
||||
# seeing its routes and would go quietly green on a dashboard that hides all of them.
|
||||
flux_ks = [(p, d) for p, d in docs_all()
|
||||
if d.get("kind") == "Kustomization"
|
||||
and str(d.get("apiVersion", "")).startswith("kustomize.toolkit.fluxcd.io")]
|
||||
# A RECONCILING Kustomization is one carrying a sourceRef. The pathless documents under
|
||||
# bootstrap/flux/patches/ are strategic-merge PATCH FRAGMENTS (they add a decryption stanza to
|
||||
# a Kustomization declared elsewhere); they name no source and reconcile nothing, so requiring
|
||||
# a path of them would fail on a file that is correct.
|
||||
reconcilers = [(p, d) for p, d in flux_ks if (d.get("spec", {}) or {}).get("sourceRef")]
|
||||
unscanned = sorted({
|
||||
f"{p.relative_to(ROOT)}:{(d.get('spec', {}) or {}).get('path', '') or '(no path)'}"
|
||||
for p, d in reconcilers
|
||||
if (d.get("spec", {}) or {}).get("path", "").lstrip("./").split("/")[0]
|
||||
not in DELIVERY_ROOTS
|
||||
})
|
||||
check(len(reconcilers) >= 10 and not unscanned,
|
||||
f"(0) all {len(reconcilers)} Flux-reconciled path(s) lie inside a scanned root — a "
|
||||
f"reconciled tree this check does not read is a service it cannot require a link for "
|
||||
f"{unscanned if unscanned else ''}")
|
||||
|
||||
# ============================================================ what the repo SERVES
|
||||
# L7: every hostname an in-repo HTTPRoute claims, and the component that owns it.
|
||||
routes: dict[str, list[tuple[Path, dict]]] = {}
|
||||
apex_inheriting: list[Path] = []
|
||||
for p, d in docs_all():
|
||||
if d.get("kind") != "HTTPRoute":
|
||||
continue
|
||||
hosts = (d.get("spec", {}) or {}).get("hostnames") or []
|
||||
if not hosts:
|
||||
# An HTTPRoute with NO hostnames inherits the hostname of every listener it attaches
|
||||
# to (runbooks/serving/httproute.yaml binds `https-apex` only, so it serves a PATH on
|
||||
# the apex). It introduces no new hostname, so it owes no tile.
|
||||
apex_inheriting.append(p)
|
||||
continue
|
||||
for h in hosts:
|
||||
routes.setdefault(str(h), []).append((p, d))
|
||||
|
||||
routed_labels = {} # sub-label -> owning component dir
|
||||
for h, owners in routes.items():
|
||||
lbl = zone_label(h)
|
||||
if lbl: # "" is the apex; None is out of zone
|
||||
routed_labels[lbl] = owners[0][0].parent
|
||||
|
||||
# L4: a Service that publishes a home-zone hostname to DNS is also "served by this repo" —
|
||||
# it is how UniFi (a KubeVirt VM behind a pinned Cilium LoadBalancer VIP on :11443) reaches
|
||||
# the LAN. external-dns's annotation IS the declaration; there is no HTTPRoute to find.
|
||||
DNS_ANN = "external-dns.alpha.kubernetes.io/hostname"
|
||||
l4_labels: dict[str, Path] = {}
|
||||
for p, d in docs_all():
|
||||
if d.get("kind") != "Service":
|
||||
continue
|
||||
ann = (d.get("metadata", {}) or {}).get("annotations", {}) or {}
|
||||
for h in re.split(r"[,\s]+", str(ann.get(DNS_ANN, ""))):
|
||||
lbl = zone_label(h) if h else None
|
||||
if lbl:
|
||||
l4_labels[lbl] = p
|
||||
|
||||
check(len(routed_labels) >= 10,
|
||||
f"(0) the repo's north-south HTTP surface is discovered "
|
||||
f"({len(routed_labels)} routed home-zone hostname(s), "
|
||||
f"{len(apex_inheriting)} apex-path route(s), {len(l4_labels)} L4-published hostname(s))")
|
||||
|
||||
# ============================================================ (1) the apex is unambiguous
|
||||
apex_claims = routes.get(TOKEN, [])
|
||||
check(len(apex_claims) == 1,
|
||||
f"(1) exactly ONE HTTPRoute claims the bare apex {TOKEN} — an ambiguous root is not a "
|
||||
f"root (found {len(apex_claims)}: {[str(p.relative_to(ROOT)) for p, _ in apex_claims]})")
|
||||
apex_path, apex_route = (apex_claims[0] if apex_claims else (None, {}))
|
||||
DASH = apex_path.parent if apex_path is not None else None
|
||||
check(DASH is not None and DASH.name == "homepage",
|
||||
f"(1) the apex route belongs to the landing-dashboard component "
|
||||
f"(found {DASH.name if DASH else None!r})")
|
||||
|
||||
# ============================================================ (2) it binds the apex listener
|
||||
parents = (apex_route.get("spec", {}) or {}).get("parentRefs") or []
|
||||
section = next((pr.get("sectionName") for pr in parents if pr.get("sectionName")), None)
|
||||
gw_name = next((pr.get("name") for pr in parents), None)
|
||||
gw_ns = next((pr.get("namespace") for pr in parents), None)
|
||||
check(section is not None,
|
||||
"(2) the apex route pins a Gateway listener by sectionName — the wildcard listener's "
|
||||
"hostname cannot match a bare apex, so an unpinned apex route binds NOTHING")
|
||||
|
||||
gw = next((d for _, d in docs_all()
|
||||
if d.get("kind") == "Gateway"
|
||||
and (d.get("metadata", {}) or {}).get("name") == gw_name
|
||||
and (gw_ns is None or (d.get("metadata", {}) or {}).get("namespace") == gw_ns)), None)
|
||||
check(gw is not None, f"(2) the parent Gateway '{gw_name}' is declared in the repo")
|
||||
listeners = ((gw or {}).get("spec", {}) or {}).get("listeners") or []
|
||||
apex_listener = next((l for l in listeners if l.get("name") == section), None)
|
||||
check(apex_listener is not None,
|
||||
f"(2) the Gateway declares the '{section}' listener the apex route binds")
|
||||
check(apex_listener is not None and apex_listener.get("hostname") == TOKEN,
|
||||
f"(2) that listener's hostname is EXACTLY the bare apex "
|
||||
f"(found {(apex_listener or {}).get('hostname')!r})")
|
||||
check(apex_listener is not None and apex_listener.get("protocol") == "HTTPS",
|
||||
"(2) the apex listener terminates HTTPS (docs/12 acceptance gate: trusted TLS on the root)")
|
||||
# Prove the sectionName is LOAD-BEARING rather than decorative: a wildcard listener exists,
|
||||
# it is a DIFFERENT listener, and its hostname does not equal the apex.
|
||||
wildcard = [l for l in listeners if str(l.get("hostname", "")) == "*." + TOKEN]
|
||||
check(bool(wildcard) and all(l.get("name") != section for l in wildcard),
|
||||
f"(2) the wildcard listener(s) {[l.get('name') for l in wildcard]} are DISTINCT from "
|
||||
f"'{section}' — which is why the apex needs its own listener, and why dropping the "
|
||||
f"sectionName would leave the root unserved")
|
||||
|
||||
# ============================================================ (3) trusted TLS on the root
|
||||
cert_secret = next((ref.get("name")
|
||||
for ref in ((apex_listener or {}).get("tls", {}) or {}).get("certificateRefs") or []),
|
||||
None)
|
||||
cert = next((d for _, d in docs_all()
|
||||
if d.get("kind") == "Certificate"
|
||||
and (d.get("spec", {}) or {}).get("secretName") == cert_secret), None)
|
||||
check(cert is not None,
|
||||
f"(3) a cert-manager Certificate produces the apex listener's Secret ({cert_secret!r})")
|
||||
check(cert is not None and TOKEN in ((cert.get("spec", {}) or {}).get("dnsNames") or []),
|
||||
"(3) the bare apex is a SAN on that Certificate — the wildcard alone does NOT cover it, "
|
||||
"so without this SAN the root serves an untrusted certificate")
|
||||
|
||||
# ============================================================ (4)(5) it is RECONCILED
|
||||
dash_rel = f"./{DASH.relative_to(ROOT)}" if DASH else None
|
||||
dash_ks = [(p, d) for p, d in flux_ks
|
||||
if (d.get("spec", {}) or {}).get("path", "").rstrip("/") == dash_rel]
|
||||
check(len(dash_ks) == 1,
|
||||
f"(4) exactly one Flux Kustomization reconciles the dashboard ({dash_rel}) — authored "
|
||||
f"but unreconciled is the 'looks done, is absent' class this repo keeps hitting "
|
||||
f"(found {len(dash_ks)})")
|
||||
if dash_ks:
|
||||
base_txt = read(ROOT / "clusters" / "base" / "kustomization.yaml")
|
||||
entry = dash_ks[0][0].name
|
||||
check(re.search(rf"^\s*-\s*{re.escape(entry)}\s*(#.*)?$", base_txt, re.M) is not None,
|
||||
f"(4) that Kustomization is listed in the base entrypoint set ({entry})")
|
||||
# (5) The dashboard's hrefs and its route's hostname are ${BASE_DOMAIN} TOKENS. Without
|
||||
# postBuild.substituteFrom the token reaches the cluster LITERALLY: the apex route binds
|
||||
# the host `${BASE_DOMAIN}` (matching no listener) and every tile links to a host that
|
||||
# does not exist — and Flux still reports the Kustomization Ready. It fails OPEN.
|
||||
subs = (((dash_ks[0][1].get("spec", {}) or {}).get("postBuild") or {}).get("substituteFrom") or [])
|
||||
check(any((s.get("name") == "cluster-vars") for s in subs),
|
||||
f"(5) {dash_ks[0][0].name} carries postBuild.substituteFrom cluster-vars. It MUST: "
|
||||
f"this component's route hostname and EVERY tile href are ${{BASE_DOMAIN}} tokens, "
|
||||
f"and an unsubstituted token is applied literally while Flux still reports Ready — "
|
||||
f"a root bound to no listener and a dashboard of dead links, with a green tier")
|
||||
|
||||
# ============================================================ (6) it ships the UI
|
||||
dash_kust = yaml.safe_load(read(DASH / "kustomization.yaml")) if DASH else None
|
||||
dash_res = set((dash_kust or {}).get("resources") or [])
|
||||
for want in ("configmap.yaml", "deployment.yaml", "service.yaml", "httproute.yaml"):
|
||||
check(want in dash_res, f"(6) the dashboard kustomization ships {want}")
|
||||
|
||||
# ================================================== the dashboard's DECLARED link set
|
||||
# Homepage's config is nested YAML inside ConfigMap string values (services.yaml,
|
||||
# bookmarks.yaml). Both are read: a bookmark is a link the root offers just as a tile is.
|
||||
cm = next((d for d in (DOCS.get(DASH / "configmap.yaml") or []) if d.get("kind") == "ConfigMap"),
|
||||
{}) if DASH else {}
|
||||
cm_data = cm.get("data") or {}
|
||||
|
||||
|
||||
def embedded(key):
|
||||
try:
|
||||
return yaml.safe_load(cm_data.get(key) or "") or []
|
||||
except yaml.YAMLError:
|
||||
return None
|
||||
|
||||
|
||||
svc_cfg = embedded("services.yaml")
|
||||
bm_cfg = embedded("bookmarks.yaml")
|
||||
check(svc_cfg is not None and bm_cfg is not None,
|
||||
"(7) the dashboard's embedded services.yaml / bookmarks.yaml parse as YAML — the app "
|
||||
"mounts these verbatim, so a malformed one is a dashboard that renders nothing")
|
||||
|
||||
|
||||
def hrefs(node):
|
||||
"""Every href anywhere in Homepage's nested group/entry structure."""
|
||||
if isinstance(node, dict):
|
||||
for k, v in node.items():
|
||||
if k == "href" and isinstance(v, str):
|
||||
yield v
|
||||
else:
|
||||
yield from hrefs(v)
|
||||
elif isinstance(node, list):
|
||||
for v in node:
|
||||
yield from hrefs(v)
|
||||
|
||||
|
||||
def host_of(url):
|
||||
m = re.match(r"^https?://([^/]+)", url or "")
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
tile_urls = list(hrefs(svc_cfg or []))
|
||||
all_urls = tile_urls + list(hrefs(bm_cfg or []))
|
||||
all_hosts = [h for h in (host_of(u) for u in all_urls) if h]
|
||||
linked = {lbl for lbl in (zone_label(h) for h in all_hosts) if lbl}
|
||||
tile_labels = {lbl for lbl in (zone_label(h) for h in (host_of(u) for u in tile_urls) if h) if lbl}
|
||||
|
||||
groups = [g for g in (svc_cfg or []) if isinstance(g, dict)]
|
||||
check(len(groups) >= 1 and len(tile_labels) >= 5,
|
||||
f"(7) the root is a grouped service catalog with links "
|
||||
f"({len(groups)} group(s), {len(tile_labels)} linked home-zone service(s))")
|
||||
|
||||
# ============================================ (8) FORWARD: nothing routed is hidden
|
||||
unsurfaced = sorted(set(routed_labels) - linked)
|
||||
check(not unsurfaced,
|
||||
"(8) every service the repo ROUTES over HTTP is surfaced on the C2 root. Unsurfaced: "
|
||||
+ ", ".join(f"{n}.{TOKEN} ({routed_labels[n].relative_to(ROOT)})" for n in unsurfaced)
|
||||
if unsurfaced else
|
||||
f"(8) every service the repo ROUTES over HTTP is surfaced on the C2 root "
|
||||
f"({len(routed_labels)} hostname(s) compared)")
|
||||
|
||||
# L4-only endpoints are reported, never required: a raw TCP/UDP port has no browsable URL.
|
||||
for lbl in sorted(set(l4_labels) - linked):
|
||||
ADVISORIES.append(
|
||||
f"{lbl}.{TOKEN} is published to DNS by an L4 Service "
|
||||
f"({l4_labels[lbl].relative_to(ROOT)}) and is not linked from the root. Not a "
|
||||
f"failure: a raw TCP/UDP endpoint has no URL a browser can follow. Verify the "
|
||||
f"service it belongs to is reachable through a tile that IS linked")
|
||||
|
||||
# ================================= (9) FORWARD: the VERSIONED onboarding contract
|
||||
cat_dir = ROOT / "catalog"
|
||||
cat_seen, cat_missing, cat_unrouted = 0, [], []
|
||||
for p in sorted(cat_dir.glob("*.yaml")):
|
||||
sd = next((d for d in (DOCS.get(p) or [])), None)
|
||||
if not isinstance(sd, dict) or sd.get("kind") != "container":
|
||||
continue # kind: vm has no north-south HTTP surface (docs/09)
|
||||
for r in sd.get("ingressRoutes") or []:
|
||||
lbl = zone_label(str(r.get("host", "")))
|
||||
if not lbl:
|
||||
continue
|
||||
if lbl not in routed_labels:
|
||||
# Declared in the catalog but not rendered into a route: an apps/ render drift
|
||||
# problem owned by R-GITOPS-02, reported here rather than silently skipped.
|
||||
cat_unrouted.append(f"{sd.get('name')}:{lbl}")
|
||||
continue
|
||||
cat_seen += 1
|
||||
if lbl not in linked:
|
||||
cat_missing.append(f"{sd.get('name')}:{lbl}")
|
||||
check(cat_seen >= 5,
|
||||
f"(9) the versioned catalog contributes routed HTTP services ({cat_seen} found) — the "
|
||||
f"C2 is tied to the onboarding contract, not only to today's rendered output")
|
||||
check(not cat_missing,
|
||||
f"(9) every catalog `kind: container` service with a route is linked from the C2 root "
|
||||
f"{cat_missing if cat_missing else ''}")
|
||||
for u in cat_unrouted:
|
||||
ADVISORIES.append(f"catalog declares an ingress host with no rendered HTTPRoute: {u} "
|
||||
f"(render drift — R-GITOPS-02 owns it, not R-C2-01)")
|
||||
|
||||
# ============================ (10)(11) REVERSE: no link points at nothing
|
||||
ann = (cm.get("metadata", {}) or {}).get("annotations", {}) or {}
|
||||
OFF_ANN = "homelab.keithjackson.dev/off-cluster-links"
|
||||
declared_off_raw = [t for t in re.split(r"[,\s]+", str(ann.get(OFF_ANN, ""))) if t]
|
||||
declared_off = {zone_label(t) for t in declared_off_raw}
|
||||
served = set(routed_labels) | set(l4_labels)
|
||||
dead = sorted(linked - served - declared_off)
|
||||
check(not dead,
|
||||
f"(10) every home-zone link the root offers resolves to something this repo serves — "
|
||||
f"an HTTPRoute (L7) or a hostname-publishing LoadBalancer Service (L4). Dead: "
|
||||
f"{[f'{d}.{TOKEN}' for d in dead] if dead else ''}")
|
||||
# The escape hatch is a DECLARATION, not a hole: same doctrine as an explicit `NONE` in
|
||||
# ci/touched-requirements.sh's path map. Empty today, and these two hold vacuously when it is
|
||||
# — they exist so it cannot be widened into a blanket without failing.
|
||||
check(all(zone_label(t) for t in declared_off_raw),
|
||||
f"(11) every declared off-cluster link is a specific home-zone hostname in TOKEN form "
|
||||
f"({declared_off_raw})")
|
||||
shadowed = sorted(declared_off & served)
|
||||
check(not shadowed,
|
||||
f"(11) no off-cluster declaration shadows a hostname the repo DOES serve "
|
||||
f"{shadowed if shadowed else ''}")
|
||||
|
||||
# ============================ (12) tokens, never literals
|
||||
literal_hosts = sorted({h for h in all_hosts if is_literal_base(h)})
|
||||
check(not literal_hosts,
|
||||
f"(12) no link on the root names a LITERAL '{LITERAL_BASE}' host — every hostname is the "
|
||||
f"${{BASE_DOMAIN}} token, so prod's front door cannot link dev's services (the "
|
||||
f"2026-09-04 class: two clusters authored to describe the same names). Literals: "
|
||||
f"{literal_hosts if literal_hosts else ''}")
|
||||
|
||||
# =====================================================================
|
||||
# R-C2-02 — platform health, and controlled operational actions
|
||||
# =====================================================================
|
||||
# docs/12 names the surfaces the C2 composes: Headlamp is "the R-C2-02 'controlled
|
||||
# operational actions where safe' surface"; Grafana is the dashboards/alerts surface. Each is
|
||||
# identified by the component that OWNS ITS ROUTE, not by a magic string in the config.
|
||||
CONSOLE, DASHBOARDS = "headlamp", "grafana"
|
||||
check(CONSOLE in linked,
|
||||
f"(13) the C2 root links the Kubernetes console ({CONSOLE}.{TOKEN}) — cluster, storage "
|
||||
f"and GitOps state, and the R-C2-02 operational-actions surface (docs/12)")
|
||||
check(DASHBOARDS in linked,
|
||||
f"(13) the C2 root links the dashboards/alerts surface ({DASHBOARDS}.{TOKEN})")
|
||||
|
||||
AGG = ("\n".join(read(ROOT / t / "kustomization.yaml")
|
||||
for t in ("platform", "platform-config", "platform-config-cdi")))
|
||||
for label, name in (("console", CONSOLE), ("dashboards", DASHBOARDS)):
|
||||
comp = routed_labels.get(name)
|
||||
check(comp is not None, f"(14) the {label} surface has an in-repo HTTPRoute")
|
||||
if comp is None:
|
||||
continue
|
||||
rel = f"./{comp.relative_to(ROOT)}"
|
||||
reconciled = any((d.get("spec", {}) or {}).get("path", "").rstrip("/") == rel
|
||||
for _, d in flux_ks)
|
||||
if not reconciled:
|
||||
# A component may reconcile as a MEMBER of an aggregate tier's kustomize tree
|
||||
# (platform / platform-config) instead of owning a Flux Kustomization. Both count;
|
||||
# being in NEITHER is "authored, never applied".
|
||||
reconciled = (f"\n - {comp.name}" in AGG
|
||||
or f"../{comp.relative_to(ROOT)}/" in AGG)
|
||||
check(reconciled,
|
||||
f"(14) the {label} surface ({comp.name}) is reconciled by Flux, not merely authored "
|
||||
f"— a link to an unreconciled component is a link to nothing")
|
||||
|
||||
# (15) the health DATA exists as code, for every axis R-C2-02 enumerates.
|
||||
rule_files = sorted({p for p, d in docs_all() if d.get("kind") == "PrometheusRule"})
|
||||
rule_groups = {g.get("name", "")
|
||||
for p, d in docs_all() if d.get("kind") == "PrometheusRule"
|
||||
for g in ((d.get("spec", {}) or {}).get("groups") or [])}
|
||||
for axis, needle in (("cluster/nodes", "nodes"),
|
||||
("storage", "longhorn"),
|
||||
("GitOps reconciliation", "flux"),
|
||||
("certificate expiry", "certificates")):
|
||||
check(any(needle in g for g in rule_groups),
|
||||
f"(15) platform-health alert rules exist for {axis} (a '*{needle}*' rule group) — "
|
||||
f"the C2 cannot surface health that is not measured")
|
||||
pc_txt = read(ROOT / "platform-config" / "kustomization.yaml")
|
||||
reconciled_rules = [p for p in rule_files if f"../{p.relative_to(ROOT)}" in pc_txt]
|
||||
check(len(reconciled_rules) >= 2,
|
||||
f"(15) those alert rules are RECONCILED — PrometheusRule is a CR of a CRD the platform "
|
||||
f"tier installs, so the rules are applied by platform-config "
|
||||
f"({len(reconciled_rules)}/{len(rule_files)} rule file(s) referenced there)")
|
||||
|
||||
# (16)(17) controlled actions: no shared cluster-admin identity; per-user OIDC instead.
|
||||
console_dir = routed_labels.get(CONSOLE)
|
||||
hr = next((d for p in (sorted(console_dir.glob("*.yaml")) if console_dir else [])
|
||||
for d in (DOCS.get(p) or []) if d.get("kind") == "HelmRelease"), None)
|
||||
check(hr is not None, "(16) the Kubernetes console is deployed by a HelmRelease")
|
||||
hv = ((hr or {}).get("spec", {}) or {}).get("values", {}) or {}
|
||||
check((hv.get("clusterRoleBinding") or {}).get("create") is False,
|
||||
"(16) the console chart is told NOT to bind its ServiceAccount to a ClusterRole. The "
|
||||
"chart's DEFAULT binds it to cluster-admin, which is exactly the 'raw cluster-admin "
|
||||
"kubeconfig in a browser' docs/12 forbids — every visitor would act as one shared "
|
||||
"omnipotent identity")
|
||||
oidc = ((hv.get("config") or {}).get("oidc") or {})
|
||||
check(bool(oidc),
|
||||
"(17) the console is configured for native OIDC login, so an action carries the "
|
||||
"operator's OWN identity (docs/12: Headlamp/Grafana use native OIDC)")
|
||||
check(bool((oidc.get("externalSecret") or {}).get("name")),
|
||||
"(17) its OIDC client credentials come from a NAMED (SOPS) Secret, not inline in the "
|
||||
"HelmRelease values — R-SEC-01")
|
||||
|
||||
# (18) rights arrive through GROUPS, at more than one privilege level.
|
||||
group_bindings = []
|
||||
for p, d in docs_all():
|
||||
if d.get("kind") not in ("ClusterRoleBinding", "RoleBinding"):
|
||||
continue
|
||||
for s in d.get("subjects") or []:
|
||||
if s.get("kind") == "Group" and str(s.get("name", "")).startswith("oidc:"):
|
||||
group_bindings.append(((d.get("metadata", {}) or {}).get("name"),
|
||||
s.get("name"),
|
||||
(d.get("roleRef") or {}).get("name")))
|
||||
check(len(group_bindings) >= 2,
|
||||
f"(18) cluster rights are bound to OIDC GROUPS, so access follows Keycloak group "
|
||||
f"membership rather than a per-user grant somebody must remember to revoke "
|
||||
f"({[(g, r) for _, g, r in group_bindings]})")
|
||||
check(any(r == "cluster-admin" for _, _, r in group_bindings)
|
||||
and any(r != "cluster-admin" for _, _, r in group_bindings),
|
||||
"(18) MORE THAN ONE privilege level exists (an admin group AND a lesser one) — "
|
||||
"least-privilege, not one tier for everybody (docs/12 access control, R-IDP-06)")
|
||||
|
||||
# (19) the root DECLARES the authenticated-group posture. DECLARED, NOT ENFORCED.
|
||||
route_ann = (apex_route.get("metadata", {}) or {}).get("annotations", {}) or {}
|
||||
check(route_ann.get("homelab.keithjackson.dev/auth-mode") in ("forward-auth", "oidc", "oauth2-proxy"),
|
||||
f"(19) the C2 root's route DECLARES an authentication mode "
|
||||
f"(found {route_ann.get('homelab.keithjackson.dev/auth-mode')!r}) "
|
||||
f"[DECLARED, NOT ENFORCED — no Gateway ext-authz attachment exists in this repo]")
|
||||
check(bool(route_ann.get("homelab.keithjackson.dev/auth-groups")),
|
||||
"(19) it DECLARES the authorized group(s) — docs/12 requires an authenticated user IN "
|
||||
"AN AUTHORIZED GROUP, not merely logged in "
|
||||
"[DECLARED, NOT ENFORCED — see the header's gap note]")
|
||||
|
||||
# ======================================================================== report
|
||||
print()
|
||||
print(f"C2 surface inventory (apex {TOKEN}, literal base default '{LITERAL_BASE}'):")
|
||||
print(f" routed over HTTP : {len(routed_labels)} -> {', '.join(sorted(routed_labels))}")
|
||||
print(f" L4-published : {len(l4_labels)} -> {', '.join(sorted(l4_labels)) or '(none)'}")
|
||||
print(f" linked from root : {len(linked)} -> {', '.join(sorted(linked))}")
|
||||
print(f" apex-path routes : {len(apex_inheriting)} -> "
|
||||
f"{', '.join(str(p.relative_to(ROOT)) for p in apex_inheriting) or '(none)'}")
|
||||
print(f" catalog services compared: {cat_seen} off-cluster declarations: "
|
||||
f"{sorted(declared_off_raw) or '(none)'}")
|
||||
|
||||
if ADVISORIES:
|
||||
print("\nADVISORIES (not failures):")
|
||||
for a in ADVISORIES:
|
||||
print(f" - {a}")
|
||||
|
||||
print()
|
||||
if not FAILURES:
|
||||
print(f"check OK: c2-root-surface — {CHECKS} assertions (R-C2-01, R-C2-02)")
|
||||
raise SystemExit(0)
|
||||
print(f"CHECK FAIL (c2-root-surface): {len(FAILURES)} failed, "
|
||||
f"{CHECKS - len(FAILURES)} passed", file=sys.stderr)
|
||||
for f in FAILURES:
|
||||
print(f" FAIL {f}", file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
PY
|
||||
@@ -85,7 +85,7 @@ done
|
||||
# second would fail.
|
||||
# Deliberately LOWERCASE: conf_normalize_reqs() upper-cases it, so the selection behaves
|
||||
# identically — but the literal here does not match ci/touched-requirements.sh's content-scan
|
||||
# pattern `R-[A-Z]+-[0-9]+`, so editing this file does not attribute a nonexistent requirement
|
||||
# pattern `R-[A-Z0-9]+-[0-9]+`, so editing this file does not attribute a nonexistent requirement
|
||||
# to the change. (Same reason the synthetic fixture below carries no requirement label at all.)
|
||||
BOGUS_REQ="r-nosuchreq-99"
|
||||
sel_without="$(bash "$RUN_SH" --layer l0 --requirements "$BOGUS_REQ" --print-selection 2>/dev/null || true)"
|
||||
|
||||
@@ -81,6 +81,18 @@ data:
|
||||
href: https://qbittorrent.${BASE_DOMAIN}
|
||||
description: Download client (VPN egress)
|
||||
icon: mdi-download-network
|
||||
# Crafty Controller — a running, north-south-exposed web UI (platform/crafty/httproute.yaml,
|
||||
# reconciled by clusters/base/crafty.yaml) that this dashboard did NOT surface until
|
||||
# 2026-09-13. R-C2-01 says the root MUST surface "the services running in the cluster and
|
||||
# links to them", so an unsurfaced route is a requirement gap, not a styling preference —
|
||||
# conformance/tests/l0-static/c2-root-surface fails on one now. Grouped under Media because
|
||||
# it manages game servers; regrouping it under Platform would be purely cosmetic. The game
|
||||
# ports themselves (crafty-mc.${BASE_DOMAIN}, TCP 25565-25567 / UDP 19132-19134) are an L4
|
||||
# LoadBalancer with no browsable URL, so this tile is how they are surfaced.
|
||||
- Crafty:
|
||||
href: https://crafty.${BASE_DOMAIN}
|
||||
description: Minecraft server manager (Keycloak-gated)
|
||||
icon: mdi-minecraft
|
||||
|
||||
- Network & Infra:
|
||||
- Pi-hole:
|
||||
|
||||
Reference in new issue
Block a user