From 18f7ba8d0f2901651d0214e09ff04bdfb5909a10 Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Sun, 8 Nov 2026 23:06:26 -0600 Subject: [PATCH] docs: apply the operator's rulings on Kasm, revocation and fault injection Retire Kasm from the catalog. Drop ACME revocation from R-TLS-02. Make L3 fault injection operator-run and exclude it from the merge gate (R-TEST-03, ADR-0026). Update the gap register to match; Homepage authentication is now ruled must-enforce and stays a gap until the implementation lands. Co-Authored-By: Claude Opus 5.5 --- decisions/README.md | 5 ++- decisions/adr-0010-conformance-testing.md | 2 +- .../adr-0011-test-tooling-kyverno-chainsaw.md | 2 +- .../adr-0026-fault-injection-operator-run.md | 37 +++++++++++++++++++ docs/00-vision-and-scope.md | 7 ++-- docs/05-gitops-and-delivery.md | 2 +- docs/07-security-and-secrets.md | 2 +- docs/08-tls-and-certificates.md | 10 ++--- docs/13-service-catalog.md | 13 +++---- docs/14-testing-and-conformance.md | 17 ++++++--- docs/15-identity-and-access.md | 2 +- docs/16-known-gaps.md | 16 +++----- roadmap/workstreams.md | 2 +- 13 files changed, 76 insertions(+), 41 deletions(-) create mode 100644 decisions/adr-0026-fault-injection-operator-run.md diff --git a/decisions/README.md b/decisions/README.md index 20629a2..5ffa368 100644 --- a/decisions/README.md +++ b/decisions/README.md @@ -59,8 +59,8 @@ pattern choices. | [0007](adr-0007-secrets-sops-age.md) | Secrets: SOPS + age | Accepted | R-SEC-01 | | [0008](adr-0008-vpn-egress-model.md) | VPN egress model: Gluetun sidecar + Cilium Egress Gateway | ~~Superseded by 0012~~ | R-NET-05/06 | | [0009](adr-0009-automated-image-updates.md) | Automated updates: Flux image automation + Renovate | ~~Superseded by 0023~~ | R-UPD-01..03 | -| [0010](adr-0010-conformance-testing.md) | Conformance testing: automated, reproducible, behavior-based | Accepted | R-TEST-01..06 | -| [0011](adr-0011-test-tooling-kyverno-chainsaw.md) | Conformance test tooling: Kyverno + Chainsaw | Accepted | R-TEST-01..05 | +| [0010](adr-0010-conformance-testing.md) | Conformance testing: automated, reproducible, behavior-based | Accepted — L3 merge gating superseded by 0026 | R-TEST-01..06 | +| [0011](adr-0011-test-tooling-kyverno-chainsaw.md) | Conformance test tooling: Kyverno + Chainsaw | Accepted — L3 CI trigger superseded by 0026 | R-TEST-01..05 | | [0012](adr-0012-vpn-egress-single-expressvpn-gateway.md) | VPN egress: single shared in-cluster ExpressVPN gateway | ~~Superseded by 0013~~ | R-NET-05/06/07 | | [0013](adr-0013-vpn-provider-pia-gluetun-wireguard.md) | VPN provider: PIA over OpenVPN-UDP + port forwarding | Accepted — provider/protocol only (client+steering superseded by 0015) | R-NET-05/06/07 | | [0014](adr-0014-idp-keycloak.md) | Identity Provider: Keycloak (SSO via OIDC + forward-auth) | Accepted — forward-auth enforcement point and app list superseded by 0022 | R-IDP-01..06 | @@ -75,3 +75,4 @@ pattern choices. | [0023](adr-0023-automated-updates-renovate.md) | Automated updates: Renovate for images and charts, every change PR-gated | Accepted | R-UPD-01, R-UPD-02, R-UPD-03, R-TEST-03 | | [0024](adr-0024-delivery-authority-ci-gate.md) | Delivery authority: the CI gate is the approver; sensitive-path review; attributed admin override | Accepted | R-TEST-03, R-TEST-06, R-GITOPS-01, R-SEC-02 | | [0025](adr-0025-in-cluster-ai-program-manager.md) | In-cluster AI Program Manager (pm-agent) | Accepted | R-GITOPS-01, R-TEST-03, R-TEST-06, R-SEC-02 | +| [0026](adr-0026-fault-injection-operator-run.md) | Fault injection is operator-run, never automated | Accepted | R-TEST-03, R-TEST-05, R-ENV-03 | diff --git a/decisions/adr-0010-conformance-testing.md b/decisions/adr-0010-conformance-testing.md index dedf84e..fc19c5a 100644 --- a/decisions/adr-0010-conformance-testing.md +++ b/decisions/adr-0010-conformance-testing.md @@ -1,5 +1,5 @@ # ADR-0010 — Conformance testing: automated, reproducible, behavior-based -- Status: Accepted +- Status: Accepted — stands, except that L3 fault injection does not gate merges ([ADR-0026](adr-0026-fault-injection-operator-run.md)) - Date: 2026-08-15 - Serves: R-TEST-01, R-TEST-02, R-TEST-03, R-TEST-04, R-TEST-05, R-TEST-06 - Supersedes: — diff --git a/decisions/adr-0011-test-tooling-kyverno-chainsaw.md b/decisions/adr-0011-test-tooling-kyverno-chainsaw.md index a7627b7..b750676 100644 --- a/decisions/adr-0011-test-tooling-kyverno-chainsaw.md +++ b/decisions/adr-0011-test-tooling-kyverno-chainsaw.md @@ -1,5 +1,5 @@ # ADR-0011 — Conformance test tooling: Kyverno + Chainsaw -- Status: Accepted +- Status: Accepted — stands, except that L3 is operator-run rather than CI-triggered ([ADR-0026](adr-0026-fault-injection-operator-run.md)) - Date: 2026-08-15 - Serves: R-TEST-01, R-TEST-02, R-TEST-03, R-TEST-04, R-TEST-05 - Refines: [ADR-0010](adr-0010-conformance-testing.md) (which fixed the *approach* and left the tool mix as the one open item) diff --git a/decisions/adr-0026-fault-injection-operator-run.md b/decisions/adr-0026-fault-injection-operator-run.md new file mode 100644 index 0000000..dde36fa --- /dev/null +++ b/decisions/adr-0026-fault-injection-operator-run.md @@ -0,0 +1,37 @@ +# ADR-0026 — Fault injection is operator-run, never automated +- Status: Accepted +- Date: 2026-11-08 +- Serves: R-TEST-03, R-TEST-05, R-ENV-03 +- Supersedes: the L3 portion of ADR-0010's merge gating and of ADR-0011's CI-triggered harness +- Accepted by: operator ruling of 2026-11-08; effective on merge. + +## Context +[ADR-0010](adr-0010-conformance-testing.md) has the conformance suite gate merges, and +[ADR-0011](adr-0011-test-tooling-kyverno-chainsaw.md) has CI trigger L2 and L3 as an in-cluster Job. +Both were written when a dev/test cluster existed to break. +[ADR-0017](adr-0017-single-environment-bare-metal-production.md) retired that cluster, so production +is now the only target. An L3 test takes the VPN tunnel down, kills a node, injects a bad update or +restores a backup. Running that automatically would put a production outage on a pipeline's +schedule, and the operator sees no reason to do so. + +## Decision +- **L3 fault-injection tests are never triggered by CI, a schedule or any other automation.** The + operator starts them deliberately, at a chosen time. +- **L3 does not gate individual merges.** The merge gate covers the automatable layers. +- **Fault injection stays required.** R-TEST-05 is unchanged: safety-critical behaviors are still + verified by fault-injection tests, using the same in-cluster Job and scoped test identity. Each + operator run is recorded in the implementation's run ledger. +- ADR-0011's decision stands for L2, and for L3's tooling and harness. + +## Alternatives considered +- **Automate L3 against production on a schedule** — rejected by the operator: a deliberate outage + has no business being unattended. +- **Rebuild a dev/test cluster to automate against** — cost with no stated need; ADR-0017 retired it. +- **Retire fault-injection tests altogether** — not ruled. R-TEST-05 protects properties such as the + VPN kill-switch and data survival, which happy-path checks cannot prove. + +## Consequences +- No credential or pipeline path can start a fault against production. +- A merge can change a safety-critical behavior without its L3 test re-running, so an L3 proof can go + stale. The implementation's status tooling reports that staleness; re-running is an operator choice. +- The wiring that remains to be built for CI is the L2 trigger only. diff --git a/docs/00-vision-and-scope.md b/docs/00-vision-and-scope.md index 0e842f5..edcef51 100644 --- a/docs/00-vision-and-scope.md +++ b/docs/00-vision-and-scope.md @@ -110,8 +110,7 @@ mandatory; `SHOULD` is strongly preferred and requires an ADR to deviate. ### TLS & certificates — `R-TLS-*` - **R-TLS-01** — Every externally reachable service MUST present TLS from a **credible, trusted, non-self-signed** authority. -- **R-TLS-02** — Certificates MUST be automatically **issued, renewed, and revoked** without manual - steps. +- **R-TLS-02** — Certificates MUST be automatically **issued and renewed** without manual steps. - **R-TLS-03** — Certificate issuance MUST work for internal-only services (no requirement to expose a service publicly just to validate a certificate). @@ -184,7 +183,9 @@ mandatory; `SHOULD` is strongly preferred and requires an ADR to deviate. any conforming cluster, producing machine-readable, requirement-tagged results. - **R-TEST-03** — A **conformance suite** MUST gate delivery: merges are blocked unless the tests for the touched requirements pass, and phase exits are blocked unless the phase's full suite - passes. "Demonstrated" means "the automated test passes," never a manual one-off. + passes. "Demonstrated" means "the automated test passes," never a manual one-off. Fault-injection + tests (R-TEST-05) are not automated: the operator runs them deliberately and records the result, + and they do not gate individual merges (ADR-0026). - **R-TEST-04** — Tests MUST be **traceable**: each test declares the requirement ID(s) it verifies, and a coverage report maps every `R-*` requirement to its verifying test(s). Uncovered requirements are a gap to close, not an acceptable state. diff --git a/docs/05-gitops-and-delivery.md b/docs/05-gitops-and-delivery.md index 18774ae..1759716 100644 --- a/docs/05-gitops-and-delivery.md +++ b/docs/05-gitops-and-delivery.md @@ -76,7 +76,7 @@ runs on every PR as a Gitea Actions job. **A red gate is unmergeable by everyone included.** The gate runs in the same pinned toolbox image locally and in CI (R-TEST-02) and establishes, in order: the adopted spec pin (R-TEST-06), the handoff log's format, lint, L0 schema + policy + repository invariants, render-equals-committed, L1 rendering, the sensitive-path approval -policy, that the live Gitea settings match their declaration, and L2/L3 conformance +policy, that the live Gitea settings match their declaration, and L2 conformance ([14](14-testing-and-conformance.md); see [known gaps](16-known-gaps.md)). Its verdict is one of three: **pass**, **blocked**, or **undetermined**. A stage that did not run is diff --git a/docs/07-security-and-secrets.md b/docs/07-security-and-secrets.md index 45c25b7..9162409 100644 --- a/docs/07-security-and-secrets.md +++ b/docs/07-security-and-secrets.md @@ -63,7 +63,7 @@ Defense in depth with declarative, auditable controls: - DNS provider API token (Cloudflare) for ACME DNS-01 + ExternalDNS. - Cloudflare Tunnel credential/token for in-cluster cloudflared (SOPS-encrypted; ADR-0016). - SMTP credentials for Alertmanager email. -- Service credentials (Nextcloud, Gitea admin, UniFi, Semaphore, DB passwords; Kasm is not onboarded). +- Service credentials (Nextcloud, Gitea admin, UniFi, Semaphore, DB passwords). - Per-app forward-auth gate credentials (IdP client secret + cookie secret; [ADR-0022](../decisions/adr-0022-forward-auth-per-app-oauth2-proxy-sidecars.md)). - WireGuard ingress server key, held in SOPS because a rebuild must restore it without re-enrolling every client (ADR-0021). diff --git a/docs/08-tls-and-certificates.md b/docs/08-tls-and-certificates.md index 5289eb4..a2058b2 100644 --- a/docs/08-tls-and-certificates.md +++ b/docs/08-tls-and-certificates.md @@ -6,7 +6,7 @@ Serves: **R-TLS-01/02/03**, **R-DNS-01**. Related: [ADR-0006 (cert-manager + Let ## Requirement recap Every externally reachable service presents TLS from a **credible, trusted, non-self-signed** -authority, with certificates **auto issued, renewed, and revoked** — including for **internal-only** +authority, with certificates **auto issued and renewed** — including for **internal-only** services (R-TLS-01/02/03). ## Solution: cert-manager + Let's Encrypt via DNS-01 @@ -39,7 +39,7 @@ Certificate issuance needs programmatic write access to the zone: ACME DNS-01 mu **Decision (OQ-4):** use a **scoped Cloudflare API token** — `Zone·DNS·Edit` limited to the `keithjackson.dev` zone — stored **SOPS-encrypted** ([07](07-security-and-secrets.md)). With it: - **cert-manager** solves DNS-01 by writing/removing TXT challenge records automatically (issue, - renew, revoke — R-TLS-02), including the wildcard `*.home.keithjackson.dev` cert and internal-only + renew — R-TLS-02), including the wildcard `*.home.keithjackson.dev` cert and internal-only services (R-TLS-03). - **ExternalDNS** manages the `*.home.keithjackson.dev` records automatically ([09](09-dns-and-ingress.md), R-DNS-02) — no manual record editing for public services. @@ -49,11 +49,11 @@ Certificate issuance needs programmatic write access to the zone: ACME DNS-01 mu committed SOPS-encrypted and never touched manually again. The token is least-privilege (single zone, DNS edit only) so its blast radius is bounded (R-SEC-02). Rotation = re-encrypt + commit. -## Renewal & revocation (R-TLS-02) +## Renewal (R-TLS-02) - cert-manager renews well before expiry automatically; no human action. -- Revocation: removing/rotating a certificate is a declarative change; cert-manager handles - issuance of replacements. Expiry monitoring is an alert condition ([10](10-observability-and-alerting.md), R-OBS-04). +- Removing or rotating a certificate is a declarative change; cert-manager issues the replacement. + The platform does not perform ACME revocation: a compromised key is handled by rotation. Expiry monitoring is an alert condition ([10](10-observability-and-alerting.md), R-OBS-04). ## Where TLS terminates diff --git a/docs/13-service-catalog.md b/docs/13-service-catalog.md index 8362440..f39dabd 100644 --- a/docs/13-service-catalog.md +++ b/docs/13-service-catalog.md @@ -79,12 +79,12 @@ and `storage[].{mountPath,subPath,readOnly}`. - Longhorn-backed config PVC; off-cluster backup; trusted TLS; auto-DNS; monitored; auto-updated on `stable`. -## The catalog (14 services) +## The catalog (13 services) Egress and dependencies below are **normative intent**; implementers render the exact policies. "Bulk" = large media tier ([04](04-storage-and-state.md)). All get trusted TLS + auto-DNS + monitoring + auto-update unless noted. **Status** gives each service's delivery path: `catalog` (rendered from -its ServiceDefinition), `platform-managed` (declared directly in git), or `not onboarded`. +its ServiceDefinition) or `platform-managed` (declared directly in git). | # | Service | Status | kind | exposure | egress | Storage | Key dependencies (explicit allows) | Notes | |---|---|---|---|---|---|---|---|---| @@ -99,9 +99,8 @@ its ServiceDefinition), `platform-managed` (declared directly in git), or `not o | 8 | **minecraft** (Crafty) | platform-managed | container | internal (game ports); public (web UI) | internet | world: longhorn (backed up) | — | Delivered as the Crafty controller, which runs the game servers; game ports on a LAN/VPN L4 address | | 9 | **unifi** | platform-managed | vm | internal | internet | VM disk: longhorn (replicated, live-migratable) | — | UniFi OS Server in a KubeVirt VM, declared directly in git (custom install flow); browser and device traffic on a LAN/VPN L4 address, no Gateway route | | 10 | **semaphore** | catalog | container | internal | internet | config/db: longhorn | → its DB | Ansible UI | -| 11 | **kasm** | not onboarded | container | internal | internet | config: longhorn | — | Kasm Workspaces (may need more resources; container-first, VM if required) | -| 12 | **gitea** | catalog | container | public | internet | repos/db: longhorn (irreplaceable → backed up) | — | Git hosting; **bootstrapping-critical** — self-hosts the platform's GitOps source + CI/CD; onboarded **early** (see below); DR-critical (off-cluster backup + mirror) | -| 13 | **qbittorrent** | catalog | container | internal | **vpn** | config: longhorn; downloads: **bulk** | ← sonarr, ← radarr | VPN + kill-switch; download client | +| 11 | **gitea** | catalog | container | public | internet | repos/db: longhorn (irreplaceable → backed up) | — | Git hosting; **bootstrapping-critical** — self-hosts the platform's GitOps source + CI/CD; onboarded **early** (see below); DR-critical (off-cluster backup + mirror) | +| 12 | **qbittorrent** | catalog | container | internal | **vpn** | config: longhorn; downloads: **bulk** | ← sonarr, ← radarr | VPN + kill-switch; download client | **VPN-class set (R-NET-05/06):** sonarr, radarr, jackett, qbittorrent — all egress via VPN with an enforced kill-switch. Their inter-dependencies (arr → qbittorrent/jackett) are **east-west allows** @@ -125,7 +124,7 @@ stall an earlier wave. VMs and leaf platform components sit on their own non-gat image import or a broken UI never holds up the app tier. **SSO integration (`auth.mode`, R-IDP-02; see [15](15-identity-and-access.md)):** -- **oidc** (native Keycloak client): gitea, nextcloud, semaphore, kasm, grafana*, headlamp* +- **oidc** (native Keycloak client): gitea, nextcloud, semaphore, grafana*, headlamp* (*platform UIs, not in the numbered app list). - **forward-auth** (per-app oauth2-proxy sidecar → Keycloak, ADR-0022): pihole, sonarr, radarr, jackett, qbittorrent, Homepage*, and the Crafty web UI* (which adds its own local login; it has no @@ -174,8 +173,6 @@ Points the schema left ambiguous, now pinned so the JSON Schema, Kyverno policie oauth2-proxy host). Everything else stays `internal`. **Plex** remote streaming uses plex.tv + an edge forward of tcp/32400 (not the tunnel), and its web UI is LAN-only. **Minecraft (crafty-mc)** and other L4 services are LAN/VPN-only. -- **Kasm** is resource-heavy and may be a candidate for `kind: vm` if container mode proves - insufficient — the ServiceDefinition makes that a one-field change. - **Gitea is bootstrapping-critical.** The platform self-hosts its git and CI/CD, so Gitea is brought online **early** (the first wave, with PiHole). Gitea + Gitea Actions are the GitOps source and CI ([05](05-gitops-and-delivery.md), [`../CONTRIBUTING.md`](../CONTRIBUTING.md)). Because a cluster diff --git a/docs/14-testing-and-conformance.md b/docs/14-testing-and-conformance.md index 6bdbe52..bb87217 100644 --- a/docs/14-testing-and-conformance.md +++ b/docs/14-testing-and-conformance.md @@ -17,7 +17,8 @@ Related: [ADR-0010](../decisions/adr-0010-conformance-testing.md). environment-agnostic. Any conforming cluster + a kubeconfig ⇒ the same pass/fail. No flakiness budget; eventual-consistency is handled with bounded polling, not sleeps-and-hope. 3. **Conformance gates delivery (R-TEST-03).** The suite blocks merges (for touched requirements) - and blocks phase exits (full phase suite). Green suite is the definition of progress. + and blocks phase exits (full phase suite). Green suite is the definition of progress. Fault + injection is operator-run and does not gate individual merges (ADR-0026). 4. **Traceable (R-TEST-04).** Each test declares its `R-*` IDs; a generated coverage report maps every requirement to its verifying test(s). Uncovered requirements are visible gaps. 5. **Fault-injection for safety-critical behavior (R-TEST-05).** Default-deny, VPN kill-switch, @@ -52,7 +53,7 @@ Related: [ADR-0010](../decisions/adr-0010-conformance-testing.md). | **L0 Static/policy** | Pre-apply, in CI | Manifests/ServiceDefinitions obey invariants | Every namespace has a default-deny baseline; no plaintext secrets; schema-valid ServiceDefinition | | **L1 Rendering** | CI, no cluster | ServiceDefinition → rendered resources match golden expectations | `egress: vpn` renders the kill-switched egress path | | **L2 Behavioral/integration** | Against the live cluster | Real runtime behavior | Default-deny probe, dependency allow/deny, egress-IP, TLS validity, DNS, PVC rebuild | -| **L3 Fault-injection** | Against the live cluster | Fail-closed / no-data-loss under failure | VPN tunnel forced down ⇒ no leak; node killed ⇒ data survives; bad update ⇒ auto-rollback; DR restore | +| **L3 Fault-injection** | Against the live cluster, operator-run only | Fail-closed / no-data-loss under failure | VPN tunnel forced down ⇒ no leak; node killed ⇒ data survives; bad update ⇒ auto-rollback; DR restore | | **L4 Conformance suite** | CI + on-demand | Aggregate, requirement-tagged gate | The union of the above, reported per `R-*` | ## Behavior test catalog (normative — implementers author these as automated tests) @@ -109,10 +110,13 @@ Each maps to requirement IDs; phrased as Given/When/Then intent. ## Gating rules (R-TEST-03) - **Per-change (merge gate):** the conformance subset tagged with the requirement IDs the change - touches MUST pass before merge. + touches MUST pass before merge, across the automatable layers (L0–L2). - **Per-phase (exit gate):** the full conformance suite for the phase's requirements MUST pass before the phase's exit criteria are considered met ([`../roadmap/phases.md`](../roadmap/phases.md)). -- **Regression:** the growing suite runs continuously against the live cluster; a previously-green +- **Fault injection (L3) is operator-run** ([ADR-0026](../decisions/adr-0026-fault-injection-operator-run.md)): + no pipeline or schedule ever starts it. The operator runs it deliberately and records each result + in the run ledger. +- **Regression:** the automatable layers run continuously; a previously-green requirement going red blocks further delivery until fixed. - **Coverage:** a requirement with no passing test is **not** satisfied, regardless of implementation state. @@ -172,7 +176,8 @@ pinned for determinism. - **L0 (Kyverno CLI) and L1 (rendering assertions)** run **in CI without a cluster** — pure manifest/ ServiceDefinition validation; fast merge-gate feedback. - **L2 behavioral and L3 fault-injection run as an in-cluster Chainsaw Job** on the live cluster. - CI **triggers** the Job (and waits for it); the Job — not CI — holds cluster access, so CI never + CI **triggers** the L2 Job (and waits for it), and only the operator starts an L3 run + (ADR-0026). The Job — not CI — holds cluster access, so CI never needs a broad kubeconfig. CI holds only a narrow trigger credential, whose reach is bounded by a fail-closed admission policy on the Job's shape. This keeps credentials in-cluster and least-privileged (R-SEC-02). @@ -194,7 +199,7 @@ pinned for determinism. ## Open questions - Tool mix is **resolved**: Kyverno (L0) + Chainsaw (L1/L2/L3), [ADR-0011](../decisions/adr-0011-test-tooling-kyverno-chainsaw.md). -- Harness location is **resolved**: L2/L3 run as an **in-cluster Chainsaw Job** triggered by CI; CI +- Harness location is **resolved**: L2/L3 run as an **in-cluster Chainsaw Job**, L2 triggered by CI and L3 by the operator; CI holds no broad cluster credential, only the admission-bounded trigger. - Kyverno-as-admission-enforcer is **resolved**: yes — the L0 policies also enforce at admission in-cluster (defense in depth), in addition to the CI gate. diff --git a/docs/15-identity-and-access.md b/docs/15-identity-and-access.md index fca01a8..b4baf69 100644 --- a/docs/15-identity-and-access.md +++ b/docs/15-identity-and-access.md @@ -21,7 +21,7 @@ Every service authenticates against Keycloak, via whichever pattern it supports: 1. **Native OIDC** — the app is configured as a Keycloak OIDC **client** and does its own login redirect. Preferred where available. Initial set: **Gitea, Nextcloud, Grafana, Semaphore, - Headlamp** (Kasm, not onboarded, joins this set when it is). + Headlamp**. 2. **Forward-auth gate** — for apps with **no** usable OIDC, a per-app identity-aware proxy (**oauth2-proxy**) runs as a sidecar in the app's pod, authenticates the user against Keycloak, and only then forwards the request ([ADR-0022](../decisions/adr-0022-forward-auth-per-app-oauth2-proxy-sidecars.md)). diff --git a/docs/16-known-gaps.md b/docs/16-known-gaps.md index b0441f5..ab07599 100644 --- a/docs/16-known-gaps.md +++ b/docs/16-known-gaps.md @@ -15,15 +15,11 @@ has yet to reach. ## Operator decisions needed -These cannot be settled by reading either repository. +This cannot be settled by reading either repository. | Topic | Question | Evidence (implementation repo) | |---|---|---| | Bulk-tier photos (R-STATE-03/04) | The bulk tier has no platform backup, and its media dataset includes a `photos/` tree. Is that tree irreplaceable, and if so, what protects it? | `docs/BULK-NFS-STORAGE.md` §2; `platform/velero/volume-policies.yaml` | -| Homepage exposure (R-IDP-02) | The apex is routed publicly with no enforced authentication. Enforce the gate (the requirement stands), or rule on an exception? | `platform/homepage/httproute.yaml`; `platform/homepage/README.md` | -| ACME revocation (R-TLS-02) | cert-manager has no revocation mechanism. Amend the requirement, or keep it as an open gap? | `conformance/tests/l0-static/tls-lifecycle-automated/chainsaw-test.yaml:36` | -| L3 in CI (R-TEST-03, R-TEST-05) | Production is the only cluster. Is automated fault injection against it acceptable once CI is wired? | `docs/plans/l2-l3-conformance-ci-wiring.md` | -| Kasm (docs/13) | Never onboarded, and no decision retires it. Onboard it, or retire it from the catalog? | `catalog/README.md:20` | ## Environment and provisioning @@ -56,7 +52,6 @@ These cannot be settled by reading either repository. | VPN port forwarding (docs/03, ADR-0015) | The spec requires a host-side PIA port-forward helper for qBittorrent and an alert on PF acquisition failure; neither exists. | `infra/ansible/roles/vpn_egress_gateway/README.md:170`, `platform/vpn-egress/README.md` | | R-NET-01 | Every namespace must carry a default-deny baseline; Kyverno generates it only for `app-namespace=true`, and no policy file exists for cert-manager, cnpg, external-dns, longhorn, velero, backup, image-automation, external-snapshotter or vpn-egress. | `platform/kyverno/policies/require-default-deny-baseline.yaml`, `decisions/2026-10-03-public-dns-wan-target.md` | | R-SEC-03, R-OBS-04 | docs/03 requires alerting on unexpected denied flows; `HubbleDeniedFlowSpike` cannot fire because no Cilium or Hubble metrics are scraped. | `platform/monitoring/alerts.yaml:1047-1050,1162-1173` | -| R-TLS-02 | Certificates must be automatically revoked; renewal is met, but cert-manager performs no ACME revocation and no layer proves it. | `conformance/tests/l0-static/tls-lifecycle-automated/chainsaw-test.yaml:36`, `DECISIONS-LOG.md:3350,3786` | | Client IP (ADR-0016) | IP-based logging and policy must read `CF-Connecting-IP` / `X-Forwarded-For` and trust them only from cloudflared; no trusted-proxy or client-IP configuration exists in `catalog/`, `platform/` or `apps/`. | `platform/cloudflared/config.yaml` (no client-IP handling downstream of it) | | R-DNS-04 | The edge must not forward 80/443; closing them is a manual router step, last recorded as not done, so the state is unverified. | `decisions/2026-11-08-dns-via-cloudflare-tunnel.md`, `runbooks/cloudflare-tunnel.md:47-55` | | R-DNS-04, R-DNS-05, R-SEC-04 | docs/09 and docs/14 require behavioural proof at the tunnel and from the LAN through a public resolver; only L0 static checks exist, and no L2 test touches the tunnel. | `conformance/tests/l0-static/cloudflared-ingress-rules`, `conformance/tests/l2-behavioral/dns-exposure/chainsaw-test.yaml` | @@ -65,7 +60,7 @@ These cannot be settled by reading either repository. | Requirement or area | Requirement and shortfall | Evidence (implementation repo) | |---|---|---| -| R-IDP-02, R-IDP-06 | No management UI may be exposed unauthenticated and the C2 requires an authorized group, but Homepage at the apex is routed straight to its Service with no gate and its `forward-auth` annotation enforces nothing. | `platform/homepage/httproute.yaml`, `platform/homepage/README.md` | +| R-IDP-02, R-IDP-06 (operator ruling 2026-11-08: enforce it) | No management UI may be exposed unauthenticated and the C2 requires an authorized group, but Homepage at the apex is routed straight to its Service with no gate and its `forward-auth` annotation enforces nothing. | `platform/homepage/httproute.yaml`, `platform/homepage/README.md` | | R-SEC-03, R-C2-02 | Operational actions must be audited, but the k3s API server has only OIDC arguments and no audit log or audit policy. | `infra/ansible/roles/k3s_server/tasks/main.yml` | | R-SEC-01 | The acceptance gate requires no plaintext secret anywhere in git history (scanned), but the check inspects the tracked tree only and no history scanner runs in CI. | `conformance/tests/l0-static/no-plaintext-secrets/check.sh`, `ci/` | | R-SEC-01 | Rotation is meant to trigger a workload restart through reconcile (e.g. a checksum annotation), but no generated mechanism exists; the only annotations are hand-stamped on Homepage and Grafana. | `platform/homepage/deployment.yaml`, `platform/monitoring/kube-prometheus-stack.yaml`, `catalog/render.py` | @@ -78,7 +73,7 @@ These cannot be settled by reading either repository. | R-OBS-03 | Per-service dashboards are specified as onboarding output, but the renderer emits no dashboards; the in-repo dashboards are node pressure, KubeVirt and Keycloak only. | `catalog/render.py`, `platform/monitoring/node-pressure-dashboard.yaml`, `platform/kubevirt/dashboard.yaml`, `platform/keycloak/grafana-dashboard.yaml` | | R-OBS-01, R-ENV-02 | Log retention is specified as an overlay setting, but Loki's retention is hardcoded with no overlay knob. | `platform/monitoring/loki.yaml`, `platform/monitoring/README.md` | | R-OBS-04 | Monitoring storage is specified on Longhorn, but Alertmanager and Grafana run without persistent volumes, so silences and notification state are lost on restart. | `platform/monitoring/kube-prometheus-stack.yaml` | -| R-OBS-02, R-OBS-04 | Acceptance requires platform alerts to fire and email in injected-fault tests, but the L3 alerting test is a scaffold and L2/L3 do not run in CI. | `conformance/tests/l3-fault-injection/alerting/chainsaw-test.yaml`, `CLAUDE.md` | +| R-OBS-02, R-OBS-04 | Acceptance requires platform alerts to fire and email in injected-fault tests, but the L3 alerting test is a scaffold. | `conformance/tests/l3-fault-injection/alerting/chainsaw-test.yaml`, `CLAUDE.md` | | R-C2-01 | Homepage is specified as a live catalog with status pings and per-service widgets, but it is a hand-maintained ConfigMap of link tiles with no status or widgets. | `platform/homepage/configmap.yaml`, `platform/homepage/README.md` | | R-C2-02 | The C2 must surface platform health and firing alerts, but none is shown on the C2 landing surface; health is reachable only by linking out to Grafana. | `platform/homepage/configmap.yaml` | | R-C2-02 | Grafana dashboards are specified as embedded in the C2, but Homepage only links to Grafana. | `platform/homepage/configmap.yaml` | @@ -89,7 +84,6 @@ These cannot be settled by reading either repository. | Requirement or area | Requirement and shortfall | Evidence (implementation repo) | |---|---|---| -| docs/13 catalog row 11 (kasm), R-IDP-02 | The catalog includes Kasm with native OIDC, and no ServiceDefinition, manifest or decision exists for it, so it is not onboarded. | `catalog/README.md:20` (only mention in the repo) | | R-STATE-03, docs/13 `backup.policy` | `backup.policy` must reference a named off-cluster schedule, and `daily-offcluster` resolves to no schedule object; it is an annotation only, while coverage comes from cluster-wide Velero schedules. | `platform/velero/README.md:33-34`, `schemas/renderer-contract.md:428`, `catalog/*.yaml` (`backup.policy`) | | R-UPD-01/02, docs/13 `update.cadence` | `update.cadence` should set how often a service checks or applies updates, and the renderer does not read it for containers or VMs. | `catalog/render.py` (no `cadence` reference), `schemas/renderer-contract.md:388-390` | @@ -97,8 +91,8 @@ These cannot be settled by reading either repository. | Requirement or area | Requirement and shortfall | Evidence (implementation repo) | |---|---|---| -| R-TEST-03 | Merges require the tests for touched requirements to pass; the gate runs L0/L1 and tagged-coverage only, because L2/L3 do not run in CI (job disabled, trigger path unimplemented, cluster not declared), and a PASS names that declared gap. | `.gitea/workflows/gate.yml` (`conformance` job `if: false`), `ci/conformance.sh`, `docs/plans/l2-l3-conformance-ci-wiring.md` | -| R-TEST-03 | Phase exits require the full suite and regression runs continuously; no pipeline runs the full-suite gate and no scheduled run exists, so L2/L3 evidence comes only from operator-recorded runs, last recorded 2026-09-19. | `conformance/run.sh` (`--require-all`, `--record`), `conformance/runs/ledger.jsonl` | +| R-TEST-03 | Merges require the tests for touched requirements to pass; the gate runs L0/L1 and tagged-coverage only, because L2 does not run in CI (job disabled, trigger path unimplemented, cluster not declared), and a PASS names that declared gap. | `.gitea/workflows/gate.yml` (`conformance` job `if: false`), `ci/conformance.sh`, `docs/plans/l2-l3-conformance-ci-wiring.md` | +| R-TEST-03 | Phase exits require the full suite and regression runs continuously; no pipeline runs the full-suite gate and no scheduled run exists, so L2 evidence comes only from operator-recorded runs, last recorded 2026-09-19. L3 is operator-run by design (ADR-0026). | `conformance/run.sh` (`--require-all`, `--record`), `conformance/runs/ledger.jsonl` | | R-TEST-01, R-TEST-04 | A requirement with no passing test is not satisfied; all 51 registry requirements have a tagged test but PROVEN is 0 (3 stale: R-NET-06, R-STATE-02, R-STATE-03; 48 not derivable from the ledger). | `conformance/lib/status.py`, `conformance/runs/ledger.jsonl` | | R-OBS-02, R-GITOPS-02, R-GITOPS-03 | Each requirement needs an executable behavior test; the only verifying tests (`alerting`, `onboarding-completeness`, `gitops-drift`) are scaffolds. | `conformance/lib/status.py`, `conformance/tests/` | | R-TEST-06 | The spec must be verifiably immutable to agents; CI never checks the spec repo out, so spec-immutability always skips there (tolerated) and only local gates with a sibling checkout enforce it. | `.gitea/workflows/gate.yml:193,227`, `ci/spec-immutability.sh`, `ci/gate.sh` (`REQUIRE_SPEC_VERIFICATION`) | diff --git a/roadmap/workstreams.md b/roadmap/workstreams.md index 889bbfb..6463f66 100644 --- a/roadmap/workstreams.md +++ b/roadmap/workstreams.md @@ -65,7 +65,7 @@ only via [`../agents/coordination-protocol.md`](../agents/coordination-protocol. 5. **qbittorrent, jackett** (VPN + kill-switch proof) → 6. **sonarr, radarr** (VPN + arr↔downloader allows) → 7. **seerr** (cross-service allows to arr + plex) → 8. **nextcloud** (DR-critical data + backups) → 9. **minecraft** (delivered as Crafty, platform-managed), **unifi** (a KubeVirt VM), -**semaphore** → **kasm** (not onboarded). +**semaphore**. As each service onboards, it adopts SSO per its `auth` mode (native OIDC or forward-auth) — [`../docs/15-identity-and-access.md`](../docs/15-identity-and-access.md).