jakman85andClaude Opus 5.5
18f7ba8d0f
Retire Kasm from the catalog. Drop ACME revocation from R-TLS-02. Make L3 fault injection operator-run and exclude it from the merge gate (R-TEST-03, ADR-0026). Update the gap register to match; Homepage authentication is now ruled must-enforce and stays a gap until the implementation lands. Co-Authored-By: Claude Opus 5.5 <[email protected]>
103 lines
15 KiB
Markdown
103 lines
15 KiB
Markdown
# 16 — Known Gaps
|
|
|
|
**Snapshot of implementation `main` at `50c8719`, taken 2026-11-08.** This register records where the
|
|
implementation does not yet meet a requirement this specification still holds. It exists so that bringing
|
|
the specification in line with the implementation did not silently delete the intent the implementation
|
|
has yet to reach.
|
|
|
|
- A gap is **not** an accepted change. Where the implementation deliberately changed direction, the
|
|
requirement or an ADR was amended instead, and the item does not appear here.
|
|
- This file is a dated snapshot, not a status surface. Every commit here costs the implementation a
|
|
`ci/SPEC_PIN` bump, so it is refreshed in batches. **Live status comes from `conformance/run.sh --status`**
|
|
in the [implementation repo](https://gitea.home.keithjackson.dev/Home-Network/home-network-k3s-impl).
|
|
- Entries were derived by reading both repositories. **Nothing here was measured against the running
|
|
platform.**
|
|
|
|
## Operator decisions needed
|
|
|
|
This cannot be settled by reading either repository.
|
|
|
|
| Topic | Question | Evidence (implementation repo) |
|
|
|---|---|---|
|
|
| Bulk-tier photos (R-STATE-03/04) | The bulk tier has no platform backup, and its media dataset includes a `photos/` tree. Is that tree irreplaceable, and if so, what protects it? | `docs/BULK-NFS-STORAGE.md` §2; `platform/velero/volume-policies.yaml` |
|
|
|
|
## Environment and provisioning
|
|
|
|
| Requirement or area | Requirement and shortfall | Evidence (implementation repo) |
|
|
|---|---|---|
|
|
| R-ENV-01, R-STATE-01 (Phase 1 exit) | A node must be destroyable and rebuildable from code with no data loss; a node was reimaged and rejoined by hand, but no automated destroy/rebuild proof exists on bare metal and the checklist item is unticked. | `infra/PROD-PROVISIONING.md` sections 0c and 9 |
|
|
| docs/02 etcd guardrail (accepted risk) | Shared-device etcd/Longhorn contention is accepted only if etcd write latency stays within budget during a Longhorn rebuild; the measurement failed on node 1 on 2026-09-07 and no dedicated etcd device was added, so the exit condition is unmet. | `infra/PROD-PROVISIONING.md` section 2 ("The accepted risk, and its exit condition"), section 0c |
|
|
| R-PLAT-03, Phase 5 exit | The VM tier must run a working application VM; at the last recorded check (2026-09-14) the UniFi guest had no IP address and its controller ports refused connections. | `apps/vms/unifi/README.md` |
|
|
|
|
## Storage, state and VMs
|
|
|
|
| Requirement or area | Requirement and shortfall | Evidence (implementation repo) |
|
|
|---|---|---|
|
|
| Off-cluster Gitea mirror (R-STATE-03; spec CONTRIBUTING, docs/05 "Circularity & DR") | An off-cluster mirror of the platform repo must outlive the cluster; Gitea has no mirrors and no mirror job exists, while prod's Flux source is in-cluster Gitea. | `catalog/gitea.yaml:52`; `decisions/2026-11-07-gitea-28.md` |
|
|
| R-STATE-03 (Gitea) | Gitea is DR-critical and its repos and SQLite are backed up by file-level kopia with no quiesce hook, and no Gitea-specific restore has been proven (the DR test restores its own scratch workload). | `apps/wave-0/gitea.yaml`; `conformance/tests/l3-fault-injection/dr-restore/chainsaw-test.yaml` |
|
|
| R-STATE-03 (UniFi) | Irreplaceable VM state must be recoverable off-cluster; the UniFi disk is excluded on the basis of a configuration export that nothing in the repo declares. | `platform/velero/helmrelease.yaml:182-188`; `apps/vms/unifi/` |
|
|
| R-STATE-01 | The per-service rebuild test is an acceptance gate; `data-survival` and `vm-data-survival` exist but have no ledger row, and per-service rebuilds are not recorded. | `conformance/runs/ledger.jsonl`; `conformance/tests/l3-fault-injection/data-survival/` |
|
|
| R-STATE-01/03 | End-to-end total-cluster recovery should be drilled at least once; bare-metal rebuild then restore has not been exercised. | `conformance/tests/l3-fault-injection/dr-restore/chainsaw-test.yaml` (KNOWN, DELIBERATE GAPS) |
|
|
| R-STATE-01 (Loki) | Destroying or scaling a workload must leave its PVC; Loki's StatefulSet retention policy deleted its PVC on scale-down (log data is reconstructable). | `decisions/2026-10-17-graceful-cluster-shutdown.md` |
|
|
| R-PLAT-03, R-PLAT-04 | VM create/start/stop/destroy must be proven; `vm-lifecycle` is logged green only on the decommissioned dev cluster and has no ledger row. | `DECISIONS-LOG.md:896`; `conformance/runs/ledger.jsonl` |
|
|
| R-NET-06 (VMs) | An `egress: vpn` VM must egress through the VPN with a working kill-switch; no VM VPN conformance test exists and `vm-demo` calls it a follow-up. | `catalog/vm-demo.yaml`; `conformance/tests/l2-behavioral/vm-lifecycle/` |
|
|
| R-UPD-* (UniFi guest) | Guest OS packages must auto-update on schedule with per-VM exceptions recorded; UniFi's cloud-init sets `package_upgrade: false` and installs no unattended-upgrades. | `apps/vms/unifi/30-virtualmachine.yaml:206-207` |
|
|
| R-OBS-* (VM guests) | A VM must expose node-exporter-class metrics from inside the guest; neither the renderer nor UniFi provides any, only KubeVirt/virt platform metrics. | `catalog/render.py`; `platform/kubevirt/servicemonitor.yaml` |
|
|
| R-STATE-03 (VM disks) | VM disks are meant to share the platform backup story; the backup path is Velero file-system backup, which upstream documents as not supporting block-mode volumes, and the default VM class is block-mode. Unmeasured here: whether any block-mode VM disk can be captured by this path. | `platform/velero/helmrelease.yaml:175`; `platform/longhorn/storageclasses.yaml` (`longhorn-vm`) |
|
|
|
|
## Networking, DNS and TLS
|
|
|
|
| Requirement or area | Requirement and shortfall | Evidence (implementation repo) |
|
|
|---|---|---|
|
|
| VPN port forwarding (docs/03, ADR-0015) | The spec requires a host-side PIA port-forward helper for qBittorrent and an alert on PF acquisition failure; neither exists. | `infra/ansible/roles/vpn_egress_gateway/README.md:170`, `platform/vpn-egress/README.md` |
|
|
| R-NET-01 | Every namespace must carry a default-deny baseline; Kyverno generates it only for `app-namespace=true`, and no policy file exists for cert-manager, cnpg, external-dns, longhorn, velero, backup, image-automation, external-snapshotter or vpn-egress. | `platform/kyverno/policies/require-default-deny-baseline.yaml`, `decisions/2026-10-03-public-dns-wan-target.md` |
|
|
| R-SEC-03, R-OBS-04 | docs/03 requires alerting on unexpected denied flows; `HubbleDeniedFlowSpike` cannot fire because no Cilium or Hubble metrics are scraped. | `platform/monitoring/alerts.yaml:1047-1050,1162-1173` |
|
|
| Client IP (ADR-0016) | IP-based logging and policy must read `CF-Connecting-IP` / `X-Forwarded-For` and trust them only from cloudflared; no trusted-proxy or client-IP configuration exists in `catalog/`, `platform/` or `apps/`. | `platform/cloudflared/config.yaml` (no client-IP handling downstream of it) |
|
|
| R-DNS-04 | The edge must not forward 80/443; closing them is a manual router step, last recorded as not done, so the state is unverified. | `decisions/2026-11-08-dns-via-cloudflare-tunnel.md`, `runbooks/cloudflare-tunnel.md:47-55` |
|
|
| R-DNS-04, R-DNS-05, R-SEC-04 | docs/09 and docs/14 require behavioural proof at the tunnel and from the LAN through a public resolver; only L0 static checks exist, and no L2 test touches the tunnel. | `conformance/tests/l0-static/cloudflared-ingress-rules`, `conformance/tests/l2-behavioral/dns-exposure/chainsaw-test.yaml` |
|
|
|
|
## Security, identity, observability and the command center
|
|
|
|
| Requirement or area | Requirement and shortfall | Evidence (implementation repo) |
|
|
|---|---|---|
|
|
| R-IDP-02, R-IDP-06 (operator ruling 2026-11-08: enforce it) | No management UI may be exposed unauthenticated and the C2 requires an authorized group, but Homepage at the apex is routed straight to its Service with no gate and its `forward-auth` annotation enforces nothing. | `platform/homepage/httproute.yaml`, `platform/homepage/README.md` |
|
|
| R-SEC-03, R-C2-02 | Operational actions must be audited, but the k3s API server has only OIDC arguments and no audit log or audit policy. | `infra/ansible/roles/k3s_server/tasks/main.yml` |
|
|
| R-SEC-01 | The acceptance gate requires no plaintext secret anywhere in git history (scanned), but the check inspects the tracked tree only and no history scanner runs in CI. | `conformance/tests/l0-static/no-plaintext-secrets/check.sh`, `ci/` |
|
|
| R-SEC-01 | Rotation is meant to trigger a workload restart through reconcile (e.g. a checksum annotation), but no generated mechanism exists; the only annotations are hand-stamped on Homepage and Grafana. | `platform/homepage/deployment.yaml`, `platform/monitoring/kube-prometheus-stack.yaml`, `catalog/render.py` |
|
|
| R-SEC-01 | VPN keys, DNS tokens and SMTP credentials should have a documented rotation procedure, but the only trace is a one-line note on the Cloudflare token and none exists for VPN keys or SMTP. | `runbooks/cert-issuance.md`, `runbooks/` |
|
|
| R-OBS-01 | Alloy is specified to ship node, pod and VM logs, but it collects pod logs only, with no node or journal source. | `platform/monitoring/alloy.yaml` |
|
|
| R-OBS-04 | NetworkPolicy reconcile errors are a specified platform alert, but no such rule exists. | `platform/monitoring/alerts.yaml` |
|
|
| R-OBS-04 | Restore-test failures are a specified platform alert, but only backup and schedule alerts exist. | `platform/monitoring/alerts.yaml` |
|
|
| R-OBS-04 | Keycloak login-endpoint failures are a specified alert, but the repo has no endpoint probe, only workload and database health rules. | `platform/monitoring/alerts.yaml` |
|
|
| R-OBS-01 | Every forward-auth gate is meant to be observed, but Crafty's gate has no metrics listener, so `OAuth2ProxyGateUnobserved` fires for it by design. | `platform/crafty/deployment.yaml`, `platform/oauth2-proxy/README.md` |
|
|
| R-OBS-03 | Per-service dashboards are specified as onboarding output, but the renderer emits no dashboards; the in-repo dashboards are node pressure, KubeVirt and Keycloak only. | `catalog/render.py`, `platform/monitoring/node-pressure-dashboard.yaml`, `platform/kubevirt/dashboard.yaml`, `platform/keycloak/grafana-dashboard.yaml` |
|
|
| R-OBS-01, R-ENV-02 | Log retention is specified as an overlay setting, but Loki's retention is hardcoded with no overlay knob. | `platform/monitoring/loki.yaml`, `platform/monitoring/README.md` |
|
|
| R-OBS-04 | Monitoring storage is specified on Longhorn, but Alertmanager and Grafana run without persistent volumes, so silences and notification state are lost on restart. | `platform/monitoring/kube-prometheus-stack.yaml` |
|
|
| R-OBS-02, R-OBS-04 | Acceptance requires platform alerts to fire and email in injected-fault tests, but the L3 alerting test is a scaffold. | `conformance/tests/l3-fault-injection/alerting/chainsaw-test.yaml`, `CLAUDE.md` |
|
|
| R-C2-01 | Homepage is specified as a live catalog with status pings and per-service widgets, but it is a hand-maintained ConfigMap of link tiles with no status or widgets. | `platform/homepage/configmap.yaml`, `platform/homepage/README.md` |
|
|
| R-C2-02 | The C2 must surface platform health and firing alerts, but none is shown on the C2 landing surface; health is reachable only by linking out to Grafana. | `platform/homepage/configmap.yaml` |
|
|
| R-C2-02 | Grafana dashboards are specified as embedded in the C2, but Homepage only links to Grafana. | `platform/homepage/configmap.yaml` |
|
|
| R-C2-02 | Flux status is specified via Headlamp's Flux view, the Flux UI or a Homepage widget, but Headlamp has no Flux plugin configured and Homepage has no widget. | `platform/headlamp/helmrelease.yaml`, `platform/homepage/configmap.yaml` |
|
|
| R-C2-02 | C2 actions are specified to include VM start/stop, Flux reconcile and alert acknowledgement, but only native Headlamp workload actions exist, with no plugins for the others. | `platform/headlamp/helmrelease.yaml` |
|
|
|
|
## Service catalog
|
|
|
|
| Requirement or area | Requirement and shortfall | Evidence (implementation repo) |
|
|
|---|---|---|
|
|
| R-STATE-03, docs/13 `backup.policy` | `backup.policy` must reference a named off-cluster schedule, and `daily-offcluster` resolves to no schedule object; it is an annotation only, while coverage comes from cluster-wide Velero schedules. | `platform/velero/README.md:33-34`, `schemas/renderer-contract.md:428`, `catalog/*.yaml` (`backup.policy`) |
|
|
| R-UPD-01/02, docs/13 `update.cadence` | `update.cadence` should set how often a service checks or applies updates, and the renderer does not read it for containers or VMs. | `catalog/render.py` (no `cadence` reference), `schemas/renderer-contract.md:388-390` |
|
|
|
|
## Delivery, updates and conformance
|
|
|
|
| Requirement or area | Requirement and shortfall | Evidence (implementation repo) |
|
|
|---|---|---|
|
|
| R-TEST-03 | Merges require the tests for touched requirements to pass; the gate runs L0/L1 and tagged-coverage only, because L2 does not run in CI (job disabled, trigger path unimplemented, cluster not declared), and a PASS names that declared gap. | `.gitea/workflows/gate.yml` (`conformance` job `if: false`), `ci/conformance.sh`, `docs/plans/l2-l3-conformance-ci-wiring.md` |
|
|
| R-TEST-03 | Phase exits require the full suite and regression runs continuously; no pipeline runs the full-suite gate and no scheduled run exists, so L2 evidence comes only from operator-recorded runs, last recorded 2026-09-19. L3 is operator-run by design (ADR-0026). | `conformance/run.sh` (`--require-all`, `--record`), `conformance/runs/ledger.jsonl` |
|
|
| R-TEST-01, R-TEST-04 | A requirement with no passing test is not satisfied; all 51 registry requirements have a tagged test but PROVEN is 0 (3 stale: R-NET-06, R-STATE-02, R-STATE-03; 48 not derivable from the ledger). | `conformance/lib/status.py`, `conformance/runs/ledger.jsonl` |
|
|
| R-OBS-02, R-GITOPS-02, R-GITOPS-03 | Each requirement needs an executable behavior test; the only verifying tests (`alerting`, `onboarding-completeness`, `gitops-drift`) are scaffolds. | `conformance/lib/status.py`, `conformance/tests/` |
|
|
| R-TEST-06 | The spec must be verifiably immutable to agents; CI never checks the spec repo out, so spec-immutability always skips there (tolerated) and only local gates with a sibling checkout enforce it. | `.gitea/workflows/gate.yml:193,227`, `ci/spec-immutability.sh`, `ci/gate.sh` (`REQUIRE_SPEC_VERIFICATION`) |
|
|
| R-UPD-01, R-UPD-02 | Hands-off updates must be proven by a behavior test; the only verifying test exercises Flux image-policy resolution, not the Renovate path that delivers updates. | `conformance/tests/l3-fault-injection/hands-off-update/chainsaw-test.yaml` |
|
|
| R-UPD-03 | A pinned service must not advance; the ServiceDefinition `update.pin` only affects a disabled render path, so it would not stop Renovate (no catalog entry uses it; a Renovate rule is the working pin). | `catalog/render.py:1731`, `ci/render-golden.sh`, `renovate.json5` |
|
|
| R-UPD-03 | A failed update must auto-roll-back; only HelmReleases have remediation, while plain Deployments stall on readiness and `autoRollbackOnFailedHealth` renders only onto the disabled image policy. | `catalog/render.py:1747`, `conformance/tests/l3-fault-injection/hands-off-update/chainsaw-test.yaml` |
|
|
| R-UPD-02, R-OBS-04 | Update failures must alert; the update alerts watch Flux objects that do not exist and so cannot fire, and nothing alerts when Renovate's schedule stops. | `platform/monitoring/alerts.yaml:1538,1552`, `.gitea/workflows/renovate.yaml` header |
|