Files
jakman85andClaude Opus 5.5 250fb54154 docs: bring the specification in line with the accepted implementation
The operator ruled the implementation repo's main (50c8719) the latest
and accepted state. Sync the spec to it.

Accepted changes are reflected: one bare-metal environment, TrueNAS bulk
storage, R2 off-cluster backup, CloudNativePG, single-node VPN gateway
with Cilium fail-closed, inbound WireGuard, per-app auth sidecars,
Renovate, the CI gate as approver, and the pm-agent. New ADR-0017..0025;
ADR-0016 corrected and accepted; R-* IDs unchanged, eight amended.

Where the implementation has simply not met a requirement, the
requirement stands and the shortfall is recorded in docs/16-known-gaps.md.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 22:46:00 -06:00

6.8 KiB
Raw Permalink Blame History

Roadmap — Delivery Phases

Delivery is phased. Each phase has an entry gate (what must be true to start), exit criteria (demonstrable outcomes tied to requirement IDs), and the workstreams (workstreams.md) that execute it. Status: Phases 0–6 are delivered and Phase 7 is open (see known gaps for what remains unproven). Phases 1–6 were first built on a dev/test cluster, decommissioned in 2026-09; prod is the only environment (ADR-0017).

A phase's exit criteria are passing automated conformance tests (per ../AGENTS.md §7 and ../docs/14-testing-and-conformance.md), not "manifests merged" and not manual demos. Every "Prove:" / "demonstrate" item below is an automated, reproducible, requirement-tagged behavior test committed with the work. A phase exits only when its full conformance subset is green. The conformance harness (WS-TEST) is stood up in Phase 0, so every subsequent phase ships with tests, not after them.


Phase 0 — Foundations & Repo Bootstrap

Entry: specs (this repo) approved. Do:

  • Create the implementation repo structure (../docs/05-gitops-and-delivery.md).
  • Stand up CI (lint, schema validation, dry-run/build) for manifests and ServiceDefinitions.
  • Author the ServiceDefinition schema + renderer contract (../docs/13-service-catalog.md).
  • Stand up the conformance harness (WS-TEST): single-command runner, reproducibility contract, requirement-tagged JUnit + coverage report, L0 policy-as-code gate, and the CI merge gate (../docs/14-testing-and-conformance.md, ADR-0010). Exit: implementation repo + CI exist; ServiceDefinition schema validated; conformance harness runs green with the coverage report wired; no cluster yet. Serves: R-GITOPS-02, R-ENV-02, R-TEST-01..04.

Phase 1 — Cluster Bring-up (IaC)

Entry: Phase 0 done. Do: node baseline + Ansible k3s HA install with Cilium; disable bundled flannel/kube-proxy/Traefik; bootstrap Flux; bootstrap SOPS age key. Exit: 3-node k3s HA cluster with Cilium up; Flux reconciling from git; a node can be destroyed and rebuilt from code. Prove: node-loss re-provision. Serves: R-PLAT-01, R-ENV-01, R-GITOPS-01, ADR-0001/0002/0004/0007.

Phase 2 — Platform Core: Network, Storage, TLS, DNS, Ingress

Entry: Phase 1 done. Do: Longhorn (3-replica) + bulk-tier plan; default-deny network baseline + Hubble; Cilium Egress Gateway; cert-manager + Let's Encrypt (DNS-01, wildcard); ExternalDNS; Gateway API listeners; split-horizon DNS foundation. Exit (demonstrate):

  • Default-deny holds: a probe pod can reach only DNS (R-NET-01).
  • A PVC survives pod destroy/rebuild (R-STATE-01).
  • A test host serves a trusted, non-self-signed cert at *.home.keithjackson.dev (R-TLS-01/03).
  • Auto DNS record appears for a test route (R-DNS-02). Serves: R-NET-01/02/03, R-STATE-01/02, R-TLS-01/02/03, R-DNS-01/02/03, ADR-0002/0003/0006.

Phase 3 — Egress & VPN

Entry: Phase 2 done; OQ-1/OQ-3 answered enough to build. Do: implement egress classes; node-level VPN egress steered by Cilium Egress Gateway with a fail-closed kill-switch (ADR-0015, ADR-0020). Exit (demonstrate):

  • vpn test workload egresses via VPN IP; internet workload via home/WAN IP (R-NET-05).
  • Kill-switch: tunnel forced down ⇒ vpn workload leaks nothing; internet unaffected (R-NET-06).
  • Flipping egress class in git re-routes without workload rewrite (R-NET-07). Serves: R-NET-04/05/06/07, ADR-0013/0015/0020.

Phase 4 — Observability, Backup/DR & Automated Updates

Entry: Phase 2 done (can overlap Phase 3). Do: kube-prometheus-stack + Loki + Alloy + Alertmanager(email); platform-health alerts; Velero and CNPG Barman off-cluster backup to Cloudflare R2; Renovate for images and charts; per-service update policy plumbing. Exit (demonstrate):

  • Injected fault emails an alert; dashboards render (R-OBS-01/02/03/04).
  • DR drill: restore a namespace + volumes from off-cluster backup (R-STATE-03).
  • A test service auto-advances a stable version on schedule with no human/AI edit; a bad update auto-rolls-back and alerts (R-UPD-01/02/03). Serves: R-OBS-, R-STATE-03, R-UPD-, ADR-0003/0018/0019/0023.

Phase 5 — IaaS Proof (KubeVirt)

Entry: Phases 2–3 done. Do: KubeVirt + CDI; run one representative application VM via ServiceDefinition kind: vm. Exit (demonstrate): create/start/stop/destroy a VM via kubectl/virtctl; Longhorn-backed disk survives rebuild; default-deny + egress apply to the VM; guest auto-updates. Serves: R-PLAT-03/04, R-STATE-01, R-NET-, R-UPD-, ADR-0005.

Phase 6 — Service Catalog Onboarding + C2 UI

Entry: Phases 2–4 done. Do: onboard the catalog services in dependency order (see workstream WS-APPS); bring Gitea and Keycloak online early in this phase — Keycloak is the foundational IdP so services adopt SSO (native OIDC / forward-auth) as they onboard (../docs/15-identity-and-access.md); and cut the GitOps source + CI/CD over to Gitea (Gitea Actions running the Kyverno gate + Chainsaw conformance Job; PRs as the enforced merge gate — ../docs/05-gitops-and-delivery.md, ../CONTRIBUTING.md); stand up the C2 UI (Homepage root + Grafana + Headlamp + Flux status). Exit (demonstrate): each service fully wired from its ServiceDefinition (compute/storage/policy/ egress/TLS/DNS/monitoring/updates) with default-deny + correct egress; the arr↔qbittorrent↔jackett↔ seerr↔plex dependency graph works and nothing undeclared does; home.keithjackson.dev serves the C2 with live links + health. Serves: R-C2-01/02, all app-facing requirements, ../docs/13-service-catalog.md.

Phase 7 — Production Readiness

Status: open; opened 2026-08-31. Entry: Phases 1–6 delivered; operator declares prod hardware ready. Do: finalize bare-metal provisioning checklist; apply the prod overlay (no base edits); full DR drill on prod; decommission dev/test. Exit: prod 3× mini-PC cluster runs the same config via the prod overlay; DR proven; monitoring + alerts live. Serves: R-ENV-02/03, R-PLAT-01. Note: This phase was explicitly opened by the operator before any build on prod hardware (../AGENTS.md §6).


Dependency flow

P0 → P1 → P2 ─┬→ P3 ─┐
              ├→ P4 ──┼→ P6 → P7
              └───────┘   ↑
                    P5 ───┘

P3, P4, P5 can proceed in parallel once P2 lands; P6 needs P2–P4; P7 needs everything proven.