jakman85andClaude Opus 5.5
250fb54154
The operator ruled the implementation repo's main (50c8719) the latest and accepted state. Sync the spec to it. Accepted changes are reflected: one bare-metal environment, TrueNAS bulk storage, R2 off-cluster backup, CloudNativePG, single-node VPN gateway with Cilium fail-closed, inbound WireGuard, per-app auth sidecars, Renovate, the CI gate as approver, and the pm-agent. New ADR-0017..0025; ADR-0016 corrected and accepted; R-* IDs unchanged, eight amended. Where the implementation has simply not met a requirement, the requirement stands and the shortfall is recorded in docs/16-known-gaps.md. Co-Authored-By: Claude Opus 5.5 <[email protected]>
118 lines
6.8 KiB
Markdown
118 lines
6.8 KiB
Markdown
# Roadmap — Delivery Phases
|
||
|
||
Delivery is phased. Each phase has an **entry gate** (what must be true to start), **exit criteria**
|
||
(demonstrable outcomes tied to requirement IDs), and the **workstreams** ([workstreams.md](workstreams.md))
|
||
that execute it. **Status: Phases 0–6 are delivered and Phase 7 is open** (see
|
||
[known gaps](../docs/16-known-gaps.md) for what remains unproven). Phases 1–6 were first built on a
|
||
dev/test cluster, decommissioned in 2026-09; prod is the only environment (ADR-0017).
|
||
|
||
A phase's exit criteria are **passing automated conformance tests** (per [`../AGENTS.md` §7](../AGENTS.md)
|
||
and [`../docs/14-testing-and-conformance.md`](../docs/14-testing-and-conformance.md)), not "manifests
|
||
merged" and not manual demos. Every "**Prove:**" / "**demonstrate**" item below is an automated,
|
||
reproducible, requirement-tagged behavior test committed with the work. A phase exits only when its
|
||
full conformance subset is green. **The conformance harness (WS-TEST) is stood up in Phase 0, so
|
||
every subsequent phase ships with tests, not after them.**
|
||
|
||
---
|
||
|
||
## Phase 0 — Foundations & Repo Bootstrap
|
||
**Entry:** specs (this repo) approved.
|
||
**Do:**
|
||
- Create the implementation repo structure ([`../docs/05-gitops-and-delivery.md`](../docs/05-gitops-and-delivery.md)).
|
||
- Stand up CI (lint, schema validation, dry-run/build) for manifests and ServiceDefinitions.
|
||
- Author the ServiceDefinition schema + renderer contract ([`../docs/13-service-catalog.md`](../docs/13-service-catalog.md)).
|
||
- **Stand up the conformance harness (WS-TEST):** single-command runner, reproducibility contract,
|
||
requirement-tagged JUnit + coverage report, L0 policy-as-code gate, and the CI merge gate
|
||
([`../docs/14-testing-and-conformance.md`](../docs/14-testing-and-conformance.md), [ADR-0010](../decisions/adr-0010-conformance-testing.md)).
|
||
**Exit:** implementation repo + CI exist; ServiceDefinition schema validated; **conformance harness
|
||
runs green with the coverage report wired**; no cluster yet.
|
||
**Serves:** R-GITOPS-02, R-ENV-02, R-TEST-01..04.
|
||
|
||
## Phase 1 — Cluster Bring-up (IaC)
|
||
**Entry:** Phase 0 done.
|
||
**Do:** node baseline + Ansible k3s HA install with **Cilium**; disable bundled
|
||
flannel/kube-proxy/Traefik; bootstrap **Flux**; bootstrap **SOPS age key**.
|
||
**Exit:** 3-node k3s HA cluster with Cilium up; Flux reconciling from git; a node can be
|
||
destroyed and rebuilt from code. **Prove:** node-loss re-provision.
|
||
**Serves:** R-PLAT-01, R-ENV-01, R-GITOPS-01, ADR-0001/0002/0004/0007.
|
||
|
||
## Phase 2 — Platform Core: Network, Storage, TLS, DNS, Ingress
|
||
**Entry:** Phase 1 done.
|
||
**Do:** Longhorn (3-replica) + bulk-tier plan; default-deny network baseline + Hubble; Cilium
|
||
Egress Gateway; cert-manager + Let's Encrypt (DNS-01, wildcard); ExternalDNS; Gateway API listeners;
|
||
split-horizon DNS foundation.
|
||
**Exit (demonstrate):**
|
||
- Default-deny holds: a probe pod can reach only DNS (R-NET-01).
|
||
- A PVC survives pod destroy/rebuild (R-STATE-01).
|
||
- A test host serves a **trusted, non-self-signed** cert at `*.home.keithjackson.dev` (R-TLS-01/03).
|
||
- Auto DNS record appears for a test route (R-DNS-02).
|
||
**Serves:** R-NET-01/02/03, R-STATE-01/02, R-TLS-01/02/03, R-DNS-01/02/03, ADR-0002/0003/0006.
|
||
|
||
## Phase 3 — Egress & VPN
|
||
**Entry:** Phase 2 done; OQ-1/OQ-3 answered enough to build.
|
||
**Do:** implement egress classes; node-level VPN egress steered by Cilium Egress Gateway with a
|
||
fail-closed **kill-switch** (ADR-0015, ADR-0020).
|
||
**Exit (demonstrate):**
|
||
- `vpn` test workload egresses via VPN IP; `internet` workload via home/WAN IP (R-NET-05).
|
||
- **Kill-switch:** tunnel forced down ⇒ `vpn` workload leaks nothing; `internet` unaffected (R-NET-06).
|
||
- Flipping `egress` class in git re-routes without workload rewrite (R-NET-07).
|
||
**Serves:** R-NET-04/05/06/07, ADR-0013/0015/0020.
|
||
|
||
## Phase 4 — Observability, Backup/DR & Automated Updates
|
||
**Entry:** Phase 2 done (can overlap Phase 3).
|
||
**Do:** kube-prometheus-stack + Loki + Alloy + Alertmanager(email); platform-health alerts; Velero
|
||
and CNPG Barman off-cluster backup to Cloudflare R2; Renovate for images and charts; per-service
|
||
update policy plumbing.
|
||
**Exit (demonstrate):**
|
||
- Injected fault emails an alert; dashboards render (R-OBS-01/02/03/04).
|
||
- DR drill: restore a namespace + volumes from off-cluster backup (R-STATE-03).
|
||
- A test service auto-advances a stable version on schedule with no human/AI edit; a bad update
|
||
auto-rolls-back and alerts (R-UPD-01/02/03).
|
||
**Serves:** R-OBS-*, R-STATE-03, R-UPD-*, ADR-0003/0018/0019/0023.
|
||
|
||
## Phase 5 — IaaS Proof (KubeVirt)
|
||
**Entry:** Phases 2–3 done.
|
||
**Do:** KubeVirt + CDI; run one representative application VM via ServiceDefinition `kind: vm`.
|
||
**Exit (demonstrate):** create/start/stop/destroy a VM via `kubectl`/`virtctl`; Longhorn-backed disk
|
||
survives rebuild; default-deny + `egress` apply to the VM; guest auto-updates.
|
||
**Serves:** R-PLAT-03/04, R-STATE-01, R-NET-*, R-UPD-*, ADR-0005.
|
||
|
||
## Phase 6 — Service Catalog Onboarding + C2 UI
|
||
**Entry:** Phases 2–4 done.
|
||
**Do:** onboard the catalog services in dependency order (see workstream WS-APPS);
|
||
**bring Gitea and Keycloak online early in this phase** — Keycloak is the foundational IdP so
|
||
services adopt SSO (native OIDC / forward-auth) as they onboard ([`../docs/15-identity-and-access.md`](../docs/15-identity-and-access.md));
|
||
and cut the GitOps source + CI/CD over to Gitea (Gitea
|
||
Actions running the Kyverno gate + Chainsaw conformance Job; PRs as the enforced merge gate —
|
||
[`../docs/05-gitops-and-delivery.md`](../docs/05-gitops-and-delivery.md), [`../CONTRIBUTING.md`](../CONTRIBUTING.md));
|
||
stand up the C2 UI (Homepage root + Grafana + Headlamp + Flux status).
|
||
**Exit (demonstrate):** each service fully wired from its ServiceDefinition (compute/storage/policy/
|
||
egress/TLS/DNS/monitoring/updates) with default-deny + correct egress; the arr↔qbittorrent↔jackett↔
|
||
seerr↔plex dependency graph works and nothing undeclared does; `home.keithjackson.dev` serves the C2
|
||
with live links + health.
|
||
**Serves:** R-C2-01/02, all app-facing requirements, [`../docs/13-service-catalog.md`](../docs/13-service-catalog.md).
|
||
|
||
## Phase 7 — Production Readiness
|
||
**Status:** open; opened 2026-08-31.
|
||
**Entry:** Phases 1–6 delivered; operator declares prod hardware ready.
|
||
**Do:** finalize bare-metal provisioning checklist; apply the **prod overlay** (no base edits);
|
||
full DR drill on prod; decommission dev/test.
|
||
**Exit:** prod 3× mini-PC cluster runs the same config via the prod overlay; DR proven; monitoring +
|
||
alerts live.
|
||
**Serves:** R-ENV-02/03, R-PLAT-01.
|
||
**Note:** This phase was explicitly opened by the operator before any build on prod hardware
|
||
([`../AGENTS.md` §6](../AGENTS.md)).
|
||
|
||
---
|
||
|
||
## Dependency flow
|
||
|
||
```
|
||
P0 → P1 → P2 ─┬→ P3 ─┐
|
||
├→ P4 ──┼→ P6 → P7
|
||
└───────┘ ↑
|
||
P5 ───┘
|
||
```
|
||
|
||
P3, P4, P5 can proceed in parallel once P2 lands; P6 needs P2–P4; P7 needs everything proven.
|