jakman85andClaude Opus 4.8
de26a53d4b
- Add CONTRIBUTING.md: interim local-git rules (no work on main; every change a feature branch merged into main) and the target state (Gitea-hosted repos, GitOps source cutover, CI/CD via Gitea Actions running the Kyverno gate + Chainsaw conformance Job, PR merge gate). Note the self-hosted-git bootstrapping circularity + off-cluster mirror/backup mitigation. - AGENTS.md: add section 12 (version-control workflow). - docs/05: source-of-truth migration (local bootstrap -> self-hosted Gitea). - docs/13 + roadmap: reprioritize Gitea as bootstrapping-critical, onboarded early (WS-APPS step 2); nextcloud onboarding split out. - task-board: T-6.1a (gitea early), T-6.1b (source+CI migration to Gitea). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
116 lines
6.1 KiB
Markdown
116 lines
6.1 KiB
Markdown
# 05 — GitOps & Delivery
|
|
|
|
Serves: **R-GITOPS-01..03**, **R-PLAT-02/04**, **R-ENV-02**. Related: [ADR-0004 (Flux)](../decisions/adr-0004-gitops-flux.md).
|
|
|
|
## Model
|
|
|
|
Pull-based GitOps with **Flux CD**. The cluster continuously reconciles its state from git. Git is
|
|
the single source of truth; the cluster conforms to git, not the reverse (R-GITOPS-01). Drift is
|
|
detected and corrected automatically (R-GITOPS-03).
|
|
|
|
## Repository layout (the implementation repo, created in a later phase)
|
|
|
|
> This spec repo (`home-network-k3s`) holds intent. The **implementation** repo(s) hold the
|
|
> reconciled manifests. Proposed structure:
|
|
|
|
```
|
|
platform/ ← cluster-wide platform services (Flux-managed)
|
|
cilium/ ← CNI, egress gateway, network baseline
|
|
longhorn/ ← storage
|
|
cert-manager/ ← TLS issuers + ClusterIssuer
|
|
externaldns/ ← DNS automation
|
|
gateway/ ← Gateway API + listeners/TLS
|
|
monitoring/ ← kube-prometheus-stack, Loki, Alloy, Alertmanager
|
|
velero/ ← backup/DR
|
|
kubevirt/ ← KubeVirt + CDI
|
|
image-automation/ ← Flux image update automation config
|
|
apps/ ← application workloads, one dir per service
|
|
plex/ sonarr/ radarr/ jackett/ seerr/ pihole/ nextcloud/
|
|
minecraft/ unifi/ semaphore/ kasm/ gitea/ qbittorrent/
|
|
catalog/ ← ServiceDefinition entries (the onboarding surface)
|
|
clusters/
|
|
dev/ ← Flux entrypoint + dev overlay/vars
|
|
prod/ ← Flux entrypoint + prod overlay/vars (design-only)
|
|
```
|
|
|
|
- **Kustomize overlays** (or equivalent) express environment differences (R-ENV-02). Base is shared;
|
|
`clusters/dev` and `clusters/prod` supply scoped values only.
|
|
- **HelmReleases** (Flux) wrap upstream charts (Cilium, Longhorn, cert-manager, kube-prometheus-
|
|
stack, etc.); app workloads use charts where good ones exist, otherwise plain manifests.
|
|
|
|
## The onboarding surface: ServiceDefinition (R-GITOPS-02)
|
|
|
|
Onboarding a service = adding one **ServiceDefinition** under `catalog/`. It is the operator-facing
|
|
abstraction that the platform compiles into the concrete resources (Deployment/VM, PVCs,
|
|
NetworkPolicies, HTTPRoute, Certificate, DNS record, ServiceMonitor, update policy). See
|
|
[`13-service-catalog.md`](13-service-catalog.md) for the schema. Intent lives in one place; the
|
|
platform renders the rest.
|
|
|
|
## Reconciliation flow
|
|
|
|
1. Operator opens a PR against the implementation repo (new/updated ServiceDefinition or platform
|
|
change).
|
|
2. CI validates (lint, policy checks, schema validation, dry-run/build).
|
|
3. Merge to the tracked branch.
|
|
4. Flux reconciles: applies manifests, reports status, corrects drift on an interval.
|
|
5. Health/rollout status surfaces in the C2 UI ([12](12-command-center-ui.md)) and alerts on failure
|
|
([10](10-observability-and-alerting.md)).
|
|
|
|
## Secrets in GitOps
|
|
|
|
Secrets are committed **encrypted** with **SOPS + age**; Flux decrypts them in-cluster at
|
|
reconciliation. No plaintext secret ever lands in git. See [`07-security-and-secrets.md`](07-security-and-secrets.md)
|
|
and [ADR-0007](../decisions/adr-0007-secrets-sops-age.md).
|
|
|
|
## Automated updates interplay (R-UPD-*)
|
|
|
|
GitOps and hands-off updates coexist:
|
|
- **Container image tags** advance automatically via **Flux image-update-automation** driven by
|
|
per-service **image policies** (semver range / stable channel) — the controller resolves and
|
|
reconciles the new version with **no human-authored change** (R-UPD-02).
|
|
- **Helm chart / manifest version bumps** are proposed by **Renovate** with auto-merge for allowed
|
|
update classes.
|
|
|
|
Full detail and the "is an automated commit a GitOps change?" nuance is in
|
|
[`11-automated-updates.md`](11-automated-updates.md) and [ADR-0009](../decisions/adr-0009-automated-image-updates.md).
|
|
|
|
## Bootstrap & recovery
|
|
|
|
- Flux is bootstrapped onto a fresh cluster by the Ansible handoff ([02](02-environments-and-provisioning.md)).
|
|
- Because desired state is in git and data is on Longhorn/backups, a rebuilt cluster reconverges by
|
|
re-bootstrapping Flux and restoring volumes ([04](04-storage-and-state.md)).
|
|
|
|
## Source-of-truth: local bootstrap → self-hosted Gitea
|
|
|
|
The platform hosts its own git, which creates a deliberate bootstrapping sequence
|
|
([`../CONTRIBUTING.md`](../CONTRIBUTING.md)):
|
|
|
|
1. **Bootstrap source.** Initially there is no remote git on this network (the prior `home-network`
|
|
Gitea is gone from here). Development is local; the cluster is first reconciled from a
|
|
**bootstrap source** (this local repo / an out-of-band source).
|
|
2. **Gitea comes online early.** **Gitea is one of the first services onboarded** ([13](13-service-catalog.md))
|
|
so the platform gains its own git host at `gitea.home.keithjackson.dev`.
|
|
3. **Cut Flux over to Gitea.** Once Gitea is stable, Flux's `GitRepository` source is migrated from
|
|
the bootstrap source to Gitea; the platform then reconciles from its own git.
|
|
4. **CI/CD moves to Gitea Actions.** The Kyverno L0 policy gate and the trigger for the in-cluster
|
|
Chainsaw conformance Job ([14](14-testing-and-conformance.md)) run as Gitea Actions pipelines on
|
|
push/PR; PRs become the enforced merge gate into `main`.
|
|
|
|
**Circularity & DR.** Gitea runs *on* the cluster Flux reconciles *from* Gitea. Mitigations:
|
|
- Keep an **off-cluster mirror** of the platform repo so a source-of-truth exists that outlives the
|
|
cluster.
|
|
- Treat **Gitea as DR-critical**: its repos/DB are backed up off-cluster ([04](04-storage-and-state.md)),
|
|
so a cluster loss never loses the git history needed to rebuild the platform.
|
|
|
|
## Guardrails
|
|
- No `kubectl apply` of app state by hand — everything flows through git/Flux (R-GITOPS-01, click-
|
|
ops-never per [AGENTS.md](../AGENTS.md)).
|
|
- Reconciliation errors and drift are **alerting conditions** (R-OBS-04).
|
|
- Base config is never forked per environment (R-ENV-02).
|
|
|
|
## Acceptance gates
|
|
- Adding a ServiceDefinition PR results in a fully wired service (compute, storage, policy, TLS, DNS,
|
|
monitoring) with no manual steps beyond the merge.
|
|
- Manually mutating a live resource is auto-reverted by Flux (drift correction proven).
|
|
- The same base config renders correctly for both dev and prod overlays.
|