jakman85andClaude Opus 4.8
e8c555cfeb
Greenfield spec set for a GitOps-driven PaaS+IaaS home platform: vision & requirements (R-* IDs), architecture, 14 domain specs, 11 ADRs, roadmap (phases + workstreams), and AI-agent coordination (roles, protocol, definition-of-done, task board). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
77 lines
4.2 KiB
Markdown
77 lines
4.2 KiB
Markdown
# 11 — Automated Updates
|
|
|
|
Serves: **R-UPD-01/02/03**. Related: [ADR-0009](../decisions/adr-0009-automated-image-updates.md).
|
|
|
|
## The requirement, precisely
|
|
|
|
> Services must update to latest **stable** versions **on a schedule**, with **no human, no AI, and
|
|
> no hand-authored GitOps change**. (R-UPD-01/02)
|
|
|
|
This has a subtle tension with GitOps (where "the desired state is what's in git"). We resolve it
|
|
explicitly below.
|
|
|
|
## Resolving "no GitOps change" vs. GitOps
|
|
|
|
There are two interpretations; we honor the operator's intent (hands-off), and ADR-0009 records the
|
|
mechanism:
|
|
|
|
- **A human/AI authoring a version bump = prohibited.** No person and no chatbot edits a tag.
|
|
- **An automation controller advancing a version under a policy declared once = permitted**, because
|
|
the *operator's* declared intent (the update policy / semver range) is the stable, human-authored
|
|
thing; the controller merely resolves it. Two mechanisms, chosen by artifact type:
|
|
|
|
### 1. Container image tags → Flux image-update-automation
|
|
- Per service, an **ImagePolicy** declares the acceptable range/channel (e.g. semver `>=1.0.0`
|
|
minor/patch, or a `stable` tag pattern). Flux's image-reflector scans the registry; image-
|
|
automation advances the running version automatically **on its scan interval** (the schedule).
|
|
- Where automation writes the resolved version back to git, that commit is machine-authored under
|
|
the declared policy — no human/AI touches it (satisfies R-UPD-02 in spirit and practice).
|
|
- Alternatively, digest-tracking of a mutable stable tag avoids per-update commits entirely. ADR-0009
|
|
picks the default and the per-service escape hatch.
|
|
|
|
### 2. Helm charts / manifests → Renovate
|
|
- **Renovate** watches chart and dependency versions and opens PRs; allowed update classes
|
|
**auto-merge** on green CI. This keeps platform components current with no human authoring.
|
|
- Chart *major* bumps (potentially breaking) can require review — configurable per policy.
|
|
|
|
### 3. VM guests → in-guest automation
|
|
- KubeVirt guests auto-update OS packages inside the guest on a schedule (unattended-upgrades / dnf-
|
|
automatic) declared in the VM's cloud-init ([06](06-vm-workloads-kubevirt.md)). Base-image
|
|
promotion follows the same automated philosophy where a golden-image pipeline exists.
|
|
|
|
## Per-service policy (R-UPD-03)
|
|
|
|
Each ServiceDefinition ([13](13-service-catalog.md)) declares an **update policy**:
|
|
- **channel/range**: which versions are "stable" and acceptable (semver range, tag regex).
|
|
- **cadence**: how often to check/apply (the schedule).
|
|
- **hold/pin**: temporarily freeze a service (e.g. a known-bad upstream release).
|
|
- **rollout & health**: how the new version rolls out and what health gates it.
|
|
- **auto-rollback**: revert on failed health so a bad upstream release self-heals.
|
|
|
|
## Safety rails (so hands-off ≠ reckless)
|
|
|
|
- **Stable only**: never track pre-release/edge tags unless explicitly opted in per service.
|
|
- **Health-gated rollout**: a failing update is caught by health checks and rolled back (R-UPD-03),
|
|
and raises an alert ([10](10-observability-and-alerting.md)).
|
|
- **Backups precede risk**: state is on Longhorn with backups, so a bad update never risks data
|
|
([04](04-storage-and-state.md)).
|
|
- **Observability**: update events (image advanced, chart merged, rollback) are visible and, on
|
|
failure, alert by email.
|
|
- **Pin escape hatch**: an operator can pin a service via git if an upstream release misbehaves —
|
|
the one sanctioned human touch, and it *stops* automation rather than performing an update.
|
|
|
|
## Interaction with default-deny & egress
|
|
Updates change image versions, not network posture. A VPN-class service stays VPN-class across
|
|
updates; policies are reconciled from the (unchanged) ServiceDefinition.
|
|
|
|
## Acceptance gates
|
|
- A service advances to a newer stable version automatically, on schedule, with no human/AI edit.
|
|
- A deliberately bad update is caught by health checks, auto-rolled-back, and alerts by email.
|
|
- A pinned service does **not** advance.
|
|
- Platform components (charts) stay current via auto-merged Renovate PRs.
|
|
|
|
## Open questions
|
|
- Exact default in ADR-0009: write-back commits vs. digest-tracking mutable tags (trade readable
|
|
history vs. zero-commit updates).
|
|
- Registry rate-limit / credential handling for image scanning at cadence.
|