Files
home-network-k3s-impl/decisions/2026-11-07-devcontainer-velero-cli.md
jakman85andClaude Opus 5.5 60e3a24652
gate / gate (pull_request) Successful in 3m1s
gate / conformance (pull_request) Skipped
images / images (pull_request) Successful in 44s
feat(devcontainer): bake velero CLI into the toolbox, pinned to prod server v1.18.2
The velero CLI and server are version-coupled, so VELERO_VERSION follows the
server running on prod (docker.io/velero/velero:v1.18.2, chart 12.2.3,
verified 2026-11-07). It has no renovate marker on purpose: bump it together
with the chart's appVersion. The verify stage asserts that
`velero version --client-only` reports the pin. The build arg reaches CI
through build-image.sh --print-build-args, so no .gitea/ or ci/ change is
needed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 23:18:21 -06:00

55 lines
3.5 KiB
Markdown

---
date: 2026-11-07
branch: feat/devcontainer-velero-cli
title: Bake the velero CLI into the dev toolbox, pinned to the prod server version
---
## What changed
- `infra/devcontainer/versions.env`: new `VELERO_VERSION=1.18.2`, with a comment explaining that
the CLI and server are version-coupled, so this pin follows the server and is not bumped on
its own. It has **no** `# renovate:` marker on purpose. When a `velero` chart bump in
`platform/velero/helmrelease.yaml` changes the appVersion, bump this pin in the same PR.
- `infra/devcontainer/Containerfile`: `ARG VELERO_VERSION`; the binary comes from the
vmware-tanzu GitHub release tarball and is installed in the same RUN as flux/helm. There is
no checksum check, which matches the other tools in that file (none of them verify one).
`velero=` is recorded in `/etc/dev-toolchain-versions`. The verify stage now checks that
`velero` is on PATH, and `check velero v${VELERO_VERSION}` asserts the output of
`velero version --client-only`.
- `infra/devcontainer/build-image.sh`: `VELERO_VERSION` added to `BUILD_ARGS`. CI reads that
list generically through `--print-build-args` (`.gitea/workflows/scripts/images.sh`), so
**no sensitive path (`.gitea/**`, `ci/**`) was touched**.
- `infra/devcontainer/README.md`: added a Backups row to the tool table.
- `runbooks/backup-failure.md`: under "First commands", a read-only
`velero backup describe --details` / `backup logs` example run through dev.sh.
## What I verified
- Live server version, read-only: `kubectl --context k3s-prod -n velero get deploy velero`
shows image `docker.io/velero/velero:v1.18.2`, and HelmRelease `velero` has
lastAttemptedRevision `12.2.3`.
- Local build: `IMAGE=localhost/home-k3s-dev:velero-wt ./build-image.sh` returned rc 0. The
verify stage printed `ok velero` (on PATH), `ok velero v1.18.2` (pin check) and
`TOOLBOX VERIFIED`. The image was built under a separate local tag and **not pushed**.
- Smoke test with that image against prod (read-only): `velero version` reports Client v1.18.2
and Server v1.18.2. `velero backup get` lists the daily backups, all `Completed` with 0 errors.
- Recurring warning (reported only, not fixed): every daily backup since
`velero-daily-20261031030050` has 1 warning. Backups up to 20261030 have 0. The warning text is:
`gitea-actions: resource: /pods name: /act-runner-5944dfdfb8-69bvq message: /Skip pod volume
data error: /pod is not in the expected status, ... phase=Failed: pod is not running`.
It is the same pod in 20261031, 20261104 and 20261108. `gitea-actions` holds 11 `Failed`
pods with reason `Evicted` from ReplicaSet `act-runner-5944dfdfb8`, ten of them started
2026-10-30T08:26Z. The running runner is `act-runner-fc597f6b5-vsvkl`. So the warning comes from a
leftover evicted (`Failed`) pod that Velero still tries to back up volumes for. Cleaning up the evicted pods is an
operator action.
- Gate: see the PR description for the toolbox `ci/gate.sh` rc and the skipped stages.
## What I did NOT verify
- The CI image build (`.gitea/workflows/images.yaml`) on this branch. It runs on push/PR and I
have not seen its result while writing this.
- An arm64 build. Only amd64 was built (`TARGETARCH` defaults to amd64).
- Any mutating velero command (`backup create`, `restore`, `delete`). None were run.
- Why only `-69bvq` of the 11 evicted act-runner pods shows up in the warning. A likely guess is
that only that pod's volumes match the backup's pod-volume selection, but I did not check.