---
date: 2026-11-07
branch: feat/devcontainer-velero-cli
title: Bake the velero CLI into the dev toolbox, pinned to the prod server version
---

## What changed

- `infra/devcontainer/versions.env`: new `VELERO_VERSION=1.18.2`, with a comment explaining that
  the CLI and server are version-coupled, so this pin follows the server and is not bumped on
  its own. It has **no** `# renovate:` marker on purpose. When a `velero` chart bump in
  `platform/velero/helmrelease.yaml` changes the appVersion, bump this pin in the same PR.
- `infra/devcontainer/Containerfile`: `ARG VELERO_VERSION`; the binary comes from the
  vmware-tanzu GitHub release tarball and is installed in the same RUN as flux/helm. There is
  no checksum check, which matches the other tools in that file (none of them verify one).
  `velero=` is recorded in `/etc/dev-toolchain-versions`. The verify stage now checks that
  `velero` is on PATH, and `check velero v${VELERO_VERSION}` asserts the output of
  `velero version --client-only`.
- `infra/devcontainer/build-image.sh`: `VELERO_VERSION` added to `BUILD_ARGS`. CI reads that
  list generically through `--print-build-args` (`.gitea/workflows/scripts/images.sh`), so
  **no sensitive path (`.gitea/**`, `ci/**`) was touched**.
- `infra/devcontainer/README.md`: added a Backups row to the tool table.
- `runbooks/backup-failure.md`: under "First commands", a read-only
  `velero backup describe --details` / `backup logs` example run through dev.sh.

## What I verified

- Live server version, read-only: `kubectl --context k3s-prod -n velero get deploy velero`
  shows image `docker.io/velero/velero:v1.18.2`, and HelmRelease `velero` has
  lastAttemptedRevision `12.2.3`.
- Local build: `IMAGE=localhost/home-k3s-dev:velero-wt ./build-image.sh` returned rc 0. The
  verify stage printed `ok velero` (on PATH), `ok velero v1.18.2` (pin check) and
  `TOOLBOX VERIFIED`. The image was built under a separate local tag and **not pushed**.
- Smoke test with that image against prod (read-only): `velero version` reports Client v1.18.2
  and Server v1.18.2. `velero backup get` lists the daily backups, all `Completed` with 0 errors.
- Recurring warning (reported only, not fixed): every daily backup since
  `velero-daily-20261031030050` has 1 warning. Backups up to 20261030 have 0. The warning text is:
  `gitea-actions: resource: /pods name: /act-runner-5944dfdfb8-69bvq message: /Skip pod volume
  data error: /pod is not in the expected status, ... phase=Failed: pod is not running`.
  It is the same pod in 20261031, 20261104 and 20261108. `gitea-actions` holds 11 `Failed`
  pods with reason `Evicted` from ReplicaSet `act-runner-5944dfdfb8`, ten of them started
  2026-10-30T08:26Z. The running runner is `act-runner-fc597f6b5-vsvkl`. So the warning comes from a
  leftover evicted (`Failed`) pod that Velero still tries to back up volumes for. Cleaning up the evicted pods is an
  operator action.
- Gate: see the PR description for the toolbox `ci/gate.sh` rc and the skipped stages.

## What I did NOT verify

- The CI image build (`.gitea/workflows/images.yaml`) on this branch. It runs on push/PR and I
  have not seen its result while writing this.
- An arm64 build. Only amd64 was built (`TARGETARCH` defaults to amd64).
- Any mutating velero command (`backup create`, `restore`, `delete`). None were run.
- Why only `-69bvq` of the 11 evicted act-runner pods shows up in the warning. A likely guess is
  that only that pod's volumes match the backup's pod-volume selection, but I did not check.