From 365e43cdea02c4c008c74a563c2181a0dbdb9fe4 Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Fri, 30 Oct 2026 01:23:36 -0500 Subject: [PATCH] fix(renovate): run Renovate as the job container; the action needs a Docker daemon jobs do not get renovatebot/github-action is a wrapper around `docker run`. On this runner a job has no Docker daemon by decision (decisions/2026-10-29-image-pipeline.md), so every run, including dispatch run 343, died before Renovate started: failed to connect to the docker API at unix:///var/run/docker.sock ... no such file or directory The pinned image ghcr.io/renovatebot/renovate:44.167.0 is now the job container, and a run step execs renovate-entrypoint.sh. actions/checkout is dropped, since Renovate clones the repo itself. An optional RENOVATE_GITHUB_COM_TOKEN secret feeds GITHUB_COM_TOKEN. Local dry run (--platform=local, uid 12021, the step body verbatim): 179 deps extracted, 42 updates, and all 4 own-image digest lookups against the Gitea registry succeed. Co-Authored-By: Claude Opus 5.5 --- .gitea/workflows/renovate.yaml | 69 +++++++++---- decisions/2026-10-30-renovate-first-run.md | 113 +++++++++++++++++++++ renovate.json5 | 7 +- 3 files changed, 167 insertions(+), 22 deletions(-) create mode 100644 decisions/2026-10-30-renovate-first-run.md diff --git a/.gitea/workflows/renovate.yaml b/.gitea/workflows/renovate.yaml index f635ebf..3e984e3 100644 --- a/.gitea/workflows/renovate.yaml +++ b/.gitea/workflows/renovate.yaml @@ -116,15 +116,37 @@ concurrency: jobs: renovate: runs-on: ubuntu-latest + # RENOVATE RUNS AS THE JOB CONTAINER, NOT THROUGH renovatebot/github-action. FIXED 2026-10-30. + # That action does exactly one thing: `docker run ghcr.io/renovatebot/renovate:`. On + # this runner a job has NO Docker daemon, by decision (decisions/2026-10-29-image-pipeline.md: + # act_runner sets container.docker_host to `-` when DOCKER_HOST is tcp://, and the DinD daemon + # is privileged, so a job that could reach it would be root on the node). So every run of the + # action died before Renovate started. Dispatch run 343 (2026-10-30), verbatim: + # [command]/usr/bin/docker run --env LOG_LEVEL ... --env RENOVATE_TOKEN=*** ... + # failed to connect to the docker API at unix:///var/run/docker.sock; ... no such file or directory + # ::error::Error: The process '/usr/bin/docker' failed with exit code 1 + # No auth, scope, config or network question was ever reached. Making the Renovate image the + # job container needs no daemon: act starts it through the runner's own DinD exactly as it + # starts the default ubuntu image, then `exec`s each step inside it. + # + # PINNED here (was the action's `renovate-version: 44.167.0`). renovate.json5 is validated + # against this exact tag — keep them in step. Renovate's github-actions manager reads + # `.gitea/workflows` and extracts `container.image`, so this pin is still self-maintained. + # + # The image runs as uid 12021 (`ubuntu`), and that is fine under act: the step command files + # ($GITHUB_STEP_SUMMARY etc.) are created mode 0666 (gitea/act v0.261.2 pkg/runner/step.go), + # HOME comes from the image env, and no step writes the workspace. There is deliberately NO + # actions/checkout: Renovate clones the repo itself from RENOVATE_ENDPOINT, and as uid 12021 + # checkout could not `git init` the root-owned workspace volume anyway. + container: + image: ghcr.io/renovatebot/renovate:44.167.0 # Renovate scans every manager across the whole repo; 40m is generous for this repo's size and # well under a runner's patience. Without a timeout a wedged run holds the `renovate` # concurrency group and silently blocks every subsequent scheduled run. timeout-minutes: 40 steps: - - name: Checkout - uses: actions/checkout@v4 - - name: Preflight - the token must exist, and its absence must be LOUD + shell: bash env: RENOVATE_TOKEN: ${{ secrets.RENOVATE_TOKEN }} run: | @@ -147,25 +169,17 @@ jobs: exit 1 fi echo "RENOVATE_TOKEN is present (value not printed)." + # Proves the step is running INSIDE the pinned Renovate image, not the default job image. + echo "renovate $(renovate --version)" - name: Run Renovate - uses: renovatebot/github-action@v41.0.9 - with: - # Pinned, and Renovate maintains this pin itself: renovate.json's enabledManagers - # includes `github-actions`, and `helpers:pinGitHubActionDigests` is in its `extends`. - # This is the one dependency in the repo whose updater is its own updater. - # MUST match the file's real name. #79 renamed it to renovate.json5 and left this line - # saying `renovate.json`; every scheduled run from 2026-10-18 then died in ~1s with - # ENOENT before Renovate started, and nothing in the gate noticed. ci/lint.sh now - # asserts that every `configurationFile:` here names a file that exists. - configurationFile: renovate.json5 - token: ${{ secrets.RENOVATE_TOKEN }} - # PINNED 2026-10-29. Unset, this action version runs Renovate `39` (action.yml - # `renovate-version` default at v41.0.9), i.e. 39.264.0, and renovate.json5 uses - # `managerFilePatterns` (>= 40.2.0): under 39.x validation fails with five errors and - # the run stops at CONFIG_VALIDATION. #79 validated against `latest`, not this. - renovate-version: 44.167.0 + shell: bash + # The repo config is discovered by Renovate from the cloned repo (renovate.json5, + # RENOVATE_REQUIRE_CONFIG=required). The action used to ALSO load it as the global config + # via RENOVATE_CONFIG_FILE; that double-loading is gone. ci/lint.sh's `configurationFile:` + # check therefore has nothing to resolve in this file, and passes. env: + RENOVATE_TOKEN: ${{ secrets.RENOVATE_TOKEN }} # Self-hosted Gitea. `platform` + `endpoint` were the two values renovate.json's own # header said were "set at runtime by the operator's renovate runner" — this is that # runner, so they are set here rather than in the repo config, which keeps renovate.json @@ -189,6 +203,23 @@ jobs: # token; it therefore needs scope `read:package` in addition to the ones listed in the # preflight above. Gitea ignores the username for token auth (services/auth/basic.go). RENOVATE_HOST_RULES: '[{"hostType":"docker","matchHost":"gitea.home.keithjackson.dev","username":"renovate","password":"${{ secrets.RENOVATE_TOKEN }}"}]' + # OPTIONAL, ADDED 2026-10-30. A github.com token (a fine-grained PAT with NO permissions, + # public-repo read only) for lookups of GitHub-hosted dependencies. Without it Renovate + # SKIPS them with `github-token-required` — measured in a local dry run: 17 deps, i.e. + # every `uses: actions/*` here plus kubevirt/kubevirt, kubevirt/containerized-data-importer + # and intel/intel-device-plugins-for-kubernetes. Unset is safe: the step below drops the + # empty variable, and those deps simply stay unmaintained, with a WARN in the log. + GITHUB_COM_TOKEN: ${{ secrets.RENOVATE_GITHUB_COM_TOKEN }} + run: | + set -euo pipefail + # An expression that evaluates to '' still EXPORTS the variable, as an empty string. + # Drop those so an unset value means "Renovate's default", never "set to empty". + for v in RENOVATE_DRY_RUN GITHUB_COM_TOKEN; do + [ -n "${!v:-}" ] || unset "$v" + done + # Through the image's entrypoint, as `docker run` would have gone: it sources the + # containerbase env and initialises the bundled tools before exec'ing Renovate. + exec renovate-entrypoint.sh renovate - name: Summary if: always() diff --git a/decisions/2026-10-30-renovate-first-run.md b/decisions/2026-10-30-renovate-first-run.md new file mode 100644 index 0000000..347e37d --- /dev/null +++ b/decisions/2026-10-30-renovate-first-run.md @@ -0,0 +1,113 @@ +--- +date: 2026-10-30 +branch: fix/renovate-first-run +title: Renovate never started because its action needs a Docker daemon jobs here do not get; it now runs as the job container +--- + +## What changed + +**`.gitea/workflows/renovate.yaml`**: the job no longer uses `renovatebot/github-action`. The +pinned Renovate image (`ghcr.io/renovatebot/renovate:44.167.0`, the same version the action was +pinned to) is now the job's `container:`. A `run:` step calls `renovate-entrypoint.sh renovate` +inside it. `actions/checkout` is gone, because Renovate clones the repo itself. +`GITHUB_COM_TOKEN` is wired from a new **optional** secret, `RENOVATE_GITHUB_COM_TOKEN`. The step +unsets it, and `RENOVATE_DRY_RUN`, when they evaluate to the empty string. The preflight also +prints `renovate --version`, which proves the step ran inside the pinned image. + +**`renovate.json5`**: one comment now names `container.image` as the version pin, in place of +`renovate-version`. The config itself is unchanged. + +### Why: the root cause + +Every run of the action died before Renovate started. That includes the operator's dispatch, +run 343 (task 555, 2026-10-30 06:14 UTC), and the 11 scheduled runs before it (312–322). From +run 343's log: + + RENOVATE_TOKEN is present (value not printed). + [command]/usr/bin/docker run --env LOG_LEVEL ... --env RENOVATE_TOKEN=*** --env RENOVATE_CONFIG_FILE=/github-action/renovate.json5 ... + failed to connect to the docker API at unix:///var/run/docker.sock; check if the path is correct and if the daemon is running: dial unix /var/run/docker.sock: connect: no such file or directory + Error: The process '/usr/bin/docker' failed with exit code 1 + +The action is only a wrapper around `docker run`. On this runner a job deliberately gets no +Docker daemon (`decisions/2026-10-29-image-pipeline.md`): act_runner maps `docker_host` to `-` when +`DOCKER_HOST` is `tcp://`, and the DinD daemon is privileged. So this was a +platform-compatibility failure. The run never reached authentication, token scope, config or the +network. **The re-minted token is not implicated, and nothing in this log can say whether its +scopes are right.** + +A job `container:` needs no daemon. act creates it through the runner's DinD the same way it +creates the default `ubuntu-latest` image, then `exec`s each step inside it. I checked this +against gitea/act v0.261.2: + +- The job container's entrypoint is `/bin/sleep` (`run_context.go`). The Renovate image has + `/usr/bin/sleep`. +- The step command files are written `0o666` (`step.go`), so the image's uid 12021 can write + `$GITHUB_STEP_SUMMARY`. +- `HOME` comes from the image env. + +## What I verified + +- **The root cause.** I read all of run 343's job log (162 lines, job 0; job 1 is 404). It has one + error, quoted above, and no Renovate output at all. Runs 312–322 are all `failure` from the + `schedule` event. +- **The config validates against the pinned tag.** `podman run ... ghcr.io/renovatebot/renovate:44.167.0 + renovate-config-validator /tmp/renovate.json5` printed `Config validated successfully`. It also + printed the known migration diff for the top-level `docker` block. +- **A dry run with the fix extracts and looks up dependencies.** This was a local reproduction. I + made a clean snapshot of this branch's tracked files. I ran it in a podman container of the pinned + image with entrypoint `/bin/sleep`, as the image's own uid 12021 with `HOME=/home/ubuntu`. + Inside it I `exec`ed the workflow step's `run:` body, extracted verbatim from the YAML, with: + - `RENOVATE_PLATFORM=local`, `RENOVATE_DRY_RUN=full` and `GITHUB_COM_TOKEN=` (empty, to + exercise the unset); + - a docker host rule for `gitea.home.keithjackson.dev` using the `gitea-admin` credential from + `~/.git-credentials`, passed through env and never printed. + + The run exited 0 and logged `Dependency extraction complete`: + - flux 15 deps / 13 files + - github-actions 24 / 4 + - kubernetes 131 / 67 + - kustomize 2 / 2 + - regex (both custom managers) 7 / 6 + - **total 179 deps / 92 files** + + It found **42 updates**: 29 kubernetes, 6 flux, 2 kustomize, 1 github-actions and 4 regex + `digest`. 7 of them are `major`. +- **The `# renovate-digest:` lookups against the Gitea registry succeed.** All three + `/v2/token?scope=repository:home-network/:pull` requests succeeded, and so did + `HEAD /v2/home-network//manifests/main`. They proposed `digest` updates for + `conformance/versions.env`, `infra/agent/versions.env`, `platform/pm-agent/configmap.yaml` and + `platform/pm-agent/deployment.yaml`, all of which carry the zero placeholder digest today. The + `kubernetes` manager's copies of those images report `disabled`, which is the intended effect of + the packageRule. +- **The new pin is still maintained by Renovate.** The github-actions manager extracts + `ghcr.io/renovatebot/renovate:44.167.0` from `renovate.yaml` as depType `container`, and proposes + 44.198.0. +- **The `regex` vs `custom.regex` question from #73/#77 is closed.** Both custom managers produced + dependencies. +- **Missing GitHub token.** The local run logged `GitHub token is required for some dependencies`. + That accounts for 17 deps skipped with `github-token-required`, which is why + `RENOVATE_GITHUB_COM_TOKEN` is offered as an optional secret. + +## What I did NOT verify + +- **The workflow on the real runner. UNVERIFIED.** My reproduction ran with podman, not + act_runner + DinD, and used `--platform=local`. The act behaviour (sleep entrypoint, `exec` as the + image user, 0666 command files) comes from reading the gitea/act v0.261.2 source, not from a run. + The first real evidence will be a `workflow_dispatch` after merge. +- **The Gitea platform path and `RENOVATE_TOKEN`'s scopes. UNVERIFIED.** I have no access token, only + the `gitea-admin` password, which Renovate's Gitea platform cannot use. So these were not + exercised: platform init (`GET /user`, which needs `read:user`), repo clone, and the registry + lookups using the **token** with `read:package` rather than a password. +- **Behaviour inside the schedule window.** A dispatch outside 03:00–06:00 America/New_York + (07:00–10:00 UTC during EDT) will extract and look up, but will log "not within schedule" + instead of `DRY-RUN: Would create ...` lines. I did not change this. +- **Renovate as uid 12021 doing artifact updates** (for example helm lockfiles) on a real, + non-dry run. `--dry-run=full` does not run the tools. +- **The self-bump.** Renovate will propose `ghcr.io/renovatebot/renovate` 44.167.0 → 44.198.0 as a + minor, which the default rule auto-merges. `.gitea/**` is a sensitive path, so it will in fact + wait for an administrator override. Nothing re-runs `renovate-config-validator` on that bump, + and no gate check runs the validator. +- **No alert for "Renovate red every day".** That gap is still open. This failure ran for 12 days + without anyone noticing. +- **Sensitive path.** `.gitea/**` is in `infra/gitea/approval-policy.json`, so CI's + `approval-policy` stage will block until the operator posts `/approve-override` as a PR review. diff --git a/renovate.json5 b/renovate.json5 index 48e5ee4..898c75e 100644 --- a/renovate.json5 +++ b/renovate.json5 @@ -29,10 +29,11 @@ // `managerFilePatterns` key ("Custom Manager contains disallowed fields", "Invalid configuration // option: flux.managerFilePatterns", ...), because 39.x only knows `fileMatch`; the rename landed // in 40.2.0. So even after the workflow found this file, every run would have stopped at -// CONFIG_VALIDATION. renovate.yaml now pins `renovate-version` to the exact release this file is -// validated against. VALIDATE WITH THAT SAME TAG: +// CONFIG_VALIDATION. renovate.yaml now pins the exact release this file is validated against — +// since 2026-10-30 as the job's `container.image` tag (the action is gone: it needed a Docker +// daemon jobs here do not get). VALIDATE WITH THAT SAME TAG: // podman run --rm -v "$PWD/renovate.json5:/tmp/renovate.json5:ro,Z" \ -// ghcr.io/renovatebot/renovate: \ +// ghcr.io/renovatebot/renovate: \ // renovate-config-validator /tmp/renovate.json5 // (Not `--strict`: that also fails on a pending migration of the pre-existing top-level // `"docker": {"enabled": true}` block at the bottom, which is a warning, not an error.) -- 2.54.0