feat(gitea-actions): act_runner capacity 2, dind sized for two concurrent jobs #146

2026-11-08T06:13:06Z

gitea-admin commented

2026-11-08 06:13:06 +00:00

What

The in-cluster Gitea act_runner now runs capacity 2, and its dind sidecar is sized for two concurrent jobs.

Until now it ran at the default capacity 1 with no config file. Gate, images and renovate runs on different refs therefore queued behind each other: waits were ~1s normally, but 29–759s when runs collided (run 472 waited 759s behind a renovate burst).

Only one file changes: apps/wave-1/gitea-actions.yaml. That file is a hand-owned EXTRA_UNITS unit in catalog/render.py, so the renderer only decides its wave. It is the source, not generated output, so no re-render was needed. render-golden passes, with 10 goldens reproduced.

Changes

ConfigMap act-runner-config containing only runner.capacity: 2.
- It is mounted at /etc/act_runner and selected with CONFIG_FILE. The v0.6.1 image's run.sh passes --config to both register and daemon.
- The v0.6.1 LoadDefault() gives a partial file the same defaults as no file, so nothing else changes.
- The generate-config example was deliberately not used:
  - its labels would override the registered labels in .runner;
  - its force_pull: true differs from the default;
  - its sample envs would be injected into every job.
Pod-template annotation act-runner-config-revision: "1". act_runner reads its config only at start, so a change to the ConfigMap alone needs a bump here to take effect.
dind resources:
- CPU request 100m -> 2, still with no CPU limit;
- memory 256Mi/2Gi -> 1Gi/4Gi.
runner resources: memory 128Mi/512Mi -> 256Mi/1Gi.

Measured (read-only, k3s-prod, 2026-11-08, capacity 1)

container	peak CPU (1m rate, 30d)	peak working set (30d)	OOMs (7d)
dind	0.69 core	447Mi	0
runner	0.24 core	314Mi (old limit 512Mi)	0

Node CPU requests, against 8 allocatable CPU on each node:

node	requested now	with this change
k3s-prod-2 (runner's node)	2945m	~4845m (61%)
k3s-prod-1	3175m	~5075m
k3s-prod-3	3840m	~5740m

The pod fits on any of them if it is rescheduled.

dind disk: 7.2G free of 11.8G on the node's ephemeral filesystem. The image cache is 2.18GB, and both jobs share it.

Concurrency semantics

Workflow concurrency: groups are enforced by Gitea on the server, not by the runner:
- gate-${{ github.ref }} with cancel-in-progress still cancels a superseded run on the same ref;
- renovate and images-<ref> still run one at a time.
The required check is still one gate job per PR head. gate.yml's conformance job has if: false.
The two jobs share one dockerd. act_runner gives each task its own GITEA-ACTIONS-TASK-<id> container prefix and its own network, and no workflow uses fixed container names or host ports.

Approval policy

apps/wave-1/** matches no sensitive_paths glob, so this PR is not a sensitive-path change.

Two comments in .gitea/workflows/ still say "capacity 1". They are left unchanged on purpose, to keep this PR out of the sensitive paths, and should be fixed in a follow-up.

Local gate

Run in the pinned toolbox from the worktree, on the committed tree, with spec repo and .git mounted: rc 3.

gitea-settings is the only stage that did not run (it needs SETTINGS_API_TOKEN, which exists only in CI).
spec-immutability passed.
render-golden, l0, l1 and approval-policy passed.
L2/L3 conformance skipped as declared (CLUSTER_AVAILABLE unset).

Not verified

The runner has not yet been observed running two jobs at once. That needs the merge and a Flux reconcile.
Gitea's runner API does not report capacity. act_runner's Declare sends only version and labels, so concurrency must be verified by observing overlapping jobs.
The rollout restarts the runner (Recreate, shutdown_timeout: 0s). Any job in flight at that moment is cancelled, possibly including the push-to-main gate run triggered by this merge.
The sizes have not been re-measured against the parallelised gate, which has not landed yet.

The operator's post-merge checklist is in decisions/2026-11-08-runner-capacity-2.md.

🤖 Generated with Claude Code

## What The in-cluster Gitea `act_runner` now runs **capacity 2**, and its dind sidecar is sized for two concurrent jobs. Until now it ran at the default capacity 1 with no config file. Gate, images and renovate runs on different refs therefore queued behind each other: waits were ~1s normally, but 29–759s when runs collided (run 472 waited 759s behind a renovate burst). Only one file changes: `apps/wave-1/gitea-actions.yaml`. That file is a **hand-owned** `EXTRA_UNITS` unit in `catalog/render.py`, so the renderer only decides its wave. It is the source, not generated output, so no re-render was needed. `render-golden` passes, with 10 goldens reproduced. ## Changes - **ConfigMap `act-runner-config`** containing only `runner.capacity: 2`. - It is mounted at `/etc/act_runner` and selected with `CONFIG_FILE`. The v0.6.1 image's `run.sh` passes `--config` to both `register` and `daemon`. - The v0.6.1 `LoadDefault()` gives a partial file the same defaults as no file, so nothing else changes. - The `generate-config` example was deliberately **not** used: - its `labels` would override the registered labels in `.runner`; - its `force_pull: true` differs from the default; - its sample `envs` would be injected into every job. - **Pod-template annotation `act-runner-config-revision: "1"`.** act_runner reads its config only at start, so a change to the ConfigMap alone needs a bump here to take effect. - **dind resources:** - CPU request `100m -> 2`, still with no CPU limit; - memory `256Mi/2Gi -> 1Gi/4Gi`. - **runner resources:** memory `128Mi/512Mi -> 256Mi/1Gi`. ## Measured (read-only, k3s-prod, 2026-11-08, capacity 1) | container | peak CPU (1m rate, 30d) | peak working set (30d) | OOMs (7d) | |---|---|---|---| | dind | 0.69 core | 447Mi | 0 | | runner | 0.24 core | 314Mi (old limit 512Mi) | 0 | **Node CPU requests**, against 8 allocatable CPU on each node: | node | requested now | with this change | |---|---|---| | k3s-prod-2 (runner's node) | 2945m | ~4845m (61%) | | k3s-prod-1 | 3175m | ~5075m | | k3s-prod-3 | 3840m | ~5740m | The pod fits on any of them if it is rescheduled. **dind disk:** 7.2G free of 11.8G on the node's ephemeral filesystem. The image cache is 2.18GB, and both jobs share it. ## Concurrency semantics - Workflow `concurrency:` groups are enforced by Gitea on the server, not by the runner: - `gate-${{ github.ref }}` with cancel-in-progress still cancels a superseded run on the same ref; - `renovate` and `images-<ref>` still run one at a time. - The required check is still one `gate` job per PR head. gate.yml's `conformance` job has `if: false`. - The two jobs share one dockerd. act_runner gives each task its own `GITEA-ACTIONS-TASK-<id>` container prefix and its own network, and no workflow uses fixed container names or host ports. ## Approval policy `apps/wave-1/**` matches no `sensitive_paths` glob, so this PR is **not** a sensitive-path change. Two comments in `.gitea/workflows/` still say "capacity 1". They are left unchanged on purpose, to keep this PR out of the sensitive paths, and should be fixed in a follow-up. ## Local gate Run in the pinned toolbox from the worktree, on the committed tree, with spec repo and `.git` mounted: **rc 3**. - `gitea-settings` is the only stage that did not run (it needs `SETTINGS_API_TOKEN`, which exists only in CI). - `spec-immutability` passed. - `render-golden`, `l0`, `l1` and `approval-policy` passed. - L2/L3 conformance skipped as declared (`CLUSTER_AVAILABLE` unset). ## Not verified - The runner has not yet been observed running two jobs at once. That needs the merge and a Flux reconcile. - Gitea's runner API does not report capacity. act_runner's `Declare` sends only version and labels, so concurrency must be verified by observing overlapping jobs. - **The rollout restarts the runner** (Recreate, `shutdown_timeout: 0s`). Any job in flight at that moment is cancelled, possibly including the push-to-main gate run triggered by this merge. - The sizes have not been re-measured against the parallelised gate, which has not landed yet. The operator's post-merge checklist is in `decisions/2026-11-08-runner-capacity-2.md`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

gitea-admin added 1 commit 2026-11-08 06:13:06 +00:00

feat(gitea-actions): act_runner capacity 2, dind sized for two concurrent jobs

gate / gate (pull_request) Successful in 2m59s

Details

gate / conformance (pull_request) Skipped

Details

b548224db8

The in-cluster runner ran at the default capacity 1 with no config file, so
gate / images / renovate runs on different refs queued behind each other
(29-759s waits; run 472 waited 759s behind a renovate burst).

- ConfigMap act-runner-config with ONLY runner.capacity: 2, selected via
  CONFIG_FILE (honoured by the v0.6.1 image's run.sh). v0.6.1 LoadDefault()
  applies the same defaults to a partial file as to no file; the
  generate-config example is deliberately not used (its labels would override
  the registered ones, force_pull differs from the default).
- Pod-template config-revision annotation so ConfigMap-only edits can roll it.
- dind: cpu request 100m -> 2 (no limit), memory 256Mi/2Gi -> 1Gi/4Gi.
  Measured at capacity 1: peak 0.69 core, 447Mi working set (30d), 0 OOMs.
- runner: memory 128Mi/512Mi -> 256Mi/1Gi (peak 314Mi at capacity 1).

apps/wave-1/gitea-actions.yaml is a hand-owned EXTRA_UNITS file (the
renderer only places it in a wave), so it is the source, not generated output.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

jakman85 merged commit 045d81c7a2 into main

2026-11-08 06:47:05 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Home-Network/home-network-k3s-impl#146