Compare commits

...
Author SHA1 Message Date
jakman85andClaude Opus 5.5 b548224db8 feat(gitea-actions): act_runner capacity 2, dind sized for two concurrent jobs
gate / gate (pull_request) Successful in 2m59s
gate / conformance (pull_request) Skipped
The in-cluster runner ran at the default capacity 1 with no config file, so
gate / images / renovate runs on different refs queued behind each other
(29-759s waits; run 472 waited 759s behind a renovate burst).

- ConfigMap act-runner-config with ONLY runner.capacity: 2, selected via
  CONFIG_FILE (honoured by the v0.6.1 image's run.sh). v0.6.1 LoadDefault()
  applies the same defaults to a partial file as to no file; the
  generate-config example is deliberately not used (its labels would override
  the registered ones, force_pull differs from the default).
- Pod-template config-revision annotation so ConfigMap-only edits can roll it.
- dind: cpu request 100m -> 2 (no limit), memory 256Mi/2Gi -> 1Gi/4Gi.
  Measured at capacity 1: peak 0.69 core, 447Mi working set (30d), 0 OOMs.
- runner: memory 128Mi/512Mi -> 256Mi/1Gi (peak 314Mi at capacity 1).

apps/wave-1/gitea-actions.yaml is a hand-owned EXTRA_UNITS file (the
renderer only places it in a wave), so it is the source, not generated output.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-08 00:09:08 -06:00
jakman85 e8d26669a4 Merge pull request 'chore(deps): Update dependency keycloak/keycloak-k8s-resources to v26.8.2' (#121) from renovate/keycloak-operator into main
gate / gate (push) Successful in 3m0s
gate / conformance (push) Skipped
Reviewed-on: #121
2026-11-08 05:53:29 +00:00
jakman85 048254bfaf Merge pull request 'feat(gitea): upgrade 1.27.3 -> 28.3.0' (#142) from feat/gitea-28 into main
gate / gate (push) Successful in 2m59s
gate / conformance (push) Skipped
Reviewed-on: #142
2026-11-08 05:41:34 +00:00
jakman85 dd94101647 Merge pull request 'feat(devcontainer): bake velero CLI into the toolbox, pinned to prod server v1.18.2' (#143) from feat/devcontainer-velero-cli into main
gate / gate (push) Successful in 3m0s
gate / conformance (push) Skipped
images / images (push) Successful in 38s
Reviewed-on: #143
2026-11-08 05:30:28 +00:00
jakman85andClaude Opus 5.5 8a9baa92bc feat(gitea): upgrade 1.27.3 -> 28.3.0
gate / gate (pull_request) Successful in 3m13s
gate / conformance (pull_request) Skipped
Gitea dropped the "1." prefix; 28.x is the successor of 1.27.
- image 28.3.0, update.range >=28.0.0 <29.0.0, Renovate ceiling <29.0.0
- drop GITEA__server__DOMAIN (no longer read in 28, ROOT_URL is the source)
- declare RUN_RETENTION_DAYS=400, EGRESS_MODE=lax (no allow list: the OIDC
  login source is not gated by the egress policy in 28.3.0), and
  DISABLE_REGISTRATION=true (28's new default)
- document autoRollbackOnFailedHealth as inert (no ImagePolicy is rendered);
  rollback is a PVC restore because the DB migration is one-way

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-11-07 22:55:06 -06:00
renovate-bot 0605ecf593 chore(deps): Update dependency keycloak/keycloak-k8s-resources to v26.8.2
gate / gate (pull_request) Successful in 2m57s
gate / conformance (pull_request) Has been skipped
2026-11-07 08:17:02 +00:00
7 changed files with 337 additions and 18 deletions

No files matched your search

+7 -3
View File
@@ -78,7 +78,7 @@ spec:
spec: spec:
containers: containers:
- name: gitea - name: gitea
image: gitea/gitea:1.27.3 image: gitea/gitea:28.3.0
ports: ports:
- containerPort: 3000 - containerPort: 3000
env: env:
@@ -89,14 +89,18 @@ spec:
secretKeyRef: secretKeyRef:
name: gitea-bootstrap name: gitea-bootstrap
key: secret-key key: secret-key
- name: GITEA__server__DOMAIN
value: gitea.${BASE_DOMAIN}
- name: GITEA__server__ROOT_URL - name: GITEA__server__ROOT_URL
value: https://gitea.${BASE_DOMAIN}/ value: https://gitea.${BASE_DOMAIN}/
- name: GITEA__actions__ENABLED - name: GITEA__actions__ENABLED
value: 'true' value: 'true'
- name: GITEA__actions__DEFAULT_ACTIONS_URL - name: GITEA__actions__DEFAULT_ACTIONS_URL
value: github value: github
- name: GITEA__actions__RUN_RETENTION_DAYS
value: '400'
- name: GITEA__security__EGRESS_MODE
value: lax
- name: GITEA__service__DISABLE_REGISTRATION
value: 'true'
- name: GITEA_ADMIN_USERNAME - name: GITEA_ADMIN_USERNAME
valueFrom: valueFrom:
secretKeyRef: secretKeyRef:
+69 -5
View File
@@ -77,6 +77,42 @@ spec:
requests: requests:
storage: 1Gi storage: 1Gi
--- ---
# act_runner config file — DELTAS ONLY. act_runner v0.6.1 LoadDefault()
# (internal/pkg/config/config.go) unmarshals this file onto a ZERO Config and then fills every
# unset field with the same default it uses when there is no file at all, so a file holding
# only `runner.capacity` changes exactly that and nothing else. Do NOT paste in the output of
# `act_runner generate-config`: that example is NOT the defaults. Its `runner.labels` would
# OVERRIDE the registered labels in /data/.runner (daemon.go: config labels win when non-empty)
# and repoint ubuntu-latest at a different job image; its `force_pull: true` differs from the
# no-file default (false); and its sample `envs` would be injected into every job.
#
# Using a file also makes act_runner IGNORE the legacy GITEA_RUNNER_CAPACITY / _FILE /
# _ENVIRON / _ENV_FILE / GITEA_DEBUG / GITEA_TRACE env vars (deprecated.go). None of them is
# set on this Deployment; the GITEA_INSTANCE_URL / _REGISTRATION_TOKEN / _NAME / _LABELS env
# below is consumed by the image's run.sh for registration, not by that legacy path.
#
# A ConfigMap edit alone does NOT restart the runner (it reads the file once at start). If a
# future change touches ONLY this ConfigMap, also bump the Deployment's pod template (e.g. the
# config-revision annotation below) or the new value stays inert until the next restart.
apiVersion: v1
kind: ConfigMap
metadata:
name: act-runner-config
namespace: gitea-actions
labels:
app.kubernetes.io/name: act-runner
data:
config.yaml: |
runner:
# Two concurrent jobs (default 1). Measured 2026-11-08: queue wait ~1s normally, but
# 29-759s whenever gate / images / renovate runs collide (run 472 waited 759s behind a
# renovate burst). Both jobs share the ONE dind daemon below; act_runner names each job's
# containers GITEA-ACTIONS-TASK-<task id> and creates a network per job, so they do not
# collide. Workflow `concurrency:` groups are enforced by Gitea server-side and are
# unaffected: gate.yml's gate-<ref> group still cancels a superseded run on the SAME ref,
# and renovate's group still serialises renovate. The dind resources are sized for TWO jobs.
capacity: 2
---
apiVersion: apps/v1 apiVersion: apps/v1
kind: Deployment kind: Deployment
metadata: metadata:
@@ -95,6 +131,10 @@ spec:
metadata: metadata:
labels: labels:
app.kubernetes.io/name: act-runner app.kubernetes.io/name: act-runner
annotations:
# Bump when act-runner-config changes without any other pod-template change, so the
# Deployment rolls and act_runner re-reads its config (see the ConfigMap header).
homelab.keithjackson.dev/act-runner-config-revision: "1" # capacity 2
spec: spec:
# CI holds no cluster credentials — do not mount a SA token into the runner or # CI holds no cluster credentials — do not mount a SA token into the runner or
# (via the shared netns) the job containers (R-SEC-02, docs/14). # (via the shared netns) the job containers (R-SEC-02, docs/14).
@@ -146,12 +186,23 @@ spec:
mountPath: /certs mountPath: /certs
- name: dind-storage - name: dind-storage
mountPath: /var/lib/docker mountPath: /var/lib/docker
# Sized for runner.capacity: 2. Every job's containers run UNDER this daemon, so this
# container's cgroup carries the whole CI workload. Measured 2026-11-08 (Prometheus,
# capacity 1): peak CPU 0.69 core (1m rate, 30d), peak working set 447Mi (30d), 0 OOMs.
# cpu request 2: two jobs at today's peak is ~1.4 core, and the gate is being
# parallelised, so per-job demand rises. The request is what the CFS share is
# weighted by under contention (100m was a 1/20 share); NO cpu limit, deliberately,
# so an idle node's spare cores are still usable. k3s-prod-2 (8 CPU) had 2945m
# requested; +1.9 core = ~4.85 (61%), and prod-1/prod-3 fit too if it reschedules.
# memory request 1Gi / limit 4Gi: two jobs at today's peak is ~0.9Gi. An OOM kill here
# takes down dockerd and with it BOTH running jobs, so the limit carries ~4x headroom
# over that for the parallelised gate. Nodes have ~31Gi allocatable.
resources: resources:
requests: requests:
cpu: 100m cpu: "2"
memory: 256Mi memory: 1Gi
limits: limits:
memory: 2Gi memory: 4Gi
startupProbe: startupProbe:
exec: exec:
command: command:
@@ -195,23 +246,36 @@ spec:
value: /certs/client value: /certs/client
- name: DOCKER_TLS_VERIFY - name: DOCKER_TLS_VERIFY
value: "1" value: "1"
# The image's run.sh passes `--config <this>` to BOTH `act_runner register` and
# `act_runner daemon` when CONFIG_FILE is set (act_runner v0.6.1 scripts/run.sh).
- name: CONFIG_FILE
value: /etc/act_runner/config.yaml
volumeMounts: volumeMounts:
- name: data - name: data
mountPath: /data mountPath: /data
- name: certs - name: certs
mountPath: /certs mountPath: /certs
readOnly: true readOnly: true
- name: config
mountPath: /etc/act_runner
readOnly: true
# Measured 2026-11-08 at capacity 1: peak CPU 0.24 core, peak working set 314Mi (30d)
# against the old 512Mi limit. It holds per-task state, log buffers and the embedded
# actions/cache server for every concurrent task, so the limit is doubled for capacity 2.
resources: resources:
requests: requests:
cpu: 100m cpu: 100m
memory: 128Mi memory: 256Mi
limits: limits:
memory: 512Mi memory: 1Gi
volumes: volumes:
- name: data - name: data
persistentVolumeClaim: persistentVolumeClaim:
claimName: act-runner-data claimName: act-runner-data
- name: certs - name: certs
emptyDir: {} emptyDir: {}
- name: config
configMap:
name: act-runner-config
- name: dind-storage - name: dind-storage
emptyDir: {} emptyDir: {}
+39 -5
View File
@@ -10,7 +10,7 @@
name: gitea name: gitea
kind: container kind: container
source: source:
image: gitea/gitea:1.27.3 # pinned recent stable tag (R-UPD-01); update.range governs advancement image: gitea/gitea:28.3.0 # pinned recent stable tag (R-UPD-01); update.range governs advancement
# Headless-install bootstrap (formerly apps/gitea-bootstrap-patch.yaml; now first-class via the # Headless-install bootstrap (formerly apps/gitea-bootstrap-patch.yaml; now first-class via the
# ServiceDefinition env/envFrom/lifecycle fields, schema >=1.2.0). Gitea must come up INSTALLED # ServiceDefinition env/envFrom/lifecycle fields, schema >=1.2.0). Gitea must come up INSTALLED
# without the interactive web wizard and with a FIXED, persistent SECRET_KEY, so all of this is # without the interactive web wizard and with a FIXED, persistent SECRET_KEY, so all of this is
@@ -23,14 +23,43 @@ env:
secretKeyRef: secretKeyRef:
name: gitea-bootstrap name: gitea-bootstrap
key: secret-key key: secret-key
- name: GITEA__server__DOMAIN # correct public clone/redirect URLs for the TLS-terminated host # ROOT_URL is the ONLY source of the instance domain since Gitea 28 (go-gitea/gitea#39400): [server]
value: gitea.${BASE_DOMAIN} # DOMAIN is no longer read, and the SSH domain defaults from ROOT_URL too. GITEA__server__DOMAIN was
# removed in the 28 upgrade; the stale DOMAIN line environment-to-ini already wrote into app.ini is
# inert. Correct public clone/redirect URLs for the TLS-terminated host come from this one value.
- name: GITEA__server__ROOT_URL - name: GITEA__server__ROOT_URL
value: https://gitea.${BASE_DOMAIN}/ value: https://gitea.${BASE_DOMAIN}/
- name: GITEA__actions__ENABLED # Phase 6 CI/CD: declared (not implicit upstream default) so the - name: GITEA__actions__ENABLED # Phase 6 CI/CD: declared (not implicit upstream default) so the
value: "true" # in-cluster act_runner (ns gitea-actions) registers against it value: "true" # in-cluster act_runner (ns gitea-actions) registers against it
- name: GITEA__actions__DEFAULT_ACTIONS_URL # resolve `uses: <org>/<action>@ref` from github.com (cluster - name: GITEA__actions__DEFAULT_ACTIONS_URL # resolve `uses: <org>/<action>@ref` from github.com (cluster
value: github # has internet egress; not air-gapped) value: github # has internet egress; not air-gapped)
# Gitea 28 added run retention (go-gitea/gitea#38855): completed runs, with their jobs, logs and
# artifacts, are deleted by the cleanup_action_runs cron after RUN_RETENTION_DAYS. Upstream's default
# (400) is kept but DECLARED, so the deletion horizon is visible here rather than implicit. The spec
# repo states no CI-history retention requirement; 0 would mean keep forever.
- name: GITEA__actions__RUN_RETENTION_DAYS
value: "400"
# Gitea 28 (go-gitea/gitea#39426) puts webhooks, the OAuth2 avatar fetch and OpenID 2.0 behind an
# egress policy: in the default lax mode, PUBLIC targets are allowed and private/loopback/CGNAT
# targets need an IP or CIDR entry in [security] ALLOWED_HOST_LIST (a hostname entry is NOT enough
# for a private address). DECLARED lax with an EMPTY allow list, deliberately:
# * the Keycloak OIDC login source is NOT gated by it. In 28.3.0 the openidConnect provider is
# goth's, built with its default HTTP client (services/auth/source/oauth2/providers_openid.go);
# egress.NewSecurityPolicy is used only by routers/web/auth/oauth.go (avatar fetch, and
# [oauth2_client] UPDATE_AVATAR is off here) and modules/auth/openid (OpenID 2.0, unused).
# So keycloak.<domain> resolving to the private Gateway address in-cluster does not matter.
# * this instance has no webhooks and no mirrors (measured 2026-11-07), and 1.27 already refused
# private webhook targets by default, so nothing that worked before is newly denied.
# Adding an entry here would only WIDEN the webhook SSRF surface. Add the narrowest CIDR (e.g. the
# Gateway address /32) only if a real private webhook target appears.
- name: GITEA__security__EGRESS_MODE
value: lax
# Gitea 28 (go-gitea/gitea#39400) disables LOCAL self-registration unless DISABLE_REGISTRATION is
# explicitly false; 1.27 defaulted it open. Declared true: accounts arrive via Keycloak only.
# OAuth2 auto-registration below is unaffected — routers/web/auth/oauth.go gates it on
# ALLOW_ONLY_INTERNAL_REGISTRATION and ENABLE_AUTO_REGISTRATION, not on DISABLE_REGISTRATION.
- name: GITEA__service__DISABLE_REGISTRATION
value: "true"
- name: GITEA_ADMIN_USERNAME # break-glass local admin creds (consumed by the postStart hook) - name: GITEA_ADMIN_USERNAME # break-glass local admin creds (consumed by the postStart hook)
valueFrom: valueFrom:
secretKeyRef: secretKeyRef:
@@ -233,9 +262,14 @@ auth:
# recovery tools (docs/15 "Availability caveat"). No secret is embedded here. # recovery tools (docs/15 "Availability caveat"). No secret is embedded here.
update: update:
channel: stable # R-UPD-01 channel: stable # R-UPD-01
range: ">=1.22.0 <2.0.0" # stay on the 1.x stable line range: ">=28.0.0 <29.0.0" # Gitea dropped the "1." prefix at 28.0.0 (successor of 1.27); stay on 28.x
cadence: "0 4 * * *" cadence: "0 4 * * *"
autoRollbackOnFailedHealth: true # R-UPD-03 # R-UPD-03. INERT FOR THIS SERVICE, and it must stay that way: this flag renders only an annotation
# on a Flux ImagePolicy, and apps/ is rendered --no-image-automation (ci/render-golden.sh), so no
# ImagePolicy exists and the Deployment (strategy Recreate) is never auto-reverted. That matters
# because a Gitea upgrade (1.27 -> 28 included) MIGRATES THE SQLITE DB ONE-WAY: an image revert onto
# a migrated DB is not a rollback. Roll Gitea back by RESTORING THE DATA PVC from its pre-upgrade backup.
autoRollbackOnFailedHealth: true
observability: observability:
metrics: true metrics: true
slis: slis:
+62
View File
@@ -0,0 +1,62 @@
---
date: 2026-11-07
branch: feat/gitea-28
title: "Gitea 1.27.3 -> 28.3.0 (upstream dropped the 1. prefix)"
---
## What changed
- `catalog/gitea.yaml`: `source.image` `gitea/gitea:1.27.3` -> `gitea/gitea:28.3.0` (still the
newest stable tag on Docker Hub and GitHub on 2026-11-07: 28.0.0, 28.1.0, 28.2.0, 28.3.0).
`update.range` `">=1.22.0 <2.0.0"` -> `">=28.0.0 <29.0.0"`.
- 28.x breaking changes handled in the env block:
- `GITEA__server__DOMAIN` **removed**: 28 no longer reads `[server] DOMAIN` (go-gitea/gitea#39400);
`ROOT_URL` was already set and is now the only source of the domain. Nothing else in the repo
reads the variable (the bootstrap script sets ROOT_URL/SSH_PORT only).
- `GITEA__actions__RUN_RETENTION_DAYS=400` **declared** (go-gitea/gitea#38855 added the 400-day
default; the spec repo states no CI-history retention requirement).
- `GITEA__security__EGRESS_MODE=lax` **declared, with no allow list.** Read from the 28.3.0
source: the Keycloak OIDC login source does NOT go through the new egress policy (goth's
openidConnect provider, default HTTP client). The policy covers webhooks, the OAuth2 avatar
fetch (`UPDATE_AVATAR` is off) and OpenID 2.0. A hostname entry would not even help for a
private address — `WithLocalNeedsIPAllow` requires an IP/CIDR entry. So the brief's
"Keycloak SSO breaks" premise does not hold for 28.3.0, and an allow-list entry would only
widen webhook SSRF reach.
- `GITEA__service__DISABLE_REGISTRATION=true` **declared** (28's new default; 1.27 defaulted it
open). OAuth2 auto-registration is gated separately and keeps working.
- `update.autoRollbackOnFailedHealth`: kept `true`, commented as **inert**. It renders only an
annotation on a Flux ImagePolicy, and `apps/` is rendered `--no-image-automation`, so no
ImagePolicy exists. The Deployment is `Recreate` with no probes, so nothing reverts the image
automatically. Rollback = restore the PVC, never an image revert onto the migrated DB.
- `apps/wave-0/gitea.yaml` re-rendered with the canonical command
(`render.py --all catalog -o apps --wave-layout --no-image-automation`, in the toolbox). Only
gitea changed.
- `renovate.json5`: gitea `allowedVersions` `<2.0.0` -> `<29.0.0`; the description no longer calls
28.x a stale tag.
## What I verified
- Docker Hub tag list and GitHub tags: 28.3.0 is the newest 28.x stable tag.
- Read the v28.3.0 source: `modules/egress/policy/policy.go`, `modules/egress/policies.go`, every
`egress.New*` caller, `services/auth/source/oauth2/providers_openid.go`,
`modules/setting/{security,actions,service}.go`, `routers/web/auth/{oauth,linkaccount}.go`,
`cmd/admin_user_list.go`, `cmd/admin_auth*.go`, the Dockerfile, and `docker/root` (the s6 setup
still runs `environment-to-ini`). The postStart hook's CLI is unchanged in 28.3.0: `admin user
list --admin` still has Username in column 2, `admin auth list` still has ID/Name in columns 1/2,
`add-oauth`/`update-oauth` keep the same flags, and curl and su-exec are still in the image.
- Live Gitea (API, 2026-11-07): version 1.27.3. Zero repo, org and system webhooks, no mirrors.
- `renovate-config-validator` at 44.167.0 (the tag renovate.yaml pins): "Config validated
successfully".
- Ran the toolbox gate on the committed tree (numbers are in the PR).
## What I did NOT verify
- **The upgrade itself.** Nothing booted 28.3.0 against a copy of the production DB. Migration
time, migration success and the post-upgrade SSO login are all UNVERIFIED, and the PR's
post-merge checklist covers them.
- **The age of the latest gitea-data Longhorn/off-cluster backup.** The cluster read failed from
the worktree (no k3s-prod context with the kubeconfig available), so the operator must confirm
it before merging.
- How the Envoy Gateway handles the new `/-/ws` WebSocket. If upgrades are not forwarded, live
notifications fall back to polling (not breaking). UNVERIFIED.
- Whether the L2/L3 conformance suite passes. It does not run in CI.
+155
View File
@@ -0,0 +1,155 @@
---
date: 2026-11-08
branch: feat/runner-capacity-2
title: act_runner runs two jobs at once, and dind is sized for two
---
## What changed
`apps/wave-1/gitea-actions.yaml` only. That file is a **hand-owned** non-catalog unit
(`EXTRA_UNITS["gitea-actions"]` in `catalog/render.py`): the renderer only decides which wave
it lands in, it does not generate its bytes, and `ci/render-golden.sh` compares only files the
catalog renders. So it is edited in place, and no re-render was needed or run.
1. **New ConfigMap `act-runner-config`** holding a config file with ONE key,
`runner.capacity: 2`, mounted read-only at `/etc/act_runner/` and selected with
`CONFIG_FILE=/etc/act_runner/config.yaml`. The image's `run.sh` (act_runner v0.6.1) turns
that into `--config` for both `register` and `daemon`.
- Deltas only, on evidence: v0.6.1 `LoadDefault()` unmarshals the file onto a zero Config and
then applies the same defaults as the no-file path, so the other settings are unchanged.
- The `generate-config` example was deliberately NOT used. Its `runner.labels` would override
the registered labels in `/data/.runner` and repoint `ubuntu-latest` at
`docker.gitea.com/runner-images`. Its `force_pull: true` is not the default. Its sample `envs`
would be injected into every job.
- With a file present, act_runner ignores the legacy `GITEA_RUNNER_CAPACITY` /
`_FILE` / `_ENVIRON` / `_ENV_FILE` / `GITEA_DEBUG` / `GITEA_TRACE` env vars. None of
them is set on this Deployment.
2. **Pod-template annotation `homelab.keithjackson.dev/act-runner-config-revision: "1"`.**
act_runner reads its config once, at start, so a change to the ConfigMap alone would never
take effect. Bump this annotation whenever you change only the ConfigMap.
3. **dind resources:** requests `cpu 100m -> 2` and `memory 256Mi -> 1Gi`; limit
`memory 2Gi -> 4Gi`. There is still no CPU limit, on purpose.
4. **runner resources:** request `memory 128Mi -> 256Mi`, limit `memory 512Mi -> 1Gi`. CPU stays
at `100m`.
Why these sizes. Measured 2026-11-08 through Prometheus, at capacity 1:
| container | peak CPU (1m rate, 30d) | peak working set (30d) | OOM events (7d) |
|---|---|---|---|
| dind | 0.69 core | 447 MiB | 0 |
| runner | 0.24 core | 314 MiB | 0 |
At the moment of measurement the pod was idle (`top`: dind 1m/138Mi, runner 1m/83Mi), on
k3s-prod-2.
- **CPU request 2.** Two jobs at today's peak is about 1.4 cores. The gate is being parallelised
in other work, so each job will use more than that. The request sets dind's CFS weight under
contention; at 100m it got a 1/20 share. With no limit, dind can still use spare cores on an
idle node.
- **Memory 1Gi / 4Gi.** Two jobs at today's peak is about 0.9 GiB. If dind is OOM-killed,
dockerd goes down and takes both running jobs with it, so the limit is about 4x that.
- **Runner limit 512Mi -> 1Gi.** Its 30-day peak was already 61% of the old limit, and with
capacity 2 it holds state for two tasks.
Scheduling headroom: CPU requests per node were measured with `describe node`. The pod's total
CPU request goes from 200m to 2100m (+1.9 cores).
| node | CPU requested now | after the change | allocatable |
|---|---|---|---|
| prod-2 (current) | 2945m | ~4845m (61%) | 8 |
| prod-1 | 3175m | ~5075m | 8 |
| prod-3 | 3840m | ~5740m (72%) | 8 |
The pod fits on any of the three if the RWO PVC moves it.
Concurrency semantics. Workflow `concurrency:` groups are enforced by Gitea on the server, not by
the runner, so capacity does not change them:
- `gate.yml` `gate-${{ github.ref }}` with cancel-in-progress still cancels a superseded run on
the same ref.
- `renovate` (group `renovate`, no cancel) and `images-${{ github.ref }}` still serialise
themselves.
What capacity 2 does change: runs on *different* refs (a PR gate next to a renovate or images
run, or two PRs) now run side by side instead of queueing. The required check
`gate / gate (pull_request)` is still one job per PR head, and gate.yml's other job
(`conformance`) is `if: false`, so it never runs.
Both jobs share one dockerd. act_runner gives each task its own container prefix
(`GITEA-ACTIONS-TASK-<task id>`) and its own network, and no workflow uses fixed container names
or host ports.
## What I verified
- **Source of truth.** `catalog/render.py` `EXTRA_UNITS` places `gitea-actions.yaml` by
anchor/offset only. `ci/render-golden.sh` iterates over rendered files only, so this file is
outside render==golden. No `--emit-waves` was run.
- **act_runner v0.6.1 behaviour, read in its source** (tag v0.6.1):
- `config.LoadDefault()` treats a partial file as described above.
- `daemon.go`: config labels override `.runner` labels only when non-empty.
- `poller.go`: a semaphore of `cfg.Runner.Capacity` bounds concurrent tasks.
- `runner.go`: per-task container name prefix.
- The image's `run.sh`, read from `gitea/act_runner:0.6.1`, honours `CONFIG_FILE`.
- **The live pod has no config file today.** `/data` holds only `.runner` and `lost+found`.
`act_runner --version` reports v0.6.1. No LimitRange or ResourceQuota exists in
`gitea-actions`.
- **Resource numbers.** All figures above come from read-only `kubectl top`, `describe node` and
Prometheus queries against k3s-prod.
- **dind disk.** The emptyDir sits on the node's 11.8G ephemeral filesystem: 4.0G used, 7.2G
free. Image cache is 2.18GB, which is the one `catthehacker/ubuntu:act-22.04` image. Two
concurrent jobs share that image.
- **Embedded config.** It parses as YAML to `{'runner': {'capacity': 2}}`. The only `$` in the
manifest is the existing `${BASE_DOMAIN}`, so nothing new is exposed to Flux postBuild
envsubst.
- **Approval policy.** `apps/wave-1/` matches no `sensitive_paths` glob in
`infra/gitea/approval-policy.json`, so this PR merges on a green gate alone.
- **Local gate.** Pinned toolbox, run from the worktree on the committed tree. The rc and skipped
stages are given in the PR description.
## What I did NOT verify
- **UNVERIFIED: that the runner actually starts with the config and runs two jobs at once.**
Nothing is applied until Flux reconciles `main`.
- **Capacity cannot be seen through Gitea's API.** act_runner v0.6.1 does not report capacity to
Gitea: `Declare` sends only version and labels. `GET /api/v1/admin/actions/runners` therefore
cannot show "capacity 2". Concurrency has to be observed instead.
- **UNVERIFIED: whether the parallelised gate fits in these sizes.** That gate has not landed.
Re-measure after it does, with
`max_over_time(sum by (container)(rate(container_cpu_usage_seconds_total{namespace="gitea-actions"}[1m]))[7d:30s])`.
- **Rollout cancels in-flight jobs.** The rollout restarts the runner pod (strategy Recreate,
`shutdown_timeout: 0s`), and any job running at that moment is cancelled, possibly including
the `push`-to-main gate run triggered by this very merge. Re-run it if that happens.
- **dind disk is not bounded by a limit.** With two jobs it holds two jobs' workspaces and layers
on the same 11.8G node filesystem. I did not add an `ephemeral-storage` limit, because an
eviction would kill both jobs. Watch kubelet DiskPressure on the runner's node.
- **Stale comments in sensitive paths.** `.gitea/workflows/gate.yml` (line ~52) and
`.gitea/workflows/README.md` (line ~86) still say "`act_runner` runs at capacity 1". I left
them alone because `.gitea/**` is a sensitive path and editing them would put this PR behind
human approval. Fix them in a separate PR.
### Operator post-merge checklist
1. Flux applied it:
`kubectl --context k3s-prod -n flux-system get kustomization apps-wave-1 -o jsonpath='{.status.lastAppliedRevision}'`
equals the merge commit.
2. The pod restarted with the config:
- `kubectl --context k3s-prod -n gitea-actions get pod -l app.kubernetes.io/name=act-runner`
shows a new pod, 2/2 Running.
- `kubectl ... exec deploy/act-runner -c runner -- cat /etc/act_runner/config.yaml` shows
`capacity: 2`.
- `kubectl ... exec deploy/act-runner -c runner -- ps` shows `act_runner daemon --config /etc/act_runner/config.yaml`.
- The logs show "declare successfully" with the two expected labels (`ubuntu-latest`,
`homelab` -> `catthehacker/ubuntu:act-22.04`). That confirms the labels did not change.
3. The runner is registered and online:
- `GET /api/v1/admin/actions/runners` lists `incluster-dind`, online, with the same ID as
before (no duplicate row).
- Capacity is NOT shown there; see above.
4. Concurrency is real: trigger two runs on different refs (e.g. two PR gates, or a PR gate plus a
`workflow_dispatch` of renovate). Then either:
- in `GET /api/v1/repos/Home-Network/home-network-k3s-impl/actions/tasks`, confirm their
`run_started_at` .. `updated_at` windows overlap; or
- check
`docker --host=tcp://127.0.0.1:2376 --tlsverify ... ps` in the dind container lists two
`GITEA-ACTIONS-TASK-*` container sets at once.
5. Resources: `kubectl --context k3s-prod -n gitea-actions describe pod` shows dind requests
cpu 2 / memory 1Gi, limit memory 4Gi, and runner limit memory 1Gi.
+3 -3
View File
@@ -28,11 +28,11 @@ resources:
# branch, never auto-merged (packageRule "keycloak-operator"). Before merging a bump, re-check # branch, never auto-merged (packageRule "keycloak-operator"). Before merging a bump, re-check
# passkey-reconciler.yaml's 26.0.6 Admin API assumptions — see README.md "Upgrading Keycloak". # passkey-reconciler.yaml's 26.0.6 Admin API assumptions — see README.md "Upgrading Keycloak".
# renovate: datasource=github-tags depName=keycloak/keycloak-k8s-resources # renovate: datasource=github-tags depName=keycloak/keycloak-k8s-resources
- https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.0.6/kubernetes/keycloaks.k8s.keycloak.org-v1.yml - https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.8.2/kubernetes/keycloaks.k8s.keycloak.org-v1.yml
# renovate: datasource=github-tags depName=keycloak/keycloak-k8s-resources # renovate: datasource=github-tags depName=keycloak/keycloak-k8s-resources
- https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.0.6/kubernetes/keycloakrealmimports.k8s.keycloak.org-v1.yml - https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.8.2/kubernetes/keycloakrealmimports.k8s.keycloak.org-v1.yml
# renovate: datasource=github-tags depName=keycloak/keycloak-k8s-resources # renovate: datasource=github-tags depName=keycloak/keycloak-k8s-resources
- https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.0.6/kubernetes/kubernetes.yml - https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/26.8.2/kubernetes/kubernetes.yml
# --- Admin (break-glass) credential — SOPS-encrypted, referenced by name (ADR-0007, R-IDP-04) --- # --- Admin (break-glass) credential — SOPS-encrypted, referenced by name (ADR-0007, R-IDP-04) ---
- keycloak-admin-credentials.sops.yaml - keycloak-admin-credentials.sops.yaml
# --- OIDC client secrets (grafana + oauth2-proxy) the realm import injects via placeholders --- # --- OIDC client secrets (grafana + oauth2-proxy) the realm import injects via placeholders ---
+2 -2
View File
@@ -316,9 +316,9 @@
"enabled": false "enabled": false
}, },
{ {
"description": "ADDED 2026-10-30. MIRRORS catalog/<svc>.yaml `update.range` — the operator's declared per-service policy (schemas/servicedefinition.schema.json), which Renovate does not read. Without it the dry run proposed gitea 28.2.0, qbittorrent 20.04.1 (both stale/odd tags, not real successors), radarr 6.x and nextcloud 35 (four majors past 31; Nextcloud cannot skip majors). KEEP IN STEP with the catalog ranges.", "description": "ADDED 2026-10-30. MIRRORS catalog/<svc>.yaml `update.range` — the operator's declared per-service policy (schemas/servicedefinition.schema.json), which Renovate does not read. Without it the dry run proposed gitea 28.2.0, qbittorrent 20.04.1 (the qbittorrent tag is a stale/odd one, not a real successor; gitea 28.x IS the real successor of 1.27 — upstream dropped the \"1.\" prefix at 28.0.0 — and the catalog moved to it on 2026-11-07, so the gitea ceiling is now <29.0.0: 28.x patches/minors are proposed, never auto-merged per the blast-radius rule, and 29 waits for a human), radarr 6.x and nextcloud 35 (four majors past 31; Nextcloud cannot skip majors). KEEP IN STEP with the catalog ranges.",
"matchPackageNames": ["gitea/gitea"], "matchPackageNames": ["gitea/gitea"],
"allowedVersions": "<2.0.0" "allowedVersions": "<29.0.0"
}, },
{ "matchPackageNames": ["nextcloud"], "allowedVersions": "<32.0.0" }, { "matchPackageNames": ["nextcloud"], "allowedVersions": "<32.0.0" },
{ "matchPackageNames": ["lscr.io/linuxserver/qbittorrent"], "allowedVersions": "<6.0.0" }, { "matchPackageNames": ["lscr.io/linuxserver/qbittorrent"], "allowedVersions": "<6.0.0" },