jakman85andClaude Opus 5
e7508931ca
Executes the FOLLOW-UP that has been carried in this manifest since the readiness fix,
now that k3s is v1.36 and sidecar containers are GA.
dind moves into initContainers as the first entry with restartPolicy: Always -- the one
field that makes an initContainer a sidecar: it starts before the regular containers,
runs alongside them, and is terminated after them. kubelet therefore enforces the
ordering that the runner used to hand-roll, so the runner drops its command: override
and runs the image's own entrypoint.
The startupProbe execs a full TLS-verified `docker ... info`, NOT a tcpSocket probe. A
tcpSocket probe would re-open the exact gap the old Stage 1 existed to close: dockerd
writes the TLS certs and only THEN starts listening, so the port can accept a connection
while the certs are absent, and act_runner hard-exits on a missing
/certs/client/ca.pem. `docker info` proves both that the certs are usable and that the
daemon answers. periodSeconds 1 x failureThreshold 60 preserves the previous 60s cap.
It also removes the one-per-start dockerd log line
http: TLS handshake error from 127.0.0.1: EOF
which was the old `nc -z` probe opening a TCP connection and closing it without a TLS
handshake -- an error we were generating ourselves and then reading in the logs.
SECONDARY WIN: deleting the shell deletes an envsubst hazard. apps-wave-1 runs postBuild
substitution over this ENTIRE manifest, comments included, and those loops depended on
bare dollar-names passing through untouched -- a braced or double-dollar spelling would
have silently unbounded the waits or failed the whole Kustomization. The only dollar left
in the file is the intended ${BASE_DOMAIN}.
LANDS ALONE, DELIBERATELY. This Deployment is the merge gate for the entire repository:
if the conversion is wrong, nothing can merge. Merge this BEFORE enabling the required
status check, and watch a real workflow run green on the converted pod before doing so --
otherwise a bad conversion locks the repo with no way to revert through the gate.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
210 lines
8.7 KiB
YAML
210 lines
8.7 KiB
YAML
---
|
|
# Gitea Actions runner (act_runner) — Phase 6 CI/CD (WS-GITOSS).
|
|
#
|
|
# WHY A DEDICATED, NON-APP NAMESPACE:
|
|
# The runner's job executor is a Docker-in-Docker (DinD) sidecar that MUST run
|
|
# `privileged: true` (dockerd needs it). The Kyverno `restrict-host-namespaces`
|
|
# policy denies privileged containers, but it (and the egress-class / auth-mode /
|
|
# default-deny-baseline policies) scope ONLY to namespaces labeled
|
|
# homelab.keithjackson.dev/app-namespace: "true". This namespace deliberately does
|
|
# NOT carry that label — it is CI plumbing, not a user-facing app — so the privileged
|
|
# DinD sidecar is admitted WITHOUT a policy exception. (Verified: a privileged Pod in
|
|
# a non-app namespace passes server-side admission; Kyverno test fixture
|
|
# `platform-workload` is likewise Excluded from restrict-host-namespaces.)
|
|
#
|
|
# EXECUTOR MODEL — Docker executor + DinD sidecar (privileged):
|
|
# act_runner runs each job step inside a container it launches via a Docker daemon.
|
|
# That daemon is the co-located `dind` sidecar (shared pod netns → tcp://localhost:2376,
|
|
# TLS via a shared emptyDir of certs). Tradeoff vs the host/self-hosted executor: DinD
|
|
# needs `privileged`, but it gives clean per-job isolation and lets the workflow pick its
|
|
# own job image (the pinned toolchain), which the host executor cannot. Privileged is
|
|
# contained to this CI-only namespace and the runner mounts NO ServiceAccount token
|
|
# (automountServiceAccountToken: false) — CI holds no cluster credentials (R-SEC-02).
|
|
apiVersion: v1
|
|
kind: Namespace
|
|
metadata:
|
|
name: gitea-actions
|
|
labels:
|
|
# NOTE: intentionally NO homelab.keithjackson.dev/app-namespace label (see header).
|
|
homelab.keithjackson.dev/ci: act-runner
|
|
---
|
|
# Default-deny ingress; broad egress (a CI runner legitimately needs the internet:
|
|
# ghcr.io / github.com / docker hub / dl.k8s.io / pypi for job images + pinned tools,
|
|
# and the Gitea ingress VIP for registration + repo clone). Nothing connects INTO the
|
|
# runner, so ingress is denied entirely.
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: default-deny-ingress-allow-egress
|
|
namespace: gitea-actions
|
|
labels:
|
|
homelab.keithjackson.dev/baseline: default-deny
|
|
spec:
|
|
podSelector: {}
|
|
policyTypes:
|
|
- Ingress
|
|
- Egress
|
|
ingress: []
|
|
egress:
|
|
# DNS (cluster + split-horizon resolution of gitea.home.keithjackson.dev).
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: kube-system
|
|
ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
# Everything else the runner + DinD need: Gitea VIP + internet registries/tool hosts.
|
|
- {}
|
|
---
|
|
# Runner state (config.yaml + the `.runner` registration file) persists here so the
|
|
# runner registers exactly ONCE and reuses its identity across restarts (no duplicate
|
|
# offline rows in Gitea). RWO → Deployment strategy must be Recreate (fix/recreate-strategy-rwo).
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: act-runner-data
|
|
namespace: gitea-actions
|
|
labels:
|
|
app.kubernetes.io/name: act-runner
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
storageClassName: longhorn
|
|
resources:
|
|
requests:
|
|
storage: 1Gi
|
|
---
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: act-runner
|
|
namespace: gitea-actions
|
|
labels:
|
|
app.kubernetes.io/name: act-runner
|
|
spec:
|
|
replicas: 1
|
|
strategy:
|
|
type: Recreate
|
|
selector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: act-runner
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app.kubernetes.io/name: act-runner
|
|
spec:
|
|
# CI holds no cluster credentials — do not mount a SA token into the runner or
|
|
# (via the shared netns) the job containers (R-SEC-02, docs/14).
|
|
automountServiceAccountToken: false
|
|
# ---- Docker-in-Docker: the job executor daemon (privileged) -------------------
|
|
#
|
|
# NATIVE SIDECAR (k3s v1.36; sidecar containers are GA). It is an initContainer with
|
|
# `restartPolicy: Always`, which means: it starts BEFORE the regular containers, keeps
|
|
# running alongside them, and is terminated after them. kubelet holds `runner` until
|
|
# this container's startupProbe passes, so the ordering is enforced by the kubelet
|
|
# instead of hand-rolled shell in the runner.
|
|
#
|
|
# This REPLACES two bounded wait-loops that used to live in the runner's `command:`.
|
|
# Deleting them also deletes an envsubst hazard: clusters/base/apps-wave-1.yaml runs
|
|
# postBuild substitution over this ENTIRE manifest, comments included, and those loops
|
|
# depended on bare dollar-names surviving it untouched. There is now no shell here at
|
|
# all, so there is nothing to escape or mis-escape.
|
|
#
|
|
# WHY THE PROBE IS `exec docker ... info` AND NOT `tcpSocket`. A tcpSocket probe would
|
|
# re-open the exact gap the old Stage 1 existed to close: dockerd writes the TLS certs
|
|
# and only THEN begins listening, so the port can accept a connection while the certs
|
|
# are absent or incomplete -- and act_runner hard-exits on a missing
|
|
# /certs/client/ca.pem. A full TLS-verified `docker info` proves BOTH that the certs
|
|
# are usable and that the daemon answers. It also removes the one-per-start
|
|
# "TLS handshake error ... EOF" that the old `nc -z` probe logged in dockerd.
|
|
#
|
|
# periodSeconds 1 x failureThreshold 60 preserves the previous 60s cap.
|
|
initContainers:
|
|
- name: dind
|
|
restartPolicy: Always # <- this field is what makes it a SIDECAR
|
|
image: docker:27-dind
|
|
securityContext:
|
|
privileged: true
|
|
env:
|
|
# Generate + publish TLS certs into the shared emptyDir; dockerd then listens
|
|
# on tcp://0.0.0.0:2376 (TLS) which the runner reaches at localhost:2376.
|
|
- name: DOCKER_TLS_CERTDIR
|
|
value: /certs
|
|
volumeMounts:
|
|
- name: certs
|
|
mountPath: /certs
|
|
- name: dind-storage
|
|
mountPath: /var/lib/docker
|
|
resources:
|
|
requests:
|
|
cpu: 100m
|
|
memory: 256Mi
|
|
limits:
|
|
memory: 2Gi
|
|
startupProbe:
|
|
exec:
|
|
command:
|
|
- docker
|
|
- --host=tcp://127.0.0.1:2376
|
|
- --tlsverify
|
|
- --tlscacert=/certs/client/ca.pem
|
|
- --tlscert=/certs/client/cert.pem
|
|
- --tlskey=/certs/client/key.pem
|
|
- info
|
|
periodSeconds: 1
|
|
failureThreshold: 60
|
|
containers:
|
|
# ---- act_runner: registers with Gitea, dispatches jobs to the DinD daemon ----
|
|
- name: runner
|
|
image: gitea/act_runner:0.2.11
|
|
# No `command:` override. The image's own entrypoint (/sbin/tini -- /opt/act/run.sh)
|
|
# registers from env on first boot and runs `act_runner daemon`. The two wait-loops
|
|
# that used to live here are gone: kubelet now holds this container until the dind
|
|
# sidecar's startupProbe passes (see initContainers above).
|
|
env:
|
|
- name: GITEA_INSTANCE_URL
|
|
value: https://gitea.${BASE_DOMAIN}
|
|
- name: GITEA_RUNNER_REGISTRATION_TOKEN
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: gitea-actions-runner
|
|
key: token
|
|
- name: GITEA_RUNNER_NAME
|
|
value: incluster-dind
|
|
# runs-on label -> job image. Both map to the catthehacker "act" image, which
|
|
# carries bash/git/curl/tar/python3+pip/node so .gitea/workflows scripts can
|
|
# install the pinned CI toolchain (ci/versions.env) and actions/checkout runs.
|
|
- name: GITEA_RUNNER_LABELS
|
|
value: "ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-22.04,homelab:docker://ghcr.io/catthehacker/ubuntu:act-22.04"
|
|
# Reach the DinD daemon over the shared pod network namespace, TLS-verified
|
|
# against the certs the sidecar drops into the shared /certs emptyDir.
|
|
- name: DOCKER_HOST
|
|
value: tcp://localhost:2376
|
|
- name: DOCKER_CERT_PATH
|
|
value: /certs/client
|
|
- name: DOCKER_TLS_VERIFY
|
|
value: "1"
|
|
volumeMounts:
|
|
- name: data
|
|
mountPath: /data
|
|
- name: certs
|
|
mountPath: /certs
|
|
readOnly: true
|
|
resources:
|
|
requests:
|
|
cpu: 100m
|
|
memory: 128Mi
|
|
limits:
|
|
memory: 512Mi
|
|
volumes:
|
|
- name: data
|
|
persistentVolumeClaim:
|
|
claimName: act-runner-data
|
|
- name: certs
|
|
emptyDir: {}
|
|
- name: dind-storage
|
|
emptyDir: {}
|