# ServiceDefinition: gitea # Source of truth: spec docs/13-service-catalog.md row 12; docs/15-identity-and-access.md; docs/05. # public git hosting | egress internet | repos+DB on Longhorn (irreplaceable, DR-critical) | auth oidc. # Bootstrapping-critical: self-hosts the platform's GitOps source + CI/CD, onboarded EARLY (docs/13 # "Gitea is bootstrapping-critical", WS-APPS step 2, right after PiHole). A cluster loss would take its # own git host with it, so repos/DB are backed up off-cluster + mirrored (R-STATE-03). # Single-container Gitea with the built-in SQLite datastore is sufficient at home scale; the repos and # DB live on one Longhorn PVC (/data), so no external PostgreSQL is required for this onboarding. # Validates against schemas/servicedefinition.schema.json. name: gitea kind: container source: image: gitea/gitea:28.3.0 # pinned recent stable tag (R-UPD-01); update.range governs advancement # Headless-install bootstrap (formerly apps/gitea-bootstrap-patch.yaml; now first-class via the # ServiceDefinition env/envFrom/lifecycle fields, schema >=1.2.0). Gitea must come up INSTALLED # without the interactive web wizard and with a FIXED, persistent SECRET_KEY, so all of this is # declared here and rendered verbatim onto the app container by catalog/render.py. env: - name: GITEA__security__INSTALL_LOCK # skip the first-run web installer (headless install) value: "true" - name: GITEA__security__SECRET_KEY # FIXED persistent key; MUST NOT regenerate per restart or valueFrom: # previously-encrypted data breaks. Pinned via the SOPS Secret. secretKeyRef: name: gitea-bootstrap key: secret-key # ROOT_URL is the ONLY source of the instance domain since Gitea 28 (go-gitea/gitea#39400): [server] # DOMAIN is no longer read, and the SSH domain defaults from ROOT_URL too. GITEA__server__DOMAIN was # removed in the 28 upgrade; the stale DOMAIN line environment-to-ini already wrote into app.ini is # inert. Correct public clone/redirect URLs for the TLS-terminated host come from this one value. - name: GITEA__server__ROOT_URL value: https://gitea.${BASE_DOMAIN}/ - name: GITEA__actions__ENABLED # Phase 6 CI/CD: declared (not implicit upstream default) so the value: "true" # in-cluster act_runner (ns gitea-actions) registers against it - name: GITEA__actions__DEFAULT_ACTIONS_URL # resolve `uses: /@ref` from github.com (cluster value: github # has internet egress; not air-gapped) # Gitea 28 added run retention (go-gitea/gitea#38855): completed runs, with their jobs, logs and # artifacts, are deleted by the cleanup_action_runs cron after RUN_RETENTION_DAYS. Upstream's default # (400) is kept but DECLARED, so the deletion horizon is visible here rather than implicit. The spec # repo states no CI-history retention requirement; 0 would mean keep forever. - name: GITEA__actions__RUN_RETENTION_DAYS value: "400" # Gitea 28 (go-gitea/gitea#39426) puts webhooks, the OAuth2 avatar fetch and OpenID 2.0 behind an # egress policy: in the default lax mode, PUBLIC targets are allowed and private/loopback/CGNAT # targets need an IP or CIDR entry in [security] ALLOWED_HOST_LIST (a hostname entry is NOT enough # for a private address). DECLARED lax with an EMPTY allow list, deliberately: # * the Keycloak OIDC login source is NOT gated by it. In 28.3.0 the openidConnect provider is # goth's, built with its default HTTP client (services/auth/source/oauth2/providers_openid.go); # egress.NewSecurityPolicy is used only by routers/web/auth/oauth.go (avatar fetch, and # [oauth2_client] UPDATE_AVATAR is off here) and modules/auth/openid (OpenID 2.0, unused). # So keycloak. resolving to the private Gateway address in-cluster does not matter. # * this instance has no webhooks and no mirrors (measured 2026-11-07), and 1.27 already refused # private webhook targets by default, so nothing that worked before is newly denied. # Adding an entry here would only WIDEN the webhook SSRF surface. Add the narrowest CIDR (e.g. the # Gateway address /32) only if a real private webhook target appears. - name: GITEA__security__EGRESS_MODE value: lax # Gitea 28 (go-gitea/gitea#39400) disables LOCAL self-registration unless DISABLE_REGISTRATION is # explicitly false; 1.27 defaulted it open. Declared true: accounts arrive via Keycloak only. # OAuth2 auto-registration below is unaffected — routers/web/auth/oauth.go gates it on # ALLOW_ONLY_INTERNAL_REGISTRATION and ENABLE_AUTO_REGISTRATION, not on DISABLE_REGISTRATION. - name: GITEA__service__DISABLE_REGISTRATION value: "true" - name: GITEA_ADMIN_USERNAME # break-glass local admin creds (consumed by the postStart hook) valueFrom: secretKeyRef: name: gitea-bootstrap key: admin-username - name: GITEA_ADMIN_PASSWORD valueFrom: secretKeyRef: name: gitea-bootstrap key: admin-password # --- Keycloak native OIDC SSO (auth.mode oidc; docs/15 "Native OIDC"; R-IDP-02/06). The confidential # `gitea` realm client's secret is mirrored into this namespace's gitea-bootstrap Secret (the same # value SOPS-held in keycloak-realm-clients); the postStart hook registers it as the "keycloak" # OAuth2 login source. The LOCAL admin above stays as the break-glass path (docs/15). --- - name: GITEA_OIDC_CLIENT_SECRET valueFrom: secretKeyRef: name: gitea-bootstrap key: oidc-client-secret # app.ini [oauth2_client]: auto-provision a Gitea account on first Keycloak login and link it to an # existing local account by verified email, so SSO users don't need a pre-created local user. - name: GITEA__oauth2_client__ENABLE_AUTO_REGISTRATION value: "true" - name: GITEA__oauth2_client__ACCOUNT_LINKING value: auto - name: GITEA__oauth2_client__USERNAME value: preferred_username lifecycle: postStart: exec: command: - /bin/sh - -c - | # Idempotent break-glass admin bootstrap. Runs alongside the gitea entrypoint # (which, with INSTALL_LOCK=true, headless-installs + migrates the DB on first # boot). Waits for the DB, then creates the admin only if it does not exist. set -u export HOME=/data/git i=0 until su-exec git gitea admin user list >/dev/null 2>&1; do i=$((i + 1)) [ "$i" -ge 150 ] && { echo "[gitea-bootstrap] DB not ready after wait; skipping" >&2; exit 0; } sleep 2 done if su-exec git gitea admin user list --admin 2>/dev/null | awk 'NR>1{print $2}' | grep -qx "$GITEA_ADMIN_USERNAME"; then echo "[gitea-bootstrap] admin '$GITEA_ADMIN_USERNAME' already present" else n=0 until su-exec git gitea admin user create \ --username "$GITEA_ADMIN_USERNAME" \ --password "$GITEA_ADMIN_PASSWORD" \ --email "$GITEA_ADMIN_USERNAME@home.local" \ --admin --must-change-password=false; do n=$((n + 1)) [ "$n" -ge 10 ] && { echo "[gitea-bootstrap] admin create failed after retries" >&2; break; } sleep 3 done echo "[gitea-bootstrap] created break-glass admin '$GITEA_ADMIN_USERNAME'" fi # --- Keycloak OIDC login source (idempotent add-or-update; R-IDP-02/06) --- # Registers the confidential `gitea` client as the OAuth2 login source "keycloak" against the # homelab realm's discovery doc. The `groups` claim maps platform-admins -> Gitea admin # (--admin-group). Missing secret => skip (break-glass admin still works), so an unfilled # secret never crashes the pod. # # HARDENED (was a silent first-boot RACE): this step used to fire before the app/network was # ready and swallow its failure (|| echo ... >&2, exit 0), so SSO silently never registered. # Now it (1) waits — bounded, ~90s — for the Keycloak discovery doc to serve HTTP 200, # (2) registers (update-in-place if present, else create), (3) VERIFIES the source actually # persisted and retries the whole cycle, and (4) on give-up logs LOUDLY to stderr AND a marker # file on the persistent /data volume WITHOUT crashlooping the pod (break-glass admin stays # usable). The verify-then-retry loop is the real guarantee against "exit 0 but nothing stuck". GITEA_DISCOVERY_URL=https://keycloak.${BASE_DOMAIN}/realms/homelab/.well-known/openid-configuration gitea_oidc_fail() { echo "[gitea-oidc] FAILED to register 'keycloak' login source: $1" >&2 { echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) [gitea-oidc] FAILED: $1"; } >> /data/gitea-oidc-bootstrap.log 2>/dev/null || true } # NOTE the `$$` escape below — it is load-bearing. apps-wave-0 declares postBuild. # substituteFrom cluster-vars, so Flux envsubst rewrites this whole manifest at RECONCILE # time, before any container runs. Written unescaped, `$${GITEA_OIDC_CLIENT_SECRET:-}` is not # a cluster-var, so envsubst blanked it and the guard reconciled as `[ -z "" ]` — ALWAYS TRUE # — making every line below (discovery wait, add/update-oauth, verify-then-retry) DEAD CODE on # every boot since 8738871 (2026-08-27). SSO kept working only because the login source was # already in the DB; a PVC rebuild or fresh cluster would have come up with NO OIDC at all. # Flux STRICT substitution cannot catch this — it EXEMPTS default-operator forms by design. # `$${VAR}` is the escape: envsubst emits it back as a plain brace expansion for the shell. # Bare `$VAR` and `$(cmd)` are left alone and need NO escaping — which is why the # `$GITEA_OIDC_CLIENT_SECRET` uses below always worked and only the guards broke. # Contract: clusters/base/apps-vms.yaml. Enforced at render time by catalog/render.py. if [ -z "$${GITEA_OIDC_CLIENT_SECRET:-}" ]; then echo "[gitea-oidc] GITEA_OIDC_CLIENT_SECRET unset; skipping OIDC source (break-glass admin only)" else # (1) Wait up to ~90s for the Keycloak discovery doc to be reachable (HTTP 200). d=0; disc_ok=0 while [ "$d" -lt 45 ]; do if [ "$(curl -sS -o /dev/null -w '%{http_code}' --max-time 5 "$GITEA_DISCOVERY_URL" 2>/dev/null)" = "200" ]; then disc_ok=1; break fi d=$((d + 1)); sleep 2 done [ "$disc_ok" = 1 ] || gitea_oidc_fail "Keycloak discovery endpoint not reachable (HTTP 200) within timeout" # (2)+(3) Register (update-in-place or create) then VERIFY it persisted; retry the whole cycle. r=0; reg_ok=0 while [ "$r" -lt 10 ]; do oidc_id=$(su-exec git gitea admin auth list 2>/dev/null | awk '$2=="keycloak"{print $1; exit}') if [ -n "$oidc_id" ]; then su-exec git gitea admin auth update-oauth --id "$oidc_id" \ --name keycloak --provider openidConnect --key gitea \ --secret "$GITEA_OIDC_CLIENT_SECRET" \ --auto-discover-url "$GITEA_DISCOVERY_URL" \ --scopes "openid email profile" --group-claim-name groups --admin-group platform-admins >/dev/null 2>&1 || true else su-exec git gitea admin auth add-oauth \ --name keycloak --provider openidConnect --key gitea \ --secret "$GITEA_OIDC_CLIENT_SECRET" \ --auto-discover-url "$GITEA_DISCOVERY_URL" \ --scopes "openid email profile" --group-claim-name groups --admin-group platform-admins >/dev/null 2>&1 || true fi # VERIFY: the "keycloak" source must actually be listed now (guards the exit-0-but-nothing-persisted race). if su-exec git gitea admin auth list 2>/dev/null | awk '$2=="keycloak"{f=1} END{exit !f}'; then reg_ok=1; break fi r=$((r + 1)); sleep 3 done if [ "$reg_ok" = 1 ]; then echo "[gitea-oidc] login source 'keycloak' registered and verified" else gitea_oidc_fail "add/update-oauth ran but the 'keycloak' source did not persist after retries" fi fi # A postStart hook MUST exit 0 — a non-zero exit makes the kubelet KILL the container # (FailedPostStartHook -> CrashLoopBackOff). This hook already ends 0 on both paths, but # it is made EXPLICIT so the guarantee does not depend on whatever statement happens to # be last. nextcloud's equivalent hook did depend on that and crashlooped on 2026-09-01, # the moment its OIDC guard was fixed and the block actually ran for the first time. exit 0 exposure: public # edge-forwarded git host at gitea.${BASE_DOMAIN} (docs/09; R-DNS/R-NET) egress: internet # normal WAN egress (docs/03; R-NET-05) storage: - name: data size: 10Gi class: longhorn # repos + SQLite DB on 3-replica Longhorn; irreplaceable (docs/04; R-STATE-02) mountPath: /data # Gitea keeps ALL state (repos, DB, app.ini, INTERNAL_TOKEN) directly under /data; # the default /data/ would strand real state on the ephemeral rootfs (R-STATE-01) dependencies: [] # default-deny: reaches only DNS; git clients reach it north-south via the Gateway (R-NET-01/02) ingressRoutes: - host: gitea.${BASE_DOMAIN} port: 3000 # Gitea HTTP web/API; TLS terminates at the shared Gateway (docs/08, docs/09) # THIS IS WHAT KEEPS PROD'S GITOPS OUT OF A DEADLOCK — it is not an incidental allowance. Remove it # and prod cannot reconcile AT ALL, in a way that is very hard to diagnose from cold. # # Prod's Flux source is THIS SERVICE, not a public hostname: # clusters/prod/kustomization.yaml -> GitRepository.spec.url = # http://gitea.gitea.svc.cluster.local:3000/Home-Network/home-network-k3s-impl.git # chosen deliberately so prod's GitOps depends on NEITHER public DNS, Cloudflare, prod's own # Gateway, NOR ${BASE_DOMAIN} — a future domain change cannot sever prod from its own source. That # file documents the other half of this pair at length; the two must be read together. # # The only other ingress this entry renders is gitea-allow-gateway-ingress, which admits # `fromEntities: [ingress]` — the shared Gateway's Envoy proxy — and nothing else. Flux's # source-controller dials the ClusterIP DIRECTLY, so under the namespace default-deny (R-NET-01) its # clone was DROPPED (`wget` from source-controller timed out; srcReady=False, "unable to list # remote"). The deadlock that made this urgent: the HTTPRoute that would have served the new public # name after the 2026-09-13 apex flip lives in a DOWNSTREAM tier, which cannot apply while the # source is down — source down -> routes never rename -> source stays down. It was broken on # 2026-09-13 with a live `kubectl apply` of a hand-written gitea-allow-flux-ingress, i.e. with a # cluster object that existed OUTSIDE git. This declaration renders that object, so one # `kubectl delete` or one cluster rebuild can no longer put prod back in the deadlock. inClusterClients: # `name` is pinned to `flux` (rather than defaulting to the namespace) so the rendered policy name # equals the live object's, `gitea-allow-flux-ingress`, and Flux ADOPTS it in place: no # near-duplicate policy beside it, and no window in which prod's own source has no allow. - name: flux namespace: flux-system # matched by NAMESPACE alone — the flux controllers' pod labels belong # to upstream's manifests, not to this platform's contract port: 3000 # ONLY the Gitea HTTP port — the same single port the Gateway allow opens # The PM agent service (platform/pm-agent/). Its dispatcher reads issues and posts comments # over this port, and every worker Job clones, pushes and opens pull requests over it. Like # Flux's source-controller above, it dials the ClusterIP DIRECTLY — deliberately, so the # agent's path to the repository depends on neither public DNS, nor Cloudflare, nor the # shared Gateway, nor ${BASE_DOMAIN}. And exactly as for Flux, that means this namespace's # own default-deny (R-NET-01) DROPS the connection unless it is declared here: without this # entry the dispatcher's egress allow is satisfied and the packet is still discarded at the # Gitea end, which presents as an authentication timeout rather than as a policy denial. # That is the 2026-09-13 lesson documented at length above, applied before it is relearned. - name: pm-agent namespace: pm-agent # matched by NAMESPACE alone, so the dispatcher Deployment and every # ephemeral worker Job are covered by the one declaration port: 3000 # ONLY the Gitea HTTP port auth: mode: oidc # native Keycloak OIDC client (docs/13 SSO row, docs/15 "Native OIDC" set; R-IDP-02) groups: - platform-admins # access scoped to an authorized group (R-IDP-06) # WIRED (native OIDC): the Keycloak `gitea` confidential client (platform/keycloak/keycloak-realm- # import.yaml + live kcadm) is registered as Gitea's "keycloak" OAuth2 login source by the postStart # hook, using the client secret mirrored into the gitea-bootstrap Secret (oidc-client-secret; same # value SOPS-held in keycloak-realm-clients). platform-admins (groups claim) -> Gitea admin. Gitea # KEEPS a LOCAL admin as the break-glass path so an IdP outage never locks the operator out of # recovery tools (docs/15 "Availability caveat"). No secret is embedded here. update: channel: stable # R-UPD-01 range: ">=28.0.0 <29.0.0" # Gitea dropped the "1." prefix at 28.0.0 (successor of 1.27); stay on 28.x cadence: "0 4 * * *" # R-UPD-03. INERT FOR THIS SERVICE, and it must stay that way: this flag renders only an annotation # on a Flux ImagePolicy, and apps/ is rendered --no-image-automation (ci/render-golden.sh), so no # ImagePolicy exists and the Deployment (strategy Recreate) is never auto-reverted. That matters # because a Gitea upgrade (1.27 -> 28 included) MIGRATES THE SQLITE DB ONE-WAY: an image revert onto # a migrated DB is not a rollback. Roll Gitea back by RESTORING THE DATA PVC from its pre-upgrade backup. autoRollbackOnFailedHealth: true observability: metrics: true slis: - up # git host down is a critical alert (docs/10; R-OBS-01) backup: policy: daily-offcluster # DR-critical: repos/DB backed up off-cluster + mirrored (docs/13, docs/04; R-STATE-03)