feat(catalog): add SD env/envFrom/lifecycle; retire gitea bootstrap patch

Extend the ServiceDefinition contract with optional per-container `env`,
`envFrom`, and `lifecycle` fields (k8s EnvVar / EnvFromSource / Lifecycle
shapes) so headless-install / DB-backed apps configure themselves through
the SD instead of a supplementary kustomize patch.

- schema: bump 1.1.0 -> 1.2.0; add optional env/envFrom/lifecycle to the
  container kind (literal + secretKeyRef/configMapKeyRef; secretRef/
  configMapRef; postStart/preStop exec via $defs/execHandler). Existing
  fields unchanged; all fields optional -> absent renders byte-identical.
- render.py: emit env/envFrom/lifecycle verbatim onto the Deployment app
  container when present; --selftest coverage for the new fields.
- catalog/gitea.yaml: fold the headless-install env (INSTALL_LOCK, fixed
  SECRET_KEY via secretKeyRef, DOMAIN/ROOT_URL, actions, break-glass admin
  creds) + the idempotent postStart break-glass admin bootstrap inline.
- apps/gitea.yaml: re-rendered with env/lifecycle inline (effective pod
  spec provably equivalent to base+patch).
- DELETE apps/gitea-bootstrap-patch.yaml + drop it from apps/kustomization
  patches. gitea-bootstrap-secret.sops.yaml kept (still sourced by the env).

Proves the renderer feature by retiring the known TODO; unblocks env + DB
onboarding for nextcloud/semaphore/unifi/minecraft. All other rendered apps
byte-identical. Gates: selftest PASS, l0 17/17 + schema, l1 goldens, lint.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

apps/gitea-bootstrap-patch.yaml

@@ -1,101 +0,0 @@
----
-# Gitea headless-install bootstrap patch (WS-APPS, Phase 6).
-#
-# WHY THIS IS A SUPPLEMENTARY MANIFEST (renderer follow-up):
-# The ServiceDefinition schema/renderer (catalog/render.py) does not yet express per-container
-# `env`, `envFrom`, `lifecycle`, or `initContainers`. Gitea needs all three to come up INSTALLED
-# without the interactive web wizard and with a persistent SECRET_KEY. Rather than widen the
-# renderer contract for a single app (out of scope for this task), this is a minimal, Gitea-only
-# kustomize strategic-merge patch layered onto the rendered apps/gitea.yaml Deployment. It is
-# fully GitOps-reconciled (Flux applies it), so nothing here is a manual pod hack Flux would revert.
-# TODO(renderer): fold generic `env`/`envFrom`/`lifecycle` support into the ServiceDefinition
-#                 schema + render.py, then delete this patch.
-#
-# WHAT IT DOES:
-#   - GITEA__security__INSTALL_LOCK=true  -> skip the first-run web installer (headless install).
-#   - GITEA__security__SECRET_KEY (from the gitea-bootstrap SOPS Secret) -> a FIXED, persistent
-#     SECRET_KEY. It must NOT regenerate per restart or previously-encrypted data breaks; sourcing
-#     it from a committed SOPS secret pins it across pod restarts/reschedules.
-#   - GITEA__server__ROOT_URL / __DOMAIN -> correct public clone/redirect URLs for the git host
-#     that TLS-terminates at the shared Gateway (gitea.home.keithjackson.dev).
-#   - GITEA__actions__ENABLED=true -> turn on Gitea Actions explicitly (Phase 6 CI/CD, WS-GITOSS).
-#     Gitea 1.21+ defaults Actions ON, but pinning it here makes the CI merge gate a declared,
-#     reviewable property of the deployment rather than an implicit upstream default that a future
-#     version bump could flip. The in-cluster act_runner (ns gitea-actions) registers against it.
-#   - GITEA__actions__DEFAULT_ACTIONS_URL=github -> resolve `uses: <org>/<action>@ref` (e.g.
-#     actions/checkout) from github.com. The cluster HAS internet egress (LE/Cloudflare/ghcr already
-#     in use), so github-sourced actions work. A fully self-hosted/air-gapped value would require
-#     mirroring each action repo into this Gitea first; the cluster is not air-gapped, so `github`
-#     is the pragmatic, working choice (WS-GITOSS Phase-6 note).
-#   - A postStart lifecycle hook creates a BREAK-GLASS local admin idempotently (only if absent),
-#     so an IdP outage never locks the operator out of recovery tools (docs/15 availability caveat).
-#     All state (repos, SQLite DB, app.ini, INTERNAL_TOKEN) lands on the /data Longhorn PVC.
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: gitea
-  namespace: gitea
-spec:
-  template:
-    spec:
-      containers:
-        - name: gitea
-          env:
-            - name: GITEA__security__INSTALL_LOCK
-              value: "true"
-            - name: GITEA__security__SECRET_KEY
-              valueFrom:
-                secretKeyRef:
-                  name: gitea-bootstrap
-                  key: secret-key
-            - name: GITEA__server__DOMAIN
-              value: gitea.home.keithjackson.dev
-            - name: GITEA__server__ROOT_URL
-              value: https://gitea.home.keithjackson.dev/
-            - name: GITEA__actions__ENABLED
-              value: "true"
-            - name: GITEA__actions__DEFAULT_ACTIONS_URL
-              value: github
-            - name: GITEA_ADMIN_USERNAME
-              valueFrom:
-                secretKeyRef:
-                  name: gitea-bootstrap
-                  key: admin-username
-            - name: GITEA_ADMIN_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: gitea-bootstrap
-                  key: admin-password
-          lifecycle:
-            postStart:
-              exec:
-                command:
-                  - /bin/sh
-                  - -c
-                  - |
-                    # Idempotent break-glass admin bootstrap. Runs alongside the gitea entrypoint
-                    # (which, with INSTALL_LOCK=true, headless-installs + migrates the DB on first
-                    # boot). Waits for the DB, then creates the admin only if it does not exist.
-                    set -u
-                    export HOME=/data/git
-                    i=0
-                    until su-exec git gitea admin user list >/dev/null 2>&1; do
-                      i=$((i + 1))
-                      [ "$i" -ge 150 ] && { echo "[gitea-bootstrap] DB not ready after wait; skipping" >&2; exit 0; }
-                      sleep 2
-                    done
-                    if su-exec git gitea admin user list --admin 2>/dev/null | awk 'NR>1{print $2}' | grep -qx "$GITEA_ADMIN_USERNAME"; then
-                      echo "[gitea-bootstrap] admin '$GITEA_ADMIN_USERNAME' already present"
-                      exit 0
-                    fi
-                    n=0
-                    until su-exec git gitea admin user create \
-                      --username "$GITEA_ADMIN_USERNAME" \
-                      --password "$GITEA_ADMIN_PASSWORD" \
-                      --email "[email protected]" \
-                      --admin --must-change-password=false; do
-                      n=$((n + 1))
-                      [ "$n" -ge 10 ] && { echo "[gitea-bootstrap] admin create failed after retries" >&2; exit 0; }
-                      sleep 3
-                    done
-                    echo "[gitea-bootstrap] created break-glass admin '$GITEA_ADMIN_USERNAME'"

apps/gitea.yaml

@@ -81,9 +81,42 @@ spec:
         image: gitea/gitea:1.24.3
         ports:
         - containerPort: 3000
+        env:
+        - name: GITEA__security__INSTALL_LOCK
+          value: 'true'
+        - name: GITEA__security__SECRET_KEY
+          valueFrom:
+            secretKeyRef:
+              name: gitea-bootstrap
+              key: secret-key
+        - name: GITEA__server__DOMAIN
+          value: gitea.home.keithjackson.dev
+        - name: GITEA__server__ROOT_URL
+          value: https://gitea.home.keithjackson.dev/
+        - name: GITEA__actions__ENABLED
+          value: 'true'
+        - name: GITEA__actions__DEFAULT_ACTIONS_URL
+          value: github
+        - name: GITEA_ADMIN_USERNAME
+          valueFrom:
+            secretKeyRef:
+              name: gitea-bootstrap
+              key: admin-username
+        - name: GITEA_ADMIN_PASSWORD
+          valueFrom:
+            secretKeyRef:
+              name: gitea-bootstrap
+              key: admin-password
         volumeMounts:
         - name: data
           mountPath: /data
+        lifecycle:
+          postStart:
+            exec:
+              command:
+              - /bin/sh
+              - -c
+              - "# Idempotent break-glass admin bootstrap. Runs alongside the gitea entrypoint\n# (which, with INSTALL_LOCK=true, headless-installs + migrates the DB on first\n# boot). Waits for the DB, then creates the admin only if it does not exist.\nset -u\nexport HOME=/data/git\ni=0\nuntil su-exec git gitea admin user list >/dev/null 2>&1; do\n  i=$((i + 1))\n  [ \"$i\" -ge 150 ] && { echo \"[gitea-bootstrap] DB not ready after wait; skipping\" >&2; exit 0; }\n  sleep 2\ndone\nif su-exec git gitea admin user list --admin 2>/dev/null | awk 'NR>1{print $2}' | grep -qx \"$GITEA_ADMIN_USERNAME\"; then\n  echo \"[gitea-bootstrap] admin '$GITEA_ADMIN_USERNAME' already present\"\n  exit 0\nfi\nn=0\nuntil su-exec git gitea admin user create \\\n  --username \"$GITEA_ADMIN_USERNAME\" \\\n  --password \"$GITEA_ADMIN_PASSWORD\" \\\n  --email \"[email protected]\" \\\n  --admin --must-change-password=false; do\n  n=$((n + 1))\n  [ \"$n\" -ge 10 ] && { echo \"[gitea-bootstrap] admin create failed after retries\" >&2; exit 0; }\n  sleep 3\ndone\necho \"[gitea-bootstrap] created break-glass admin '$GITEA_ADMIN_USERNAME'\"\n"
       volumes:
       - name: data
         persistentVolumeClaim:

apps/kustomization.yaml

@@ -17,7 +17,8 @@ resources:
   - qbittorrent.yaml
   - jackett.yaml
   - seerr.yaml
-# Gitea-only supplementary patch: headless-install env + break-glass admin postStart hook.
+# Gitea's headless-install env + break-glass admin postStart hook are now first-class in the
-# The renderer does not yet emit env/lifecycle (renderer follow-up); see gitea-bootstrap-patch.yaml.
+# ServiceDefinition (catalog/gitea.yaml env/lifecycle, schema >=1.2.0) and rendered inline into
-patches:
+# apps/gitea.yaml, so the former gitea-bootstrap-patch.yaml strategic-merge patch is retired.
-  - path: gitea-bootstrap-patch.yaml
+# gitea-bootstrap-secret.sops.yaml is KEPT — the env still sources the fixed SECRET_KEY + admin
+# creds from it via secretKeyRef.

catalog/gitea.yaml

@@ -11,6 +11,69 @@ name: gitea
 kind: container
 source:
   image: gitea/gitea:1.24.3     # pinned recent stable tag (R-UPD-01); update.range governs advancement
+# Headless-install bootstrap (formerly apps/gitea-bootstrap-patch.yaml; now first-class via the
+# ServiceDefinition env/envFrom/lifecycle fields, schema >=1.2.0). Gitea must come up INSTALLED
+# without the interactive web wizard and with a FIXED, persistent SECRET_KEY, so all of this is
+# declared here and rendered verbatim onto the app container by catalog/render.py.
+env:
+  - name: GITEA__security__INSTALL_LOCK      # skip the first-run web installer (headless install)
+    value: "true"
+  - name: GITEA__security__SECRET_KEY        # FIXED persistent key; MUST NOT regenerate per restart or
+    valueFrom:                               # previously-encrypted data breaks. Pinned via the SOPS Secret.
+      secretKeyRef:
+        name: gitea-bootstrap
+        key: secret-key
+  - name: GITEA__server__DOMAIN              # correct public clone/redirect URLs for the TLS-terminated host
+    value: gitea.home.keithjackson.dev
+  - name: GITEA__server__ROOT_URL
+    value: https://gitea.home.keithjackson.dev/
+  - name: GITEA__actions__ENABLED            # Phase 6 CI/CD: declared (not implicit upstream default) so the
+    value: "true"                            # in-cluster act_runner (ns gitea-actions) registers against it
+  - name: GITEA__actions__DEFAULT_ACTIONS_URL  # resolve `uses: <org>/<action>@ref` from github.com (cluster
+    value: github                              # has internet egress; not air-gapped)
+  - name: GITEA_ADMIN_USERNAME               # break-glass local admin creds (consumed by the postStart hook)
+    valueFrom:
+      secretKeyRef:
+        name: gitea-bootstrap
+        key: admin-username
+  - name: GITEA_ADMIN_PASSWORD
+    valueFrom:
+      secretKeyRef:
+        name: gitea-bootstrap
+        key: admin-password
+lifecycle:
+  postStart:
+    exec:
+      command:
+        - /bin/sh
+        - -c
+        - |
+          # Idempotent break-glass admin bootstrap. Runs alongside the gitea entrypoint
+          # (which, with INSTALL_LOCK=true, headless-installs + migrates the DB on first
+          # boot). Waits for the DB, then creates the admin only if it does not exist.
+          set -u
+          export HOME=/data/git
+          i=0
+          until su-exec git gitea admin user list >/dev/null 2>&1; do
+            i=$((i + 1))
+            [ "$i" -ge 150 ] && { echo "[gitea-bootstrap] DB not ready after wait; skipping" >&2; exit 0; }
+            sleep 2
+          done
+          if su-exec git gitea admin user list --admin 2>/dev/null | awk 'NR>1{print $2}' | grep -qx "$GITEA_ADMIN_USERNAME"; then
+            echo "[gitea-bootstrap] admin '$GITEA_ADMIN_USERNAME' already present"
+            exit 0
+          fi
+          n=0
+          until su-exec git gitea admin user create \
+            --username "$GITEA_ADMIN_USERNAME" \
+            --password "$GITEA_ADMIN_PASSWORD" \
+            --email "[email protected]" \
+            --admin --must-change-password=false; do
+            n=$((n + 1))
+            [ "$n" -ge 10 ] && { echo "[gitea-bootstrap] admin create failed after retries" >&2; exit 0; }
+            sleep 3
+          done
+          echo "[gitea-bootstrap] created break-glass admin '$GITEA_ADMIN_USERNAME'"
 exposure: public                # edge-forwarded git host at gitea.home.keithjackson.dev (docs/09; R-DNS/R-NET)
 egress: internet                # normal WAN egress (docs/03; R-NET-05)
 storage:

catalog/render.py

@@ -218,6 +218,16 @@ def r_deployment(s: dict) -> dict:
     if ports:
         container["ports"] = [{"containerPort": p["port"]} for p in ports]
 
+    # env / envFrom (schema-optional): the k8s EnvVar / EnvFromSource lists are emitted
+    # VERBATIM onto the app container. This is what lets a headless-install/DB-backed app
+    # (gitea, nextcloud, semaphore, unifi, minecraft) configure itself and pull a fixed
+    # secret (e.g. gitea's SECRET_KEY via secretKeyRef) without a supplementary patch.
+    # Absent => the key is never set, so apps that declare no env render byte-identical.
+    if s.get("env"):
+        container["env"] = s["env"]
+    if s.get("envFrom"):
+        container["envFrom"] = s["envFrom"]
+
     volume_mounts, volumes = [], []
     for vol in s["storage"]:
         # Default mount is /data/<name>; an explicit `mountPath` (schema-optional) overrides
@@ -234,6 +244,12 @@ def r_deployment(s: dict) -> dict:
     if volume_mounts:
         container["volumeMounts"] = volume_mounts
 
+    # lifecycle hooks (schema-optional): the k8s Lifecycle (postStart/preStop exec) is
+    # emitted VERBATIM. Used for idempotent break-glass/bootstrap steps (e.g. gitea's
+    # postStart exec that seeds a local admin). Absent => byte-identical for other apps.
+    if s.get("lifecycle"):
+        container["lifecycle"] = s["lifecycle"]
+
     pod_spec = {"containers": [container]}
     if volumes:
         pod_spec["volumes"] = volumes
@@ -1100,6 +1116,55 @@ def _selftest() -> int:
     check("VirtualMachine" in vm_kinds, "vm: missing VirtualMachine")
     check("DataVolume" in vm_kinds, "vm: missing CDI DataVolume")
 
+    # 6) env / envFrom / lifecycle -> emitted VERBATIM onto the app container (T-APPS).
+    envd_docs = render(
+        {
+            "name": "envd",
+            "source": {"image": "a:1"},
+            "env": [
+                {"name": "PLAIN", "value": "literal"},
+                {
+                    "name": "SECRET",
+                    "valueFrom": {"secretKeyRef": {"name": "sec", "key": "k"}},
+                },
+            ],
+            "envFrom": [{"secretRef": {"name": "bulk"}}],
+            "lifecycle": {
+                "postStart": {"exec": {"command": ["/bin/sh", "-c", "echo hi"]}}
+            },
+        }
+    )
+    ec = next(
+        d[1] for d in envd_docs if d[1]["kind"] == "Deployment"
+    )["spec"]["template"]["spec"]["containers"][0]
+    check(
+        ec.get("env") == [
+            {"name": "PLAIN", "value": "literal"},
+            {"name": "SECRET", "valueFrom": {"secretKeyRef": {"name": "sec", "key": "k"}}},
+        ],
+        "env must render verbatim onto the container (literal + secretKeyRef)",
+    )
+    check(
+        ec.get("envFrom") == [{"secretRef": {"name": "bulk"}}],
+        "envFrom must render verbatim onto the container",
+    )
+    check(
+        ec.get("lifecycle")
+        == {"postStart": {"exec": {"command": ["/bin/sh", "-c", "echo hi"]}}},
+        "lifecycle must render verbatim onto the container",
+    )
+    # Absence => the keys are never set (existing apps stay byte-identical).
+    plain_c = next(
+        d[1] for d in render({"name": "plain", "source": {"image": "a:1"}})
+        if d[1]["kind"] == "Deployment"
+    )["spec"]["template"]["spec"]["containers"][0]
+    check(
+        "env" not in plain_c
+        and "envFrom" not in plain_c
+        and "lifecycle" not in plain_c,
+        "an SD without env/envFrom/lifecycle must not gain those container keys",
+    )
+
     if failures:
         for f in failures:
             sys.stderr.write(f"SELFTEST FAIL: {f}\n")

schemas/servicedefinition.schema.json

@@ -2,7 +2,7 @@
   "$schema": "https://json-schema.org/draft/2020-12/schema",
   "$id": "https://home.keithjackson.dev/schemas/servicedefinition.schema.json",
   "title": "ServiceDefinition",
-  "description": "Operator-facing onboarding contract for the home k3s + Longhorn platform. One ServiceDefinition per file under catalog/. Captures INTENT; the platform compiles it into concrete resources (see schemas/renderer-contract.md). Canonical source of truth: spec docs/13-service-catalog.md. Serves R-GITOPS-02; ties into R-NET-*, R-STATE-*, R-TLS-*, R-DNS-*, R-OBS-*, R-UPD-*, R-IDP-*. Schema version: 1.1.0 (adds shape invariant exposure:public => auth.mode!=none, docs/13 v1.1 / docs/15 / R-IDP-02).",
+  "description": "Operator-facing onboarding contract for the home k3s + Longhorn platform. One ServiceDefinition per file under catalog/. Captures INTENT; the platform compiles it into concrete resources (see schemas/renderer-contract.md). Canonical source of truth: spec docs/13-service-catalog.md. Serves R-GITOPS-02; ties into R-NET-*, R-STATE-*, R-TLS-*, R-DNS-*, R-OBS-*, R-UPD-*, R-IDP-*. Schema version: 1.2.0 (adds optional per-container env/envFrom/lifecycle for the container kind — k8s EnvVar/EnvFromSource/Lifecycle shapes emitted verbatim onto the app container; unblocks headless-install/DB-backed apps e.g. gitea/nextcloud/semaphore/unifi/minecraft). 1.1.0 added the shape invariant exposure:public => auth.mode!=none (docs/13 v1.1 / docs/15 / R-IDP-02).",
   "type": "object",
   "additionalProperties": false,
   "required": ["name", "source"],
@@ -38,6 +38,93 @@
         }
       }
     },
+    "env": {
+      "type": "array",
+      "description": "Optional per-container environment variables for the container kind (docs/13). Mirrors the k8s EnvVar shape and is emitted verbatim onto the app container: each entry is either a literal {name, value} or a {name, valueFrom: {secretKeyRef|configMapKeyRef}} sourced from a Secret/ConfigMap key. Absent => no env (existing apps render byte-identical). Ignored for kind=vm (VMs configure via cloud-init).",
+      "default": [],
+      "items": {
+        "type": "object",
+        "additionalProperties": false,
+        "required": ["name"],
+        "properties": {
+          "name": {
+            "type": "string",
+            "description": "Environment variable name (C_IDENTIFIER).",
+            "pattern": "^[A-Za-z_][A-Za-z0-9_]*$"
+          },
+          "value": {
+            "type": "string",
+            "description": "Literal value. Mutually exclusive with valueFrom."
+          },
+          "valueFrom": {
+            "type": "object",
+            "description": "Source the value from a Secret or ConfigMap key (exactly one of secretKeyRef/configMapKeyRef).",
+            "additionalProperties": false,
+            "minProperties": 1,
+            "maxProperties": 1,
+            "properties": {
+              "secretKeyRef": {
+                "type": "object",
+                "additionalProperties": false,
+                "required": ["name", "key"],
+                "properties": {
+                  "name": { "type": "string", "minLength": 1 },
+                  "key": { "type": "string", "minLength": 1 }
+                }
+              },
+              "configMapKeyRef": {
+                "type": "object",
+                "additionalProperties": false,
+                "required": ["name", "key"],
+                "properties": {
+                  "name": { "type": "string", "minLength": 1 },
+                  "key": { "type": "string", "minLength": 1 }
+                }
+              }
+            }
+          }
+        },
+        "oneOf": [
+          { "required": ["value"], "not": { "required": ["valueFrom"] } },
+          { "required": ["valueFrom"], "not": { "required": ["value"] } }
+        ]
+      }
+    },
+    "envFrom": {
+      "type": "array",
+      "description": "Optional bulk env sources for the container kind. Mirrors the k8s EnvFromSource shape and is emitted verbatim onto the app container: each entry pulls ALL keys from a Secret or ConfigMap (exactly one of secretRef/configMapRef). Absent => no envFrom (existing apps render byte-identical). Ignored for kind=vm.",
+      "default": [],
+      "items": {
+        "type": "object",
+        "additionalProperties": false,
+        "minProperties": 1,
+        "maxProperties": 1,
+        "properties": {
+          "secretRef": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": ["name"],
+            "properties": { "name": { "type": "string", "minLength": 1 } }
+          },
+          "configMapRef": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": ["name"],
+            "properties": { "name": { "type": "string", "minLength": 1 } }
+          }
+        }
+      }
+    },
+    "lifecycle": {
+      "type": "object",
+      "description": "Optional container lifecycle hooks for the container kind (docs/13). Mirrors the k8s Lifecycle shape and is emitted verbatim onto the app container. Used for idempotent break-glass/bootstrap steps (e.g. gitea's postStart exec that seeds a local admin). Absent => no lifecycle (existing apps render byte-identical). Ignored for kind=vm.",
+      "additionalProperties": false,
+      "minProperties": 1,
+      "properties": {
+        "postStart": { "$ref": "#/$defs/execHandler" },
+        "preStop": { "$ref": "#/$defs/execHandler" }
+      }
+    },
     "exposure": {
       "type": "string",
       "description": "internal = LAN-only (internal DNS + Gateway on LAN); public = additionally edge port-forwarded 80/443 (docs/09; R-DNS/R-NET).",
@@ -264,5 +351,28 @@
         }
       }
     }
-  ]
+  ],
+  "$defs": {
+    "execHandler": {
+      "type": "object",
+      "description": "A container lifecycle hook. Only the exec form is supported (a command run inside the container); mirrors k8s LifecycleHandler.exec.",
+      "additionalProperties": false,
+      "required": ["exec"],
+      "properties": {
+        "exec": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": ["command"],
+          "properties": {
+            "command": {
+              "type": "array",
+              "description": "Command line to execute inside the container. Not run in a shell unless the command itself invokes one.",
+              "minItems": 1,
+              "items": { "type": "string" }
+            }
+          }
+        }
+      }
+    }
+  }
 }