fix(keycloak): one owner for the keycloak ServiceMonitor (26.8 operator 409 loop blocks Flux)

The 26.x Keycloak operator creates its own ServiceMonitor named after the
CR (keycloak/keycloak) when metrics are enabled; spec.serviceMonitor.enabled
defaults true. That collides with the Flux-owned platform/keycloak/
servicemonitor.yaml: the operator's PATCH 409s, it exhausts retries, the CR
stays Ready=False/HasErrors=True, and platform-config (wait: true) stalls
every downstream Kustomization.

Keep the repo's ServiceMonitor (conformance L0/L2 and the dashboard are
written against it; the operator's scrapes the same port 9000 /metrics) and
set spec.serviceMonitor.enabled: false.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

decisions/2026-11-08-keycloak-servicemonitor-owner.md

@@ -0,0 +1,60 @@
+---
+date: 2026-11-08
+branch: fix/keycloak-servicemonitor-owner
+title: One owner for the keycloak ServiceMonitor — disable the 26.8 operator's (409 loop stalled platform-config)
+---
+
+## What changed
+
+- `platform/keycloak/keycloak-instance.yaml`: added `spec.serviceMonitor.enabled: false`.
+  The 26.x Keycloak operator creates its own ServiceMonitor, named after the CR (`keycloak/keycloak`),
+  whenever `metrics-enabled=true` and the ServiceMonitor CRD exists. `serviceMonitor.enabled` defaults
+  to `true` in the 26.8.2 CRD, both in v2alpha1 and in v2beta1 (v2beta1 is now the storage version).
+  That is the same name and namespace as the repo's Flux-owned `platform/keycloak/servicemonitor.yaml`.
+  After the 26.0.6 -> 26.8.2 bump (#121/#144) the operator's PATCH against it returned
+  `409 Conflict` and it ran out of retries. The CR then sat at `Ready=False`/`HasErrors=True`, and
+  `platform-config` (`wait: true`, which health-checks every object it applies, this CR included)
+  blocked everything that `dependsOn` it: velero, apps-vms, apps-wave-0 (and so waves 1/2 and
+  pm-agent), homepage, oauth2-proxy, headlamp, conformance, vpn-ingress, crafty, runbooks.
+- `platform/keycloak/servicemonitor.yaml`: a comment saying it is the sole owner.
+- **Why ours is kept:** these are written against it: the L0 `servicemonitor-ports-resolve`
+  allow-list entry `platform/keycloak/servicemonitor.yaml:keycloak`, the L2 `keycloak-monitored`
+  chainsaw test, the README's ownership table, and the dashboard's `up{namespace="keycloak"}`.
+  The operator's version (upstream `KeycloakServiceMonitorDependentResource` @26.8.2) scrapes the same
+  target: port name `management` (9000 on `keycloak-service`), path `/metrics`, interval 30s,
+  selector = the operator's instance labels. It offers nothing ours lacks, so having two owners
+  would add a fight and gain nothing.
+
+## What I verified
+
+- Live, read-only, `k3s-prod` ~06:25Z: the CRD schema field `spec.serviceMonitor.enabled`
+  (`default: true`) exists in both served versions. The live `keycloak` ServiceMonitor has exactly
+  **one** managedFields manager, `kustomize-controller` (Apply, 2026-09-05), **no ownerReferences**,
+  and labels `kustomize.toolkit.fluxcd.io/name=platform-config`, `release=kube-prometheus-stack`. So
+  the operator's apply never landed, and nothing on the object ties it to the Keycloak CR.
+- The ServiceMonitor will not be deleted when the operator is switched off: the operator's dependent
+  informer is filtered on `Constants.DEFAULT_LABELS_AS_STRING`
+  (`app.kubernetes.io/managed-by=keycloak-operator`). Our object does not carry that label and has
+  no ownerReference, so the operator cannot see it as its secondary and cannot delete or GC it.
+  This conclusion comes from reading the code. It was not observed.
+- The other operator dependents in ns `keycloak` do not collide with anything: `keycloak-service`,
+  `keycloak-discovery`, the StatefulSet `keycloak`, `keycloak-initial-admin`, and
+  `keycloak-network-policy` (owner Keycloak, no Flux label). The repo declares none of these names.
+  It only *references* `keycloak-service` by name (HTTPRoute, reconcilers). There are no PDBs in the
+  namespace, and the Ingress is already `enabled: false`. The only NetworkPolicy in the namespace is
+  the operator's. The repo has no Keycloak NetworkPolicy, and no Kyverno-generated default-deny is
+  present there. NetworkPolicies add up, so the operator's allow-policy cannot fight another owner.
+  Left at its default.
+- Local gate: see the PR body. It records the rc and stages.
+
+## What I did NOT verify
+
+- That the CR reaches `Ready=True`/`HasErrors=False` after merge. The status also says
+  "Performing Keycloak upgrade, scaling down the deployment". I am inferring that this half clears
+  once the workflow stops erroring. It is not proven.
+- That the operator stops issuing the PATCH once `enabled: false` is applied. Watch the operator
+  logs for 409.
+- That `platform-config` and its downstream Kustomizations go Ready on the new revision, and that
+  Prometheus keeps exactly one `serviceMonitor/keycloak/keycloak/0` pool with targets up.
+- The operator's actual label set for `DEFAULT_LABELS_AS_STRING`. I took it from the observed labels
+  on operator-created objects, not from `Constants.java`.

decisions/2026-11-08-renovate-config-cleanup.md

@@ -0,0 +1,65 @@
+---
+date: 2026-11-08
+branch: chore/renovate-config-cleanup
+title: Renovate config — hand-migrate the docker key, track gitea/runner, keep HelmRelease range lower bounds
+---
+
+## What changed
+
+- **`renovate.json5`: the deprecated top-level `"docker": {"enabled": true}` is hand-migrated** to
+  `packageRules[0] = {"matchCategories": ["docker"], "enabled": true}`, the exact shape Renovate's
+  own migration produced on `origin/renovate/migrate-config` (PR #139). Its comment moved with it.
+  This replaces #139, which was closed because its rewrite deleted every comment in the file. No
+  other comment was removed or reflowed (diff is additions plus the moved block).
+- **`gitea/runner` added to the blast-radius rule** beside `gitea/act_runner`, with a comment:
+  act_runner was renamed upstream to gitea/runner after 0.6.1; Renovate does not follow renames,
+  so the actual image migration is a separate manual PR.
+- **Two new packageRules for flux/helm `>=A <B` ranges:** `rangeStrategy: "bump"`, plus
+  `matchUpdateTypes: ["bump"] -> enabled: false`, so in-range releases (which Flux adopts on its
+  own) do not become PRs. Cause of the bug, from the 44.167.0 source: rangeStrategy is unset
+  (`auto`); the flux manager has no `getRangeStrategy`/`updateLockedDependency`, so
+  `lib/modules/manager/index.ts getRangeStrategy` resolves `auto` to `replace`; helm versioning is
+  the npm api verbatim, and npm `getNewValue` under `replace` turns `>=1.12.0 <1.13.0` into
+  `<1.14.0` (what PRs #131, #130 and #115 did).
+- **`platform/headlamp/helmrelease.yaml`: `version: "<0.47.0"` -> `">=0.46.0 <0.47.0"`**, comment
+  refreshed. The live chart is 0.46.0, so the set of installable charts is unchanged.
+
+## What I verified
+
+- **Semantic diff of #139:** I parsed `renovate.json5` on `origin/renovate/migrate-config` and on its
+  parent `5b484f6` with json5 (the copy inside the renovate image). With `docker` removed from the
+  parent and `packageRules[0]` removed from the migrated file, the two are identical. So the only
+  change was `{"matchCategories":["docker"],"enabled":true}`, inserted at packageRules[0].
+- **Validator, `ghcr.io/renovatebot/renovate:44.167.0`** (the tag that `.gitea/workflows/renovate.yaml`
+  pins as `container.image`): `renovate-config-validator` gives `Config validated successfully`
+  (rc 0), and so does `--strict` (rc 0), both as a file path and in repo mode. **`--strict` on
+  origin/main's file gives rc 1** and prints the `docker` migration diff. Nothing is left to migrate.
+- **`getNewValue` called in the 44.167.0 image** (helm versioning), currentValue
+  `>=1.12.0 <1.13.0`, newVersion 1.13.0: replace `<1.14.0`, widen `>=1.12.0 <1.14.0`,
+  bump `>=1.13.0 <1.14.0`, pin/update-lockfile `<1.14.0`. On `<0.47.0`, bump returns the value
+  unchanged, so it would never propose anything for that range. That is why the rule is scoped to
+  the `>=A <B` shape.
+- **Local dry run** (`--platform=local`, `RENOVATE_DRY_RUN=lookup`, 44.167.0, live chart repos)
+  against a fixture repo of four HelmReleases under `platform/`:
+  - With this branch's config: `No config migration necessary`. longhorn `>=1.9.0 <1.10.0` ->
+    `>=1.13.0 <1.14.0` (minor). kube-prometheus-stack `>=88.0.0 <89.0.0` -> `>=93.1.0 <94.0.0`
+    (major). The in-range bump `>=88.6.5 <89.0.0` was dropped (`Filtered out 1 disabled update(s)`).
+    headlamp `>=0.46.0 <0.47.0` and `<0.47.0`: no update.
+  - Control run with origin/main's config: `Config migration necessary`, longhorn -> `<1.14.0`,
+    kube-prometheus-stack -> `<94.0.0`. This reproduces the reported bug.
+- Live headlamp HelmRelease (read-only kubectl, k3s-prod): spec `<0.47.0`, history chartVersion
+  `0.46.0`, Ready `True`.
+
+## What I did NOT verify
+
+- **I have not seen a real scheduled Renovate run against this repo with this config.** The claim that
+  Renovate stops reopening the migrate-config PR rests on the validator and on `No config migration
+  necessary` in the local dry run, not on the Gitea bot. After merge, watch the next run, and if
+  `renovate/migrate-config` is still on the remote, delete it.
+- **Under bump, the new lower bound is the release Renovate proposes**, e.g. `>=1.13.2 <1.14.0`, not
+  `>=1.13.0`. Flux still installs the newest release in the window. Ranges with other shapes keep the
+  default `replace`: `3.9.x`, `~`, ceiling-only, and exact pins.
+- **The open PRs #131, #115 and any others still carry unbounded `<X` values.** They were generated
+  under the old strategy, and Renovate should rebase or regenerate them under bump on its next run.
+  I have not checked that it does.
+- The gitea/runner rename itself (a new image reference and any config changes) is not done here.

platform/headlamp/helmrelease.yaml

@@ -40,7 +40,11 @@ spec:
       # values (-session-ttl + the -oidc-* args from the externalSecret; useCookie stays default-false
       # so `-oidc-use-cookie` is NOT emitted) is supported by the v0.41.0 backend. Renovate maintains
       # within 0.41.x; the appVersion continues to pin the Headlamp image.
-      version: "<0.47.0"
+      # 2026-11-08: NOW 0.46.x. PR #130 (Renovate, default rangeStrategy `replace`) rewrote
+      # `>=0.41.0 <0.42.0` to the unbounded-below `<0.47.0`; the lower bound is restored here at the
+      # chart that was live when it merged (0.46.0). renovate.json5 now bumps `>=A <B` ranges as a window,
+      # so the next widen PR proposes `>=0.47.x <0.48.0` rather than `<0.48.0`.
+      version: ">=0.46.0 <0.47.0"
       sourceRef:
         kind: HelmRepository
         name: headlamp

platform/keycloak/keycloak-instance.yaml

@@ -38,6 +38,20 @@ spec:
   ingress:
     enabled: false                         # exposed via Gateway API HTTPRoute (httproute.yaml), not the
                                            # operator's built-in Ingress
+  # --- ONE OWNER for the `keycloak` ServiceMonitor: this repo (servicemonitor.yaml), not the operator.
+  #     Since 26.x the operator creates its OWN ServiceMonitor, named after this CR (`keycloak`, i.e.
+  #     the SAME name/namespace as ours), whenever metrics-enabled=true and the CRD exists —
+  #     spec.serviceMonitor.enabled defaults TRUE. On 2026-11-08, right after the 26.0.6 -> 26.8.2
+  #     bump, its server-side apply onto our Flux-owned object failed `409 Conflict`, the operator
+  #     exhausted retries, the CR sat Ready=False/HasErrors=True, and `platform-config` (wait: true,
+  #     so it health-checks this CR) stalled every Kustomization downstream of it. Ours is kept
+  #     because the L0 servicemonitor-ports-resolve and L2 keycloak-monitored conformance tests and
+  #     the dashboard's up{namespace="keycloak"} are written against it; the operator's would scrape
+  #     the same thing (management port 9000, /metrics, 30s). Do NOT flip this to true without
+  #     deleting servicemonitor.yaml in the same change — two owners of one object is a fight
+  #     (cf. the default-deny NetworkPolicy drift loop).
+  serviceMonitor:
+    enabled: false
   # --- Hardening + observability options (docs/15). Brute-force detection is a REALM setting
   #     (bruteForceProtected in the realm import); these enable health + metrics endpoints. ---
   additionalOptions:

platform/keycloak/servicemonitor.yaml

@@ -8,6 +8,9 @@
 # /metrics) once metrics-enabled/health-enabled are set (see keycloak-instance.yaml additionalOptions).
 # We select the operator-created keycloak-service (which selects the Keycloak pods) and scrape the pods'
 # management port by number via targetPort.
+# SOLE OWNER: the operator's own ServiceMonitor (same name `keycloak`) is switched OFF in
+#   keycloak-instance.yaml (spec.serviceMonitor.enabled: false) — 26.x defaults it ON and its apply
+#   409-looped against this object on 2026-11-08. Keep exactly one of the two.
 # TODO(WS-SEC, bring-up): confirm the management port number/name the operator exposes (9000 default)
 #   and adjust targetPort if the pinned operator changes it. The KeycloakDown / KeycloakDatabaseDown
 #   alerts (platform/monitoring/alerts.yaml) key on kube-state-metrics and fire regardless of this SM.

renovate.json5

@@ -37,6 +37,8 @@
 //     renovate-config-validator /tmp/renovate.json5
 // (Not `--strict`: that also fails on a pending migration of the pre-existing top-level
 // `"docker": {"enabled": true}` block at the bottom, which is a warning, not an error.)
+// RESOLVED 2026-11-08: that block is now hand-migrated to the first packageRule (`matchCategories:
+// ["docker"]`), the exact shape Renovate's own migration produced, so no migration is pending.
 //
 // `regex` vs `custom.regex` in enabledManagers, RESOLVED FROM SOURCE (Renovate 39.264.0 and
 // later): both work. lib/config/migrations/custom/enabled-managers-migration.ts rewrites
@@ -196,6 +198,22 @@
     }
   ],
   "packageRules": [
+    // The `docker` manager (Dockerfiles, docker-compose) and the `kubernetes` manager above were
+    // BOTH previously `enabled: false`, solely so Renovate would not fight Flux image-automation
+    // over a running tag. That controller was never enabled and never will be here (see _comment),
+    // so the only thing those two flags were achieving was leaving every plain `image:` tag in the
+    // repo unmanaged. `docker` is left out of enabledManagers because this repo contains no
+    // Dockerfile that ships to the cluster; `kubernetes` is now ON and is the one that matters.
+    // MOVED HERE 2026-11-08 from a top-level `"docker": {"enabled": true}` key, which Renovate
+    // deprecated (the top-level language objects became `matchCategories` rules). Hand-migrated to
+    // the exact output of Renovate's own config migration (branch renovate/migrate-config, PR #139:
+    // a json5 semantic diff of that file against its parent shows THIS rule, inserted at
+    // packageRules[0], as the ONLY change). #139 itself was closed unmerged because its rewrite
+    // dropped every comment in this file. With nothing left to migrate, Renovate stops reopening it.
+    {
+      "matchCategories": ["docker"],
+      "enabled": true
+    },
     {
       "description": "Auto-merge the allowed low-risk classes on green CI \u2014 no human authoring (R-UPD-02). The Gitea merge gate (make -C ci gate) is the required check.",
       "matchUpdateTypes": [
@@ -252,6 +270,11 @@
         // Keycloak's (the IdP) included, now that imageName is tracked.
         "gitea/gitea",
         "gitea/act_runner",
+        // ADDED 2026-11-08. act_runner was RENAMED upstream to gitea/runner after 0.6.1
+        // (https://gitea.com/gitea/runner/releases). Renovate does not follow a rename — it will
+        // never propose gitea/act_runner -> gitea/runner — so that migration is a manual PR. Listed
+        // now so the new name is blast-radius from its first reference, not after an auto-merge.
+        "gitea/runner",
         "ghcr.io/cloudnative-pg/postgresql"
       ],
       "automerge": false,
@@ -368,6 +391,45 @@
       "automerge": false,
       "addLabels": ["needs-review", "range-widen"]
     },
+    // ADDED 2026-11-08 — KEEP THE LOWER BOUND WHEN A MINOR-LOCKED RANGE MOVES. MEASURED PROBLEM:
+    // PRs #131 (longhorn), #130 (headlamp) and #115 (kube-prometheus-stack) turned
+    // `">=1.12.0 <1.13.0"` into `"<1.14.0"` — an UNBOUNDED-below range that also admits every
+    // older chart (and #130 merged: platform/headlamp/helmrelease.yaml carried `<0.47.0`). WHY,
+    // from the 44.167.0 source this file is validated against: no rangeStrategy is set, so it is
+    // `auto`; the flux manager has no getRangeStrategy and no updateLockedDependency, so
+    // lib/modules/manager/index.ts getRangeStrategy resolves `auto` -> `replace`. The helm
+    // datasource's defaultVersioning is `helm`, which is the npm versioning api verbatim
+    // (lib/modules/versioning/helm/index.ts: `api = { ...npm }`), and npm getNewValue under
+    // `replace` rewrites `>=A <B` to just `<C`. Calling that getNewValue in the 44.167.0 image for
+    // currentValue `>=1.12.0 <1.13.0`, newVersion 1.13.0:
+    //   replace -> `<1.14.0`   widen -> `>=1.12.0 <1.14.0`   bump -> `>=1.13.0 <1.14.0`
+    // Only `bump` keeps a one-minor window. (widen keeps the bound but grows the window by a minor
+    // on every PR — the opposite of minor-locking.) Under bump the new lower bound is the release
+    // proposed (e.g. 1.13.2 -> `>=1.13.2 <1.14.0`); Flux still picks the newest that satisfies it.
+    // Scoped to the bounded `>=A <B` shape only: on a ceiling-only range like `<0.47.0` bump
+    // returns the value unchanged, so Renovate would silently propose NOTHING for it, ever.
+    {
+      "description": "ADDED 2026-11-08. HelmRelease `version: \">=A <B\"` ranges move as a whole window (rangeStrategy bump), so a range-widen PR yields `>=1.13.0 <1.14.0`, not `<1.14.0`. See the comment above for the measured cause.",
+      "matchManagers": ["flux"],
+      "matchDatasources": ["helm"],
+      "matchCurrentValue": "/^>=\\S+ <\\S+$/",
+      "rangeStrategy": "bump"
+    },
+    // ...AND DO NOT LET bump ADD NOISE. Under bump Renovate also raises the lower bound for a
+    // release ALREADY INSIDE the range (lib/workers/repository/process/lookup/generate.ts sets
+    // isBump; e.g. 1.12.5 against `>=1.12.0 <1.13.0` -> `>=1.12.5 <1.13.0`), which `replace` never
+    // proposed. Flux already installs that release with no PR, and the range-widen rule above
+    // would hold every one for review. util/package-rules/update-types.ts matches isBump as
+    // updateType `bump`, so exactly those updates are switched off here; out-of-range moves are
+    // ordinary minor/major updates and are unaffected.
+    {
+      "description": "ADDED 2026-11-08. No PR for a chart release the HelmRelease range already admits — Flux adopts it on its own.",
+      "matchManagers": ["flux"],
+      "matchDatasources": ["helm"],
+      "matchCurrentValue": "/^>=\\S+ <\\S+$/",
+      "matchUpdateTypes": ["bump"],
+      "enabled": false
+    },
     {
       "description": "ADDED 2026-10-30. velero-plugin-for-aws is pinned to a RELEASE CANDIDATE (v1.14.3-rc.2) and Renovate proposed NOTHING for it, although v1.14.3 and v1.14.4 are stable on Docker Hub. Docker versioning treats everything after the first `-` as a compatibility SUFFIX, so only other `*-rc.2` tags qualify and the RC is a fossil forever. Semver versioning reads it as a prerelease, and a prerelease upgrades to the stable release.",
       "matchPackageNames": ["velero/velero-plugin-for-aws"],
@@ -439,14 +501,5 @@
     "conformance/tests/**",
     "platform/image-automation/per-service/**",
     "platform/kyverno/tests/**"
-  ],
-  // The `docker` manager (Dockerfiles, docker-compose) and the `kubernetes` manager above were
-  // BOTH previously `enabled: false`, solely so Renovate would not fight Flux image-automation
-  // over a running tag. That controller was never enabled and never will be here (see _comment),
-  // so the only thing those two flags were achieving was leaving every plain `image:` tag in the
-  // repo unmanaged. `docker` is left out of enabledManagers because this repo contains no
-  // Dockerfile that ships to the cluster; `kubernetes` is now ON and is the one that matters.
-  "docker": {
-    "enabled": true
-  }
+  ]
 }