feat(cnpg): required pod anti-affinity + switchover primary rolls

`instances: 3` alone does not keep the three instances apart. All three
Clusters used CNPG's default `podAntiAffinityType: preferred`, which is a
scheduler HINT — two instances may share a node whenever that scores better,
and they WILL on the path that matters: when CNPG recreates an instance after
a node loss, the surviving nodes are the only candidates and one of them
already runs an instance. `preferred` is how a 3-instance cluster decays back
into the single-node failure domain the previous commit exists to remove.

`required` makes it a scheduling constraint. On a 3-node cluster with no
spare, the cost is that an instance whose node is gone stays Pending until a
node returns. That is correct: a replacement squeezed onto an already-occupied
node dies with that node while making the cluster LOOK three-way replicated.
2 of 3 instances still serve and CNPG still promotes.

ALSO SETS primaryUpdateMethod: switchover. This is not cosmetic. Changing the
affinity changes the instance pod spec, so CNPG performs a rolling update, and
the CRD default for primaryUpdateMethod is `restart` (verified against the
live CNPG 1.30.0 CRD on both clusters) — a delete-and-recreate of the PRIMARY
pod, i.e. tens of seconds of no Postgres. On keycloak-db that is an SSO
outage, which also means kubectl OIDC, Grafana, Headlamp, Gitea and
forward-auth. With standbys now present, `switchover` promotes an
already-updated streaming replica instead, reducing the roll to a connection
reset. Known constraint documented in the manifest: under `switchover` the
operator rejects an update that changes imageName and a postgresql parameter
in the same reconcile.

Verified satisfiable before merge (read-only): k3s-prod and k3s-dev each have
3 Ready, schedulable, untainted nodes, and every pgdata PVC already has a
Longhorn replica on all three (replica-soft-anti-affinity=false), so any
instance can attach from any node.

DEV IMPACT: identical, and dev can absorb it. Longhorn dev nodes are 124.9 GiB
each with 152.0 GiB already scheduled under a 200% over-provisioning ceiling
(249.8 GiB); the six new 5Gi volumes add 30 GiB scheduled per node -> 182.0,
still inside the ceiling, and only ~600 MiB per volume is really consumed.
Prod is far looser: 239 -> 269 GiB scheduled against a 829 GiB ceiling.

THIS IS THE DISRUPTIVE COMMIT of the three. Land it on its own reconcile,
after the scale-out has produced two Ready standbys per Cluster, so the blast
radius is one clean switchover per database rather than a scale-out and a roll
at once. Expect a brief SSO interruption for keycloak-db.

Verified: yamllint clean; both tiers render the new fields; field names,
types and enum values checked against the live clusters.postgresql.cnpg.io
CRD (CNPG 1.30.0, identical on prod and dev).

Co-Authored-By: Claude Opus 5 <[email protected]>
(cherry picked from commit e1d364238d73e6ee99c9a50108fe8b4e09481a7e)

apps/wave-0/nextcloud-db.yaml

@@ -54,6 +54,21 @@ spec:
   inheritedMetadata:
     labels:
       app.kubernetes.io/name: nextcloud-db
+  # HARD anti-affinity — the three instances MUST land on three DIFFERENT nodes; and roll the
+  # primary by SWITCHOVER rather than CNPG's default in-place `restart`, which would delete and
+  # recreate the primary pod. Full rationale in platform/keycloak/postgres-cluster.yaml. Short
+  # version: CNPG's default `preferred` is a scheduler hint, so when CNPG recreates an instance
+  # after a node loss it may stack two instances on one node — which is how `instances: 3` decays
+  # back into the single-node failure domain this change removes. `required` makes it a constraint;
+  # the cost is that an instance whose node is gone stays Pending, which is correct (2 of 3 still
+  # serve). Satisfiable on both clusters: prod and dev each have 3 schedulable, untainted nodes.
+  #
+  # APPLYING THIS CHANGES THE POD SPEC, so CNPG rolls the instances: standbys first, then one
+  # switchover. Expect a single brief connection reset for Nextcloud.
+  affinity:
+    enablePodAntiAffinity: true
+    podAntiAffinityType: required
+  primaryUpdateMethod: switchover
   storage:
     storageClass: longhorn
     size: 5Gi

apps/wave-0/semaphore-db.yaml

@@ -55,6 +55,21 @@ spec:
   inheritedMetadata:
     labels:
       app.kubernetes.io/name: semaphore-db
+  # HARD anti-affinity — the three instances MUST land on three DIFFERENT nodes; and roll the
+  # primary by SWITCHOVER rather than CNPG's default in-place `restart`, which would delete and
+  # recreate the primary pod. Full rationale in platform/keycloak/postgres-cluster.yaml. Short
+  # version: CNPG's default `preferred` is a scheduler hint, so when CNPG recreates an instance
+  # after a node loss it may stack two instances on one node — which is how `instances: 3` decays
+  # back into the single-node failure domain this change removes. `required` makes it a constraint;
+  # the cost is that an instance whose node is gone stays Pending, which is correct (2 of 3 still
+  # serve). Satisfiable on both clusters: prod and dev each have 3 schedulable, untainted nodes.
+  #
+  # APPLYING THIS CHANGES THE POD SPEC, so CNPG rolls the instances: standbys first, then one
+  # switchover. Expect a single brief connection reset for Semaphore.
+  affinity:
+    enablePodAntiAffinity: true
+    podAntiAffinityType: required
+  primaryUpdateMethod: switchover
   storage:
     storageClass: longhorn
     size: 5Gi

platform/keycloak/postgres-cluster.yaml

@@ -42,6 +42,39 @@ spec:
   # cause a switchover — see it for the blast radius.)
   instances: 3
   imageName: ghcr.io/cloudnative-pg/postgresql:16.4
+  # HARD anti-affinity — the three instances MUST land on three DIFFERENT nodes (2026-09-07).
+  #
+  # CNPG's default is `preferred`, which is only a scheduler HINT: the scheduler may put two
+  # instances on one node whenever that scores better, and it WILL do so on the path that matters —
+  # when CNPG recreates an instance after losing a node, the surviving nodes are the only candidates
+  # and one of them already runs an instance. So `preferred` is precisely how `instances: 3` decays
+  # back into the single-node failure domain this whole change exists to remove; nothing pins the
+  # good layout in place. `required` makes it a scheduling CONSTRAINT instead of a preference.
+  #
+  # THE TRADE, STATED PLAINLY: on a 3-node cluster with no spare node, an instance whose node is
+  # gone stays Pending until a node is available. That is the CORRECT behaviour here. A replacement
+  # squeezed onto a node that already hosts an instance buys nothing — both die with that node —
+  # while making the cluster LOOK three-way replicated. Availability is unaffected either way:
+  # 2 of 3 instances still serve and CNPG promotes one of them.
+  #
+  # Satisfiable on both clusters: prod and dev each have 3 schedulable, untainted nodes (checked
+  # read-only before merge). topologyKey defaults to kubernetes.io/hostname — the failure domain
+  # that matters here. enablePodAntiAffinity defaults to true; stated explicitly so this block can
+  # be read without knowing the default.
+  affinity:
+    enablePodAntiAffinity: true
+    podAntiAffinityType: required
+  # Roll the PRIMARY by SWITCHOVER, not by restarting it in place. CNPG's default for
+  # primaryUpdateMethod is `restart`, which deletes and recreates the primary pod — tens of seconds
+  # with no Postgres serving, and on THIS database that is an SSO outage. With standbys present
+  # (instances: 3 above) a switchover promotes an already-updated, already-streaming replica and
+  # demotes the old primary, so the interruption is a connection reset rather than a cold start.
+  # This is the field that makes the anti-affinity roll below a switchover rather than an outage.
+  #
+  # KNOWN CONSTRAINT: under `switchover` the operator REJECTS an update that changes `imageName`
+  # AND a postgresql parameter in the same reconcile. Renovate bumping the image alone is fine;
+  # never combine an image bump with a postgresql.parameters change in one commit.
+  primaryUpdateMethod: switchover
   storage:
     storageClass: longhorn
     size: 5Gi