Files
jakman85andClaude Opus 5 cf7d97e546 fix(obs): twelve app SLI rules read a signal that exists; kill the dead ServiceMonitors
Twelve `<app>-down` SLI rules were STRUCTURALLY INERT. Each read `up{job="<app>"} == 0`,
and the ServiceMonitor rendered beside it named `endpoints[].port: metrics` — a port NO
rendered app Service defined. Every monitor discovered ZERO targets, so no `up{job="<app>"}`
series existed and the rule could not fire. Verified on prod before this change:
`count by (job) (up)` listed 24 jobs and not one of the twelve appeared; `up{job="plex"}`
and `up{job="gitea"}` both returned nothing. `PrometheusTargetsMissing` could not see it
either — a target that was never discovered has no `up` series to be 0.

THE PORT NAME WAS NEVER THE MISSING PIECE. Ground truth per app: Plex, Pi-hole, Jackett,
qBittorrent, Sonarr, Radarr, Seerr, Semaphore and Nextcloud export NO Prometheus metrics
without a separate exporter sidecar; Gitea can but only with app.ini metrics enabled;
Keycloak does, on the operator's management port, already scraped by
platform/keycloak/servicemonitor.yaml; vm-demo is a KubeVirt VM with no Service at all, so
its monitor selected nothing whatsoever. `metrics: true` was asserting a fact none of them
had. There was also a THIRD layer of inertness nobody had hit yet: every app namespace ships
default-deny with `ingress: []` and the renderer emitted no monitoring allow, so even a
correct port would have been dropped by policy — while renderer-contract.md claimed the
baseline carried such an allow.

FIXED IN THE GENERATOR, not in twelve rendered files:
  * `slis: [up]` now renders a `<app>-down` alert against kube-state-metrics readiness of the
    service's own workload object —
    `kube_deployment_status_replicas_available{namespace,deployment} == 0 or absent(...)`,
    `kube_statefulset_status_replicas_ready` for a StatefulSet, and
    `absent(kubevirt_vmi_info{namespace,name,phase="running"})` for a VM. The same idiom
    KeycloakDown has used since it was fixed. kube-state-metrics IS scraped, so this needs no
    exporter and no new target. The `or absent(...)` half is load-bearing: the signal
    DISAPPEARING now fires the alert instead of silencing it.
  * schema 1.6.0 adds `observability.metricsEndpoint` — WHERE metrics are served. A
    ServiceMonitor is rendered ONLY from it, and it renders all three halves of a working
    scrape together: the named Service port, the monitor endpoint, and the
    monitoring-namespace CiliumNetworkPolicy allow. No catalog entry declares one, so twelve
    dead ServiceMonitors are gone. `metrics: true` keeps meaning logs-to-Loki plus
    kube-state-metrics liveness.
  * schema 1.6.0 also adds `observability.livenessWorkload`, and catalog/keycloak.yaml uses
    it. Keycloak's server is a StatefulSet run by the operator; the default Deployment shape
    would have made its `absent()` half fire FOREVER — the other failure mode this repo has
    shipped.
  * renderer selftest gains cases 9/9a/9b/9c: no metricsEndpoint => no monitor; the up SLI
    must not read `up{job=...}`; endpoint port must exist on the Service; the metrics port
    must NOT reach the Gateway allow; an unknown livenessWorkload.kind is REFUSED.

PROOF, both directions, for all twelve (apps/wave-{0,1,2}/tests/app-slis.test.yaml, run by
platform/monitoring/tests/run.sh over the RENDERED manifests):
  * quiet on the healthy state; firing on no-ready-replica; firing when the series is ABSENT;
    the VM asserted both gone and in a non-running phase; per-app isolation asserted, and
    keycloak asserted un-maskable by its neighbours (keycloak-operator Deployment,
    keycloak-db StatefulSet ready in the same namespace).
  * mutation-checked: six mutations, each turning its suite red — reverting plex to
    `up{job=...} == 0`; dropping seerr's `absent()` half; pointing keycloak at a Deployment;
    capitalising the VMI phase to `Running`; widening keycloak's matcher to the namespace;
    inverting plex's `== 0` to `!= 0`. Control run green.
  * live on prod with `promtool query instant`: all ten deployed app Deployments export
    `kube_deployment_status_replicas_available`,
    `kube_statefulset_status_replicas_ready{namespace="keycloak",statefulset="keycloak"}` = 2,
    `kubevirt_vmi_info` present with a lowercase `phase="running"`; and evaluating the twelve
    SHIPPED expressions returns no series for eleven (quiet, not permanently firing). The
    twelfth is vm-demo, which is not in apps/wave-0/kustomization.yaml and so is not deployed
    — its rule is not applied either.

DELETION WAS NOT THE ANSWER, and the evidence is recorded rather than assumed:
kube-prometheus-stack's `kubernetes-apps` group IS live on prod (17 rules, KubePodNotReady /
KubeDeploymentReplicasMismatch among them), so app pods are not unmonitored — but that is a
generic warning keyed to a namespace, not the per-service CRITICAL SLI R-OBS-01 declares.
platform/monitoring/alerts.yaml carries no generic app-readiness rule at all.

GUARD SO IT CANNOT RECUR SILENTLY: new whole-repo L0 invariant
conformance/tests/l0-static/servicemonitor-ports-resolve. It walks the MONITORS, not the
alerts, so it needs no `up{job=...}` rule to reach one, and resolves each endpoint by SELECTOR
across the whole tree rather than by same-file adjacency. Fixing the alerts DELETED the only
thing that made the broken monitors visible to the sibling check, which is exactly why this
axis needed its own home. It declares `conformance.home.k3s/scope: whole-repo` and adds no
ci/l0.sh stage. Proven to FAIL on the pre-fix tree: 15 failures — eleven port disagreements,
vm-demo's monitor matching no Service at all, and the three generator/contract assertions —
and to pass after. Its resolver is additionally proven on in-check fixtures (agreeing pair
accepted, disagreeing pair rejected, target-less endpoint rejected), because after the fix
there are zero in-repo monitors left for it to judge and a guard whose only evidence is
"nothing failed" cannot be told apart from one that stopped working. It also asserts
r_servicemonitor does not HARDCODE a port name — the mechanism by which one edit gave twelve
services the same wrong one.

Also fixed, found on the way:
  * platform/monitoring/tests/run.sh discovered the SUITE convention only inside
    platform/monitoring/tests/ while documenting it as general — a suite added anywhere else
    was never run and still looked like coverage. Now discovered tree-wide; a misplaced
    *.test.yaml fails loudly instead of vanishing.
  * KNOWN_UNWIRED_JOBS in alert-metrics-have-scrape-targets is now EMPTY, and the twelve
    apps/* entries in alert-rules-unit-tested's UNTESTED_FILES are gone — both because the
    gaps were CLOSED, not waived. Both registers are shrink-only and a stale entry fails.
  * renderer-contract.md claimed the default-deny baseline carried a standing monitoring
    scrape allow. It never did; the claim is corrected and the allow is now real, per declared
    metrics endpoint and scoped to that one port.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-13 17:11:30 -05:00

315 lines
17 KiB
YAML

---
# Namespace (trust zone; default-deny + policies apply)
apiVersion: v1
kind: Namespace
metadata:
name: nextcloud
labels:
homelab.keithjackson.dev/app-namespace: 'true'
gateway-access: shared
annotations:
homelab.keithjackson.dev/backup-policy: daily-offcluster
---
# Default-deny baseline (R-NET-01): deny all, allow only DNS
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: nextcloud
labels:
homelab.keithjackson.dev/baseline: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress: []
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
# PersistentVolume nextcloud-legacy-archive - static external NFS (bulk tier)
apiVersion: v1
kind: PersistentVolume
metadata:
name: nextcloud-legacy-archive
labels:
app.kubernetes.io/name: nextcloud
spec:
capacity:
storage: 270Gi
accessModes:
- ReadOnlyMany
persistentVolumeReclaimPolicy: Retain
storageClassName: ''
mountOptions:
- nfsvers=4.1
- hard
- noatime
- timeo=600
- retrans=2
nfs:
server: ${BULK_NFS_SERVER}
path: /mnt/Storage/nextcloud
readOnly: true
claimRef:
namespace: nextcloud
name: nextcloud-legacy-archive
---
# PVC nextcloud-data (R-STATE)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nextcloud-data
namespace: nextcloud
labels:
app.kubernetes.io/name: nextcloud
spec:
accessModes:
- ReadWriteOnce
storageClassName: longhorn
resources:
requests:
storage: 20Gi
---
# PVC nextcloud-legacy-archive (R-STATE)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nextcloud-legacy-archive
namespace: nextcloud
labels:
app.kubernetes.io/name: nextcloud
spec:
accessModes:
- ReadOnlyMany
storageClassName: ''
volumeName: nextcloud-legacy-archive
resources:
requests:
storage: 270Gi
---
# Deployment (R-PLAT-02)
apiVersion: apps/v1
kind: Deployment
metadata:
name: nextcloud
namespace: nextcloud
labels:
app.kubernetes.io/name: nextcloud
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: nextcloud
template:
metadata:
labels:
app.kubernetes.io/name: nextcloud
homelab.keithjackson.dev/egress-class: internet
homelab.keithjackson.dev/auth-mode: oidc
annotations:
homelab.keithjackson.dev/auth-mode: oidc
homelab.keithjackson.dev/auth-groups: platform-admins
spec:
containers:
- name: nextcloud
image: nextcloud:31-apache
ports:
- containerPort: 80
env:
- name: POSTGRES_HOST
value: nextcloud-db-rw.nextcloud-db.svc.cluster.local
- name: POSTGRES_DB
value: nextcloud
- name: POSTGRES_USER
value: nextcloud
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: nextcloud-bootstrap
key: db-password
- name: NEXTCLOUD_ADMIN_USER
valueFrom:
secretKeyRef:
name: nextcloud-bootstrap
key: admin-username
- name: NEXTCLOUD_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: nextcloud-bootstrap
key: admin-password
- name: NEXTCLOUD_TRUSTED_DOMAINS
value: nextcloud.${BASE_DOMAIN}
- name: OVERWRITEHOST
value: nextcloud.${BASE_DOMAIN}
- name: OVERWRITEPROTOCOL
value: https
- name: OVERWRITECLIURL
value: https://nextcloud.${BASE_DOMAIN}
- name: NEXTCLOUD_OIDC_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: nextcloud-bootstrap
key: oidc-client-secret
volumeMounts:
- name: data
mountPath: /var/www/html
- name: legacy-archive
mountPath: /mnt/legacy-nextcloud
subPath: nextcloud
readOnly: true
lifecycle:
postStart:
exec:
command:
- /bin/sh
- -c
- "# Idempotent Keycloak OIDC wiring (R-IDP-02/06). Runs alongside the nextcloud entrypoint\n# (which installs/upgrades on first boot). Installs+enables the user_oidc app and creates OR\n# updates the \"keycloak\" login provider (occ user_oidc:provider is create-or-update by\n# identifier). Auto-discovery pulls the realm's endpoints; the `groups` claim drives Nextcloud\n# group provisioning. occ runs as www-data (owns config.php); the client secret is passed via\n# --clientsecret-env so it never lands on a command line. Safe to re-run every start; a missing\n# secret or not-yet-installed instance is skipped so the pod never crashloops (the local\n# break-glass admin still works).\nset -u\n# OCC runs occ as www-data (owns config.php). NOTE: this wrapper word-splits its args via $*,\n# so it is used ONLY for single-token subcommands (status/app:list/app:install/...). The\n# provider call below carries a MULTI-WORD value (--scope 'openid email profile') and is run as\n# ONE su -c with the scope single-quoted, so the inner shell keeps it a single argument.\nOCC() { su -s /bin/sh www-data -c \"php /var/www/html/occ $*\"; }\nNEXTCLOUD_DISCOVERY_URL=https://keycloak.${BASE_DOMAIN}/realms/homelab/.well-known/openid-configuration\nnextcloud_oidc_fail() {\n echo \"[nextcloud-oidc] FAILED to configure 'keycloak' provider: $1\" >&2\n { echo \"$(date -u +%Y-%m-%dT%H:%M:%SZ) [nextcloud-oidc] FAILED: $1\"; } >> /var/www/html/nextcloud-oidc-bootstrap.log 2>/dev/null || true\n}\n# (a) Wait for the instance to be installed AND out of maintenance mode. The entrypoint holds\n# maintenance mode during first-boot install/upgrade; occ writes made in that window do not\n# stick \u2014 a second race on top of the quoting bug fixed at (c).\ni=0\nuntil OCC status 2>/dev/null | grep -q \"installed: true\" && ! OCC status 2>/dev/null | grep -q \"maintenance: true\"; do\n i=$((i + 1))\n [ \"$i\" -ge 300 ] && { echo \"[nextcloud-oidc] occ not ready (installed/maintenance) after wait; skipping\" >&2; exit 0; }\n sleep 2\ndone\n# Let Nextcloud's OWN server-side HTTP client reach the private-network IdP. The Keycloak\n# FQDN resolves (split-horizon) to the PRIVATE Cilium gateway VIP (RFC1918), and Nextcloud's\n# SSRF guard otherwise REFUSES that request \u2014 OCP\\Http\\Client\\LocalServerException \"violates\n# local access rules\" \u2014 during user_oidc's discovery/JWKS/token fetch AT LOGIN, surfacing to\n# the user as \"Could not reach the provider\". (The (b) curl check below uses system libcurl,\n# NOT this guard, so it passes while the browser login still failed.) Idempotent; re-applied\n# every start so it survives pod rolls / PVC rebuilds. Confirmed root cause 2026-08-27.\nOCC config:system:set allow_local_remote_servers --value=true --type=boolean >/dev/null 2>&1 \\\n || echo \"[nextcloud-oidc] WARN: failed to set allow_local_remote_servers\" >&2\n# NOTE the `$$` escape below \u2014 it is load-bearing. apps-wave-1 declares postBuild.\n# substituteFrom cluster-vars, so Flux envsubst rewrites this whole manifest at RECONCILE\n# time, before any container runs. Written unescaped, `$${NEXTCLOUD_OIDC_CLIENT_SECRET:-}` is\n# not a cluster-var, so envsubst blanked it and the guard reconciled as `[ -z \"\" ]` \u2014 ALWAYS\n# TRUE \u2014 so this hook `exit 0`d here on EVERY boot since 8738871 (2026-08-27), never\n# installing/enabling user_oidc nor registering the provider. SSO kept working only because\n# the provider was already in the DB; a PVC rebuild or fresh cluster would have come up with\n# NO OIDC at all. Flux STRICT substitution cannot catch this \u2014 it EXEMPTS default-operator\n# forms by design. `$${VAR}` is the escape: envsubst emits it back as a plain brace expansion\n# for the shell. Bare `$VAR` and `$(cmd)` are left alone and need NO escaping \u2014 which is why\n# `--clientsecret-env` and the bare uses below always worked and only the guard broke.\n# Contract: clusters/base/apps-vms.yaml. Enforced at render time by catalog/render.py.\nif [ -z \"$${NEXTCLOUD_OIDC_CLIENT_SECRET:-}\" ]; then\n echo \"[nextcloud-oidc]\
\ client secret unset; skipping OIDC (break-glass admin only)\"\n exit 0\nfi\nif ! OCC app:list 2>/dev/null | grep -q \"user_oidc\"; then\n OCC app:install user_oidc || echo \"[nextcloud-oidc] app:install user_oidc failed\" >&2\nfi\nOCC app:enable user_oidc >/dev/null 2>&1 || true\n# (b) Wait up to ~90s for the Keycloak discovery doc to be reachable (HTTP 200) before registering.\nd=0; disc_ok=0\nwhile [ \"$d\" -lt 45 ]; do\n if [ \"$(curl -sS -o /dev/null -w '%{http_code}' --max-time 5 \"$NEXTCLOUD_DISCOVERY_URL\" 2>/dev/null)\" = \"200\" ]; then\n disc_ok=1; break\n fi\n d=$((d + 1)); sleep 2\ndone\n[ \"$disc_ok\" = 1 ] || nextcloud_oidc_fail \"Keycloak discovery endpoint not reachable (HTTP 200) within timeout\"\n# (c) Configure (create-or-update by identifier) then VERIFY it persisted; retry the whole cycle.\n# HARDENED: run as ONE su -c so www-data's shell parses --scope='openid email profile' as a\n# single argument. The previous `OCC user_oidc:provider ... --scope=\"openid email profile\"`\n# form fed the value back through the wrapper's $*, which SPLIT it into extra positionals ->\n# occ \"Too many arguments\" -> silent failure on EVERY boot. Secret via --clientsecret-env\n# (never on argv). The verify-then-retry loop guarantees the provider actually persisted.\nr=0; reg_ok=0\nwhile [ \"$r\" -lt 10 ]; do\n su -s /bin/sh www-data -c \"php /var/www/html/occ user_oidc:provider keycloak \\\n --clientid=nextcloud \\\n --clientsecret-env=NEXTCLOUD_OIDC_CLIENT_SECRET \\\n --discoveryuri=$NEXTCLOUD_DISCOVERY_URL \\\n --scope='openid email profile' \\\n --unique-uid=0 \\\n --mapping-uid=preferred_username \\\n --mapping-email=email \\\n --mapping-display-name=name \\\n --group-provisioning=1 \\\n --mapping-groups=groups\" >/dev/null 2>&1 || true\n if OCC user_oidc:provider 2>/dev/null | grep -q \"keycloak\"; then\n reg_ok=1; break\n fi\n r=$((r + 1)); sleep 3\ndone\nif [ \"$reg_ok\" = 1 ]; then\n echo \"[nextcloud-oidc] provider 'keycloak' configured and verified\"\nelse\n nextcloud_oidc_fail \"user_oidc:provider ran but the provider did not persist after retries\"\nfi\n# (d) Promote Keycloak 'platform-admins' members to Nextcloud admins (#5). user_oidc 8.11 has\n# NO native group->admin mapping (occ user_oidc:provider offers group PROVISIONING and a\n# login WHITELIST only), so mirror the provisioned 'platform-admins' membership into the\n# built-in 'admin' group (the sole source of admin rights in Nextcloud). Idempotent every\n# start: occ group:adduser is a no-op for an existing member. A user is promotable only\n# AFTER their first OIDC login provisions them into the group, so a brand-new admin gains\n# rights on their next login + the following pod start (or run `occ group:adduser admin\n# <uid>` once). The LOCAL break-glass admin is unaffected (docs/15). The group name is\n# passed to php as argv (never interpolated) so group:list JSON parses robustly.\nADMIN_SRC_GROUP=platform-admins\n# HARDENED (verify-and-retry, like step (c)): the previous single pass ran `occ group:adduser`\n# exactly once with its output swallowed (>/dev/null 2>&1) and NO retry, so an early-boot\n# transient failure (DB / maintenance race in the first seconds after container start) silently\n# dropped the promotion until the NEXT pod restart. This loop re-reads group:list and re-adds\n# any platform-admins member still missing from 'admin' until all are verified (bounded ~30s);\n# it is a no-op for members already in 'admin'. Group names are passed to php as argv (never\n# interpolated) so the JSON parses robustly.\nparse_group() { su -s /bin/sh www-data -c \"php /var/www/html/occ group:list --output=json\" 2>/dev/null | php -r '$d=json_decode(stream_get_contents(STDIN),true);$g=$argv[1];if(is_array($d)&&!empty($d[$g])&&is_array($d[$g])){foreach($d[$g] as $u){echo $u,\"\\n\";}}' -- \"$1\" 2>/dev/null; }\np=0\nwhile [ \"$p\" -lt 15 ]; do\n current_admins=$(parse_group admin)\n missing=0\n for uid in $(parse_group \"$ADMIN_SRC_GROUP\"\
); do\n if printf '%s\\n' \"$current_admins\" | grep -qxF -- \"$uid\"; then\n continue\n fi\n missing=1\n if OCC group:adduser admin \"$uid\" >/dev/null 2>&1; then\n echo \"[nextcloud-oidc] ensured '$uid' is a Nextcloud admin (member of $ADMIN_SRC_GROUP)\"\n fi\n done\n [ \"$missing\" = 0 ] && break\n p=$((p + 1)); sleep 2\ndone\nif [ \"$p\" -ge 15 ]; then\n nextcloud_oidc_fail \"could not promote all $ADMIN_SRC_GROUP members into 'admin' after retries\"\nfi\n# A postStart hook MUST exit 0. A non-zero exit makes the kubelet KILL the container\n# (FailedPostStartHook -> CrashLoopBackOff), which is exactly what happened on 2026-09-01:\n# the final statement used to be `[ \"$p\" -ge 15 ] && nextcloud_oidc_fail ...`, so on the\n# SUCCESS path the test returned 1 and became the script's exit status. That was latent for\n# as long as the OIDC guard above was dead code and exited 0 first; fixing the guard\n# exposed it immediately. Every failure path here logs loudly and writes a marker file and\n# deliberately does NOT crash the pod \u2014 the break-glass local admin must stay usable.\nexit 0\n"
volumes:
- name: data
persistentVolumeClaim:
claimName: nextcloud-data
- name: legacy-archive
persistentVolumeClaim:
claimName: nextcloud-legacy-archive
---
# Service
apiVersion: v1
kind: Service
metadata:
name: nextcloud
namespace: nextcloud
labels:
app.kubernetes.io/name: nextcloud
spec:
selector:
app.kubernetes.io/name: nextcloud
ports:
- name: port-80
port: 80
targetPort: 80
---
# CiliumNetworkPolicy egress allow (R-NET-02): nextcloud-allow-nextcloud-db-5432
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: nextcloud-allow-nextcloud-db-5432
namespace: nextcloud
labels:
app.kubernetes.io/name: nextcloud
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: nextcloud
egress:
- toEndpoints:
- matchLabels:
app.kubernetes.io/name: nextcloud-db
k8s:io.kubernetes.pod.namespace: nextcloud-db
toPorts:
- ports:
- port: '5432'
protocol: TCP
---
# CiliumNetworkPolicy gateway-ingress allow (R-NET-01/02): nextcloud-allow-gateway-ingress
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: nextcloud-allow-gateway-ingress
namespace: nextcloud
labels:
app.kubernetes.io/name: nextcloud
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: nextcloud
ingress:
- fromEntities:
- ingress
toPorts:
- ports:
- port: '80'
protocol: TCP
---
# CiliumNetworkPolicy world egress allow (egress class governs the path; R-NET-05): nextcloud-allow-internet-egress
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: nextcloud-allow-internet-egress
namespace: nextcloud
labels:
app.kubernetes.io/name: nextcloud
spec:
endpointSelector:
matchLabels:
app.kubernetes.io/name: nextcloud
egress:
- toEntities:
- world
- toEndpoints:
- matchLabels:
io.kubernetes.pod.namespace: keycloak
app: keycloak
---
# HTTPRoute nextcloud-80 (R-TLS/R-DNS)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: nextcloud-80
namespace: nextcloud
labels:
app.kubernetes.io/name: nextcloud
annotations:
homelab.keithjackson.dev/auth-mode: oidc
homelab.keithjackson.dev/auth-groups: platform-admins
spec:
parentRefs:
- name: shared
namespace: gateway
sectionName: https
hostnames:
- nextcloud.${BASE_DOMAIN}
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: nextcloud
port: 80
---
# PrometheusRule SLI alerts (R-OBS-01)
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: nextcloud-slis
namespace: nextcloud
labels:
release: kube-prometheus-stack
app.kubernetes.io/name: nextcloud
spec:
groups:
- name: nextcloud.slis
rules:
- alert: nextcloud-down
expr: kube_deployment_status_replicas_available{namespace="nextcloud", deployment="nextcloud"} == 0 or absent(kube_deployment_status_replicas_available{namespace="nextcloud", deployment="nextcloud"})
for: 5m
labels:
severity: critical
service: nextcloud
annotations:
summary: nextcloud is down
description: 'nextcloud: its Deployment has no available/ready replica for 5m, or the signal is gone entirely (SLI: up, via kube_deployment_status_replicas_available).'