merge: alert when a backup schedule has NEVER run, and when one is paused
gate / gate (push) Has been cancelled
gate / conformance (push) Has been cancelled

VeleroNoRecentBackup could not see the failure that actually happened:
velero-daily sat paused having never produced a successful backup, so
velero_backup_last_successful_timestamp{schedule=...} did not exist and
time() - <absent series> yields no series. Prod had no volume backups and
nothing alerted, further masked by a resources-only schedule succeeding
daily. No velero_* metric describes a Schedule OBJECT, proven by scraping a
throwaway paused Schedule and finding zero lines, so this exposes the CRs
via kube-state-metrics customresource_state and keys on
defaultVolumesToFsBackup — the property, not a name list. Adds the repo's
first prometheus rule-unit-test harness, mutation-checked both ways.
This commit is contained in:
jakman85 committed 2026-09-12 23:49:26 -05:00
commit 4865734c5f
10 files changed
+787 -3

No files matched your search

+5 -1
View File
@@ -22,6 +22,7 @@ Individual stages:
make -C ci lint # yaml/lint (no cluster; live now)
make -C ci l0 # schema + Kyverno + L0 static conformance (no cluster; live now)
make -C ci l1 # L1 rendering checks (no cluster; live now)
make -C ci prom-rules # promtool rule-unit tests on alerts (no cluster; live now)
make -C ci conformance # L2/L3 in-cluster Chainsaw Job (SKIPS unless CLUSTER_AVAILABLE=true)
make -C ci test # unit tests for these scripts themselves (ci/tests/*.test.sh)
```
@@ -102,13 +103,16 @@ in that case — nothing goes untested — and says plainly that the *scoped* ch
```
ci/
Makefile targets: lint · l0 · l1 · conformance · gate · touched · test · tools · versions
Makefile targets: lint · l0 · l1 · prom-rules · conformance · gate · touched · test · tools · versions
versions.env PINNED tool versions (R-TEST-02) — single source of truth
lib.sh shared helpers: logging, tool checks, cluster detection, artifact paths
yamllint.yaml lint config for the yaml/lint stage
lint.sh Stage: yaml/lint
l0.sh Stage: schema validation + Kyverno L0 gate + L0 static conformance
l1.sh Stage: L1 rendering checks (Chainsaw L1 via the conformance runner)
prom-rules.sh Stage: `promtool test rules` over platform/monitoring's PrometheusRules
(harness: platform/monitoring/tests/). Catches the two alerting bugs no
other stage can see — permanently-firing and structurally INERT rules.
conformance.sh Stage: L2/L3 in-cluster Chainsaw Job (trigger + collect)
touched-requirements.sh compute the R-* subset a change touches (drives the R-TEST-03 filter);
exits 3 LOUDLY rather than guessing when the diff base is unreliable