# =============================================================================
# L0 STATIC — every metric an alert rule queries is CLAIMED by a scrape target this repo configures
# Catalog: docs/14 L0 row ("Pre-apply, in CI") — runs WITHOUT a cluster.
#
# THE DEFECT THIS CLOSES. An alert rule whose metric is never scraped is not a broken alert — it is
# an ABSENT alert that reads as a present one. `longhorn_volume_robustness == 2` over a TSDB with no
# longhorn_* series returns no vector, so the rule is permanently quiet, permanently "healthy", and
# indistinguishable from a correct rule on a healthy cluster. It survives review (the PromQL is
# right), it survives `promtool test rules` (the unit test supplies the series itself), and it
# survives every dashboard (nobody looks for a panel that was never built).
#
# MEASURED ON PROD, 2026-09-13, before this change — 13 rules across three families were inert:
#   count(longhorn_volume_robustness)                        => (empty)   4 rules
#   count({__name__=~"certmanager.*"})                       => (empty)   3 rules
#   count(hubble_drop_total)                                 => (empty)   1 rule
#   count by (job) (up)                                      => 20 jobs, none Longhorn/cert-manager/Hubble
# So storage health was unmonitored on a cluster whose prod-1 had just had a failing SSD
# (DECISIONS-LOG), TLS expiry was unmonitored with the live `wildcard-home` wildcard 89 days from
# expiry, and the platform had no policy-drop visibility at all. The Flux family (gotk_*, 5 rules)
# was the same defect and was fixed separately in a79693d.
#
# GIVEN every alerting rule in every PrometheusRule in the repo (platform/**, apps/wave-*),
# WHEN  each rule's PromQL is parsed for the metric names it selects,
# THEN  (1) every metric name is CLAIMED by exactly one declared scrape owner, and each owner's
#           claim is PROVEN AGAINST THE REPO — the file that enables that scrape must still contain
#           the enabling declaration, so deleting `metrics.serviceMonitor.enabled` from
#           platform/longhorn/helmrelease.yaml fails this check instead of silently re-inerting four
#           storage alerts;
#       (2) an owner's NEGATIVE conditions hold too, where the enablement can be defeated without
#           being deleted — cert-manager's chart calls `fail` if podmonitor and servicemonitor are
#           both set, Longhorn's own NetworkPolicies drop the scrape unless
#           networkpolicy-metrics.yaml is wired, and kube_node_labels does not exist at all unless
#           kube-state-metrics' metricLabelsAllowlist names the label;
#       (3) a metric with NO owner FAILS, unless it is declared in KNOWN_UNWIRED_METRICS — a
#           shrink-only register (hubble_drop_total today, with the CNI-rollout reason recorded);
#       (4) `up{job="X"}` is audited on its own axis, because `up` always exists and so the metric
#           check above can never see this failure: where the repo authors a Service and a
#           ServiceMonitor in the SAME file, the monitor's endpoint port must be a port that Service
#           actually defines. All 12 apps/* `<app>-down` SLI rules used to fail this (the monitors
#           named port `metrics`; no app Service defined one) and were registered in
#           KNOWN_UNWIRED_JOBS with the live proof that their `up{job=...}` series did not exist.
#           THAT REGISTER IS NOW EMPTY BECAUSE THE TWELVE RULES WERE FIXED, not waived: they read
#           kube-state-metrics readiness of their own workload object instead (2026-09-13), the dead
#           ServiceMonitors are gone, and so is this axis' only subject. The monitor side therefore
#           moved into its own check — tests/l0-static/servicemonitor-ports-resolve — which needs no
#           alert rule to reach a monitor, resolves by selector across the whole tree instead of by
#           same-file adjacency, and proves its resolver on fixtures so it cannot pass merely for
#           lack of subjects;
#   AND (5) the check is LOAD-BEARING: this Test DECLARES `conformance.home.k3s/scope: whole-repo`,
#           so ci/l0.sh Stage 3b's `--include-scope whole-repo` unions it into EVERY gate
#           invocation, and ci/gate.sh blocks the merge when the l0 stage fails.
#
# WHY BOTH REGISTERS ARE RATCHETS AND NOT BLANKET FAILURES: the same reasoning as
# alert-rules-unit-tested. Failing outright on the 12 app SLI rules would have made the gate red
# until exporters existed for Plex, Pi-hole, Jackett, qBittorrent, Sonarr, Radarr, Seerr and friends
# — none of which speak Prometheus natively — so the gate would have been disabled, and a disabled
# gate protects nothing. Merely printing the gap is how these 13 rules stayed inert for three phases.
# (The ratchet did its job: the twelve were closed a day later without an exporter, by reading a
# signal that already existed. The register is empty now, and that is the outcome it was for.)
# So each gap is DECLARED, counted on every run, and CANNOT GROW: a new alert on an unclaimed
# metric FAILS, a new Service+ServiceMonitor pair with a mismatched port FAILS, and a STALE
# register entry (the gap was closed, or the alert deleted) FAILS too, so neither register can rot
# into fiction.
#
# STATUS: EXECUTABLE. `check.sh` is an offline assertion script (PyYAML, the same dependency as
# catalog/render.py and dev-nodes-provisioned-as-code); conformance/run.sh --layer l0 and ci/l0.sh
# run it directly, and this Chainsaw Test wraps the SAME script.
#
# NOT proven (all named in check.sh's header, which carries the full list). The one worth stating
# here because it bit this very repo: an owner claims metrics by PREFIX, so this proves the SCRAPE
# TARGET exists, never that the exporter emits that exact SERIES NAME.
# `kubevirt_vmi_phase_count{phase="Failed"}` sat inside the scraped kubevirt_ family on an UP target
# and did not exist at all on the deployed KubeVirt (corrected here to
# `kubevirt_vmi_info{phase="failed"}`, lowercase — a capital-F matcher is equally inert). Catching
# that needs the exporter's own metric list, i.e. a live read; it belongs to a future L1/L3 check
# that diffs alert metrics against /api/v1/label/__name__/values.
#
# Also not proven: that a target is UP. This is static — it asserts the
# enabling declaration is in git, never that the endpoint answers, that the chart still honours the
# value key, or that a NetworkPolicy is sufficient. Those were established by live prod reads
# recorded in the commit message (longhorn_volume_robustness 44 series, certmanager_* 104 series,
# all six new targets up==1). It also cannot check a monitor whose Service comes from a Helm chart
# rather than from this repo — the port name lives outside the tree — which is why (4) is scoped to
# same-file Service+ServiceMonitor pairs. And it does not judge thresholds or reachability-to-human
# (R-OBS-02 / the L3 alerting test).
# =============================================================================
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
  name: alert-metrics-have-scrape-targets
  labels:
    conformance.home.k3s/R-OBS-04: "true"
  annotations:
    conformance.home.k3s/requirements: "R-OBS-04"
    conformance.home.k3s/layer: "L0"
    # WHOLE-REPO SCOPE — this invariant quantifies over every PrometheusRule in the tree AND over
    # every scrape-enabling declaration that feeds them, which live in eight different components
    # (platform/monitoring, platform/longhorn, platform/cert-manager, platform/velero,
    # platform/kubevirt, platform/keycloak, platform/cnpg, apps/wave-*). No single requirement
    # path-maps to that set.
    #
    # WHY IT MUST NOT BE REQUIREMENT-SCOPED: ci/touched-requirements.sh maps apps/* to NOTHING and
    # platform/monitoring/* to R-OBS-01/R-OBS-02 — not R-OBS-04. A change that adds an alert under
    # apps/ or platform/kubevirt/, or one that flips a chart's `serviceMonitor.enabled` back to
    # false under platform/longhorn/, therefore attributes a requirement set that would FILTER THIS
    # CHECK OUT of exactly the change that re-inerts the alerts. Declaring the scope makes
    # ci/l0.sh Stage 3b union it into every run instead, and makes an honest SKIP loud (rc 3 =
    # UNWIRED, never green) rather than silently green.
    conformance.home.k3s/scope: "whole-repo"
    conformance.home.k3s/safety-critical: "false"
    conformance.home.k3s/executable: "true"
    conformance.home.k3s/intent: >-
      Every metric name queried by every alert rule is claimed by a scrape target whose enabling
      declaration is still present in this repo, every `up{job=...}` selector names a job a repo
      monitor can actually produce, and the two gaps that remain are enumerated in registers that
      can only shrink — so an alert can never again be authored, reviewed and merged against a
      metric nothing scrapes.
spec:
  steps:
    - name: audit-every-alert-metric-against-the-configured-scrape-targets
      description: >-
        Parse every metric name out of every alerting rule, resolve each against the declared
        scrape-owner table, prove each owner's enabling declaration (and its negative conditions)
        still hold in git, audit `up{job=...}` against same-file Service/ServiceMonitor port
        agreement, and ratchet both registers of declared gaps.
      try:
        - script:
            content: bash ./check.sh