--- # cert-manager — issues trusted Let's Encrypt certs via ACME DNS-01 (ADR-0006). Phase 0 STUB. apiVersion: helm.toolkit.fluxcd.io/v2 kind: HelmRelease metadata: name: cert-manager namespace: cert-manager spec: interval: 30m chart: spec: chart: cert-manager version: ">=1.21.0 <1.22.0" # Renovate-maintained range (bring-up is past; team uses ranges, not exact pins) sourceRef: kind: HelmRepository name: jetstack namespace: cert-manager install: remediation: retries: 3 upgrade: remediation: retries: 3 values: crds: enabled: true # install CRDs with the chart # DNS-01 self-check via PUBLIC resolvers. The k3s-dev cluster is NAT'd and its DNS can't resolve # the public _acme-challenge TXT that cert-manager creates in Cloudflare (VERIFIED: the record IS # publicly resolvable via 1.1.1.1/8.8.8.8, but the in-cluster self-check timed out — "not yet # propagated"). Point the propagation check straight at public resolvers so it never relies on # cluster/split-horizon DNS (docs/09 split-horizon note; realises the TODO that was here). dns01RecursiveNameservers: "1.1.1.1:53,8.8.8.8:53" dns01RecursiveNameserversOnly: true # --- SCRAPE THE TLS ISSUER, or every certificate alert is INERT (added 2026-09-13) ---------- # platform/monitoring/alerts.yaml ships CertificateExpiringSoon, CertificateExpiryImminent # (critical) and CertificateNotReady — and NONE of them could ever fire. The chart exposes the # controller's metrics port by default (`prometheus.enabled: true`, confirmed on prod: # Service/cert-manager listens on 9402) but defaults `prometheus.servicemonitor.enabled: false`, # and no ServiceMonitor existed in ns cert-manager. VERIFIED live on prod 2026-09-13: # `count({__name__=~"certmanager.*"})` returned NOTHING and `count by (job) (up)` listed no # cert-manager job — so certmanager_certificate_expiration_timestamp_seconds and # certmanager_certificate_ready_status were never in the TSDB. # # WHY THIS IS PART OF THE TLS PATH: the live wildcard `wildcard-home` (ns gateway) expires # 2026-12-12T02:16:57Z and is renewed by an ACME DNS-01 order that can fail quietly — a Cloudflare # token rotation, a DNS-01 self-check timeout (the very failure the two lines above work around), # a rate limit. With no scrape, the FIRST signal of a failed renewal is a browser TLS error for # every *.${BASE_DOMAIN} service. That is a user-discovered outage on a fully automated path, # which is exactly what R-TLS-01/R-OBS-04 exist to prevent. # # Chart mechanics, verified by rendering cert-manager v1.21.2 (the version this range resolves to): # * templates/servicemonitor.yaml renders on `prometheus.enabled AND # prometheus.servicemonitor.enabled`. Measured: `helm template --set # prometheus.servicemonitor.enabled=true` emits exactly 1 ServiceMonitor, 0 without it. # `prometheus.enabled` is already true by chart default, so this one line is sufficient. # * podmonitor and servicemonitor are MUTUALLY EXCLUSIVE — the chart calls `fail` if both are # set. Do not add `prometheus.podmonitor.enabled` alongside this. # * the rendered ServiceMonitor selects the controller/webhook/cainjector Services by # app.kubernetes.io/{name,instance,component} on targetPort `http-metrics`, and sets # `jobLabel: app.kubernetes.io/name` — so the scrape job lands as `cert-manager`. # * it carries the label `prometheus: default` (servicemonitor.prometheusInstance). Harmless # here: platform/monitoring sets serviceMonitorSelectorNilUsesHelmValues false, so Prometheus # selects every ServiceMonitor regardless of labels. Left at the chart default deliberately — # overriding it would imply a label-selecting Prometheus this cluster does not have. # Cost: 3 scrape targets, one certmanager_* series set — negligible. # # PROVEN END TO END ON PROD, 2026-09-13 21:0xZ, by applying the rendered ServiceMonitor by hand # (then removing it, so the merge's Helm upgrade owns it) — no NetworkPolicy was needed, as # neither ns cert-manager nor ns monitoring carries one: # up{namespace="cert-manager"} => 3 targets, all 1 # (jobs cert-manager / cainjector / webhook, # because jobLabel is app.kubernetes.io/name) # count({__name__=~"certmanager.*"}) => 104 (was EMPTY) # (certmanager_certificate_expiration_timestamp_seconds - time())/86400 # => 89.2 days for `wildcard-home` (ns gateway, # ClusterIssuer letsencrypt-prod) # certmanager_certificate_ready_status{condition="True"} => 1 # So the three Certificate* rules now evaluate against a real series and are correctly QUIET: # the wildcard is healthy and 89 days out, which is exactly what CertificateExpiringSoon's 14-day # threshold is meant to catch and could not have. prometheus: servicemonitor: enabled: true