# ============================================================================= # L2 BEHAVIORAL — certificate lifecycle is automatic (R-TLS-02) # [RUNNABLE / NOT YET EXECUTED — see STATUS] # Catalog: docs/14 "Trusted TLS (R-TLS-01/02)" # Spec: docs/00 R-TLS-02 — "Certificates MUST be automatically **issued, renewed, # and revoked** without manual steps." # docs/08 §"Renewal & revocation (R-TLS-02)" — "cert-manager renews well # before expiry automatically; no human action." / "Revocation: removing/ # rotating a certificate is a declarative change; cert-manager handles # issuance of replacements." # docs/08 acceptance gate — "Certificates renew automatically ahead of expiry # with no manual step (**observed across a renewal**)." # Impl: platform/cert-manager/**, platform/gateway/wildcard-certificate.yaml. # Sibling: tests/l0-static/tls-lifecycle-automated proves the "without manual steps" # clause statically (in the merge gate). This test observes the three VERBS. # # ENVIRONMENT-AGNOSTIC (R-ENV-02): NOTHING about either environment is written down. The # serving certificate is DISCOVERED — find the Gateway whose listener terminates TLS, follow # its certificateRefs Secret, and read that Secret's `cert-manager.io/certificate-name` # annotation to reach the Certificate. No hostname, no namespace list, no issuer name, no # address: dev proves dev's cert and prod proves prod's from the same authored assertions. # # GIVEN cert-manager reconciled by Flux and a Gateway terminating TLS, # WHEN the serving Certificate's provenance and renewal schedule are read, and a NEW # Certificate is then requested, left to renew itself, and removed, # THEN 1. the serving credential was produced by the CONTROLLER, not a human: its # Secret carries cert-manager's certificate-name annotation, its # CertificateRequest was submitted by cert-manager's own ServiceAccount and # was APPROVED automatically, and its issuer is a Ready ACME ClusterIssuer; # 2. an unattended RENEWAL IS SCHEDULED: status.renewalTime is in the future and # strictly before status.notAfter, so no human is in the renewal path; # 3. ISSUED — a newly declared Certificate becomes Ready and its Secret appears # with certificate material, with no step between declaring and having it; # 4. RENEWED — that certificate, given a renewal window that opens one minute # after issuance, REISSUES ITSELF at least twice (status.revision >= 3) and # the serving bytes actually change, with no human action. This is docs/08's # "observed across a renewal", fast-forwarded; # 5. REMOVED/REVOKED — deleting the Certificate withdraws its serving material. # # WHY A HERMETIC SELF-SIGNED ISSUER FOR STEPS 3-5, AND WHAT THAT COSTS # ------------------------------------------------------------------- # Steps 3-5 need a certificate lifecycle that completes in minutes and repeats on every # run. Driving that through the real ACME issuer would fire a public Let's Encrypt order # (and a Cloudflare _acme-challenge write) per conformance run, burning issuance rate # limits for evidence that is not about ACME: cert-manager's renewal scheduler and # ownership logic are issuer-independent. So steps 3-5 use a `selfSigned` Issuer created # in this test's OWN namespace, and the ACME/DNS-01 path is proven elsewhere — step 1-2 # read the REAL wildcard certificate the Gateway serves, and l2-behavioral/trusted-tls # proves a publicly-trusted LE certificate is actually served (R-TLS-01/03). # STATED PLAINLY: steps 3-5 do NOT prove the ACME order path renews; they prove the # CONTROLLER's issue/renew/withdraw automation runs unattended in this cluster, while # steps 1-2 prove the real serving certificate is on exactly that automated path. # # STATUS: RUNNABLE, NOT YET EXECUTED. Steps 1-2 are read-only. Steps 3-5 CREATE an # Issuer/Certificate (and DELETE the Certificate they created) inside chainsaw's # per-test namespace — the same class of mutation every other L2 test here performs, # self-cleaning via namespace teardown, touching no platform object. WS-TEST authored # them against live dev/prod read-only inspection under an explicit instruction not to # mutate either cluster, so THE MUTATING STEPS HAVE NOT BEEN RUN. Nothing about the # suite gates on them: ci/conformance.sh only runs L2/L3 when CLUSTER_AVAILABLE=true. # # EXPECTED FAILURE, DELIBERATELY NOT SOFTENED (R-TEST-06) # ------------------------------------------------------ # Step 5 WILL FAIL as the platform is authored today. cert-manager only garbage-collects # a Certificate's Secret when the controller runs with `enableCertificateOwnerRef: true`; # platform/cert-manager/helmrelease.yaml does not set it (chart default false), and the # live wildcard Secret on both clusters has NO ownerReferences (verified read-only, # 2026-09-05). So a removed Certificate leaves its credential in place, and "revoked # without manual steps" does not hold. That is a finding about the platform, not a # defect in this test: the assertion stays as the requirement reads and is escalated to # the PM. (ACME revocation proper is not implemented by cert-manager at all.) # # KNOWN, DELIBERATE GAPS (this test claims none of these): # * Not proven: that the CA would refuse a revoked certificate — there is no # revocation request to make, so nothing observable exists to assert. # * Not proven: renewal of the REAL wildcard cert (its window opens 30 days before a # 90-day expiry; a conformance run cannot wait, and forcing it would rotate the # platform's serving credential). Step 2 asserts the schedule; step 4 asserts the # mechanism that will act on it. # * Not proven: certificate expiry ALERTING — docs/08 delegates that to R-OBS-04 # (l3-fault-injection/alerting). # ============================================================================= apiVersion: chainsaw.kyverno.io/v1alpha1 kind: Test metadata: name: cert-lifecycle-automated labels: conformance.home.k3s/R-TLS-02: "true" annotations: conformance.home.k3s/requirements: "R-TLS-02" conformance.home.k3s/layer: "L2" conformance.home.k3s/safety-critical: "false" conformance.home.k3s/executable: "true" conformance.home.k3s/intent: >- Given cert-manager and a TLS-terminating Gateway, when the serving certificate's provenance and renewal schedule are read and a new certificate is requested, left to renew, and removed, then issuance, renewal and withdrawal all happen with no human step — the serving cert was controller-issued and auto-approved, its renewal is scheduled ahead of expiry, and a declared certificate issues and reissues itself unattended. spec: timeouts: apply: 60s assert: 3m exec: 10m steps: # ---------------------------------------------------------------- READ-ONLY HALF - name: given-cert-manager-is-reconciled-and-healthy description: >- The controller that issues and renews must be running, and must have been installed by Flux rather than by hand — otherwise "automatic" has no engine. try: - script: content: | set -u # Discover cert-manager's Deployments by chart label, in whatever namespace # the release lives (nothing about the environment is assumed). sel='app.kubernetes.io/instance=cert-manager' rows="$(kubectl get deploy -A -l "$sel" \ -o custom-columns='NS:.metadata.namespace,N:.metadata.name,RDY:.status.readyReplicas' \ --no-headers 2>/dev/null || true)" echo "$rows" [ -n "$rows" ] || { echo "FAIL: no cert-manager Deployment found ($sel)"; exit 1; } n=0 echo "$rows" | while read -r ns name rdy; do [ "${rdy:-0}" -ge 1 ] || { echo "FAIL: $ns/$name has no ready replica"; exit 1; } done || exit 1 n="$(echo "$rows" | wc -l)" # controller + webhook + cainjector: without the webhook a Certificate is # rejected, without cainjector the webhook has no CA — all three or nothing. [ "$n" -ge 3 ] || { echo "FAIL: expected >=3 cert-manager Deployments, saw $n"; exit 1; } # GitOps-managed, not hand-installed: Flux labels what it applies. ns="$(echo "$rows" | head -1 | awk '{print $1}')" lbl="$(kubectl get deploy -n "$ns" -l "$sel" \ -o jsonpath='{.items[0].metadata.labels.helm\.toolkit\.fluxcd\.io/name}' 2>/dev/null || true)" [ -n "$lbl" ] || { echo "FAIL: cert-manager is not owned by a Flux HelmRelease"; exit 1; } echo "CERT_MANAGER_HEALTHY_OK ($n deployments, flux HelmRelease $lbl)" check: (contains($stdout, 'CERT_MANAGER_HEALTHY_OK')): true - name: then-the-serving-certificate-was-issued-by-the-controller description: >- Follow the Gateway's terminating listener to the Secret it serves, and prove that Secret's provenance is cert-manager — annotation, CertificateRequest requester, automatic approval, and a Ready ACME issuer. try: - script: content: | set -u # --- discover the serving Secret from the Gateway itself ------------------- gw="$(kubectl get gateway -A \ -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{"\n"}{end}' \ 2>/dev/null | head -1)" [ -n "$gw" ] || { echo "FAIL: no Gateway found — nothing terminates TLS"; exit 1; } gns="${gw% *}"; gname="${gw#* }" sec="$(kubectl -n "$gns" get gateway "$gname" \ -o jsonpath='{.spec.listeners[?(@.tls.mode=="Terminate")].tls.certificateRefs[0].name}' \ 2>/dev/null | awk '{print $1}')" [ -n "$sec" ] || { echo "FAIL: Gateway $gns/$gname terminates no TLS listener"; exit 1; } echo "gateway=$gns/$gname serving-secret=$sec" # --- provenance (1): only the controller writes this annotation ------------ cert="$(kubectl -n "$gns" get secret "$sec" \ -o jsonpath='{.metadata.annotations.cert-manager\.io/certificate-name}' 2>/dev/null || true)" [ -n "$cert" ] || { echo "FAIL: Secret $gns/$sec carries no cert-manager.io/certificate-name annotation — it was not produced by cert-manager"; exit 1; } crt="$(kubectl -n "$gns" get secret "$sec" -o jsonpath='{.data.tls\.crt}' 2>/dev/null || true)" [ -n "$crt" ] || { echo "FAIL: Secret $gns/$sec holds no tls.crt"; exit 1; } # --- provenance (2): the Certificate is Ready and has a revision ----------- ready="$(kubectl -n "$gns" get certificate "$cert" \ -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}' 2>/dev/null || true)" rev="$(kubectl -n "$gns" get certificate "$cert" -o jsonpath='{.status.revision}' 2>/dev/null || true)" echo "certificate=$gns/$cert ready=$ready revision=$rev" [ "$ready" = "True" ] || { echo "FAIL: Certificate $cert is not Ready"; exit 1; } [ -n "$rev" ] && [ "$rev" -ge 1 ] || { echo "FAIL: Certificate $cert has no issued revision"; exit 1; } # --- provenance (3): the request was made BY THE CONTROLLER and auto-approved # spec.username is the identity that created the CertificateRequest. A human # operator would appear as a user/OIDC identity, not a ServiceAccount. cr="$(kubectl -n "$gns" get certificaterequest \ -o jsonpath="{range .items[?(@.metadata.annotations.cert-manager\.io/certificate-name=='${cert}')]}{.metadata.name}{' '}{.spec.username}{'\n'}{end}" \ 2>/dev/null | tail -1)" [ -n "$cr" ] || { echo "FAIL: no CertificateRequest for $cert — cannot show who requested it"; exit 1; } crname="${cr% *}"; requester="${cr#* }" echo "certificaterequest=$crname requester=$requester" case "$requester" in system:serviceaccount:*) : ;; *) echo "FAIL: CertificateRequest $crname was submitted by '$requester', not a controller ServiceAccount"; exit 1 ;; esac appr="$(kubectl -n "$gns" get certificaterequest "$crname" \ -o jsonpath='{.status.conditions[?(@.type=="Approved")].status}' 2>/dev/null || true)" [ "$appr" = "True" ] || { echo "FAIL: CertificateRequest $crname is not Approved=True (approval would be a manual step)"; exit 1; } # --- provenance (4): an automated ACME issuer, and it is Ready ------------- ikind="$(kubectl -n "$gns" get certificate "$cert" -o jsonpath='{.spec.issuerRef.kind}')" iname="$(kubectl -n "$gns" get certificate "$cert" -o jsonpath='{.spec.issuerRef.name}')" echo "issuer=$ikind/$iname" if [ "$ikind" = "ClusterIssuer" ]; then iready="$(kubectl get clusterissuer "$iname" -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}' 2>/dev/null || true)" iacme="$(kubectl get clusterissuer "$iname" -o jsonpath='{.spec.acme.server}' 2>/dev/null || true)" else iready="$(kubectl -n "$gns" get issuer "$iname" -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}' 2>/dev/null || true)" iacme="$(kubectl -n "$gns" get issuer "$iname" -o jsonpath='{.spec.acme.server}' 2>/dev/null || true)" fi [ "$iready" = "True" ] || { echo "FAIL: issuer $iname is not Ready — renewal would need a human to fix it"; exit 1; } [ -n "$iacme" ] || { echo "FAIL: issuer $iname is not an ACME issuer"; exit 1; } echo "SERVING_CERT_CONTROLLER_ISSUED_OK" check: (contains($stdout, 'SERVING_CERT_CONTROLLER_ISSUED_OK')): true - name: then-an-unattended-renewal-is-already-scheduled description: >- cert-manager writes status.renewalTime only after computing when it will renew. Asserting it is in the future and before expiry proves the renewal is scheduled and that nothing is waiting on a human (docs/08 "renews well before expiry"). try: - script: content: | set -u gw="$(kubectl get gateway -A \ -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{"\n"}{end}' | head -1)" gns="${gw% *}"; gname="${gw#* }" sec="$(kubectl -n "$gns" get gateway "$gname" \ -o jsonpath='{.spec.listeners[?(@.tls.mode=="Terminate")].tls.certificateRefs[0].name}' | awk '{print $1}')" cert="$(kubectl -n "$gns" get secret "$sec" \ -o jsonpath='{.metadata.annotations.cert-manager\.io/certificate-name}')" renew="$(kubectl -n "$gns" get certificate "$cert" -o jsonpath='{.status.renewalTime}')" after="$(kubectl -n "$gns" get certificate "$cert" -o jsonpath='{.status.notAfter}')" [ -n "$renew" ] || { echo "FAIL: Certificate $cert has no status.renewalTime — no renewal is scheduled"; exit 1; } [ -n "$after" ] || { echo "FAIL: Certificate $cert has no status.notAfter"; exit 1; } now_s="$(date -u +%s)" r_s="$(date -u -d "$renew" +%s)" a_s="$(date -u -d "$after" +%s)" echo "now=$(date -u -d "@$now_s" +%FT%TZ) renewalTime=$renew notAfter=$after" echo "renewal is in $(( (r_s - now_s) / 86400 ))d; expiry in $(( (a_s - now_s) / 86400 ))d" [ "$r_s" -gt "$now_s" ] || { echo "FAIL: renewalTime is in the PAST — the scheduled renewal did not happen"; exit 1; } [ "$r_s" -lt "$a_s" ] || { echo "FAIL: renewalTime is not before notAfter — renewal would come too late"; exit 1; } echo "RENEWAL_SCHEDULED_OK" check: (contains($stdout, 'RENEWAL_SCHEDULED_OK')): true # ------------------------------------------------- MUTATING HALF (hermetic namespace) - name: when-a-certificate-is-declared-then-it-is-issued-with-no-human-step description: >- Declare an Issuer and a Certificate in this test's own namespace, with a renewal window that opens one minute after issuance. Nothing else is done to them for the rest of the test — every later transition is the controller acting on its own. try: - apply: resource: # selfSigned: no external CA, no rate limit, no DNS write. See the header for # what this proves (controller automation) and what it does not (the ACME path). apiVersion: cert-manager.io/v1 kind: Issuer metadata: name: conformance-lifecycle spec: selfSigned: {} - apply: resource: apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: conformance-lifecycle spec: secretName: conformance-lifecycle-tls # 1h is cert-manager's minimum duration; renewing 59m before expiry puts the # renewal instant ~1 minute after each issuance, so an unattended renewal is # observable inside a test run instead of 30 days from now. duration: 1h renewBefore: 3540s commonName: conformance-lifecycle.invalid dnsNames: - conformance-lifecycle.invalid privateKey: rotationPolicy: Always issuerRef: kind: Issuer name: conformance-lifecycle - assert: resource: apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: conformance-lifecycle status: conditions: - type: Ready status: "True" revision: 1 - script: content: | set -u crt="$(kubectl -n "$NAMESPACE" get secret conformance-lifecycle-tls \ -o jsonpath='{.data.tls\.crt}' 2>/dev/null || true)" key="$(kubectl -n "$NAMESPACE" get secret conformance-lifecycle-tls \ -o jsonpath='{.data.tls\.key}' 2>/dev/null || true)" [ -n "$crt" ] && [ -n "$key" ] || { echo "FAIL: no certificate material was issued into conformance-lifecycle-tls"; exit 1; } # Record the serving bytes so the renewal step can prove they CHANGED. echo "$crt" | cksum > /tmp/conformance-cert-r1.cksum echo "revision-1 cert fingerprint: $(cat /tmp/conformance-cert-r1.cksum)" echo "CERT_ISSUED_OK" check: (contains($stdout, 'CERT_ISSUED_OK')): true - name: then-it-renews-itself-before-expiry-with-no-human-step description: >- Nothing touches the Certificate here. Poll until the controller has reissued it at least twice on its own and the serving bytes differ from revision 1 — docs/08's "observed across a renewal", and every request auto-approved. try: - script: content: | set -u want=3 # revision 1 = issuance; >=3 means TWO unattended renewals for i in $(seq 1 30); do rev="$(kubectl -n "$NAMESPACE" get certificate conformance-lifecycle \ -o jsonpath='{.status.revision}' 2>/dev/null || echo 0)" echo "poll $i: revision=${rev:-0}" [ "${rev:-0}" -ge "$want" ] && break sleep 20 done rev="$(kubectl -n "$NAMESPACE" get certificate conformance-lifecycle \ -o jsonpath='{.status.revision}' 2>/dev/null || echo 0)" [ "${rev:-0}" -ge "$want" ] || { echo "FAIL: revision reached only ${rev:-0} — cert-manager did not renew unattended"; exit 1; } # The serving material must actually have rotated, not just the revision count. now="$(kubectl -n "$NAMESPACE" get secret conformance-lifecycle-tls \ -o jsonpath='{.data.tls\.crt}' | cksum)" was="$(cat /tmp/conformance-cert-r1.cksum)" echo "r1=$was now=$now" [ "$now" != "$was" ] || { echo "FAIL: the Secret still holds the revision-1 certificate — nothing was actually reissued"; exit 1; } # Every renewal request was approved by the controller, never by a person. bad=0 for cr in $(kubectl -n "$NAMESPACE" get certificaterequest -o name 2>/dev/null); do st="$(kubectl -n "$NAMESPACE" get "$cr" \ -o jsonpath='{.status.conditions[?(@.type=="Approved")].status}' 2>/dev/null || true)" who="$(kubectl -n "$NAMESPACE" get "$cr" -o jsonpath='{.spec.username}' 2>/dev/null || true)" echo " $cr approved=$st requester=$who" [ "$st" = "True" ] || bad=1 case "$who" in system:serviceaccount:*) : ;; *) bad=1 ;; esac done [ "$bad" = 0 ] || { echo "FAIL: a CertificateRequest was not auto-approved, or was not submitted by a controller"; exit 1; } echo "CERT_RENEWED_UNATTENDED_OK (revision=$rev)" check: (contains($stdout, 'CERT_RENEWED_UNATTENDED_OK')): true - name: then-removing-the-certificate-withdraws-the-serving-material description: >- docs/08 reads revocation as "removing/rotating a certificate is a declarative change". The declarative change is the deletion; withdrawal of the credential must follow with no manual cleanup. EXPECTED TO FAIL TODAY — cert-manager needs `enableCertificateOwnerRef: true` for the Secret to be garbage-collected, and the tracked HelmRelease does not set it. Kept as the requirement reads (R-TEST-06). try: - delete: ref: apiVersion: cert-manager.io/v1 kind: Certificate name: conformance-lifecycle - script: content: | set -u for i in $(seq 1 15); do if ! kubectl -n "$NAMESPACE" get secret conformance-lifecycle-tls >/dev/null 2>&1; then echo "CERT_WITHDRAWN_OK (serving material gone after $((i * 4))s)" exit 0 fi sleep 4 done owners="$(kubectl -n "$NAMESPACE" get secret conformance-lifecycle-tls \ -o jsonpath='{.metadata.ownerReferences}' 2>/dev/null || true)" echo "FAIL: the Certificate was deleted but Secret conformance-lifecycle-tls still" echo " holds usable certificate material (ownerReferences=${owners:-none})." echo " Removing a certificate therefore does NOT withdraw the credential, so" echo " R-TLS-02's 'revoked ... without manual steps' does not hold. FIX:" echo " set enableCertificateOwnerRef: true in platform/cert-manager values" echo " (owner decision — WS-TEST does not weaken the assertion instead)." exit 1 check: (contains($stdout, 'CERT_WITHDRAWN_OK')): true finally: # Namespace teardown removes both objects anyway; explicit so a leftover Secret # from the expected failure above is never left holding key material. - script: content: | kubectl -n "$NAMESPACE" delete secret conformance-lifecycle-tls --ignore-not-found --wait=false || true kubectl -n "$NAMESPACE" delete certificate conformance-lifecycle --ignore-not-found --wait=false || true kubectl -n "$NAMESPACE" delete issuer conformance-lifecycle --ignore-not-found --wait=false || true rm -f /tmp/conformance-cert-r1.cksum || true