Wire L1 render conformance tests to the renderer (R-NET-01, R-NET-05/06/07)

Implements the two L1 rendering tests that were failing scaffolds:
- egress-class-render (R-NET-05/06/07): renders vpn + internet fixtures, diffs each
  against a committed golden, asserts the vpn steering label is present on vpn and
  absent on internet (fail-closed), and that flipping the class changes ONLY the
  egress path (R-NET-07). This fixes the --requirements R-NET-05 merge gate.
- default-deny-baseline-render (R-NET-01): asserts every render carries the
  namespaced default-deny baseline, shaped deny-all + both directions.

Each test carries a pure-offline render-check.sh (renderer + golden diff, no
cluster). run.sh runs L1 render checks offline and emits a REAL pass/fail JUnit
(docs/14: L1 rendering assertions run in CI without a cluster) instead of skipping
L1 with the cluster-gated layers; the chainsaw-test.yaml wrappers invoke the same
render-check.sh so an in-cluster run executes the identical assertion.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed 2026-08-24 22:19:08 -05:00
1 parent 6b2c04ed5c
commit 4a017c590e
12 files changed
+851 -26

No files matched your search

@@ -11,7 +11,11 @@
# the L0 Kyverno gate (T-0.5). Rendering the baseline is necessary but NOT
# sufficient — enforcement is proven behaviorally at L2 (R-TEST-01).
#
# STATUS: scaffold — needs T-0.3 (renderer) + T-2.2 (baseline definition).
# STATUS: IMPLEMENTED (T-0.3). render-check.sh renders a service and asserts the
# namespaced default-deny baseline is present + shaped (deny-all, both directions) —
# a pure offline check (renderer + golden diff, no cluster). conformance/run.sh --layer
# l1 and ci/l1.sh run it directly; this Chainsaw Test wraps the SAME script so an
# in-cluster run executes the identical assertion. Rendering != enforcement (proven L2).
# =============================================================================
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
@@ -27,11 +31,10 @@ metadata:
Rendering any service's namespace yields a default-deny baseline policy.
spec:
steps:
- name: given-a-rendered-namespace
- name: render-and-assert-default-deny
description: >-
Render a service and assert the namespaced default-deny baseline is present and
shaped (deny-all ingress, both Ingress+Egress), matching the golden.
try:
- script:
content: 'echo "TODO(T-0.3/T-2.2): render a service; capture namespaced policies"'
- name: then-render-includes-default-deny
try:
- assert:
file: golden/default-deny-baseline.expected.yaml # TODO: author golden
content: bash ./render-check.sh
@@ -0,0 +1,8 @@
# L1 fixture: a minimal ServiceDefinition. Any onboarded service — regardless of its
# dependencies — must render the namespaced default-deny baseline (R-NET-01). This fixture
# declares NO dependencies, so its render must still carry the deny-all + DNS-only baseline.
# Validates against schemas/servicedefinition.schema.json.
name: baseline-demo
kind: container
source:
image: nginx:1.27
@@ -0,0 +1,103 @@
---
# Namespace (trust zone; default-deny + policies apply)
apiVersion: v1
kind: Namespace
metadata:
name: baseline-demo
labels:
homelab.keithjackson.dev/app-namespace: 'true'
---
# Default-deny baseline (R-NET-01): deny all, allow only DNS
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: baseline-demo
labels:
homelab.keithjackson.dev/baseline: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress: []
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
# Deployment (R-PLAT-02)
apiVersion: apps/v1
kind: Deployment
metadata:
name: baseline-demo
namespace: baseline-demo
labels:
app.kubernetes.io/name: baseline-demo
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: baseline-demo
template:
metadata:
labels:
app.kubernetes.io/name: baseline-demo
homelab.keithjackson.dev/egress-class: internet
homelab.keithjackson.dev/auth-mode: none
spec:
containers:
- name: baseline-demo
image: nginx:1.27
---
# ServiceMonitor (R-OBS-01)
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: baseline-demo
namespace: baseline-demo
labels:
release: kube-prometheus-stack
app.kubernetes.io/name: baseline-demo
spec:
selector:
matchLabels:
app.kubernetes.io/name: baseline-demo
namespaceSelector:
matchNames:
- baseline-demo
endpoints:
- port: metrics
interval: 30s
---
# Flux ImageRepository (R-UPD)
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
name: baseline-demo
namespace: baseline-demo
spec:
image: nginx
interval: 1h0m0s
---
# Flux ImagePolicy (R-UPD)
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
name: baseline-demo
namespace: baseline-demo
annotations:
homelab.keithjackson.dev/update-channel: stable
homelab.keithjackson.dev/auto-rollback: 'true'
spec:
imageRepositoryRef:
name: baseline-demo
policy:
semver:
range: '>=0.0.0'
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# =============================================================================
# L1 render check (offline, no cluster) — default-deny-baseline-render
# Serves: R-NET-01 (every namespace renders a default-deny baseline). Run by
# conformance/run.sh --layer l1 and ci/l1.sh. Pure bash + catalog/render.py.
#
# GIVEN any onboarded service (this fixture declares NO dependencies),
# WHEN its resources are rendered,
# THEN (1) the render matches the committed golden byte-for-byte, and
# (2) it includes a NetworkPolicy labelled baseline=default-deny that denies
# all ingress (ingress: []) and covers BOTH Ingress and Egress — the
# render-time partner of the Kyverno L0 gate and the L2 behavioral test.
# =============================================================================
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
RENDER="${ROOT}/catalog/render.py"
PYTHON="${PYTHON:-python3}"
LBL="homelab.keithjackson.dev"
tmp="$(mktemp -d)"
trap 'rm -rf "${tmp}"' EXIT
fail() { echo "RENDER-CHECK FAIL (default-deny-baseline-render): $*" >&2; exit 1; }
"${PYTHON}" "${RENDER}" "${DIR}/fixtures/svcdef.yaml" > "${tmp}/out.yaml"
# (1) golden match.
diff -u "${DIR}/golden/expected.yaml" "${tmp}/out.yaml" || fail "render drifted from golden"
# (2) the baseline is present and shaped as deny-all + both directions.
grep -q "${LBL}/baseline: default-deny" "${tmp}/out.yaml" \
|| fail "render is missing the default-deny baseline NetworkPolicy (R-NET-01)"
# Structural assertion via the renderer's own parse (deterministic, no cluster):
# find the default-deny NetworkPolicy and assert its shape.
"${PYTHON}" - "${tmp}/out.yaml" <<'PY' || fail "default-deny baseline is malformed (R-NET-01)"
import sys, yaml
docs = [d for d in yaml.safe_load_all(open(sys.argv[1])) if d]
np = [d for d in docs
if d.get("kind") == "NetworkPolicy"
and d.get("metadata", {}).get("labels", {}).get(
"homelab.keithjackson.dev/baseline") == "default-deny"]
assert len(np) == 1, f"expected exactly one default-deny baseline, got {len(np)}"
spec = np[0]["spec"]
assert spec.get("ingress") == [], "baseline must deny ALL ingress (ingress: [])"
assert set(spec.get("policyTypes", [])) == {"Ingress", "Egress"}, \
"baseline must cover BOTH Ingress and Egress"
PY
echo "render-check OK: default-deny-baseline-render (R-NET-01)"
@@ -13,9 +13,12 @@
# (and `egress: internet` does NOT), and flipping the class only changes the
# egress path (R-NET-07), not the workload spec.
#
# STATUS: scaffold — needs T-0.3 (renderer contract) + T-3.1/T-3.2 (egress design).
# Rendering assertions can run without live behavior, but Chainsaw applies to a
# cluster; until Phase 1 this is skipped by run.sh like every other layer.
# STATUS: IMPLEMENTED (T-0.3). The render assertion is a pure offline check
# (render-check.sh: renderer + golden diff, no cluster). conformance/run.sh --layer l1
# and ci/l1.sh run render-check.sh directly WITHOUT a cluster (docs/14: "L1 rendering
# assertions run in CI without a cluster"). This Chainsaw Test wraps the SAME
# render-check.sh in a script step so an in-cluster Chainsaw run executes the identical
# assertion — rendering the policy, NOT enforcing it (enforcement is proven at L2/L3).
# =============================================================================
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
@@ -33,19 +36,11 @@ metadata:
egress: vpn renders the egress-gateway + kill-switch path; internet does not.
spec:
steps:
- name: given-vpn-servicedefinition-render
- name: render-and-assert-egress-class
description: >-
Render the vpn + internet fixtures and assert: each matches its golden; the vpn
render carries the egress-class=vpn steering label and internet does not; and the
two renders differ ONLY in the egress-class value (R-NET-07).
try:
- script:
# TODO(T-0.3): invoke the renderer on a fixture ServiceDefinition
# (egress: vpn) and capture rendered YAML to $RENDER_OUT.
content: 'echo "TODO: render fixtures/svcdef-egress-vpn.yaml -> rendered.yaml"'
- name: then-render-contains-killswitched-egress
try:
- assert:
# TODO(T-3.x): assert the rendered set includes the egress-gateway
# steering + kill-switch construct for a vpn-class workload.
file: golden/egress-vpn.expected.yaml # TODO: author golden file
- name: then-internet-render-omits-vpn-path
try:
- script:
content: 'echo "TODO: render egress: internet fixture; assert NO vpn/gateway steer"'
content: bash ./render-check.sh
@@ -0,0 +1,30 @@
# L1 fixture: an internet-class ServiceDefinition. BYTE-IDENTICAL to svcdef-egress-vpn.yaml
# EXCEPT `egress: internet`. The two renders must differ ONLY in the egress path (R-NET-07):
# the internet render carries NO vpn steering label. Validates against the schema.
name: sonarr
kind: container
source:
image: linuxserver/sonarr
exposure: internal
egress: internet
storage:
- name: config
size: 2Gi
class: longhorn
ingressRoutes:
- host: sonarr.home.keithjackson.dev
port: 8989
auth:
mode: forward-auth
groups:
- media-admins
update:
channel: stable
cadence: "0 4 * * *"
autoRollbackOnFailedHealth: true
observability:
metrics: true
slis:
- up
backup:
policy: daily-offcluster
@@ -0,0 +1,31 @@
# L1 fixture: a vpn-class ServiceDefinition. Identical to svcdef-egress-internet.yaml
# EXCEPT `egress: vpn`, so the render diff isolates the egress path (R-NET-05/07).
# sonarr is in the normative vpn-class set (docs/03/13). Validates against
# schemas/servicedefinition.schema.json.
name: sonarr
kind: container
source:
image: linuxserver/sonarr
exposure: internal
egress: vpn
storage:
- name: config
size: 2Gi
class: longhorn
ingressRoutes:
- host: sonarr.home.keithjackson.dev
port: 8989
auth:
mode: forward-auth
groups:
- media-admins
update:
channel: stable
cadence: "0 4 * * *"
autoRollbackOnFailedHealth: true
observability:
metrics: true
slis:
- up
backup:
policy: daily-offcluster
@@ -0,0 +1,200 @@
---
# Namespace (trust zone; default-deny + policies apply)
apiVersion: v1
kind: Namespace
metadata:
name: sonarr
labels:
homelab.keithjackson.dev/app-namespace: 'true'
gateway-access: shared
annotations:
homelab.keithjackson.dev/backup-policy: daily-offcluster
---
# Default-deny baseline (R-NET-01): deny all, allow only DNS
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: sonarr
labels:
homelab.keithjackson.dev/baseline: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress: []
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
# PVC sonarr-config (R-STATE)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: sonarr-config
namespace: sonarr
labels:
app.kubernetes.io/name: sonarr
spec:
accessModes:
- ReadWriteOnce
storageClassName: longhorn
resources:
requests:
storage: 2Gi
---
# Deployment (R-PLAT-02)
apiVersion: apps/v1
kind: Deployment
metadata:
name: sonarr
namespace: sonarr
labels:
app.kubernetes.io/name: sonarr
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: sonarr
template:
metadata:
labels:
app.kubernetes.io/name: sonarr
homelab.keithjackson.dev/egress-class: internet
homelab.keithjackson.dev/auth-mode: forward-auth
annotations:
homelab.keithjackson.dev/auth-mode: forward-auth
homelab.keithjackson.dev/auth-groups: media-admins
spec:
containers:
- name: sonarr
image: linuxserver/sonarr
ports:
- containerPort: 8989
volumeMounts:
- name: config
mountPath: /data/config
volumes:
- name: config
persistentVolumeClaim:
claimName: sonarr-config
---
# Service
apiVersion: v1
kind: Service
metadata:
name: sonarr
namespace: sonarr
labels:
app.kubernetes.io/name: sonarr
spec:
selector:
app.kubernetes.io/name: sonarr
ports:
- name: port-8989
port: 8989
targetPort: 8989
---
# HTTPRoute sonarr-8989 (R-TLS/R-DNS)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: sonarr-8989
namespace: sonarr
labels:
app.kubernetes.io/name: sonarr
annotations:
homelab.keithjackson.dev/auth-mode: forward-auth
homelab.keithjackson.dev/auth-groups: media-admins
spec:
parentRefs:
- name: shared
namespace: gateway
sectionName: https
hostnames:
- sonarr.home.keithjackson.dev
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: sonarr
port: 8989
---
# ServiceMonitor (R-OBS-01)
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: sonarr
namespace: sonarr
labels:
release: kube-prometheus-stack
app.kubernetes.io/name: sonarr
spec:
selector:
matchLabels:
app.kubernetes.io/name: sonarr
namespaceSelector:
matchNames:
- sonarr
endpoints:
- port: metrics
interval: 30s
---
# PrometheusRule SLI alerts (R-OBS-01)
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: sonarr-slis
namespace: sonarr
labels:
release: kube-prometheus-stack
app.kubernetes.io/name: sonarr
spec:
groups:
- name: sonarr.slis
rules:
- alert: sonarr-down
expr: up{job="sonarr"} == 0
for: 5m
labels:
severity: critical
service: sonarr
annotations:
summary: sonarr is down
description: 'sonarr has been unreachable for 5m (SLI: up).'
---
# Flux ImageRepository (R-UPD)
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
name: sonarr
namespace: sonarr
spec:
image: linuxserver/sonarr
interval: 1h0m0s
---
# Flux ImagePolicy (R-UPD)
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
name: sonarr
namespace: sonarr
annotations:
homelab.keithjackson.dev/update-channel: stable
homelab.keithjackson.dev/auto-rollback: 'true'
spec:
imageRepositoryRef:
name: sonarr
policy:
semver:
range: '>=0.0.0'
@@ -0,0 +1,200 @@
---
# Namespace (trust zone; default-deny + policies apply)
apiVersion: v1
kind: Namespace
metadata:
name: sonarr
labels:
homelab.keithjackson.dev/app-namespace: 'true'
gateway-access: shared
annotations:
homelab.keithjackson.dev/backup-policy: daily-offcluster
---
# Default-deny baseline (R-NET-01): deny all, allow only DNS
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: sonarr
labels:
homelab.keithjackson.dev/baseline: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress: []
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
# PVC sonarr-config (R-STATE)
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: sonarr-config
namespace: sonarr
labels:
app.kubernetes.io/name: sonarr
spec:
accessModes:
- ReadWriteOnce
storageClassName: longhorn
resources:
requests:
storage: 2Gi
---
# Deployment (R-PLAT-02)
apiVersion: apps/v1
kind: Deployment
metadata:
name: sonarr
namespace: sonarr
labels:
app.kubernetes.io/name: sonarr
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: sonarr
template:
metadata:
labels:
app.kubernetes.io/name: sonarr
homelab.keithjackson.dev/egress-class: vpn
homelab.keithjackson.dev/auth-mode: forward-auth
annotations:
homelab.keithjackson.dev/auth-mode: forward-auth
homelab.keithjackson.dev/auth-groups: media-admins
spec:
containers:
- name: sonarr
image: linuxserver/sonarr
ports:
- containerPort: 8989
volumeMounts:
- name: config
mountPath: /data/config
volumes:
- name: config
persistentVolumeClaim:
claimName: sonarr-config
---
# Service
apiVersion: v1
kind: Service
metadata:
name: sonarr
namespace: sonarr
labels:
app.kubernetes.io/name: sonarr
spec:
selector:
app.kubernetes.io/name: sonarr
ports:
- name: port-8989
port: 8989
targetPort: 8989
---
# HTTPRoute sonarr-8989 (R-TLS/R-DNS)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: sonarr-8989
namespace: sonarr
labels:
app.kubernetes.io/name: sonarr
annotations:
homelab.keithjackson.dev/auth-mode: forward-auth
homelab.keithjackson.dev/auth-groups: media-admins
spec:
parentRefs:
- name: shared
namespace: gateway
sectionName: https
hostnames:
- sonarr.home.keithjackson.dev
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: sonarr
port: 8989
---
# ServiceMonitor (R-OBS-01)
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: sonarr
namespace: sonarr
labels:
release: kube-prometheus-stack
app.kubernetes.io/name: sonarr
spec:
selector:
matchLabels:
app.kubernetes.io/name: sonarr
namespaceSelector:
matchNames:
- sonarr
endpoints:
- port: metrics
interval: 30s
---
# PrometheusRule SLI alerts (R-OBS-01)
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: sonarr-slis
namespace: sonarr
labels:
release: kube-prometheus-stack
app.kubernetes.io/name: sonarr
spec:
groups:
- name: sonarr.slis
rules:
- alert: sonarr-down
expr: up{job="sonarr"} == 0
for: 5m
labels:
severity: critical
service: sonarr
annotations:
summary: sonarr is down
description: 'sonarr has been unreachable for 5m (SLI: up).'
---
# Flux ImageRepository (R-UPD)
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
name: sonarr
namespace: sonarr
spec:
image: linuxserver/sonarr
interval: 1h0m0s
---
# Flux ImagePolicy (R-UPD)
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
name: sonarr
namespace: sonarr
annotations:
homelab.keithjackson.dev/update-channel: stable
homelab.keithjackson.dev/auto-rollback: 'true'
spec:
imageRepositoryRef:
name: sonarr
policy:
semver:
range: '>=0.0.0'
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# =============================================================================
# L1 render check (offline, no cluster) — egress-class-render
# Serves: R-NET-05 (egress is a declared property), R-NET-06/07 (vpn steering /
# flip only changes the egress path). Run by conformance/run.sh --layer l1 and by
# ci/l1.sh. Pure bash + the renderer (catalog/render.py); no cluster, no chainsaw.
#
# GIVEN two ServiceDefinitions identical except `egress: vpn` vs `egress: internet`,
# WHEN the renderer produces resources,
# THEN (1) each matches its committed golden byte-for-byte,
# (2) the vpn render carries the egress-class=vpn steering label and the
# internet render does NOT (fail-closed: no silent vpn path on internet),
# (3) the two renders differ ONLY in the egress-class value (R-NET-07):
# flipping the class changes the egress path, not the workload spec.
# =============================================================================
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "${DIR}/../../../.." && pwd)" # repo root
RENDER="${ROOT}/catalog/render.py"
PYTHON="${PYTHON:-python3}"
LBL="homelab.keithjackson.dev"
tmp="$(mktemp -d)"
trap 'rm -rf "${tmp}"' EXIT
fail() { echo "RENDER-CHECK FAIL (egress-class-render): $*" >&2; exit 1; }
# (1) golden match for both classes.
"${PYTHON}" "${RENDER}" "${DIR}/fixtures/svcdef-egress-vpn.yaml" > "${tmp}/vpn.yaml"
"${PYTHON}" "${RENDER}" "${DIR}/fixtures/svcdef-egress-internet.yaml" > "${tmp}/internet.yaml"
diff -u "${DIR}/golden/egress-vpn.expected.yaml" "${tmp}/vpn.yaml" || fail "vpn render drifted from golden"
diff -u "${DIR}/golden/egress-internet.expected.yaml" "${tmp}/internet.yaml" || fail "internet render drifted from golden"
# (2) vpn steering label present on vpn, absent on internet (R-NET-05, fail-closed).
grep -q "${LBL}/egress-class: vpn" "${tmp}/vpn.yaml" \
|| fail "vpn render is missing the egress-class=vpn steering label"
if grep -q "${LBL}/egress-class: vpn" "${tmp}/internet.yaml"; then
fail "internet render must NOT carry the vpn steering label (fail-open leak)"
fi
grep -q "${LBL}/egress-class: internet" "${tmp}/internet.yaml" \
|| fail "internet render is missing the egress-class=internet label"
# (3) R-NET-07: the only difference is the egress-class value. Neutralize the class
# value in each render; the rest MUST be byte-identical.
sed 's#egress-class: vpn#egress-class: CLASS#' "${tmp}/vpn.yaml" > "${tmp}/vpn.norm"
sed 's#egress-class: internet#egress-class: CLASS#' "${tmp}/internet.yaml" > "${tmp}/internet.norm"
diff -u "${tmp}/vpn.norm" "${tmp}/internet.norm" \
|| fail "flipping egress class changed more than the egress path (R-NET-07 violated)"
echo "render-check OK: egress-class-render (R-NET-05/06/07)"