From c6a480eed1fad052013608a5b5e768c6ad6212d0 Mon Sep 17 00:00:00 2001 From: Keith Jackson Date: Tue, 25 Aug 2026 00:10:56 -0500 Subject: [PATCH] =?UTF-8?q?test(net):=20L2=20conformance=20=E2=80=94=20exp?= =?UTF-8?q?osed=20app=20reachable=20through=20Gateway=20under=20default-de?= =?UTF-8?q?ny=20(R-NET-03)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New runnable chainsaw test gateway-ingress-reachable (R-NET-01/02/03): stands up an exposed-app fixture (namespace default-deny + the renderer's additive gateway-ingress CiliumNetworkPolicy allowing fromEntities:[ingress] to the declared port 8080 only, a backend listening on 8080 + an undeclared 9091, and declared/undeclared HTTPRoutes on the shared Gateway) and proves BEHAVIOR through the REAL Gateway VIP: * then-reachable-through-gateway -> the Gateway (ingress identity) reaches backend:8080 through default-deny (backend marker returned) [R-NET-02/03] * then-undeclared-port-denied -> port 9091 (no allow) is NOT reachable through the Gateway (marker never returned; envoy 503) [R-NET-02] * then-east-west-still-denied -> a non-ingress in-namespace peer is DROPPED on 8080; the allow is least-privilege, east-west default-deny intact (hermetic) [R-NET-01] Verified live: reachable=REACHED (200, marker), undeclared=DENIED, east-west=DENIED. Self-skipping: the gateway probes classify each attempt REACHED / DENIED / NO-GATEWAY (curl 000). NO-GATEWAY self-skips (green + note) because a node-local Cilium L7LB/envoy desync can make pods on an affected node unable to reach the Gateway VIP at all — a SEPARATE platform fault, not default-deny — so this test never false-fails on it. The east-west denial is hermetic and always asserts. VIP discovered per-step via a kubectl output binding (env-agnostic; chainsaw bindings are step-scoped). L2 layer stays gated by CLUSTER_AVAILABLE in run.sh. chainsaw v0.2.12 quirks honored (scalar/JMESPath asserts, no readinessProbe under default-deny, step-level finally delete). Co-Authored-By: Claude Opus 4.8 --- .../chainsaw-test.yaml | 302 ++++++++++++++++++ .../fixtures/app-and-probe.yaml | 178 +++++++++++ 2 files changed, 480 insertions(+) create mode 100644 conformance/tests/l2-behavioral/gateway-ingress-reachable/chainsaw-test.yaml create mode 100644 conformance/tests/l2-behavioral/gateway-ingress-reachable/fixtures/app-and-probe.yaml diff --git a/conformance/tests/l2-behavioral/gateway-ingress-reachable/chainsaw-test.yaml b/conformance/tests/l2-behavioral/gateway-ingress-reachable/chainsaw-test.yaml new file mode 100644 index 0000000..151aa03 --- /dev/null +++ b/conformance/tests/l2-behavioral/gateway-ingress-reachable/chainsaw-test.yaml @@ -0,0 +1,302 @@ +# ============================================================================= +# L2 BEHAVIORAL — Exposed app reachable through the Gateway under default-deny +# (R-NET-01, R-NET-02, R-NET-03) [RUNNABLE] +# Catalog: docs/14 (default-deny + explicit allow family) +# Spec: docs/03 "Default-deny baseline" / "Explicit allow rules"; +# docs/09 "Interaction with default-deny" (ingress opens NORTH-SOUTH only). +# +# THE GAP THIS CLOSES: an onboarded app's rendered default-deny has `ingress: []` +# (deny ALL), so the Cilium Gateway (Envoy, reserved identity `ingress`) cannot +# reach the backend — an Accepted HTTPRoute is still unreachable. The renderer now +# also emits an ADDITIVE gateway-ingress CiliumNetworkPolicy allowing ONLY the +# `ingress` identity to ONLY the declared port. This test proves that end-to-end +# through the REAL shared Gateway: +# +# GIVEN an exposed app namespace with default-deny + the gateway-ingress allow +# (fromEntities: [ingress] -> declared port 8080 only), attached to the +# shared Gateway with a declared route (8080) and an undeclared route (9091), +# WHEN a client dials the Gateway VIP for the DECLARED host -> THEN it reaches the +# backend (proves the additive allow opens the north-south path, R-NET-02/03); +# WHEN a client dials the Gateway VIP for the UNDECLARED port -> THEN the backend is +# NOT reached (default-deny still drops it, R-NET-02); +# WHEN an in-namespace peer (NOT the `ingress` identity) dials the declared port +# east-west -> THEN it is DENIED (the allow is least-privilege / gateway-only; +# default-deny east-west is intact, R-NET-01). +# +# This asserts observable BEHAVIOR through the live Gateway (green Job = the intended +# verdict), not object existence (R-TEST-01). +# +# TWO FAILURE MODES ARE DISTINGUISHED (per the WS-NET pod->Gateway-VIP reachability +# finding, DECISIONS-LOG): the gateway probes classify each attempt as +# * REACHED (backend marker in the body) -> the intended verdict for the declared host; +# * DENIED (a real HTTP status, e.g. 503, but NO backend marker) -> default-deny drop +# (the intended verdict for the undeclared port); +# * NO-GATEWAY (curl code 000 / connect timeout) -> the probe's node cannot reach the +# Gateway VIP AT ALL. That is a SEPARATE platform fault (a node-local Cilium L7LB/ +# envoy desync makes pods on an affected node unable to reach the Gateway VIP or +# ClusterIP), NOT default-deny. The probe SELF-SKIPS that case (exits 0 with a SKIP +# note) so this test never false-fails on the unrelated reachability bug. The +# hermetic east-west denial (below) needs no Gateway and always asserts. +# +# SELF-SKIPPING: the whole L2 layer is gated by CLUSTER_AVAILABLE in conformance/ +# run.sh (skipped with no cluster). Within the test, `given` asserts the shared +# Gateway is Programmed (a missing Gateway fails fast/diagnostically), and the gateway +# probes self-skip the node-local reachability fault as above. The backend uses NO +# readinessProbe: the default-deny baseline blocks the kubelet node->pod probe, so a +# probe-bearing pod would never go Ready (DECISIONS-LOG K4); `phase: Running` is the +# correct readiness signal. Cleanup deletes both fixture namespaces in `finally`. +# ============================================================================= +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: gateway-ingress-reachable + labels: + conformance.home.k3s/R-NET-01: "true" + conformance.home.k3s/R-NET-02: "true" + conformance.home.k3s/R-NET-03: "true" + annotations: + conformance.home.k3s/requirements: "R-NET-01, R-NET-02, R-NET-03" + conformance.home.k3s/layer: "L2" + conformance.home.k3s/safety-critical: "false" + conformance.home.k3s/executable: "true" + conformance.home.k3s/intent: >- + An exposed app is reachable through the shared Gateway under default-deny via an + additive gateway-ingress allow (ingress identity -> declared port only); an + undeclared port is not reachable through the Gateway and east-west stays denied. +spec: + # A generous timeout: pulling images + programming two HTTPRoutes on the Gateway. + timeouts: + apply: 60s + assert: 180s + steps: + # -- GIVEN: the exposed-app fixture (default-deny + gateway-ingress allow + backend + + # Service + declared/undeclared HTTPRoutes) and a probe namespace. Assert the shared + # Gateway is Programmed (precondition) and the backend pod is Running (listeners up). + # Each gateway-probe step below re-captures the Gateway VIP into a binding (chainsaw + # output bindings are step-scoped) so probes dial the real ingress address that + # split-horizon DNS resolves service hostnames to — env-agnostic, no hardcoded IP. + - name: given-exposed-app-under-default-deny + try: + - apply: + file: fixtures/app-and-probe.yaml + - assert: + resource: + apiVersion: gateway.networking.k8s.io/v1 + kind: Gateway + metadata: + name: shared + namespace: gateway + # JMESPath binding (parenthesized key) rather than a list literal: chainsaw does + # STRICT length-equal list matching, so asserting one element against the full + # conditions[] would fail (DECISIONS-LOG K4). This pinpoints Programmed=True. + status: + (conditions[?type == 'Programmed'] | [0].status): "True" + - assert: + resource: + apiVersion: v1 + kind: Pod + metadata: + labels: + app.kubernetes.io/name: gw-backend + namespace: conformance-gw-ingress + status: + # scalar match — Running means both httpd listeners are up (no readinessProbe + # under default-deny; DECISIONS-LOG K4). + phase: Running + + # -- THEN(reachable): a client in the unrestricted probe namespace dials the Gateway VIP + # for the DECLARED host. --connect-to sets SNI/Host to the hostname but dials the VIP + # (the real ingress address; -k because we test REACHABILITY, not TLS — trusted-tls + # covers the chain). The backend's marker in the body proves the Gateway (identity + # `ingress`) reached the backend THROUGH default-deny (green Job = REACHED). If the + # probe's node cannot reach the VIP at all (code 000), self-skip (green + SKIP note): + # that is the separate pod->Gateway-VIP platform fault, not default-deny. + - name: then-reachable-through-gateway + try: + # chainsaw output bindings are step-scoped, so re-capture the VIP in this step. + - command: + entrypoint: kubectl + args: [-n, gateway, get, gateway, shared, -o, "jsonpath={.status.addresses[0].value}"] + outputs: + - name: gwvip + value: ($stdout) + - apply: + resource: + apiVersion: batch/v1 + kind: Job + metadata: + name: probe-reachable + namespace: conformance-gw-probe + spec: + backoffLimit: 6 + activeDeadlineSeconds: 180 + template: + spec: + restartPolicy: Never + containers: + - name: probe + image: curlimages/curl:8.11.1 + env: + - name: GWVIP + value: ($gwvip) + command: + - sh + - -c + - | + set -u + host="gw-declared-conformance.home.keithjackson.dev" + last=000 + i=0 + while [ "$i" -lt 30 ]; do + code="$(curl -k -sS --max-time 8 -o /tmp/body \ + -w '%{http_code}' \ + --connect-to "${host}:443:${GWVIP}:443" \ + "https://${host}/" 2>/dev/null || echo 000)" + last="$code" + if grep -q GW_BACKEND_REACHED /tmp/body 2>/dev/null; then + echo "REACHED: Gateway reached the backend through default-deny (code ${code})" + exit 0 + fi + i=$((i+1)); sleep 5 + done + if [ "$last" = "000" ]; then + echo "SKIP: this node cannot reach the Gateway VIP (code 000) —" 1>&2 + echo " separate pod->Gateway-VIP platform fault, NOT default-deny" 1>&2 + exit 0 + fi + echo "FAIL: Gateway returned ${last} but the backend marker was absent — default-deny drop of the declared port?" 1>&2 + exit 1 + - assert: + resource: + apiVersion: batch/v1 + kind: Job + metadata: { name: probe-reachable, namespace: conformance-gw-probe } + status: { succeeded: 1 } + + # -- THEN(undeclared port denied through the Gateway): same Gateway VIP, same backend, the + # UNDECLARED host -> port 9091. Routing is identical to the declared route (proven to + # reach the backend), so the ONLY difference is the missing allow on 9091: the Gateway + # Envoy's upstream connect is DROPPED by default-deny. The backend marker MUST be + # absent. A leak (the 9091 marker appears) fails the Job. If the node cannot reach the + # VIP at all (code 000), self-skip (same platform fault as above). + - name: then-undeclared-port-denied-through-gateway + try: + # chainsaw output bindings are step-scoped, so re-capture the VIP in this step. + - command: + entrypoint: kubectl + args: [-n, gateway, get, gateway, shared, -o, "jsonpath={.status.addresses[0].value}"] + outputs: + - name: gwvip + value: ($stdout) + - apply: + resource: + apiVersion: batch/v1 + kind: Job + metadata: + name: probe-undeclared + namespace: conformance-gw-probe + spec: + backoffLimit: 4 + activeDeadlineSeconds: 150 + template: + spec: + restartPolicy: Never + containers: + - name: probe + image: curlimages/curl:8.11.1 + env: + - name: GWVIP + value: ($gwvip) + command: + - sh + - -c + - | + set -u + host="gw-undeclared-conformance.home.keithjackson.dev" + reached_gw=no + i=0 + while [ "$i" -lt 8 ]; do + code="$(curl -k -sS --max-time 8 -o /tmp/body \ + -w '%{http_code}' \ + --connect-to "${host}:443:${GWVIP}:443" \ + "https://${host}/" 2>/dev/null || echo 000)" + if grep -q GW_BACKEND_UNDECLARED /tmp/body 2>/dev/null; then + echo "LEAK: undeclared port 9091 reachable through the Gateway (code ${code})" 1>&2 + exit 1 + fi + [ "$code" != "000" ] && reached_gw=yes + i=$((i+1)); sleep 5 + done + if [ "$reached_gw" = "yes" ]; then + echo "DENIED: undeclared port not reachable through the Gateway (backend marker never returned)" + exit 0 + fi + echo "SKIP: this node cannot reach the Gateway VIP (code 000) —" 1>&2 + echo " separate pod->Gateway-VIP platform fault, NOT default-deny" 1>&2 + exit 0 + - assert: + resource: + apiVersion: batch/v1 + kind: Job + metadata: { name: probe-undeclared, namespace: conformance-gw-probe } + status: { succeeded: 1 } + + # -- THEN(east-west still denied): HERMETIC (no Gateway needed). An in-namespace peer + # (NOT the `ingress` identity), covered by the namespace-wide default-deny, dials the + # DECLARED port east-west. The gateway-ingress allow is least-privilege (only + # fromEntities: [ingress]), so this connect MUST be dropped — default-deny east-west + # is intact (R-NET-01). The probe EXITS 0 only when the connection is BLOCKED (green + # Job = denied). This step always runs the real assertion (independent of the pod->VIP + # reachability fault above). + - name: then-east-west-still-denied + try: + - apply: + resource: + apiVersion: batch/v1 + kind: Job + metadata: + name: probe-eastwest + namespace: conformance-gw-ingress + spec: + backoffLimit: 4 + activeDeadlineSeconds: 120 + template: + metadata: + labels: { app: eastwest-peer } + spec: + restartPolicy: Never + containers: + - name: probe + image: busybox:1.36.1 + command: + - sh + - -c + - | + set -u + # DNS resolves (baseline allows it); the TCP connect must drop. + if nc -w 5 gw-backend 8080 /dev/null; then + echo "LEAK: east-west reached the declared port under default-deny" 1>&2 + exit 1 + fi + echo "DENIED: east-west blocked; gateway allow is least-privilege" + exit 0 + - assert: + resource: + apiVersion: batch/v1 + kind: Job + metadata: { name: probe-eastwest, namespace: conformance-gw-ingress } + status: { succeeded: 1 } + # -- Cleanup: delete both fixture namespaces (fixed names, outside chainsaw's managed + # namespace). finally accepts delete (DECISIONS-LOG: step-level finally = script/ + # command/delete/wait only). + finally: + - delete: + ref: + apiVersion: v1 + kind: Namespace + name: conformance-gw-ingress + - delete: + ref: + apiVersion: v1 + kind: Namespace + name: conformance-gw-probe diff --git a/conformance/tests/l2-behavioral/gateway-ingress-reachable/fixtures/app-and-probe.yaml b/conformance/tests/l2-behavioral/gateway-ingress-reachable/fixtures/app-and-probe.yaml new file mode 100644 index 0000000..02f83a8 --- /dev/null +++ b/conformance/tests/l2-behavioral/gateway-ingress-reachable/fixtures/app-and-probe.yaml @@ -0,0 +1,178 @@ +# Fixtures for gateway-ingress-reachable (R-NET-01/02/03). +# +# Reproduces the WS-APPS onboarding shape the renderer emits for an EXPOSED service: +# an app namespace carrying the default-deny baseline (deny ALL ingress) PLUS the +# additive gateway-ingress CiliumNetworkPolicy that allows ONLY the reserved `ingress` +# identity (the Cilium Gateway/Envoy proxy) to the ONE declared service port. A second +# port listens but has NO allow -> it is the "undeclared port" that must stay denied +# even through the Gateway. +# +# The namespace opts into the shared Gateway (gateway-access: shared) exactly like a +# rendered app namespace, so its HTTPRoutes attach to the real platform Gateway and the +# probe exercises the true north-south path (Gateway Envoy -> backend), not a stand-in. +--- +apiVersion: v1 +kind: Namespace +metadata: + name: conformance-gw-ingress + labels: + homelab.keithjackson.dev/app-namespace: "true" + gateway-access: "shared" +--- +# Default-deny baseline: deny ALL ingress + egress, allow ONLY DNS egress. Byte-shape +# matches the renderer's r_default_deny / the platform baseline (R-NET-01). podSelector {} +# => every pod in the namespace, including the backend. +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: default-deny + namespace: conformance-gw-ingress +spec: + podSelector: {} + policyTypes: [Ingress, Egress] + ingress: [] + egress: + - to: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: kube-system + - podSelector: + matchLabels: + k8s-app: kube-dns + ports: + - protocol: UDP + port: 53 + - protocol: TCP + port: 53 +--- +# The ADDITIVE gateway-ingress allow the renderer now emits (render.py r_gateway_ingress_policy): +# ingress from the reserved `ingress` identity (the Cilium Gateway Envoy proxy) to ONLY the +# declared service port 8080. Port 9091 is deliberately NOT allowed (the undeclared port). +apiVersion: cilium.io/v2 +kind: CiliumNetworkPolicy +metadata: + name: gw-backend-allow-gateway-ingress + namespace: conformance-gw-ingress +spec: + endpointSelector: + matchLabels: + app.kubernetes.io/name: gw-backend + ingress: + - fromEntities: + - ingress + toPorts: + - ports: + - port: "8080" + protocol: TCP +--- +# Backend: two busybox httpd listeners. 8080 = the DECLARED/allowed port; 9091 = an +# UNDECLARED port that LISTENS (so a block is caused by POLICY, not a missing listener, +# per the deny-undeclared rationale). No readinessProbe: the default-deny baseline blocks +# the kubelet node->pod probe, so probe-bearing pods would never go Ready (DECISIONS-LOG K4). +apiVersion: apps/v1 +kind: Deployment +metadata: + name: gw-backend + namespace: conformance-gw-ingress +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: gw-backend + template: + metadata: + labels: + app.kubernetes.io/name: gw-backend + spec: + containers: + - name: declared + image: busybox:1.36.1 + command: + - sh + - -c + - | + mkdir -p /www + echo GW_BACKEND_REACHED > /www/index.html + exec httpd -f -p 8080 -h /www + - name: undeclared + image: busybox:1.36.1 + command: + - sh + - -c + - | + mkdir -p /www + echo GW_BACKEND_UNDECLARED > /www/index.html + exec httpd -f -p 9091 -h /www +--- +# Service addresses both listeners (addressability); POLICY, not the Service, decides what +# the Gateway may reach. 8080 declared+allowed; 9091 undeclared (no allow). +apiVersion: v1 +kind: Service +metadata: + name: gw-backend + namespace: conformance-gw-ingress +spec: + selector: + app.kubernetes.io/name: gw-backend + ports: + - name: declared + port: 8080 + targetPort: 8080 + - name: undeclared + port: 9091 + targetPort: 9091 +--- +# DECLARED HTTPRoute -> allowed port 8080. Attaches to the real shared Gateway. +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + name: gw-declared + namespace: conformance-gw-ingress +spec: + parentRefs: + - name: shared + namespace: gateway + sectionName: https + hostnames: + - gw-declared-conformance.home.keithjackson.dev + rules: + - matches: + - path: + type: PathPrefix + value: / + backendRefs: + - name: gw-backend + port: 8080 +--- +# UNDECLARED HTTPRoute -> undeclared port 9091. The route is Accepted (routing works), but +# the Gateway Envoy's connection to backend:9091 is DROPPED by default-deny (no allow) — so +# the Gateway cannot reach it. Proves an undeclared port stays denied even through ingress. +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + name: gw-undeclared + namespace: conformance-gw-ingress +spec: + parentRefs: + - name: shared + namespace: gateway + sectionName: https + hostnames: + - gw-undeclared-conformance.home.keithjackson.dev + rules: + - matches: + - path: + type: PathPrefix + value: / + backendRefs: + - name: gw-backend + port: 9091 +--- +# Probe namespace: NO default-deny, so probes can reach the Gateway Service (simulating a +# LAN/edge client). The Gateway then proxies to the backend as identity `ingress` — the +# path under test. Kept separate from the app namespace so the app's default-deny stays +# namespace-wide (podSelector {}) exactly like production. +apiVersion: v1 +kind: Namespace +metadata: + name: conformance-gw-probe