jakman85andClaude Opus 5
f92707f38c
Lets the operator's Android / Linux / Windows / macOS devices dial in from anywhere and behave like LAN machines: full reach to 192.168.2.0/24 plus internet egress out the house WAN. Opposite direction to platform/vpn-egress, which sends pods OUT through PIA; the two share nothing but the word "VPN". WireGuard because it is the only option with a first-party client on all four named platforms, and because its handshake MAC is keyed with the responder's own public key: an unauthenticated packet gets NO reply, so udp/51820 is indistinguishable from a closed port. That, not "it is encrypted", is what makes an unfiltered WAN port defensible, and why there is no source allowlist. hostNetwork: true is REQUIRED, not convenient. A pod-networked server must SNAT clients to its own pod IP, so every remote client would carry that pod's Cilium identity -- and a pod-sourced request to the shared Gateway VIP returns 403 from Envoy for ALL 16 hostnames (measured 2026-10-04), because Cilium's Gateway preserves the original source identity and enforces the destination's ingress policy. The obvious build would have shipped a healthy-looking tunnel on which every cluster web UI was 403 and *.home.keithjackson.dev did not even resolve. In the host netns traffic is SNATed to the node and arrives as host/remote-node, exactly like a LAN client. It also sidesteps the unsafe ip_forward sysctl, the L2-announcement lease lottery, and double encapsulation (hence MTU 1420). Permitted by restrict-host-namespaces' own carve-out for platform namespaces. The cost is stated in networkpolicy.yaml rather than papered over: no NetworkPolicy constrains a hostNetwork pod. A DaemonSet so the hand-made WAN forward cannot be invalidated by a reschedule; all three nodes answer the same keys, so failover is a one-line router change. Peer PUBLIC keys are plaintext in peers.yaml, which makes "who can get into my house" a reviewable PR diff. The only secret -- the server's private key -- is an out-of-band Secret: an agent must not mint a long-lived private key, and the two halves are joined only at pod start into a tmpfs that dies with the pod. Ships INERT-BUT-HEALTHY with zero peers. THREE off-repo manual steps remain (kernel module, server key, WAN forward) -- runbooks/vpn-remote-access.md. Touches clusters/** , so ci/approval-policy.sh will block pending the operator's /approve-override. Deliberately not posted by me (CLAUDE.md §1). Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
82 lines
7.2 KiB
YAML
82 lines
7.2 KiB
YAML
---
|
|
# Base Flux entrypoint set. Shared by BOTH environment overlays (clusters/dev, clusters/prod).
|
|
# Overlays reference this base and patch ONLY the cluster-vars ConfigMap (R-ENV-02).
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
kind: Kustomization
|
|
resources:
|
|
- flux-system.yaml # GitRepository (reconciliation source)
|
|
# Cilium (the CNI) — its OWN Kustomization, split OUT of `platform` (2026-09-01). The mirror image
|
|
# of the oauth2-proxy/headlamp splits: those are leaves that must never gate a tier (wait:false);
|
|
# Cilium is the one component that legitimately SHOULD gate everything, so it keeps wait:true and
|
|
# `platform` dependsOn it. Splitting it out buys a per-CNI Ready condition, CNI-appropriate
|
|
# interval/timeout, and `flux suspend kustomization cilium` as an emergency valve that does not
|
|
# freeze the other eleven platform components. See clusters/base/cilium.yaml.
|
|
- cilium.yaml # Flux Kustomization -> ./platform/cilium (dependsOn flux-system, wait:true)
|
|
# Intel GPU device plugin — its OWN Kustomization (dependsOn cilium, wait:true), NOT a line in
|
|
# platform/kustomization.yaml. Same reasoning as the Cilium split-out, applied to a leaf: inside
|
|
# that wait:true tier a wedged 3-node DaemonSet would be indistinguishable from eleven unrelated
|
|
# components AND would stall platform-config + every apps wave on "dependency not ready" — so a
|
|
# component added to fix a silent no-op (Plex software transcoding) could cause a real outage.
|
|
# wait:true here is a HEALTH SIGNAL, not a gate: nothing in the graph dependsOn it.
|
|
# It advertises `gpu.intel.com/i915`; the renderer change that REQUESTS that resource must land
|
|
# strictly later. See clusters/base/intel-gpu-plugin.yaml.
|
|
- intel-gpu-plugin.yaml # Flux Kustomization -> ./platform/intel-gpu-plugin (dependsOn cilium)
|
|
- platform.yaml # Flux Kustomization -> ./platform (CONTROLLERS: HelmReleases / CRDs)
|
|
- platform-config.yaml # Flux Kustomization -> ./platform-config (CONFIGS: CRs; dependsOn platform)
|
|
# SECOND-ORDER CDI configs — CRs whose CRD is created by a controller AT RUNTIME, not by any
|
|
# manifest in git. cdi-operator.yaml registers only `cdis.cdi.kubevirt.io`; `storageprofiles` (and
|
|
# datavolumes, datasources, …) appear only after cdi-operator reconciles the `CDI` CR that
|
|
# platform-config applies. Because Flux dry-runs every object in a pass before applying any of them,
|
|
# keeping the StorageProfiles in platform-config was a TRUE DEADLOCK (unknown kind -> pass aborts ->
|
|
# CR never applied -> CRD never created -> tier never converges), invisible on dev and fatal on any
|
|
# cold cluster — observed on prod 2026-09-05. This tier is the durable fix; `standup.sh crds` stays
|
|
# as the belt-and-braces repair for clusters already wedged. See clusters/base/platform-config-cdi.yaml.
|
|
- platform-config-cdi.yaml # Flux Kustomization -> ./platform-config-cdi (dependsOn platform-config)
|
|
# App tier, split into dependency-ordered WAVES (was the single ./apps `apps` Kustomization).
|
|
# Each apps-wave-<N> dependsOn wave N-1 (wave 0 dependsOn platform-config) and health-gates
|
|
# (wait:true) ONLY its own wave, so a crashloop in a later wave can't stall an earlier one.
|
|
# GENERATED by `catalog/render.py --emit-waves`; add a line here only when a NEW deepest wave
|
|
# is introduced (a service deeper than any existing wave). See DESIGN-waves.md.
|
|
- apps-wave-0.yaml # Flux Kustomization -> ./apps/wave-0 (dependsOn platform-config)
|
|
- apps-wave-1.yaml # Flux Kustomization -> ./apps/wave-1 (dependsOn apps-wave-0)
|
|
- apps-wave-2.yaml # Flux Kustomization -> ./apps/wave-2 (dependsOn apps-wave-1)
|
|
# VM tier (KubeVirt) — non-gating (wait:false), separate lifecycle from the app waves.
|
|
- apps-vms.yaml # Flux Kustomization -> ./apps/vms (dependsOn platform-config)
|
|
- conformance.yaml # Flux Kustomization -> ./conformance/incluster (non-gating; R-SEC-02 trigger identity)
|
|
# oauth2-proxy forward-auth — POST-platform tier (dependsOn platform-config, wait:false). A leaf
|
|
# service that needs Keycloak + trusted TLS + split-horizon DNS must NOT sit in the gating
|
|
# `platform` tier, where it self-gated its own dependencies into a cold-start deadlock (twice).
|
|
- oauth2-proxy.yaml # Flux Kustomization -> ./platform/oauth2-proxy (dependsOn platform-config)
|
|
# headlamp dashboard — POST-platform tier (dependsOn platform-config, wait:false). A leaf UI whose
|
|
# HelmRelease can crashloop (upstream headlamp-0.40.1 `-session-ttl` chart/image desync) must NOT
|
|
# sit in the gating `platform` tier, where its unhealth stalled platform-config + oauth2-proxy +
|
|
# every apps wave/vm on "dependency not ready" (DECISIONS-LOG: "Headlamp gating stall").
|
|
- headlamp.yaml # Flux Kustomization -> ./platform/headlamp (dependsOn platform-config)
|
|
# Homepage apex landing dashboard — POST-platform tier (dependsOn platform-config, wait:false).
|
|
# A leaf UI at home.keithjackson.dev; same non-gating rationale as headlamp/oauth2-proxy (a UI must
|
|
# never gate the platform chain). Carries no SOPS secret, so clusters/dev adds no decryption patch.
|
|
- homepage.yaml # Flux Kustomization -> ./platform/homepage (dependsOn platform-config)
|
|
# Crafty Controller — Minecraft management UI, web UI Keycloak-gated by a per-pod oauth2-proxy
|
|
# reverse-proxy sidecar (Option B). POST-platform tier (dependsOn platform-config, wait:false); same
|
|
# non-gating leaf-UI rationale as headlamp/homepage/oauth2-proxy (a UI must never gate the platform).
|
|
- crafty.yaml # Flux Kustomization -> ./platform/crafty (dependsOn platform-config)
|
|
# Alert runbooks — POST-platform tier (dependsOn platform-config, wait:false). Serves the 34
|
|
# runbooks the alert rules' `runbook_url` annotations point at, at <apex>/runbooks/<name>; before
|
|
# this tier existed that path was a 404 and every alert handed the operator a dead link. Same
|
|
# non-gating leaf rationale as headlamp/homepage/crafty. Carries NO postBuild substitution on
|
|
# purpose — it renders documentation containing `${...}` shell examples that envsubst would empty;
|
|
# see clusters/base/runbooks.yaml.
|
|
- runbooks.yaml # Flux Kustomization -> ./runbooks (dependsOn platform-config)
|
|
# Velero backup controller — POST-platform tier (dependsOn platform-config, wait:false). A backup
|
|
# controller must NEVER gate the platform chain; same non-gating rationale as headlamp/oauth2-proxy.
|
|
# Carries velero-cloud.sops.yaml, so clusters/dev adds a SOPS decryption patch for it (by name).
|
|
- backup.yaml # Flux Kustomization -> ./platform/velero (dependsOn platform-config)
|
|
# WireGuard remote-access VPN (north-south INGRESS, udp/51820) — POST-platform tier
|
|
# (dependsOn platform-config, wait:false). NOT the same thing as platform/vpn-egress, which sends
|
|
# pods OUT through PIA; this lets remote CLIENTS IN. Non-gating is load-bearing rather than
|
|
# conventional here: its DaemonSet cannot start until the operator creates the out-of-band
|
|
# `wg-server-key` Secret, so in the gating `platform` tier an unfinished manual step would have
|
|
# stalled every apps wave. Carries no SOPS secret, so neither overlay adds a decryption patch.
|
|
- vpn-ingress.yaml # Flux Kustomization -> ./platform/vpn-ingress (dependsOn platform-config)
|
|
- cluster-vars.yaml # overlay knob surface (base defaults)
|