Merge credential-forward correction + SOPS activation into main

This commit is contained in:
jakman85 committed 2026-08-16 22:09:21 -05:00
commit ae36d67dc3
23 files changed
+448 -198

No files matched your search

+40
View File
@@ -152,6 +152,46 @@ until their hand-offs land.
`helm`-installed with a release secret, or CLI/kubectl?) — verify before creating the cilium Kustomization.
4. Create the dev root Kustomization (the actual un-suspend) — deliberately NOT done here.
## CORRECTION — credential-forward, unified graph (operator directive, 2026-08-16)
**Operator pushback:** "Are we making implementation choices solely on the fact that you have to ask
for credentials? It is better to get the credentials upfront (encrypted file) and keep the rollout in
line with the existing architecture." **Correct — and it exposed a scope distortion.**
- **A2 SUPERSEDED / D-P2A REVERTED.** Choosing components by "needs no operator secret" inverted the
roadmap: `roadmap/phases.md` **Phase 2 = Network, Storage, TLS, DNS, Ingress** → the credential-gated
cert-manager/externaldns/gateway ARE the Phase-2 core, while **monitoring is Phase 4** (I'd pulled it
forward). The per-component dev divergence (`clusters/dev/rollout/`) existed only to dodge asking for
the Cloudflare token.
- **Reverted to the existing architecture:** deleted `clusters/dev/rollout/` + the `$patch: delete`s;
restored the unified base graph (dev = ../base + cluster-vars patches + SOPS decryption patches, one
`platform` Kustomization, R-ENV-02 intact — **removes the earlier architect-review flag**). Reverted
the monitoring Grafana/SMTP dodges (it's Phase 4). Kept the non-credential-driven improvements: tofu
for_each/rejoin (F1/F2), and Kyverno parametrization + webhook namespace-excludes (Kyverno IS Phase 2).
- **DECISION D-P2B — phase the unified `platform/kustomization.yaml` (operator-approved).** Components
are enabled in the single shared list **per phase** (the same pattern that deferred kyverno):
Phase-2 set now active = **kyverno, cilium, longhorn, cert-manager, externaldns, gateway**; monitoring/
image-automation (P4), velero (P4, OQ-BKP), kubevirt (P5), keycloak/oauth2-proxy (P6) are commented
with phase markers. dev+prod render the same list. Verified: `kubectl kustomize platform` renders only
the 6 Phase-2 namespaces; dev shows the unified platform+apps Kustomizations; prod builds; gate PASS.
- **DECISION D-SEC1 — provision ALL credentials upfront via SOPS; operator encrypts (both operator-chosen).**
Activated SOPS: `.sops.yaml` installed at repo root (recipient still REPLACE-ME — operator fills after
age-keygen). Authored `bootstrap/sops/secret-templates/*.yaml.example` (cloudflare-api-token ×2 ns,
grafana-admin, alertmanager-smtp, keycloak-admin, oauth2-proxy, pia — all placeholder-only, `.example`
so kustomize/SOPS ignore them) + `bootstrap/sops/CREDENTIALS.md` (end-to-end checklist). Pre-wired
**commented** `- cloudflare-api-token.sops.yaml` lines in cert-manager + externaldns kustomizations
(operator uncomments post-encrypt). Filled the non-secret ACME email ([email protected],
confirm/change). **Agents never touch real secret values** (per `.sops.yaml`).
### Operator TODO to reach Phase-2 deploy (see bootstrap/sops/CREDENTIALS.md)
1. age-keygen → fill root `.sops.yaml` recipient → install `sops-age` on cluster.
2. Provision `cloudflare-api-token` (Zone:DNS:Edit + Zone:Read on keithjackson.dev) in **both**
cert-manager + external-dns; uncomment the two kustomization lines.
3. Confirm ACME email. 4. Push `main` to host Gitea. 5. Create the dev root Kustomization (the un-suspend).
6. (Upfront, optional now) encrypt the Phase-3/4/6 secrets so later phases are credential-ready.
7. **Chart-key verification** (flagged in templates): keycloak + oauth2-proxy `existingSecret` key names,
Keycloak DB secret, and PIA/Gluetun protocol (≠ TCP) — verify at each component's phase.
## KNOWN ISSUES to fix before Phase-2 un-suspend (logged, not yet fixed)
| # | Issue | Impact | Fix |
|---|---|---|---|
+74
View File
@@ -0,0 +1,74 @@
# Credentials & SOPS — operator provisioning checklist
All platform secrets are **SOPS-encrypted (age)**, committed to git, and decrypted by Flux at
reconcile (ADR-0007, R-SEC-01). **Agents never handle real secret values** — encryption is done by
you (the operator). This file is the end-to-end flow. Templates live in
[`secret-templates/`](secret-templates/); the master `.sops.yaml` is at the **repo root**.
Credentials are provisioned **upfront** so component rollout is gated only by *phase/readiness*,
never by a missing secret.
---
## One-time SOPS setup (do this first)
1. **Generate the age keypair** (private key stays on your machine + goes to the cluster; never in git):
```bash
./bootstrap/sops/age-keygen.sh # prints the PUBLIC recipient (age1…) + writes the private key
```
2. **Fill the recipient** into the repo-root `.sops.yaml` — replace **both** `REPLACE-ME-AGE-PUBLIC-RECIPIENT`
occurrences with the `age1…` public key from step 1. (This file governs encryption for `platform/**`,
`apps/**`, `catalog/**`, and any `secrets/**`.)
3. **Install the private key on the cluster** as the `sops-age` Secret so Flux can decrypt:
```bash
./bootstrap/sops/install-age-key.sh # creates secret/sops-age in flux-system (see age-key-secret.md)
```
The dev/prod overlays already point the `platform`/`apps` Flux Kustomizations at `decryption.secretRef: sops-age`.
> Keep the age **private** key out of the repo (same discipline as the ansible-vault passphrase at
> `~/.config/home-k3s/ansible-vault-pass`). If it leaks, rotate: new key → re-encrypt all `*.sops.yaml` → re-install.
---
## Per-secret flow (repeat for each secret below)
```bash
# 1. copy the template into its component dir under its real name
cp bootstrap/sops/secret-templates/<template>.yaml.example platform/<component>/<name>.sops.yaml
# 2. fill the real value(s) in stringData:
$EDITOR platform/<component>/<name>.sops.yaml
# 3. encrypt in place (SOPS picks up the root .sops.yaml rules automatically)
sops -e -i platform/<component>/<name>.sops.yaml
# 4. wire it: uncomment the pre-added `- <name>.sops.yaml` line in platform/<component>/kustomization.yaml
# 5. verify it still builds, then commit the CIPHERTEXT
kubectl kustomize platform/<component> >/dev/null && git add -A && git commit
```
---
## Secret inventory (provision all now; wire per phase)
| Secret | Namespace | Keys | Consumer | Phase | Template |
|---|---|---|---|---|---|
| `cloudflare-api-token` | `cert-manager` | `api-token` | cert-manager DNS-01 ClusterIssuer | **2** | `cloudflare-api-token.cert-manager.yaml.example` |
| `cloudflare-api-token` | `external-dns` | `api-token` | ExternalDNS (`CF_API_TOKEN`) — **same token**, 2nd copy | **2** | `cloudflare-api-token.external-dns.yaml.example` |
| `grafana-admin-credentials` | `monitoring` | `admin-user`, `admin-password` | Grafana admin | 4 | `grafana-admin-credentials.yaml.example` |
| `alertmanager-smtp` | `monitoring` | `smtp-password` | Alertmanager email | 4 | `alertmanager-smtp.yaml.example` |
| `keycloak-admin-credentials` | `keycloak` | `admin-user`, `admin-password` ⚠️verify chart keys; DB secret likely also needed | Keycloak IdP | 6 | `keycloak-admin-credentials.yaml.example` |
| `oauth2-proxy-credentials` | `oauth2-proxy` | `client-id`, `client-secret`, `cookie-secret` ⚠️verify chart keys | oauth2-proxy forward-auth | 6 | `oauth2-proxy-credentials.yaml.example` |
| `pia-credentials` | *(Phase-3 VPN ns, TBA)* | `username`, `password` | Gluetun VPN egress (protocol ≠ TCP) — **not wired yet** | 3 | `pia-credentials.yaml.example` |
**Cloudflare token scope:** one API token with **Zone → DNS → Edit** + **Zone → Read** on the
`keithjackson.dev` zone serves both cert-manager (DNS-01) and ExternalDNS. Same value in both namespace copies.
---
## Phase-2 blockers (needed to un-suspend the Phase-2 platform tier)
- [ ] SOPS one-time setup (age key → `.sops.yaml` → `sops-age` on cluster).
- [ ] `cloudflare-api-token` provisioned + wired in **both** `cert-manager` and `external-dns`.
- [ ] ACME account email set in `platform/cert-manager/clusterissuers.yaml` (both issuers) — non-secret; a sensible default may already be filled, confirm it.
- [ ] Push current `main` to the host Gitea (Flux source), then create the dev root Kustomization to reconcile `clusters/dev`.
Later-phase secrets (Grafana/SMTP, Keycloak, oauth2-proxy, PIA) can be encrypted now and simply wired
when their component is enabled in `platform/kustomization.yaml`.
+67
View File
@@ -0,0 +1,67 @@
# SOPS Secret Templates
This directory holds **reference templates** for the SOPS-encrypted secrets that
platform components consume. They are **not applied and not encrypted**.
## Why `.yaml.example`?
Every file here uses a `.yaml.example` extension so that:
- **kustomize never loads them** (they are not `.yaml` / not listed in any
`kustomization.yaml`), and
- **SOPS never auto-encrypts them** (the repo's `.sops.yaml` creation rules match
`*.sops.yaml`, not `*.yaml.example`).
They exist purely so the operator can see the exact Kubernetes Secret shape
(name, namespace, keys) each component expects. **They contain only placeholder
values** (`REPLACE-ME`, `<fill me>`) — never real secrets.
## Operator flow (per secret)
For each `<name>.yaml.example`:
1. **Copy** it to the target path shown in the file's top comment, renaming to
`<name>.sops.yaml`. Example:
```bash
cp bootstrap/sops/secret-templates/grafana-admin-credentials.yaml.example \
platform/monitoring/grafana-admin-credentials.sops.yaml
```
2. **Fill** in the real values (replace every `REPLACE-ME` / `<fill me>`).
3. **Encrypt in place** with SOPS + age:
```bash
sops -e -i platform/monitoring/grafana-admin-credentials.sops.yaml
```
4. **Wire it** into the component by adding the resource line (shown in the
file's top comment) under `resources:` in that component's
`kustomization.yaml`:
```yaml
resources:
- grafana-admin-credentials.sops.yaml
```
5. **Commit** the encrypted `.sops.yaml` (ciphertext is safe to commit) and the
`kustomization.yaml` change. Flux decrypts it at reconcile via the cluster
age key.
## Files in this directory
| Template | Secret / namespace | Phase |
|---|---|---|
| `cloudflare-api-token.cert-manager.yaml.example` | `cloudflare-api-token` / `cert-manager` | 2 |
| `cloudflare-api-token.external-dns.yaml.example` | `cloudflare-api-token` / `external-dns` (same token) | 2 |
| `grafana-admin-credentials.yaml.example` | `grafana-admin-credentials` / `monitoring` | 4 |
| `alertmanager-smtp.yaml.example` | `alertmanager-smtp` / `monitoring` | 4 |
| `keycloak-admin-credentials.yaml.example` | `keycloak-admin-credentials` / `keycloak` | 6 |
| `oauth2-proxy-credentials.yaml.example` | `oauth2-proxy-credentials` / `oauth2-proxy` | 6 |
| `pia-credentials.yaml.example` | `pia-credentials` / TBD (Phase-3 VPN) | 3 |
## Master checklist
See [`bootstrap/sops/CREDENTIALS.md`](../CREDENTIALS.md) for the full,
end-to-end credential provisioning flow and checklist.
@@ -0,0 +1,26 @@
# =============================================================================
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize
# and never auto-encrypted by SOPS). Reference scaffold only.
#
# (a) Copy to: platform/monitoring/alertmanager-smtp.sops.yaml
# (b) kustomization: add this line under `resources:` in
# platform/monitoring/kustomization.yaml
# - alertmanager-smtp.sops.yaml
# (c) Encrypt: sops -e -i platform/monitoring/alertmanager-smtp.sops.yaml
# (d) Consumed by: Alertmanager (kube-prometheus-stack). Only the SMTP
# PASSWORD lives here. It is mounted at
# /etc/alertmanager/secrets/alertmanager-smtp/smtp-password. [Phase 4]
#
# NOTE: The non-secret SMTP endpoint fields — smtp_smarthost, smtp_from,
# smtp_auth_username, and the receiver `to:` addresses — are configured in
# platform/monitoring/kube-prometheus-stack.yaml (currently REPLACE-ME
# placeholders there). Only the password is this secret.
# =============================================================================
apiVersion: v1
kind: Secret
metadata:
name: alertmanager-smtp
namespace: monitoring
type: Opaque
stringData:
smtp-password: REPLACE-ME
@@ -0,0 +1,23 @@
# =============================================================================
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize
# and never auto-encrypted by SOPS). Reference scaffold only.
#
# (a) Copy to: platform/cert-manager/cloudflare-api-token.sops.yaml
# (b) kustomization: add this line under `resources:` in
# platform/cert-manager/kustomization.yaml
# - cloudflare-api-token.sops.yaml
# (c) Encrypt: sops -e -i platform/cert-manager/cloudflare-api-token.sops.yaml
# (d) Consumed by: cert-manager ClusterIssuers (DNS-01 solver). The Cloudflare
# DNS-01 solver's apiTokenSecretRef points at secret
# `cloudflare-api-token`, key `api-token`. [Phase 2]
#
# The token must be a Cloudflare API token with Zone:DNS:Edit + Zone:Read scope.
# =============================================================================
apiVersion: v1
kind: Secret
metadata:
name: cloudflare-api-token
namespace: cert-manager
type: Opaque
stringData:
api-token: REPLACE-ME-CLOUDFLARE-DNS-EDIT-TOKEN
@@ -0,0 +1,27 @@
# =============================================================================
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize
# and never auto-encrypted by SOPS). Reference scaffold only.
#
# (a) Copy to: platform/externaldns/cloudflare-api-token.sops.yaml
# (component DIR is platform/externaldns/, but the Kubernetes
# NAMESPACE is external-dns — note the hyphen.)
# (b) kustomization: add this line under `resources:` in
# platform/externaldns/kustomization.yaml
# - cloudflare-api-token.sops.yaml
# (c) Encrypt: sops -e -i platform/externaldns/cloudflare-api-token.sops.yaml
# (d) Consumed by: external-dns reads env CF_API_TOKEN via secretKeyRef
# name `cloudflare-api-token`, key `api-token`. [Phase 2]
#
# NOTE: This is the SAME Cloudflare token as the cert-manager copy — just a
# second copy living in the external-dns namespace. ClusterIssuer secrets and
# external-dns run in different namespaces, so the token must exist in both. A
# single Cloudflare API token with Zone:DNS:Edit + Zone:Read scope serves both.
# =============================================================================
apiVersion: v1
kind: Secret
metadata:
name: cloudflare-api-token
namespace: external-dns
type: Opaque
stringData:
api-token: REPLACE-ME-CLOUDFLARE-DNS-EDIT-TOKEN
@@ -0,0 +1,21 @@
# =============================================================================
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize
# and never auto-encrypted by SOPS). Reference scaffold only.
#
# (a) Copy to: platform/monitoring/grafana-admin-credentials.sops.yaml
# (b) kustomization: add this line under `resources:` in
# platform/monitoring/kustomization.yaml
# - grafana-admin-credentials.sops.yaml
# (c) Encrypt: sops -e -i platform/monitoring/grafana-admin-credentials.sops.yaml
# (d) Consumed by: kube-prometheus-stack -> grafana.admin.existingSecret
# (userKey: admin-user, passwordKey: admin-password). [Phase 4]
# =============================================================================
apiVersion: v1
kind: Secret
metadata:
name: grafana-admin-credentials
namespace: monitoring
type: Opaque
stringData:
admin-user: admin
admin-password: REPLACE-ME
@@ -0,0 +1,33 @@
# =============================================================================
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize
# and never auto-encrypted by SOPS). Reference scaffold only.
#
# (a) Copy to: platform/keycloak/keycloak-admin-credentials.sops.yaml
# (b) kustomization: add this line under `resources:` in
# platform/keycloak/kustomization.yaml
# - keycloak-admin-credentials.sops.yaml
# (c) Encrypt: sops -e -i platform/keycloak/keycloak-admin-credentials.sops.yaml
# (d) Consumed by: Keycloak Helm chart -> auth.existingSecret. [Phase 6]
#
# !! VERIFY KEY NAMES BEFORE ENCRYPTING !!
# The exact stringData key names depend on the pinned Keycloak chart version.
# Bitnami Keycloak commonly reads `admin-password`; some versions also read
# `admin-user` (and older/other variants use `adminPassword`). Check the pinned
# chart's values (auth.existingSecret / auth.passwordSecretKey) and adjust the
# keys below to match before running `sops -e -i`.
#
# !! SEPARATE DB SECRET LIKELY NEEDED !!
# The Keycloak helmrelease references "Admin + DB credentials". Keycloak needs a
# PostgreSQL DB credential in addition to these admin creds. A separate DB secret
# (or the chart's postgresql.auth.existingSecret) will likely be required at
# Phase 6 — flag and provision it then.
# =============================================================================
apiVersion: v1
kind: Secret
metadata:
name: keycloak-admin-credentials
namespace: keycloak
type: Opaque
stringData:
admin-user: admin
admin-password: REPLACE-ME
@@ -0,0 +1,32 @@
# =============================================================================
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize
# and never auto-encrypted by SOPS). Reference scaffold only.
#
# (a) Copy to: platform/oauth2-proxy/oauth2-proxy-credentials.sops.yaml
# (b) kustomization: add this line under `resources:` in
# platform/oauth2-proxy/kustomization.yaml
# - oauth2-proxy-credentials.sops.yaml
# (c) Encrypt: sops -e -i platform/oauth2-proxy/oauth2-proxy-credentials.sops.yaml
# (d) Consumed by: oauth2-proxy Helm chart -> existingSecret. [Phase 6]
#
# VALUE NOTES:
# - client-id / client-secret: come from the Keycloak OIDC client created for
# oauth2-proxy (Phase 6, after Keycloak is up).
# - cookie-secret: must be a 16, 24, or 32-byte value, base64-encoded.
# Generate with: openssl rand -base64 32
#
# !! VERIFY KEY NAMES BEFORE ENCRYPTING !!
# Confirm the exact stringData key names against the pinned oauth2-proxy chart's
# existingSecret contract (common keys: client-id, client-secret, cookie-secret,
# but chart versions vary). Adjust below to match before running `sops -e -i`.
# =============================================================================
apiVersion: v1
kind: Secret
metadata:
name: oauth2-proxy-credentials
namespace: oauth2-proxy
type: Opaque
stringData:
client-id: REPLACE-ME
client-secret: REPLACE-ME
cookie-secret: REPLACE-ME
@@ -0,0 +1,32 @@
# =============================================================================
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize
# and never auto-encrypted by SOPS). Reference scaffold only.
#
# (a) Copy to: (TO BE DECIDED AT PHASE 3 — VPN component not authored yet)
# (b) kustomization: DO NOT WIRE YET.
# (c) Encrypt: sops -e -i <target>.sops.yaml (once the target path exists)
# (d) Consumed by: Phase-3 Gluetun VPN egress gateway. The consuming
# component AND its namespace DO NOT EXIST yet — this template
# is provisioned upfront. [Phase 3]
#
# !! NAMESPACE UNKNOWN — the metadata.namespace below is a placeholder. Set it to
# the VPN gateway's namespace when that component is authored at Phase 3. !!
#
# VALUE NOTES:
# - username: PIA (Private Internet Access) user, form pNNNNNNN.
# - password: PIA password.
# - Gluetun typically reads OPENVPN_USER / OPENVPN_PASSWORD (OpenVPN) or
# WireGuard credentials. Confirm the chosen protocol when wiring — the
# requirement says NOT TCP. The env var names / key names the Gluetun
# deployment expects may differ from username/password below; adjust to match
# the chosen protocol before encrypting.
# =============================================================================
apiVersion: v1
kind: Secret
metadata:
name: pia-credentials
namespace: REPLACE-ME-PHASE-3-VPN-NAMESPACE
type: Opaque
stringData:
username: REPLACE-ME
password: REPLACE-ME
-2
View File
@@ -27,8 +27,6 @@ data:
MONITORING_PROMETHEUS_RETENTION: "7d" # dev short | prod long
MONITORING_PROMETHEUS_PVC_SIZE: "10Gi"
MONITORING_LOKI_PVC_SIZE: "10Gi"
MONITORING_STORAGE_CLASS: "longhorn" # PVC tier for Prometheus/Loki (dev may override)
GRAFANA_ADMIN_SECRET: "grafana-admin-credentials" # by-name SOPS secret (prod); dev="" → chart default
# --- admission enforcement (Kyverno, ADR-0011). Base defaults are the STRICT/prod posture;
# the dev overlay softens them for bootstrap (Audit / Ignore / 1) so a cold cluster can
+18 -20
View File
@@ -2,38 +2,39 @@
# clusters/dev — the ACTIVE build target overlay.
#
# `flux bootstrap ... --path=clusters/dev` points the dev cluster's Flux here.
# TRANSITIONAL (Phase-2 bootstrap): this overlay reuses the base source + cluster-vars, but
# REPLACES the monolithic base `platform`/`apps` Kustomizations with a per-component NO-SECRET
# rollout (./rollout) — so dev brings up cilium/longhorn/monitoring/kyverno in order WITHOUT
# dragging in the secret-blocked components. It patches cluster-vars with dev values (R-ENV-02)
# and softens Kyverno for bootstrap. CONVERGENCE: delete ./rollout + the two $patch:delete
# below to return to the unified base graph once all operator hand-offs land. See DECISIONS-LOG.
# This overlay adds NO resources of its own. Its ONLY job is to patch the shared cluster-vars
# ConfigMap with dev-scoped values (R-ENV-02). The base reconciliation graph is reused verbatim.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../base
- rollout
patches:
# --- Drop the monolithic base platform/apps Kustomizations. They reconcile ALL 11 components
# as one unit (incl. secret-blocked cert-manager/externaldns/velero/keycloak/oauth2-proxy/
# image-automation/gateway/kubevirt); ./rollout replaces them with the per-component
# no-secret set for dev bootstrap. Remove these two deletes to re-adopt the unified graph.
# (No decryption patch is needed here: the rollout components carry no SOPS content, so
# the `sops-age` Secret is not required for the dev bootstrap tier.) ---
# --- SOPS decryption + bootstrap source (PM-inlined from bootstrap/flux/patches/, 2026-08-16) ---
# Inlined (not cross-dir file refs) to stay within kustomize's default load restrictions.
# Realises the commented decryption TODOs in clusters/base and sets the real source URL.
# The `sops-age` Secret is created out-of-band (bootstrap/sops/install-age-key.sh) — name only.
- patch: |-
$patch: delete
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: platform
namespace: flux-system
spec:
decryption:
provider: sops
secretRef:
name: sops-age
- patch: |-
$patch: delete
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: apps
namespace: flux-system
spec:
decryption:
provider: sops
secretRef:
name: sops-age
- patch: |-
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
@@ -84,6 +85,8 @@ patches:
path: /data/VELERO_RETENTION
value: "72h"
# dev-softened Kyverno for cold-start (base defaults Enforce/Fail/3 = prod posture).
# Not credential-driven — bootstrap-safety knobs so a cold cluster comes up before
# admission is proven/HA; flip dev to the strict posture once green.
- op: replace
path: /data/KYVERNO_VALIDATION_ACTION
value: "Audit"
@@ -93,8 +96,3 @@ patches:
- op: replace
path: /data/KYVERNO_ADMISSION_REPLICAS
value: "1"
# dev: no Grafana admin Secret yet → empty makes the chart use its default admin password
# (dev-only, NAT-isolated). Prod keeps the by-name SOPS secret from the base default.
- op: replace
path: /data/GRAFANA_ADMIN_SECRET
value: ""
-25
View File
@@ -1,25 +0,0 @@
---
# Cilium — ADOPT the running Ansible-installed CNI. Parity between platform/cilium and the
# live config was verified (k8sServiceHost/port via pod env, kube-proxy-replacement, gateway-api,
# egress+bpf masquerade, hubble, l2, cgroup, v1.16.5) → this reconcile is a NO-OP.
# prune: false — Cilium was bootstrapped out-of-band; adopt its config, never own-delete the CNI.
# No decryption (no SOPS content). No dependsOn (it is the datapath everything else rides on).
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: cilium
namespace: flux-system
spec:
interval: 30m
retryInterval: 2m
timeout: 10m
prune: false
wait: true
sourceRef:
kind: GitRepository
name: flux-system
path: ./platform/cilium
postBuild:
substituteFrom:
- kind: ConfigMap
name: cluster-vars
-20
View File
@@ -1,20 +0,0 @@
---
# Phase-2 NO-SECRET incremental rollout — DEV bootstrap scaffold.
#
# Per-component Flux Kustomizations for the components that need NO operator secret:
# kyverno (admission, dev-softened) · cilium (adopt running CNI) · longhorn (storage)
# · monitoring (dependsOn longhorn).
# Secret-blocked components (cert-manager, externaldns, gateway, velero, keycloak,
# oauth2-proxy, image-automation, kubevirt) are intentionally NOT created until their
# operator hand-offs arrive (DECISIONS-LOG A2 / hand-off table).
#
# TRANSITIONAL: this is the dev bootstrap graph. Once every hand-off lands, dev drops
# ./rollout and re-adopts the unified base monolith (clusters/base/platform.yaml) so dev
# and prod share one reconciliation graph again (R-ENV-02 convergence). See DECISIONS-LOG.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- kyverno.yaml # first: admission present (Audit/Ignore in dev — non-blocking) before workloads
- cilium.yaml # adopt the running CNI (parity verified — no-op reconcile)
- longhorn.yaml # storage; monitoring depends on its StorageClass
- monitoring.yaml # dependsOn longhorn
-25
View File
@@ -1,25 +0,0 @@
---
# Kyverno — in-cluster admission enforcer. DEV-SOFTENED via cluster-vars:
# KYVERNO_VALIDATION_ACTION=Audit, KYVERNO_WEBHOOK_FAILURE_POLICY=Ignore,
# KYVERNO_ADMISSION_REPLICAS=1 (base defaults Enforce/Fail/3 for prod).
# Non-blocking on today's cluster (policies match only `app-namespace=true` namespaces,
# none exist yet). No SOPS content. Listed first so admission is present before workloads.
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: kyverno
namespace: flux-system
spec:
interval: 1h
retryInterval: 2m
timeout: 10m
prune: true
wait: true
sourceRef:
kind: GitRepository
name: flux-system
path: ./platform/kyverno
postBuild:
substituteFrom:
- kind: ConfigMap
name: cluster-vars
-25
View File
@@ -1,25 +0,0 @@
---
# Longhorn — block storage / CSI (default `longhorn` StorageClass). No SOPS content.
# No dependsOn. The bulk-NFS StorageClass in this component carries REPLACE-ME (OQ-2) but is
# an isolated, unreferenced object — it does NOT block Longhorn's HelmRelease reconcile.
# PRE-DEPLOY CHECK (runtime, out of git): confirm `iscsid` active + /var/lib/longhorn mounted
# on all 3 nodes (cloud-init provides both; verify after the T-1.5 rebuild).
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: longhorn
namespace: flux-system
spec:
interval: 10m
retryInterval: 2m
timeout: 10m
prune: true
wait: true
sourceRef:
kind: GitRepository
name: flux-system
path: ./platform/longhorn
postBuild:
substituteFrom:
- kind: ConfigMap
name: cluster-vars
-26
View File
@@ -1,26 +0,0 @@
---
# Monitoring — kube-prometheus-stack + Loki + Alloy. Prometheus/Loki PVCs bind the
# `longhorn` StorageClass, so this dependsOn longhorn (Flux waits for it Ready first).
# Grafana admin password (chart default in dev) and Alertmanager SMTP (deferred to the
# Phase-4 hand-off) are handled WITHOUT an operator secret in dev → NO decryption block.
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: monitoring
namespace: flux-system
spec:
interval: 10m
retryInterval: 2m
timeout: 15m
prune: true
wait: true
dependsOn:
- name: longhorn
sourceRef:
kind: GitRepository
name: flux-system
path: ./platform/monitoring
postBuild:
substituteFrom:
- kind: ConfigMap
name: cluster-vars
+2 -2
View File
@@ -12,7 +12,7 @@ metadata:
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: [email protected] # TODO(WS-SEC): ACME account email
email: [email protected] # ACME account email (LE expiry notices) — operator: confirm/change
privateKeySecretRef:
name: letsencrypt-staging-account-key
solvers:
@@ -32,7 +32,7 @@ metadata:
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: [email protected] # TODO(WS-SEC): ACME account email
email: [email protected] # ACME account email (LE expiry notices) — operator: confirm/change
privateKeySecretRef:
name: letsencrypt-prod-account-key
solvers:
+4
View File
@@ -6,3 +6,7 @@ resources:
- helmrepository.yaml
- helmrelease.yaml
- clusterissuers.yaml
# SOPS-encrypted Cloudflare DNS-01 token (ClusterIssuer apiTokenSecretRef). Operator flow:
# copy bootstrap/sops/secret-templates/cloudflare-api-token.cert-manager.yaml.example here as
# cloudflare-api-token.sops.yaml, fill + `sops -e -i`, then uncomment. See bootstrap/sops/CREDENTIALS.md.
# - cloudflare-api-token.sops.yaml
+4
View File
@@ -5,3 +5,7 @@ resources:
- namespace.yaml
- helmrepository.yaml
- helmrelease.yaml
# SOPS-encrypted Cloudflare token (env CF_API_TOKEN) — SAME token as cert-manager, second copy in
# the external-dns namespace. Copy bootstrap/sops/secret-templates/cloudflare-api-token.external-dns.yaml.example
# here as cloudflare-api-token.sops.yaml, fill + `sops -e -i`, then uncomment. See bootstrap/sops/CREDENTIALS.md.
# - cloudflare-api-token.sops.yaml
+19 -21
View File
@@ -2,27 +2,25 @@
# Top-level platform kustomization — the entrypoint the Flux `platform` Kustomization renders
# (clusters/base/platform.yaml, path: ./platform).
#
# Phase 0 scaffolding: lists every platform component EXCEPT kyverno (owned by the policy engineer,
# WS-TEST — see platform/README.md).
#
# ⚠️ INTEGRATION TODO(PM): once platform/kyverno/ is authored by the policy engineer, uncomment the
# `- kyverno` line below. Keep it FIRST so admission policies are present before other components
# reconcile. This workstream must NOT create files under platform/kyverno/.
# UNIFIED GRAPH (R-ENV-02): dev and prod both render THIS single list; overlays differ only by
# cluster-vars. Components are enabled here PER PHASE as their phase opens (roadmap/phases.md) —
# the same pattern by which kyverno was previously deferred. Credentials each component needs are
# provisioned upfront as SOPS-encrypted Secrets (bootstrap/sops/, .sops.yaml), so enabling a
# component is never gated on a credential ask — only on its phase and readiness.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
# - kyverno # DEFERRED to Phase 2 (T-2.7): admission enforcement needs a cluster.
# Scaffolding is present & buildable at platform/kyverno/; enable it first
# (before other components) when T-2.7 activates. Kyverno's CI L0 gate is
# already live now via .gitea/ + ci/ (no cluster needed).
- cilium
- longhorn
- cert-manager
- externaldns
- gateway
- monitoring
- velero
- kubevirt
- image-automation
- keycloak
- oauth2-proxy
# ---- Phase 2 — Platform Core: Network, Storage, TLS, DNS, Ingress ----
- kyverno # admission enforcer FIRST — policies present before other components reconcile
- cilium # CNI (adopts the running Ansible-installed release; parity verified)
- longhorn # replicated block storage / CSI
- cert-manager # Let's Encrypt DNS-01 wildcard — needs SOPS `cloudflare-api-token`
- externaldns # auto DNS records — needs SOPS `cloudflare-api-token`
- gateway # Gateway API listeners (ingress; replaces Traefik)
# ---- Later phases: enabled when their phase opens (creds already provisioned via SOPS) ----
# - monitoring # Phase 4 (Observability): kube-prometheus-stack + Loki + Alloy + Alertmanager
# - velero # Phase 4 (Backup/DR): BLOCKED on OQ-BKP (off-cluster backup target decision)
# - image-automation # Phase 4 (Automated updates): Flux image-automation + Renovate
# - kubevirt # Phase 5 (IaaS proof): KubeVirt + CDI
# - keycloak # Phase 6 (Catalog + C2 UI): foundational IdP — needs SOPS `keycloak-admin-credentials`
# - oauth2-proxy # Phase 6: forward-auth — needs SOPS `oauth2-proxy-credentials`
+25 -31
View File
@@ -62,7 +62,7 @@ spec:
storageSpec:
volumeClaimTemplate:
spec:
storageClassName: ${MONITORING_STORAGE_CLASS} # replicated tier (ADR-0003); cluster-vars knob
storageClassName: longhorn # replicated tier (ADR-0003)
accessModes: ["ReadWriteOnce"]
resources:
requests:
@@ -80,24 +80,22 @@ spec:
memory: 32Mi
limits:
memory: 96Mi
# SMTP is an OPERATOR HAND-OFF DEFERRED to Phase 4 — no creds yet, and the by-name
# `alertmanager-smtp` Secret does NOT exist. Mounting a missing Secret wedges Alertmanager
# in ContainerCreating, so the mount is DISABLED until the hand-off. Re-enable verbatim
# (uncomment) when the SOPS `alertmanager-smtp` Secret is delivered. TODO(Phase-4 SMTP).
# secrets:
# - alertmanager-smtp
# Routing stays live; RECEIVERS ARE NULL until the SMTP hand-off (alerts still fire, route,
# and show in the Alertmanager UI + Grafana — they just aren't emailed). The smtp_* globals
# and email_configs are retained below, commented, so re-enabling is a pure uncomment.
# SMTP credentials are an OPERATOR HAND-OFF: a SOPS-encrypted Secret, referenced BY NAME
# only (docs/07). Mounted at /etc/alertmanager/secrets/<name>/ so the config below can read
# the password from a file (never inline plaintext).
secrets:
- alertmanager-smtp
# Routing/receivers. Non-secret SMTP endpoint fields (smarthost/from/to/username) are
# operator-supplied placeholders (REPLACE-ME, same convention as velero); the PASSWORD is
# read from the mounted SOPS secret via *_file. Both severities email; subject/route differ.
config:
global:
resolve_timeout: 5m
# TODO(Phase-4 SMTP hand-off): uncomment together with the alertmanager-smtp Secret mount.
# smtp_smarthost: "smtp.example.com:587"
# smtp_from: "[email protected]"
# smtp_auth_username: "<smtp-login>"
# smtp_auth_password_file: /etc/alertmanager/secrets/alertmanager-smtp/smtp-password
# smtp_require_tls: true
smtp_smarthost: "REPLACE-ME:587" # operator hand-off (e.g. smtp.gmail.com:587)
smtp_from: "REPLACE-ME" # operator hand-off (From: address)
smtp_auth_username: "REPLACE-ME" # operator hand-off (SMTP login)
smtp_auth_password_file: /etc/alertmanager/secrets/alertmanager-smtp/smtp-password
smtp_require_tls: true
route:
group_by: ["alertname", "namespace"]
group_wait: 30s
@@ -111,29 +109,25 @@ spec:
group_wait: 10s
repeat_interval: 1h
receivers:
# NULL receivers until Phase-4 SMTP. email_configs retained (commented) for verbatim re-enable.
- name: email-warning
# email_configs:
# - to: "<alert-inbox>"
# send_resolved: true
# headers:
# subject: '[WARNING] {{ .CommonLabels.alertname }} ({{ .CommonLabels.namespace }})'
email_configs:
- to: "REPLACE-ME" # operator hand-off (alert inbox)
send_resolved: true
headers:
subject: '[WARNING] {{ .CommonLabels.alertname }} ({{ .CommonLabels.namespace }})'
- name: email-critical
# email_configs:
# - to: "<alert-inbox>"
# send_resolved: true
# headers:
# subject: '[CRITICAL] {{ .CommonLabels.alertname }} ({{ .CommonLabels.namespace }})'
email_configs:
- to: "REPLACE-ME" # operator hand-off (alert inbox)
send_resolved: true
headers:
subject: '[CRITICAL] {{ .CommonLabels.alertname }} ({{ .CommonLabels.namespace }})'
# --- Grafana: dashboards (R-OBS-03), surfaced in the C2 UI (docs/12) behind Keycloak SSO ---
grafana:
enabled: true
# Admin bootstrap + (later) OIDC client are SOPS secrets referenced BY NAME (WS-SEC/ADR-0014).
# cluster-vars knob: prod = the SOPS `grafana-admin-credentials` Secret; dev = "" (empty),
# which makes the chart fall back to its default admin password (dev-only, NAT-isolated) so
# Grafana starts without an operator secret. TODO: supply a dev Secret if wider dev access.
admin:
existingSecret: ${GRAFANA_ADMIN_SECRET}
existingSecret: grafana-admin-credentials
userKey: admin-user
passwordKey: admin-password
# DEV TRIM: emptyDir (default). Dashboards are provisioned from ConfigMaps (sidecar), so no
+1 -1
View File
@@ -60,7 +60,7 @@ spec:
replicas: 1
persistence:
enabled: true
storageClass: ${MONITORING_STORAGE_CLASS} # replicated tier (ADR-0003); cluster-vars knob
storageClass: longhorn # replicated tier (ADR-0003)
size: ${MONITORING_LOKI_PVC_SIZE} # overlay knob (dev 5Gi / prod large)
resources:
requests: