6 Commits
Author SHA1 Message Date
jakman85andClaude Opus 4.8 a720c73fcd monitoring: node CPU-pressure alerts + PSI/load dashboard (#4)
Adds NodeCPUPressureHigh (CPU PSI 'some' > 15% for 15m) and NodeCPULoadHigh (load5/core > 1 for 15m)
to platform-health.nodes, plus a 'Nodes — CPU / Memory Pressure' Grafana dashboard (PSI + load-per-core
+ utilization). These surface the bursty CPU contention that averaged CPU%/kubectl top hides — the
signal behind 'spikes cause blips of delay'. Makes the 6->8 vCPU resize measurable.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-30 21:51:15 -05:00
jakman85andClaude Opus 4.8 ed6a772599 feat(keycloak): provision SSO client secrets + re-enable oauth2-proxy (R-IDP-01/02/04)
Generate the 3 OIDC client secrets (SOPS-encrypted, minimal manifests) so the realm
import can inject the grafana + oauth2-proxy client secrets and both consumers match:
- platform/keycloak/keycloak-realm-clients.sops.yaml (grafana + oauth2-proxy client secrets)
- platform/oauth2-proxy/oauth2-proxy-credentials.sops.yaml (client-id/secret/cookie)
- platform/monitoring/grafana-oidc-credentials.sops.yaml (grafana client secret)
Wire each into its component kustomization and re-enable oauth2-proxy in platform +
platform-config (reverses the temp deferral now that its secret exists).

Values generated by PM per operator instruction (random; recorded out-of-repo at
~/.config/home-k3s/keycloak-sso-secrets.txt for later rotation). Encrypted via sops in the
dev container to the age recipient in .sops.yaml; plaintext never committed.

Serves: R-IDP-01, R-IDP-02, R-IDP-04, R-SEC-01.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-08-24 22:08:04 -05:00
jakman85andClaude Opus 4.8 eeb97208fa fix(monitoring): move PrometheusRule alerts to platform-config (CRD ordering)
Enabling monitoring deadlocked the platform Kustomization: alerts.yaml ships
PrometheusRule CRs whose monitoring.coreos.com CRDs are installed by the
kube-prometheus-stack HelmRelease in the SAME pass → 'no matches for kind
PrometheusRule' → atomic apply failure → NO monitoring HelmReleases created.
Same controllers/configs fix as cert-manager/kyverno: drop alerts.yaml from
platform/monitoring and apply it via platform-config (dependsOn platform), so the
monitoring CRDs exist first. Verified: platform has 0 PrometheusRule (keeps the 3
monitoring HRs); platform-config has the 8 alert rules; gate PASS.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-21 22:30:01 -05:00
jakman85andClaude Opus 4.8 dfde5292d7 feat(phase4): deploy monitoring (kube-prometheus-stack + Loki + Alloy + Alertmanager)
Enable monitoring in platform/kustomization.yaml (Phase 4). Grafana admin +
Alertmanager SMTP secrets are provisioned (SOPS) and wired; Longhorn SC is up for
Prometheus/Loki PVCs. Set the Alertmanager receiver 'to:' to a valid inbox
([email protected]) so the config loads (REPLACE-ME is not a valid address);
operator can change it. NOTE: smtp_from ([email protected]) differs from
the Gmail auth user — Gmail may reject/rewrite unless a send-as alias is configured;
that only affects email delivery at alert time, not stack health.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-21 22:06:23 -05:00
jakman85andClaude Opus 4.8 cc89b4b830 Autonomous session: Phase-2 platform authoring + Flux bootstrap progress
Author-only Phase-2 components (platform suspended; cluster untouched):
- platform/longhorn: 3-replica SC + bulk external-mount indirection (R-STATE)
- platform/monitoring: kube-prometheus-stack + Loki + Alloy (dev-trimmed) + 8 alert rules (R-OBS)
- platform/cilium: parity fix (k8sServiceHost=10.73.0.11 + masquerade) safe to adopt Phase 2
- conformance: partial L2 tests (worker hit session limit; incomplete — see DECISIONS-LOG K2)
- bootstrap/gitea: fix rootless-podman perms (host bind-mount -> named volume)

Flux installed on the cluster; host Gitea up + repo pushed; GitRepository BLOCKED by
host firewalld (VM->10.73.0.1:3000) — needs one operator sudo command (DECISIONS-LOG).
Known issue K1: cluster-vars substitution-var case mismatch (fix before un-suspend).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-16 14:32:47 -05:00
jakman85andClaude Opus 4.8 af4f6c07ae T-0.1 Flux GitOps layout + dev/prod overlays + platform component skeletons (R-GITOPS-01/03, R-ENV-02, R-PLAT-01)
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-08-16 00:08:31 -05:00