feat(bootstrap): automate the Kyverno CRD pre-apply — the last cold-start blocker
`platform/kyverno/admission/helmrelease.yaml` sets `crds: {install: false}`
because the CRD bundle is 5,636,155 bytes, far past kubectl's 256 KiB
last-applied-configuration limit — a client-side apply fails with
"metadata.annotations: Too long" and STALLS the HelmRelease.
The consequence nobody automated: with CRD install off, NOTHING installed
them. `platform` has `wait: true` and every other tier chains transitively
through it, so on a cold cluster Kyverno's API types never exist, `platform`
never goes Ready, and all 10 downstream tiers stall. Dev only ever had these
CRDs because they were applied BY HAND — exactly the undocumented manual step
this repo exists to eliminate.
New `bootstrap/flux/preapply-crds.sh`, wired into bootstrap.sh as step [2/4]
(after `flux install`, before the graph is applied) and standalone-safe so an
already-bootstrapped cluster can be unblocked without re-running bootstrap.
NO SECOND PIN: all four chart coordinates are read with yq out of the same
HelmRelease/HelmRepository pair Flux reconciles. The version is passed through
as the CONSTRAINT ("3.9.x"), not a resolved tag — helm and helm-controller both
use Masterminds/semver, so the render is the version Flux installs. Renovate
bumps one line and this step follows; there is nothing here to drift from.
Only `kind: CustomResourceDefinition` documents are applied. The by-hand
command recorded in the HelmRelease comment piped the WHOLE template into
kubectl, which would create Deployments/webhooks outside Helm's ownership and
then fight the HelmRelease over field managers.
THREE GUARDS, because a silent skip here reproduces a failure mode this repo
has been bitten by repeatedly:
1. any derived coordinate that is empty/null/multi-valued aborts;
2. rendered CRD count must be > 0 — this caught a real bug in the first
draft: on an empty render `yq .metadata.name` emits one BLANK line, which
counted as 1 CRD and sailed past the guard into an apply of an empty file
returning 0, i.e. the exact silent-success trap. Blank/null lines are now
dropped and a chart with no CRDs exits 1;
3. every applied CRD is waited on for condition=Established, so "kubectl
returned 0" is not the only evidence.
No opt-out flag, deliberately: a skipped CRD apply yields a cluster that looks
bootstrapped and reconciles nothing.
`roles/flux_bootstrap` is untouched — it already shells to bootstrap.sh, the
documented single entrypoint, so it inherits the step with no dev/prod fork
(R-ENV-02). A second role task would double-apply.
MEASURED, not assumed — reproduced under the PINNED toolchain (helm 3.16.0,
yq 4.44.6) in infra/devcontainer, matching an independent run on helm 3.20.0
byte-for-byte: 3.9.x resolves to 3.9.0, 22 CRDs, 5,636,155 bytes CRD-only
(5,870,019 full render). Largest single CRD document is 1,378,544 bytes —
under etcd's 1.5 MiB default, but not by much; worth watching on chart bumps.
NOT YET EXERCISED against a live API server: the server-side apply and the
Established wait were tested with a stub kubectl. velero and
kube-prometheus-stack also say "CRDs pre-applied" but both still get theirs
from Helm's native crds/ dir on FIRST install, so neither is a cold-start
blocker; CRD_HELMRELEASE makes extending this a one-liner if that changes.
Refs R-GITOPS-01, R-ENV-02.
Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed
1 parent
9137bbd0c3
commit
13081a863f
4 files changed
+294
-13
No files matched your search
@@ -11,7 +11,8 @@ Spec: [ADR-0004 (Flux)](../../../home-network-k3s/decisions/adr-0004-gitops-flux
|
||||
## Files
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `bootstrap.sh` | **The single bootstrap entrypoint** the Infra worker's final Ansible play calls. render+apply the Flux controllers (`components/`) → verify `sops-age` Secret → `kubectl apply -k clusters/dev`. |
|
||||
| `bootstrap.sh` | **The single bootstrap entrypoint** the Infra worker's final Ansible play calls. render+apply the Flux controllers (`components/`) → **pre-apply chart CRDs (`preapply-crds.sh`)** → verify `sops-age` Secret → `kubectl apply -k ${FLUX_ENV_PATH}`. |
|
||||
| `preapply-crds.sh` | **Server-side pre-apply of the Kyverno CRD bundle** — the CRDs `platform/kyverno/admission/helmrelease.yaml` deliberately tells Helm *not* to install (`crds: {install: false}`, because the ~5.6 MiB bundle exceeds kubectl's 256 KiB last-applied annotation limit and stalls the release). Nothing else installs them, so without this step `platform` never goes Ready on a cold cluster. Chart name/version/repo are **derived from that HelmRelease** — no second pin. Idempotent; runnable standalone. |
|
||||
| `components/kustomization.yaml` | **Customization of the Flux CONTROLLERS themselves.** Overlays `flux install --export` output; currently enables `StrictPostBuildSubstitutions` on kustomize-controller. See "Customizing the controllers" below. |
|
||||
| `patches/platform-sops-decryption.yaml` | Strategic-merge patch enabling `decryption.provider: sops` + `secretRef: sops-age` on the `platform` Kustomization (realises the base TODO). |
|
||||
| `patches/apps-sops-decryption.yaml` | Same, for the app-wave Kustomizations. There is **no single `apps` Kustomization** any more — the apps tier is now `apps-wave-0/1/2` + `apps-vms`, and SOPS decryption is layered on **per wave** (each wave Kustomization is patched by name in `clusters/dev`). |
|
||||
@@ -74,6 +75,49 @@ flux get kustomizations -A # all 11 must return
|
||||
**Rollback** is the same command with `FLUX_INSTALL_PLAIN=1 bash bootstrap/flux/bootstrap.sh`, or
|
||||
simply `flux install --namespace=flux-system --components-extra=…` (drops the arg back off).
|
||||
|
||||
## The CRD pre-apply step (`preapply-crds.sh`) — why bootstrap has to do this
|
||||
|
||||
`platform/kyverno/admission/helmrelease.yaml` sets `crds: {install: false}`. That is correct and must
|
||||
stay: the Kyverno CRD bundle is **5,636,155 bytes across 22 CRDs** at chart 3.9.0 (measured with
|
||||
`helm template … --include-crds`, 2026-09-04), so Helm's client-side apply hits kubectl's 256 KiB
|
||||
`last-applied-configuration` annotation limit — `metadata.annotations: Too long` — which stalls the
|
||||
HelmRelease and blocks `platform` plus every tier that `dependsOn` it.
|
||||
|
||||
The part that was never automated: **with CRD install off, nothing installs the CRDs.** Kyverno's
|
||||
Deployments start, its API types do not exist, `platform` never reaches Ready. Dev has them only
|
||||
because somebody once ran the command in a shell. `preapply-crds.sh` is that command, in git:
|
||||
|
||||
```
|
||||
helm template <chart> --repo <url> --version <constraint> --include-crds --set crds.install=true
|
||||
| yq 'select(.kind == "CustomResourceDefinition")'
|
||||
| kubectl apply --server-side --force-conflicts
|
||||
```
|
||||
|
||||
- **Version derivation, not a second pin.** `chart`, `version`, `namespace` and the repo `url` are
|
||||
read out of the HelmRelease/HelmRepository pair with `yq`. `spec.chart.spec.version` is a semver
|
||||
**constraint** (`3.9.x`), and `helm template --version` accepts the same constraint syntax
|
||||
helm-controller resolves — so the rendered CRDs are the ones Flux will install, and a Renovate bump
|
||||
of that one line carries this step with it. There is nothing here that can drift.
|
||||
- **CRDs only.** Piping the *whole* `helm template` output into kubectl (the by-hand command in the
|
||||
HelmRelease comment) would create Deployments/webhooks outside Helm's ownership and then fight the
|
||||
HelmRelease over field managers. The `yq` filter is load-bearing.
|
||||
- **Idempotent.** Server-side apply is an upsert; `--force-conflicts` adopts the fields dev's
|
||||
hand-applied CRDs left owned by `kubectl-client-side-apply`. Re-running prints
|
||||
`serverside-applied` and exits 0.
|
||||
- **Cannot degrade to a no-op.** Three guards: every derived coordinate must be non-empty and not
|
||||
`null`; the rendered CRD count must be `> 0` (an empty stream would otherwise `apply` cleanly and
|
||||
exit 0 — the exact silent-success bug this repo keeps hitting); and after the apply every CRD is
|
||||
waited on for `condition=Established`. There is **no skip flag** on purpose.
|
||||
- **Environment-agnostic (R-ENV-02).** It names no environment and takes no overlay input — it reads
|
||||
repo files and acts on `$KUBECONFIG`. Dev and prod run the identical step.
|
||||
|
||||
Standalone use (an already-bootstrapped cluster whose `platform` is blocked on the missing CRDs):
|
||||
|
||||
```bash
|
||||
# inside infra/devcontainer, KUBECONFIG pointed at the target cluster
|
||||
bash bootstrap/flux/preapply-crds.sh
|
||||
```
|
||||
|
||||
## Why `flux install` + apply, not `flux bootstrap git` (Phase 1)
|
||||
|
||||
The reconciliation graph is **already authored** in `clusters/base` (read-only to this worker):
|
||||
@@ -98,7 +142,7 @@ tofu apply → cloud-init → Ansible (k3s + Cilium) [Infra worker, T-1
|
||||
▼ (final Ansible play calls the entrypoints below)
|
||||
1. bootstrap/sops/age-keygen.sh # operator generates age key OUTSIDE repo
|
||||
2. bootstrap/sops/age-key-secret.md # operator installs private key as sops-age Secret
|
||||
3. bootstrap/flux/bootstrap.sh # flux install → verify sops-age → apply -k clusters/dev
|
||||
3. bootstrap/flux/bootstrap.sh # flux install → preapply-crds.sh → verify sops-age → apply -k ${FLUX_ENV_PATH}
|
||||
│
|
||||
▼
|
||||
Flux reconciles: GitRepository(flux-system) → Kustomization(platform) → Kustomization(platform-config)
|
||||
@@ -118,8 +162,10 @@ live in `infra/devcontainer/versions.env`).
|
||||
template). `bootstrap.sh` **fails fast** if this Secret is absent.
|
||||
3. `bootstrap/flux/bootstrap.sh` (this entrypoint).
|
||||
|
||||
`bootstrap.sh` is idempotent-friendly (`flux install` and `kubectl apply -k` re-run safely) and
|
||||
never handles key material — it only *verifies* the Secret exists.
|
||||
`bootstrap.sh` is idempotent-friendly (`flux install`, `preapply-crds.sh`'s server-side apply, and
|
||||
`kubectl apply -k` all re-run safely) and never handles key material — it only *verifies* the Secret
|
||||
exists. Beyond `flux`/`kubectl` it now also needs **`helm` + `yq`** on PATH (step 2's CRD render;
|
||||
both are in `infra/devcontainer`) and outbound network to the Kyverno chart repo.
|
||||
|
||||
## Handoff — clusters owner / PM (files this worker may not edit)
|
||||
|
||||
|
||||
@@ -33,10 +33,19 @@
|
||||
# the single apps.yaml was retired). Letting
|
||||
# bootstrap regenerate them would DUPLICATE/conflict. `flux install` +
|
||||
# applying the authored clusters/dev overlay is the reconciling path.
|
||||
# 2. sops-age Secret — the operator installs the age PRIVATE key out-of-band BEFORE reconcile so
|
||||
# 2. CRD pre-apply — ./preapply-crds.sh applies the Kyverno CRD bundle SERVER-SIDE, because
|
||||
# platform/kyverno/admission/helmrelease.yaml sets `crds: {install: false}`
|
||||
# (the ~5.6 MiB bundle blows kubectl's 256 KiB last-applied annotation limit,
|
||||
# which STALLS the HelmRelease and blocks the `platform` Kustomization and
|
||||
# every tier below it). With CRD install off, NOTHING else installs them, so
|
||||
# a cold cluster can never reconcile `platform` without this step. The chart
|
||||
# version is DERIVED from that same HelmRelease — no second pin. Dev only
|
||||
# ever had these CRDs because they were applied BY HAND; this is that step,
|
||||
# in git, idempotent and version-derived. See ./preapply-crds.sh.
|
||||
# 3. sops-age Secret — the operator installs the age PRIVATE key out-of-band BEFORE reconcile so
|
||||
# decryption works on first pass (../sops/age-key-secret.md). This script
|
||||
# only VERIFIES it exists; it never handles key material.
|
||||
# 3. `kubectl apply -k ${FLUX_ENV_PATH}` — materialises the authored reconciliation graph:
|
||||
# 4. `kubectl apply -k ${FLUX_ENV_PATH}` — materialises the authored reconciliation graph:
|
||||
# GitRepository `flux-system` (source) -> Kustomization `platform` ->
|
||||
# `platform-config` -> the app-wave Kustomizations `apps-wave-0/1/2`
|
||||
# (chained dependsOn) + `apps-vms`, with cluster-vars overlay
|
||||
@@ -71,7 +80,10 @@ repo_root="$(git -C "${script_dir}" rev-parse --show-toplevel 2>/dev/null || (cd
|
||||
echo "==> Flux + SOPS bootstrap — path=${FLUX_ENV_PATH}"
|
||||
|
||||
# --- 0. tooling present (from infra/devcontainer; versions pinned there) ------
|
||||
for bin in flux kubectl; do
|
||||
# helm + yq are needed by step 2 (./preapply-crds.sh renders the CRD bundle from the authored
|
||||
# HelmRelease). Checked HERE as well as in that script so a missing binary stops the run BEFORE the
|
||||
# Flux controllers are installed, instead of half-way through.
|
||||
for bin in flux kubectl helm yq; do
|
||||
command -v "$bin" >/dev/null 2>&1 || { echo "ERROR: '$bin' not found (use infra/devcontainer)." >&2; exit 1; }
|
||||
done
|
||||
|
||||
@@ -99,7 +111,7 @@ done
|
||||
#
|
||||
# ESCAPE HATCH: FLUX_INSTALL_PLAIN=1 falls back to a bare `flux install` (no customization, so NO
|
||||
# strict substitution). Use only to unblock a bring-up if the render path misbehaves, and say so.
|
||||
echo "==> [1/3] installing Flux controllers${FLUX_COMPONENTS_EXTRA:+ (+ extra: ${FLUX_COMPONENTS_EXTRA})}"
|
||||
echo "==> [1/4] installing Flux controllers${FLUX_COMPONENTS_EXTRA:+ (+ extra: ${FLUX_COMPONENTS_EXTRA})}"
|
||||
flux_install_args=(--namespace="${FLUX_NAMESPACE}")
|
||||
[ -n "${FLUX_COMPONENTS_EXTRA}" ] && flux_install_args+=(--components-extra="${FLUX_COMPONENTS_EXTRA}")
|
||||
|
||||
@@ -130,18 +142,29 @@ else
|
||||
kubectl apply -f "${render_dir}/rendered.yaml"
|
||||
fi
|
||||
|
||||
# --- 2. verify the out-of-band age key Secret exists -------------------------
|
||||
# --- 2. pre-apply the CRDs Helm is not allowed to install ---------------------
|
||||
# platform/kyverno/admission/helmrelease.yaml sets `crds: {install: false}`, so the 22 Kyverno CRDs
|
||||
# must exist BEFORE the platform Kustomization reconciles that HelmRelease — otherwise Kyverno's
|
||||
# controllers come up with no API types, `platform` never goes Ready, and all of its dependents stall.
|
||||
# ./preapply-crds.sh derives the chart version from the HelmRelease itself (no second pin), applies
|
||||
# only the CRD documents SERVER-SIDE (no 256 KiB annotation limit), and is idempotent. It FAILS HARD
|
||||
# rather than skipping — there is deliberately NO opt-out flag: a skipped CRD apply produces a
|
||||
# cluster that looks bootstrapped and reconciles nothing.
|
||||
echo "==> [2/4] pre-applying chart CRDs (server-side)"
|
||||
bash "${script_dir}/preapply-crds.sh"
|
||||
|
||||
# --- 3. verify the out-of-band age key Secret exists -------------------------
|
||||
# We DO NOT create it here (it holds the private key). The operator installs it per
|
||||
# ../sops/age-key-secret.md. Fail fast if missing so decryption can't silently no-op.
|
||||
echo "==> [2/3] verifying ${SOPS_SECRET_NAME} Secret exists in ${FLUX_NAMESPACE}"
|
||||
echo "==> [3/4] verifying ${SOPS_SECRET_NAME} Secret exists in ${FLUX_NAMESPACE}"
|
||||
if ! kubectl -n "${FLUX_NAMESPACE}" get secret "${SOPS_SECRET_NAME}" >/dev/null 2>&1; then
|
||||
echo "ERROR: Secret ${FLUX_NAMESPACE}/${SOPS_SECRET_NAME} not found." >&2
|
||||
echo " Install the age private key first: bootstrap/sops/age-key-secret.md" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- 3. apply the authored environment entrypoint ----------------------------
|
||||
echo "==> [3/3] kubectl apply -k ${FLUX_ENV_PATH}"
|
||||
# --- 4. apply the authored environment entrypoint ----------------------------
|
||||
echo "==> [4/4] kubectl apply -k ${FLUX_ENV_PATH}"
|
||||
kubectl apply -k "${repo_root}/${FLUX_ENV_PATH}"
|
||||
|
||||
echo
|
||||
|
||||
Executable
+169
@@ -0,0 +1,169 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# preapply-crds.sh — server-side pre-apply of the CRDs that Helm is NOT allowed to install.
|
||||
# ENVIRONMENT-AGNOSTIC: it reads the chart coordinates out of the authored HelmRelease and talks to
|
||||
# whatever cluster $KUBECONFIG points at, so dev and prod run the SAME step with no fork (R-ENV-02).
|
||||
# Called by ./bootstrap.sh (step 2/4); also safe to run STANDALONE against an already-bootstrapped
|
||||
# cluster (that is how prod gets unblocked without re-running the whole bootstrap).
|
||||
#
|
||||
# R-GITOPS-01 (the graph must reconcile from git, not from someone's shell history) · R-ENV-02.
|
||||
# =============================================================================
|
||||
# *** AUTHORED, NOT RUN. Agents must not execute it — it MUTATES a cluster. ***
|
||||
# =============================================================================
|
||||
#
|
||||
# WHY THIS EXISTS
|
||||
#
|
||||
# platform/kyverno/admission/helmrelease.yaml sets `crds: {install: false}`. That is deliberate:
|
||||
# the Kyverno CRD bundle is ~5.6 MiB, far past kubectl's 256 KiB last-applied-configuration
|
||||
# annotation limit, so Helm's client-side apply fails with "metadata.annotations: Too long" and
|
||||
# STALLS the HelmRelease — which blocks the `platform` Kustomization and every tier that dependsOn
|
||||
# it (the kps-cascade failure mode; see the HelmRelease's own comment).
|
||||
#
|
||||
# The consequence nobody automated: with CRD install off, NOTHING installs those CRDs. Kyverno's
|
||||
# Deployments come up but its API types do not exist, so `platform` never goes Ready on a cold
|
||||
# cluster. The dev cluster only has them because they were once applied BY HAND — exactly the
|
||||
# undocumented manual step this repo exists to eliminate. This script is that step, in git,
|
||||
# version-derived and idempotent.
|
||||
#
|
||||
# HOW THE VERSION IS DERIVED (there is deliberately NO second copy of it)
|
||||
#
|
||||
# Everything comes out of the HelmRelease/HelmRepository pair that Flux itself reconciles:
|
||||
#
|
||||
# chart name spec.chart.spec.chart (HelmRelease)
|
||||
# chart version spec.chart.spec.version (HelmRelease) e.g. "3.9.x"
|
||||
# chart repo url spec.url (HelmRepository)
|
||||
# namespace metadata.namespace (HelmRelease)
|
||||
#
|
||||
# The version is a semver CONSTRAINT, not a fixed tag. `helm template --version` accepts the same
|
||||
# constraint syntax Flux's helm-controller resolves (both use Masterminds/semver), so passing it
|
||||
# through UNCHANGED renders the very chart version Flux will install. Renovate bumps one line in
|
||||
# one file and this step follows automatically — it cannot drift, because there is nothing here to
|
||||
# drift from.
|
||||
#
|
||||
# WHY ONLY THE CRD DOCUMENTS ARE APPLIED
|
||||
#
|
||||
# The by-hand command recorded in the HelmRelease comment piped the WHOLE `helm template` output
|
||||
# into kubectl. Do not do that: it would create Deployments/Services/webhooks outside Helm's
|
||||
# ownership, which then fight the HelmRelease over field managers and leave orphans behind. We
|
||||
# filter to `kind: CustomResourceDefinition` and apply nothing else. Values are the chart DEFAULTS
|
||||
# (plus an explicit crds.install=true) — NOT the repo's values, which switch CRD rendering off.
|
||||
#
|
||||
# IDEMPOTENCY
|
||||
#
|
||||
# `kubectl apply --server-side --force-conflicts` is a declarative upsert: the first run creates,
|
||||
# every later run reports "serverside-applied" and exits 0, and field ownership is taken from
|
||||
# whatever previously wrote those objects (that is what --force-conflicts is for — it resolves the
|
||||
# hand-applied dev CRDs' `kubectl-client-side-apply` manager without a human deciding anything).
|
||||
# Server-side apply does not write the last-applied annotation, which is the whole reason the
|
||||
# 5.6 MiB bundle can be applied at all.
|
||||
#
|
||||
# LOUD FAILURE (this script must never degrade to a successful no-op)
|
||||
#
|
||||
# `set -euo pipefail`, plus three explicit guards, because a silent skip here reproduces the exact
|
||||
# class of bug this repo has been bitten by more than once:
|
||||
# 1. every derived coordinate must be non-empty and not "null" — a moved YAML path fails here;
|
||||
# 2. the rendered CRD count must be > 0 — if a future chart moves/renames its CRD templates the
|
||||
# filter would match nothing and `kubectl apply` on an empty stream EXITS 0. Guarded;
|
||||
# 3. after applying, every CRD is waited on for condition=Established — so "applied" is proven
|
||||
# to mean "the API type exists", not merely "kubectl returned 0".
|
||||
#
|
||||
set -euo pipefail
|
||||
|
||||
# --- config (env-overridable) ------------------------------------------------
|
||||
# The single source of truth for the chart coordinates. Repo-relative on purpose: this is the file
|
||||
# Flux reconciles, so it is the file this step must agree with.
|
||||
CRD_HELMRELEASE="${CRD_HELMRELEASE:-platform/kyverno/admission/helmrelease.yaml}"
|
||||
# How long to wait for the applied CRDs to become Established (apiextensions publishes the type).
|
||||
CRD_ESTABLISH_TIMEOUT="${CRD_ESTABLISH_TIMEOUT:-120s}"
|
||||
|
||||
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
repo_root="$(git -C "${script_dir}" rev-parse --show-toplevel 2>/dev/null || (cd "${script_dir}/../.." && pwd))"
|
||||
|
||||
echo "==> CRD pre-apply — source of truth: ${CRD_HELMRELEASE}"
|
||||
|
||||
# --- 0. tooling present (from infra/devcontainer; versions pinned there) ------
|
||||
# helm renders the bundle, yq selects the CRD documents, kubectl applies them server-side. All three
|
||||
# are baked into infra/devcontainer (HELM_VERSION / YQ_VERSION / KUBECTL_VERSION in versions.env).
|
||||
for bin in helm yq kubectl; do
|
||||
command -v "$bin" >/dev/null 2>&1 || { echo "ERROR: '$bin' not found (use infra/devcontainer)." >&2; exit 1; }
|
||||
done
|
||||
|
||||
helmrelease="${repo_root}/${CRD_HELMRELEASE}"
|
||||
[ -f "${helmrelease}" ] || {
|
||||
echo "ERROR: ${helmrelease} not found — CRD_HELMRELEASE points nowhere." >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
# --- 1. derive the chart coordinates from the authored manifests --------------
|
||||
chart_name="$(yq eval-all --no-doc 'select(.kind == "HelmRelease") | .spec.chart.spec.chart' "${helmrelease}")"
|
||||
chart_version="$(yq eval-all --no-doc 'select(.kind == "HelmRelease") | .spec.chart.spec.version' "${helmrelease}")"
|
||||
chart_ns="$(yq eval-all --no-doc 'select(.kind == "HelmRelease") | .metadata.namespace' "${helmrelease}")"
|
||||
chart_repo="$(yq eval-all --no-doc 'select(.kind == "HelmRepository") | .spec.url' "${helmrelease}")"
|
||||
|
||||
# Guard 1: a moved/renamed YAML path must STOP the bootstrap, not render a bogus chart. yq prints
|
||||
# "null" for a missing path and an empty string for a missing document, so both are rejected.
|
||||
for pair in "chart:${chart_name}" "version:${chart_version}" "namespace:${chart_ns}" "repo:${chart_repo}"; do
|
||||
key="${pair%%:*}"; val="${pair#*:}"
|
||||
case "${val}" in
|
||||
""|null|*$'\n'*)
|
||||
echo "ERROR: could not derive a single '${key}' from ${CRD_HELMRELEASE} (got: '${val}')." >&2
|
||||
echo " The HelmRelease/HelmRepository shape changed — fix the yq paths in $(basename "${BASH_SOURCE[0]}")." >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo " chart=${chart_name} version=${chart_version} repo=${chart_repo} namespace=${chart_ns}"
|
||||
echo " (version is the constraint Flux resolves — NOT a second copy pinned here)"
|
||||
|
||||
# --- 2. render the chart and keep ONLY the CRD documents ----------------------
|
||||
render_dir="$(mktemp -d)"
|
||||
trap 'rm -rf "${render_dir}"' EXIT
|
||||
crd_file="${render_dir}/crds.yaml"
|
||||
|
||||
# --include-crds covers charts that ship a native crds/ dir; crds.install=true covers Kyverno, whose
|
||||
# CRDs are subchart TEMPLATES that the repo's own values switch off. Chart defaults otherwise.
|
||||
echo "==> rendering ${chart_name} ${chart_version} (CRD documents only)"
|
||||
helm template "${chart_name}" "${chart_name}" \
|
||||
--repo "${chart_repo}" \
|
||||
--version "${chart_version}" \
|
||||
--namespace "${chart_ns}" \
|
||||
--include-crds \
|
||||
--set crds.install=true \
|
||||
| yq eval-all 'select(.kind == "CustomResourceDefinition")' - > "${crd_file}"
|
||||
|
||||
# Blank/"null" lines are dropped on purpose: an EMPTY render makes `yq .metadata.name` emit one
|
||||
# empty line, which would otherwise count as a CRD and sail straight past guard 2 below.
|
||||
mapfile -t crd_names < <(yq eval-all --no-doc '.metadata.name' "${crd_file}" | grep -vx -e '' -e 'null' || true)
|
||||
crd_count="${#crd_names[@]}"
|
||||
|
||||
# Guard 2: an empty stream applies cleanly and exits 0 — the silent-no-op trap. Refuse it.
|
||||
[ "${crd_count}" -gt 0 ] || {
|
||||
echo "ERROR: rendered 0 CustomResourceDefinitions from ${chart_name} ${chart_version}." >&2
|
||||
echo " The chart's CRD layout or the crds.install value key changed. Refusing to continue:" >&2
|
||||
echo " applying nothing would look like success and leave 'platform' blocked forever." >&2
|
||||
exit 1
|
||||
}
|
||||
echo " ${crd_count} CRDs, $(wc -c < "${crd_file}") bytes (>256 KiB — this is why it must be server-side)"
|
||||
|
||||
# --- 3. server-side apply (idempotent upsert) ---------------------------------
|
||||
# --server-side: no last-applied annotation, so the size limit does not apply.
|
||||
# --force-conflicts: adopt fields owned by an earlier manager (e.g. dev's hand-applied CRDs, or
|
||||
# Helm's own manager if a past chart version did install them). Makes a re-run a plain no-op.
|
||||
echo "==> kubectl apply --server-side --force-conflicts (${crd_count} CRDs)"
|
||||
kubectl apply --server-side --force-conflicts -f "${crd_file}"
|
||||
|
||||
# --- 4. prove the API types actually exist -----------------------------------
|
||||
# Without this, "kubectl returned 0" is the only evidence. Establishment is what Kyverno's
|
||||
# controllers and the platform Kustomization actually need.
|
||||
echo "==> waiting for condition=Established (timeout ${CRD_ESTABLISH_TIMEOUT})"
|
||||
crd_refs=()
|
||||
for n in "${crd_names[@]}"; do crd_refs+=("crd/${n}"); done
|
||||
kubectl wait --for=condition=Established --timeout="${CRD_ESTABLISH_TIMEOUT}" \
|
||||
"${crd_refs[@]}" >/dev/null || {
|
||||
echo "ERROR: not all ${crd_count} CRDs reached Established within ${CRD_ESTABLISH_TIMEOUT}." >&2
|
||||
echo " Inspect: kubectl get crd | grep ${chart_name}" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
echo "==> ${crd_count} ${chart_name} CRDs applied and Established."
|
||||
Reference in new issue
Block a user