feat(pihole): declare the admin password and adlists in git
Prerequisite for running more than one PiHole replica. A second replica with a fresh PVC would have come up serving DNS and blocking nothing, with a different admin password -- answers varying by which pod replied. pihole.toml's own ### CHANGED markers gave the exact inventory: four settings already came from env (declared), one came from the image, and exactly ONE operator-set value lived only on the PVC -- webserver.api.pwhash, a one-way hash nobody could recover. Separately the adlists were undeclared: one list and 72,234 gravity domains that nothing in this repo named. Declares both: - A SOPS secret feeding FTLCONF_webserver_api_password. webserver.api.password takes cleartext and FTL hashes it itself (verified: pihole setpassword IS setFTLConfigValue on that key), so the PVC's pwhash becomes derived state. - A level-triggered postStart hook that reconciles the declared adlist set into gravity.db -- adds, re-enables, and disables (never deletes) undeclared lists, running pihole -g only when something moved. A file would not work: v6's adlists.list import is migration-only, guarded on the DB not existing. The rendered hook was run in the live pod against throwaway copies of gravity.db, exercising all four paths. A fresh DB built from the image's own schema template has ZERO adlists, confirming this is load-bearing rather than cosmetic. Password kept rather than set empty: the pihole Service still publishes port 80 past the oauth2-proxy, closed only by policy, and default-deny has drifted here before. Multi-replica is phase 2 and deliberately not in this branch -- it needs a schema field, a StatefulSet (the config PVC is RWO), anti-affinity and a PDB. Adds a *.sops.yaml -- a sensitive path -- so approval-policy will block pending an operator /approve-override. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
jakman85andClaude Opus 5 committed
1 parent
939461120a
commit
3b1d6ae115
5 files changed
+336
No files matched your search
@@ -14,6 +14,7 @@ resources:
|
||||
- nextcloud-db-backup-credentials-prod.sops.yaml
|
||||
- nextcloud-db-backup-credentials.sops.yaml
|
||||
- nextcloud-db.yaml
|
||||
- pihole-bootstrap-secret.sops.yaml
|
||||
- pihole-dns.yaml
|
||||
- pihole-oauth2-proxy-credentials.sops.yaml
|
||||
- pihole.yaml
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
# =================================================================================================
|
||||
# PiHole WEB/API ADMIN PASSWORD — the one piece of PiHole state that was NOT declared anywhere.
|
||||
# =================================================================================================
|
||||
# WHY THIS FILE EXISTS. Measured on live prod 2026-10-04, `pihole.toml` marks every non-default
|
||||
# setting with `### CHANGED`, and names the ones that came from the environment `### CHANGED (env)`.
|
||||
# The complete list was:
|
||||
#
|
||||
# dns.upstreams ### CHANGED (env) <- declared, catalog/pihole.yaml
|
||||
# dns.listeningMode ### CHANGED (env) <- declared, catalog/pihole.yaml
|
||||
# dns.rateLimit.count ### CHANGED (env) <- declared, catalog/pihole.yaml
|
||||
# misc.dnsmasq_lines ### CHANGED (env) <- declared, catalog/pihole.yaml
|
||||
# files.macvendor ### CHANGED <- set by the image itself, container path
|
||||
# webserver.api.pwhash ### CHANGED <- SET OUT OF BAND. Declared NOWHERE. This file.
|
||||
#
|
||||
# So exactly ONE operator-set value lived only on the Longhorn PVC: the admin password, as a
|
||||
# one-way BALLOON-SHA256 hash. Nobody could recover it (a hash does not reverse), nothing in git
|
||||
# produced it, and a rebuild from git would have come up with a DIFFERENT password — or none.
|
||||
#
|
||||
# THE REASON THIS BLOCKS HA, which is why it is fixed first. The next step is running more than one
|
||||
# PiHole replica, and a replica with a fresh PVC would come up with its own unrelated password. The
|
||||
# admin UI would then accept or reject the same credential depending on which pod answered. Declaring
|
||||
# it here makes every replica, and every rebuild, identical by construction.
|
||||
#
|
||||
# MECHANISM: `webserver.api.password` is a real, settable FTL config key — verified in the running
|
||||
# image, where `pihole setpassword` is itself just `setFTLConfigValue "webserver.api.password"` with
|
||||
# the comment "pihole-FTL will automatically hash the password". So FTL takes CLEARTEXT on this key
|
||||
# and writes the hash into `webserver.api.pwhash` itself. catalog/pihole.yaml therefore passes this
|
||||
# value as `FTLCONF_webserver_api_password`, and the hash on the PVC becomes a derived artefact
|
||||
# rather than the source of truth.
|
||||
#
|
||||
# DEFENCE IN DEPTH, NOT THE ONLY GATE. The admin UI is already behind the oauth2-proxy sidecar
|
||||
# (`--allowed-group=platform-admins`), so this password is a SECOND gate. It is kept rather than set
|
||||
# empty because the `pihole` Service still publishes port 80 (PiHole direct, bypassing the proxy on
|
||||
# :4180) and only the CiliumNetworkPolicy closes it — and the default-deny policy in this repo has a
|
||||
# documented drift history. An unauthenticated admin UI that can rewrite LAN DNS is not a good thing
|
||||
# to leave resting on one policy staying correct. Narrowing that Service port is tracked follow-up.
|
||||
#
|
||||
# ROTATION: `sops` this file in your own terminal, change the value, commit. Flux re-applies the
|
||||
# Secret, the pod restarts, FTL re-hashes. Do NOT `sops -d` it into a shared terminal or a log
|
||||
# (CLAUDE.md section 6); read it in an editor. The value below was generated randomly at authoring
|
||||
# time and has never been printed to a transcript.
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: pihole-bootstrap
|
||||
namespace: pihole
|
||||
labels:
|
||||
app.kubernetes.io/name: pihole
|
||||
type: Opaque
|
||||
stringData:
|
||||
#ENC[AES256_GCM,data:TMOuRyJeb5IVn3lZPXY7lgh1KGxu7GY9+WIOLHpMB50JqzmM2tHIaWUWKTt1JtZoX/E5/Y7WCUTnfkJLhRBN2zV2MwWRBSLYVy5CMg7xV8PTLRZJYAVUYS5Trg==,iv:q+DpfT6PQfY5ZuNsmP7Ddl+PDLjSm89fJICAhYoW64k=,tag:axijaO3DaxIZgZ3LeK2hXA==,type:comment]
|
||||
webui-password: ENC[AES256_GCM,data:7FcK/bUW79M46f/MlouAmATKbMnoUTNuMUjtqJ5vpU4=,iv:I8DBFB39uXZV42Ie2IPwSefmMgdKEtiDknfc9Tu7h5E=,tag:L423784UrDU3TvylpHc6KQ==,type:str]
|
||||
sops:
|
||||
kms: []
|
||||
gcp_kms: []
|
||||
azure_kv: []
|
||||
hc_vault: []
|
||||
age:
|
||||
- recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
|
||||
enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB0MkRSNjZLZlpXYkcvaVJL
|
||||
cjlqb2wrcWREZnZLdGpsTHNPWEpseHlRZnpjCnJVM1FDQ2M2dXlaS1JBZ2k2V1BP
|
||||
Yi9QUDZSMzZsL09pTi93RTdjdGxtY2MKLS0tIDIrQTZIVWdhMGIxSCtlUFN2Mlho
|
||||
SWVybld4VjJyaHRSKzZZU0xsUUV2SEEKbDYHV/qBZVspK7DpBW3vYjRMJ/HmAN+9
|
||||
G9bOlNEp0jAtcNZEn6lpQ9ORgvmIrhJHoBQRLabdMWzNWSZsNn56tA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
lastmodified: "2026-10-04T06:01:27Z"
|
||||
mac: ENC[AES256_GCM,data:rhTMTOwLOtHXHgqI5M2ZW/6w+UpLpxBlTk7NoN0wmGSqOcE+OGrXA1DhkZrJweyQT0RSNSSBm/6QMztLP81eepOzdp2rEkwvbQPK/l6K9g16Ww9c1IatS3NQsKEB8AoMDx7jBQ6exXYgggPx3MJsNYbE+WyloQ96U9KPdrecAM8=,iv:n34v2zX9D5xsaR1PFCH94mNMiuRfhUyI2lseUCoqiTA=,tag:9YmN9nIVLXfsGYIb0bUl2w==,type:str]
|
||||
pgp: []
|
||||
encrypted_regex: ^(data|stringData)$
|
||||
version: 3.9.4
|
||||
@@ -90,9 +90,21 @@ spec:
|
||||
value: '0'
|
||||
- name: FTLCONF_misc_dnsmasq_lines
|
||||
value: address=/${BASE_DOMAIN}/${GATEWAY_VIP};address=/unifi.${BASE_DOMAIN}/${UNIFI_L4_VIP};address=/crafty-mc.${BASE_DOMAIN}/${CRAFTY_L4_VIP};address=/pihole-dns.${BASE_DOMAIN}/${PIHOLE_DNS_VIP}
|
||||
- name: FTLCONF_webserver_api_password
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: pihole-bootstrap
|
||||
key: webui-password
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /data/config
|
||||
lifecycle:
|
||||
postStart:
|
||||
exec:
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- "# Assert the declared adlists in gravity.db. Idempotent: steady state writes nothing and\n# does NOT run a gravity rebuild.\nset -u\nGRAVITY=/etc/pihole/gravity.db\n\n# THE DECLARED SET \u2014 one URL per line, the source of truth for what PiHole blocks.\n# To add a list: add a line here, commit, let Flux reconcile. Nothing else to do.\nDECLARED='https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts'\n\n# On a fresh PVC, FTL creates gravity.db a moment after the container starts. Wait, bounded,\n# and NEVER block or fail the start: a missing DB here must not crashloop DNS for the LAN.\ni=0\nwhile [ ! -s \"$GRAVITY\" ] && [ \"$i\" -lt 90 ]; do sleep 2; i=$(expr $i + 1); done\nif [ ! -s \"$GRAVITY\" ]; then\n echo \"pihole-adlists: gravity.db absent after 180s - leaving adlists untouched\" >&2\n exit 0\nfi\n\nsql() { pihole-FTL sqlite3 \"$GRAVITY\" \"$1\"; }\nCHANGED=0\n\n# 1) Every declared URL must be present AND enabled.\necho \"$DECLARED\" | while IFS= read -r url; do\n [ -n \"$url\" ] || continue\n printf '%s\\n' \"$url\"\ndone > /tmp/declared.txt\nwhile IFS= read -r url; do\n [ -n \"$url\" ] || continue\n # Single quotes inside a URL would break this SQL; adlist URLs do not contain them.\n have=$(sql \"select count(*) from adlist where address='$url';\")\n if [ \"$have\" = \"0\" ]; then\n echo \"pihole-adlists: adding $url\"\n sql \"insert into adlist (address,enabled,comment) values ('$url',1,'declared in catalog/pihole.yaml');\"\n CHANGED=1\n else\n en=$(sql \"select enabled from adlist where address='$url';\")\n if [ \"$en\" != \"1\" ]; then\n echo \"pihole-adlists: re-enabling $url\"\n sql \"update adlist set enabled=1 where address='$url';\"\n CHANGED=1\n fi\n fi\ndone < /tmp/declared.txt\n\n# 2) Anything enabled in the DB but NOT declared gets disabled (never deleted - see above).\nsql \"select address from adlist where enabled=1;\" | while IFS= read -r addr; do\n [ -n \"$addr\" ] || continue\n if ! grep -qxF \"$addr\" /tmp/declared.txt; then\n echo \"pihole-adlists: disabling undeclared $addr\"\n sql \"update adlist set enabled=0 where address='$addr';\"\n echo changed >> /tmp/adlist-changed\n fi\ndone\n[ -s /tmp/adlist-changed ] && CHANGED=1\nrm -f /tmp/adlist-changed /tmp/declared.txt\n\n# 3) Rebuild gravity ONLY if the declared set actually moved.\nif [ \"$CHANGED\" = \"1\" ]; then\n echo \"pihole-adlists: declared set changed - running gravity update\"\n pihole -g || echo \"pihole-adlists: gravity update failed; existing gravity.db retained\" >&2\nelse\n echo \"pihole-adlists: already matches the declared set - nothing to do\"\nfi\nexit 0\n"
|
||||
- name: oauth2-proxy
|
||||
image: quay.io/oauth2-proxy/oauth2-proxy:v7.8.1
|
||||
imagePullPolicy: IfNotPresent
|
||||
|
||||
@@ -100,6 +100,20 @@ env:
|
||||
# while resolution for the zone does not work. Confirmed both ways against the live image.
|
||||
- name: FTLCONF_misc_dnsmasq_lines
|
||||
value: "address=/${BASE_DOMAIN}/${GATEWAY_VIP};address=/unifi.${BASE_DOMAIN}/${UNIFI_L4_VIP};address=/crafty-mc.${BASE_DOMAIN}/${CRAFTY_L4_VIP};address=/pihole-dns.${BASE_DOMAIN}/${PIHOLE_DNS_VIP}"
|
||||
# THE ADMIN PASSWORD, previously the ONE operator-set value declared nowhere. Sourced from
|
||||
# apps/wave-0/pihole-bootstrap-secret.sops.yaml (SOPS/age, same wave => same reconcile), which
|
||||
# carries the full rationale and the measurement that found it.
|
||||
#
|
||||
# `webserver.api.password` takes CLEARTEXT and FTL hashes it itself into `webserver.api.pwhash`.
|
||||
# Verified in the running image, not assumed: `pihole setpassword` is itself
|
||||
# `setFTLConfigValue "webserver.api.password" "$PASSWORD"`, commented upstream as "pihole-FTL will
|
||||
# automatically hash the password". So the BALLOON-SHA256 hash on the PVC is now a DERIVED artefact
|
||||
# and this is its source of truth. Rotating the Secret rotates the password.
|
||||
- name: FTLCONF_webserver_api_password
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: pihole-bootstrap
|
||||
key: webui-password
|
||||
ingressRoutes:
|
||||
- host: pihole.${BASE_DOMAIN}
|
||||
port: 80 # admin web UI, protected at the Gateway by forward-auth (docs/09)
|
||||
@@ -114,6 +128,110 @@ auth:
|
||||
# rather than a renderer constant: the measurement that justifies the flag on two apps is
|
||||
# evidence about those two apps. Its oauth2-proxy log holds 0 unauthenticated requests across
|
||||
# the current pod's whole 23h lifetime (measured 2026-10-01), so there is nothing to act on.
|
||||
lifecycle:
|
||||
# LEVEL-TRIGGERED ADLIST ASSERTION, run on EVERY container start. Same pattern, and the same
|
||||
# reasoning, as catalog/qbittorrent.yaml's postStart: desired state lives in git and is
|
||||
# RE-ASSERTED unconditionally rather than set once by hand.
|
||||
#
|
||||
# WHY THIS IS NEEDED AT ALL — the blocklists were NOT reproducible from git. Measured on live prod
|
||||
# 2026-10-04: `gravity.db` held exactly ONE adlist (StevenBlack's hosts, enabled) and 72,234
|
||||
# gravity domains, and NOTHING in this repo named it. Worse, a FRESH database does not seed it:
|
||||
# the image's own schema template `/etc/.pihole/advanced/Templates/gravity.db.sql` creates the
|
||||
# `adlist` table and inserts NO rows into it (grepped in the running image — there is no
|
||||
# `INSERT INTO adlist` at all). So a rebuild from git, or a replica with a new PVC, would come up
|
||||
# SERVING DNS AND BLOCKING NOTHING, silently and with every object reporting healthy.
|
||||
#
|
||||
# WHY A postStart HOOK AND NOT A FILE. PiHole v6 keeps adlists as DB rows, not config. The
|
||||
# `/etc/pihole/adlists.list` file still exists in `gravity.sh` but is MIGRATION-ONLY: its import
|
||||
# sits inside `migrate_to_database()`, guarded by `if [ ! -e "$gravityDBfile" ]`, so it fires only
|
||||
# while CREATING a database and never again on an existing one. A ConfigMap mounted there would
|
||||
# therefore be silently ignored on every pod this cluster currently runs. There is also no
|
||||
# `pihole adlist` CLI subcommand (only allow/deny for domains), so SQL against the DB is the
|
||||
# actual interface. FTL ships an embedded shell (`pihole-FTL sqlite3`, SQLite 3.53.1) so no extra
|
||||
# tooling is needed in the image.
|
||||
#
|
||||
# IT RECONCILES, IT DOES NOT JUST ADD: adlists present in the DB but absent from DECLARED below are
|
||||
# DISABLED (not deleted — deleting cascades to `gravity` rows and would force a full re-download of
|
||||
# everything on a typo). So this list is the source of truth in both directions, and `pihole -g`
|
||||
# runs ONLY when something actually changed, which keeps the steady-state start free of a 70k-domain
|
||||
# rebuild.
|
||||
#
|
||||
# NO BRACED DOLLAR EXPANSION ANYWHERE IN THIS SCRIPT, DELIBERATELY — the same contract
|
||||
# catalog/qbittorrent.yaml documents at length. apps-wave-0 declares postBuild.substituteFrom
|
||||
# cluster-vars, so Flux runs envsubst over this whole manifest at reconcile time; a `${VAR}` form
|
||||
# would behave differently on the cluster than under a plain `kubectl apply` of the golden. Bare
|
||||
# `$VAR` and `$(cmd)` are untouched by envsubst and so behave identically however this is run.
|
||||
# `render.py`'s assert_substitution_safe refuses to emit a manifest that breaks this.
|
||||
postStart:
|
||||
exec:
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- |
|
||||
# Assert the declared adlists in gravity.db. Idempotent: steady state writes nothing and
|
||||
# does NOT run a gravity rebuild.
|
||||
set -u
|
||||
GRAVITY=/etc/pihole/gravity.db
|
||||
|
||||
# THE DECLARED SET — one URL per line, the source of truth for what PiHole blocks.
|
||||
# To add a list: add a line here, commit, let Flux reconcile. Nothing else to do.
|
||||
DECLARED='https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts'
|
||||
|
||||
# On a fresh PVC, FTL creates gravity.db a moment after the container starts. Wait, bounded,
|
||||
# and NEVER block or fail the start: a missing DB here must not crashloop DNS for the LAN.
|
||||
i=0
|
||||
while [ ! -s "$GRAVITY" ] && [ "$i" -lt 90 ]; do sleep 2; i=$(expr $i + 1); done
|
||||
if [ ! -s "$GRAVITY" ]; then
|
||||
echo "pihole-adlists: gravity.db absent after 180s - leaving adlists untouched" >&2
|
||||
exit 0
|
||||
fi
|
||||
|
||||
sql() { pihole-FTL sqlite3 "$GRAVITY" "$1"; }
|
||||
CHANGED=0
|
||||
|
||||
# 1) Every declared URL must be present AND enabled.
|
||||
echo "$DECLARED" | while IFS= read -r url; do
|
||||
[ -n "$url" ] || continue
|
||||
printf '%s\n' "$url"
|
||||
done > /tmp/declared.txt
|
||||
while IFS= read -r url; do
|
||||
[ -n "$url" ] || continue
|
||||
# Single quotes inside a URL would break this SQL; adlist URLs do not contain them.
|
||||
have=$(sql "select count(*) from adlist where address='$url';")
|
||||
if [ "$have" = "0" ]; then
|
||||
echo "pihole-adlists: adding $url"
|
||||
sql "insert into adlist (address,enabled,comment) values ('$url',1,'declared in catalog/pihole.yaml');"
|
||||
CHANGED=1
|
||||
else
|
||||
en=$(sql "select enabled from adlist where address='$url';")
|
||||
if [ "$en" != "1" ]; then
|
||||
echo "pihole-adlists: re-enabling $url"
|
||||
sql "update adlist set enabled=1 where address='$url';"
|
||||
CHANGED=1
|
||||
fi
|
||||
fi
|
||||
done < /tmp/declared.txt
|
||||
|
||||
# 2) Anything enabled in the DB but NOT declared gets disabled (never deleted - see above).
|
||||
sql "select address from adlist where enabled=1;" | while IFS= read -r addr; do
|
||||
[ -n "$addr" ] || continue
|
||||
if ! grep -qxF "$addr" /tmp/declared.txt; then
|
||||
echo "pihole-adlists: disabling undeclared $addr"
|
||||
sql "update adlist set enabled=0 where address='$addr';"
|
||||
echo changed >> /tmp/adlist-changed
|
||||
fi
|
||||
done
|
||||
[ -s /tmp/adlist-changed ] && CHANGED=1
|
||||
rm -f /tmp/adlist-changed /tmp/declared.txt
|
||||
|
||||
# 3) Rebuild gravity ONLY if the declared set actually moved.
|
||||
if [ "$CHANGED" = "1" ]; then
|
||||
echo "pihole-adlists: declared set changed - running gravity update"
|
||||
pihole -g || echo "pihole-adlists: gravity update failed; existing gravity.db retained" >&2
|
||||
else
|
||||
echo "pihole-adlists: already matches the declared set - nothing to do"
|
||||
fi
|
||||
exit 0
|
||||
update:
|
||||
channel: stable # R-UPD-01
|
||||
range: ">=2024.0.0"
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
---
|
||||
date: 2026-10-04
|
||||
branch: feat/pihole-declare-state
|
||||
title: PiHole's operator-set state is now declared in git — the prerequisite for running more than one replica
|
||||
---
|
||||
|
||||
## What changed
|
||||
|
||||
Phase 1 of making LAN DNS survive a node failure. The operator asked for multi-replica PiHole;
|
||||
measurement showed that cannot be done safely first, because PiHole's important state was not
|
||||
reproducible from git. A second replica with a fresh PVC would have come up **serving DNS and
|
||||
blocking nothing, with a different admin password** — answers varying by which pod replied. This
|
||||
branch closes that; the replica work is phase 2 and is deliberately NOT here.
|
||||
|
||||
**The undeclared-state inventory, measured not guessed.** `pihole.toml` marks every non-default
|
||||
setting `### CHANGED`, and those that came from the environment `### CHANGED (env)`. The complete
|
||||
live list on 2026-10-04:
|
||||
|
||||
| Setting | Status |
|
||||
|---|---|
|
||||
| `dns.upstreams`, `dns.listeningMode`, `dns.rateLimit.count`, `misc.dnsmasq_lines` | `### CHANGED (env)` — already declared in `catalog/pihole.yaml` |
|
||||
| `files.macvendor` | `### CHANGED` — set by the image itself (container path), not operator state |
|
||||
| `webserver.api.pwhash` | **`### CHANGED` — set out of band. Declared NOWHERE.** |
|
||||
|
||||
So exactly **one** operator-set value lived only on the PVC. Separately, the **adlists** were also
|
||||
undeclared: `gravity.db` held one list (StevenBlack's hosts) and 72,234 gravity domains, with nothing
|
||||
in this repo naming it.
|
||||
|
||||
- **`apps/wave-0/pihole-bootstrap-secret.sops.yaml` (new, SOPS/age).** Secret `pihole-bootstrap`,
|
||||
key `webui-password`. Named to the `<svc>-bootstrap-secret.sops.yaml` convention so
|
||||
`render.py --emit-waves` places it with no renderer change. The value is a 32-char random password
|
||||
generated at authoring time and **never printed to a transcript**.
|
||||
- **`catalog/pihole.yaml`:** `FTLCONF_webserver_api_password` sourced from that Secret.
|
||||
`webserver.api.password` takes CLEARTEXT and FTL hashes it itself — verified in the running image,
|
||||
where `pihole setpassword` *is* `setFTLConfigValue "webserver.api.password"`, commented upstream
|
||||
"pihole-FTL will automatically hash the password". The PVC's `pwhash` becomes a derived artefact.
|
||||
- **`catalog/pihole.yaml`:** a `lifecycle.postStart` hook that **reconciles** the declared adlist set
|
||||
into `gravity.db` on every container start — adds missing, re-enables disabled, and **disables**
|
||||
(never deletes) anything enabled but undeclared, running `pihole -g` only when something moved.
|
||||
Same level-triggered idiom, and the same reasoning, as `catalog/qbittorrent.yaml`'s postStart.
|
||||
- **`apps/wave-0/pihole.yaml`, `apps/wave-0/kustomization.yaml`:** re-rendered / secret registered.
|
||||
|
||||
**Why a postStart hook and not a file.** PiHole v6 keeps adlists as DB rows. `/etc/pihole/adlists.list`
|
||||
still exists in `gravity.sh` but is **migration-only**: its import sits inside `migrate_to_database()`
|
||||
behind `if [ ! -e "$gravityDBfile" ]`, so it fires only while *creating* a database. A ConfigMap
|
||||
mounted there would be silently ignored on every pod this cluster runs today. There is also no
|
||||
`pihole adlist` CLI subcommand, so SQL is the actual interface; FTL ships an embedded shell
|
||||
(`pihole-FTL sqlite3`) so nothing had to be added to the image.
|
||||
|
||||
**The password is kept rather than set empty, deliberately.** The admin UI is already behind the
|
||||
oauth2-proxy sidecar (`--allowed-group=platform-admins`), so an empty password would have been
|
||||
simpler. But the `pihole` Service still publishes **port 80** (PiHole direct, bypassing the proxy on
|
||||
`:4180`) and only the CiliumNetworkPolicy closes it — and this repo has a documented default-deny
|
||||
drift history. An unauthenticated admin UI that can rewrite LAN DNS should not rest on one policy
|
||||
staying correct. Narrowing that Service port is follow-up, not done here.
|
||||
|
||||
**SENSITIVE PATHS — this PR adds a `**/*.sops.yaml`, so `ci/approval-policy.sh` will BLOCK it.** It
|
||||
needs the operator's `/approve-override <reason>` as a PR **review** of type "Comment". Not posted by
|
||||
me: CLAUDE.md section 1 forbids an agent clearing this on its own PR.
|
||||
|
||||
## What I verified
|
||||
|
||||
Live prod, 2026-10-04, read-only except where noted.
|
||||
|
||||
- **The undeclared-state inventory above** — read from `pihole.toml`'s own `### CHANGED` markers, with
|
||||
the password value masked. This is the measurement the whole branch rests on.
|
||||
- **A fresh `gravity.db` seeds NO adlists.** The image's own schema template
|
||||
`/etc/.pihole/advanced/Templates/gravity.db.sql` creates the `adlist` table and contains no
|
||||
`INSERT INTO adlist` at all — grepped in the running image. Confirmed empirically below (0 rows).
|
||||
This is why the adlist declaration is load-bearing and not cosmetic.
|
||||
- **The adlists.list path really is migration-only** — read the guard in `/opt/pihole/gravity.sh`
|
||||
(`migrate_to_database()`, `if [ ! -e "$gravityDBfile" ]`).
|
||||
- **`webserver.api.password` is a real settable key** — `pihole-FTL --config webserver.api.password`
|
||||
returns a masked value, and `/usr/local/bin/pihole`'s `setpassword` sets exactly that key.
|
||||
- **THE RENDERED HOOK ACTUALLY WORKS — all four paths, extracted from `apps/wave-0/pihole.yaml` and
|
||||
run in the live pod against THROWAWAY COPIES of `gravity.db` in `/tmp`** (production `gravity.db`
|
||||
was never written, and `pihole -g` was stubbed so no 70k-domain download could start):
|
||||
1. *Steady state* → `already matches the declared set - nothing to do`. No writes, no rebuild.
|
||||
2. *True fresh DB built from the image's schema template* → 0 adlists before; hook added the
|
||||
declared list and triggered the rebuild. **This is the fresh-PVC/new-replica case.**
|
||||
3. *An undeclared list enabled* → disabled it (`enabled=0`), left the declared one alone, and did
|
||||
not delete the row.
|
||||
4. *Declared list present but disabled* → re-enabled it.
|
||||
- **Required tools exist in the image:** `bash`, `expr`, `grep`, `printf`, `sleep`, `pihole-FTL`.
|
||||
- **The Secret is genuinely encrypted** to the repo's age recipient; `stringData.webui-password`
|
||||
starts `ENC[`. The gate's `no-plaintext-secrets (R-SEC-01)` L0 check passes.
|
||||
- **Schema:** `check-jsonschema` against `schemas/servicedefinition.schema.json` passes.
|
||||
- **Render determinism:** the canonical
|
||||
`render.py --all catalog -o apps --wave-layout --no-image-automation` changed **only**
|
||||
`apps/wave-0/pihole.yaml`. `render-golden` passes, so the hook's text also satisfies
|
||||
`assert_substitution_safe` (no braced dollar expansion — the Flux envsubst contract).
|
||||
- **Toolbox gate:** rc **3**, with `spec-immutability`, `decisions-log`, `lint`, `l0`,
|
||||
`render-golden`, `l1`, `approval-policy` all **pass**. The rc 3 is the two usual environmental
|
||||
skips — `gitea-settings` (no local `SETTINGS_API_TOKEN`) and `conformance` L2/L3
|
||||
(`CLUSTER_AVAILABLE` unset). **Not green; reported as rc 3.**
|
||||
|
||||
## What I did NOT verify
|
||||
|
||||
- **Nothing here has been reconciled.** Flux has not applied any of it, so the password has never
|
||||
been set by this mechanism and the hook has never run during a real container start. Everything
|
||||
above is the rendered artifact tested out-of-band, plus the image's own source.
|
||||
- **THE CURRENT ADMIN PASSWORD WILL STOP WORKING, and that is the intent.** FTL re-hashes
|
||||
`webserver.api.password` on start, overwriting the existing `pwhash`. Whoever knows the present
|
||||
password (nobody, which is why this started) loses it; the declared one replaces it. I have not
|
||||
observed that replacement happen.
|
||||
- **Cold-start cost is unmeasured and could matter.** `postStart` must complete before the container
|
||||
is reported Running. On a fresh PVC the hook waits for `gravity.db` (bounded, up to 180s) and then
|
||||
runs a real `pihole -g` (72k domains, network downloads). `apps-wave-0` is `wait: true` and
|
||||
`apps-wave-1` `dependsOn` it, so a slow cold start **delays the app tier**. The steady-state path
|
||||
exits in about a second (scenario 1), so this is a rebuild/first-boot cost only — but it is not
|
||||
measured, and the stub in testing means `pihole -g`'s real duration here is unknown.
|
||||
- **Whether `pihole -g` succeeds from inside the hook's environment.** Its failure is caught and the
|
||||
existing `gravity.db` retained, so a failure degrades rather than breaks — untested live.
|
||||
- **Concurrency.** Nothing serialises the hook against FTL's own scheduled gravity update. SQLite
|
||||
handles the locking, and the hook's writes are three small statements, but no interleaving was
|
||||
exercised.
|
||||
- **MULTI-REPLICA — PHASE 2, NOT STARTED.** This branch only removes the blocker. The remaining work,
|
||||
and it is substantial: a `replicas`/placement field on the ServiceDefinition schema (currently
|
||||
`additionalProperties: false` with no such field), `render.py` emitting a StatefulSet with
|
||||
`volumeClaimTemplates` (the config PVC is **ReadWriteOnce**, so a 2-replica Deployment cannot
|
||||
work), pod anti-affinity, a PodDisruptionBudget (there is none today), re-rendered goldens, and
|
||||
`conformance/**` updates — the last making phase 2 sensitive-path as well. The PVC rename a
|
||||
StatefulSet forces will discard the existing `gravity.db`, which is only safe **because** of this
|
||||
branch, and should be done only after this one is confirmed working in-cluster.
|
||||
- **The real risk the operator named is NOT fixed by phase 2 either.** Replicas address *node*
|
||||
failure. LAN DNS still depends on the cluster as a whole — etcd quorum, Cilium, Longhorn, a bad
|
||||
reconcile. Quantified on 2026-10-04: with `tolerationSeconds=300` plus node-NotReady detection
|
||||
plus a Longhorn RWO force-detach, a failure of the node running the pod is roughly **6-10 minutes
|
||||
of no LAN DNS**; a failure of the node holding the `pihole-dns-*` L2 lease (currently a
|
||||
*different* node — lease on k3s-prod-2, pod on k3s-prod-1) is tens of seconds. An out-of-cluster
|
||||
secondary resolver carrying the same `address=` lines is the only option that removes the
|
||||
dependency, and it was considered and not chosen.
|
||||
- **Port 80 on the `pihole` Service is still published** and still closed only by policy. Not narrowed.
|
||||
Reference in new issue
Block a user