jakman85andClaude Opus 5
ff0533106b
1. PROD NODE DNS = 8.8.8.8 + 1.1.1.1 (operator, 2026-09-01). Two INDEPENDENT public
upstreams, so the single-upstream SPOF that broke containerd pulls and CoreDNS
forwarding on dev cannot recur. Prod deliberately differs from dev, which is
PiHole-first because dev needs the local jackson.lan zone.
SCOPE CORRECTED IN THE COMMENTS. Three resolvers answer *.home.keithjackson.dev and
only one is configured here:
pods -> CoreDNS -> platform/coredns/coredns-custom.yaml, whose `template` plugin
answers the WHOLE zone with ${GATEWAY_VIP}. Self-contained in-cluster;
PiHole is NOT in this path. Prod is already correct (.20).
LAN -> PiHole. Not a cluster concern.
nodes -> THIS FILE.
The one thing public node DNS breaks: the k3s API SERVER is a node process carrying
oidc-issuer-url=https://keycloak.home.keithjackson.dev/realms/homelab and reaches the
issuer through the NODE resolver, so prod would get the PUBLIC record instead of the
prod Gateway VIP — breaking kubectl OIDC SSO. Dev avoids this only because its node
resolver IS PiHole. Not blocking (prod has no cluster or Keycloak yet); options are
recorded in PROD-PROVISIONING.md §4 for the operator to pick.
2. SERVICE VIPs DECIDED: UNIFI_L4_VIP 192.168.2.21, CRAFTY_L4_VIP 192.168.2.22 — the
first two addresses after GATEWAY_VIP (.20), mirroring dev's .53/.54/.55 ordering
within the .20-.30 pool. S3_BACKUP_ENDPOINT stays TBD-PROD-* : no prod backup target
exists, and dev's versitygw is across the pfSense boundary on the dev host, so it is
not DR for prod.
3. PROD BECOME VAULT committed, ansible-vault encrypted, matching the hypervisor and
proxmox_hosts pattern. Verified working: `ansible k3s_prod -m command -a 'id -un'
--become` returns root. The L0 no-plaintext-secrets check asserts every ansible
vault.yml starts with $ANSIBLE_VAULT, so this stays honest by gate.
4. TOOLCHAIN GAP FIXED: the devcontainer image does NOT ship the Galaxy collections, so
EVERY ansible play died at parse time on community.general.modprobe. ansible.cfg now
sets collections_path = collections, and the collections are installed there from the
existing collections/requirements.yml (gitignored — pinned by requirements, fetched at
setup, vendored bytes are not source).
5. DRIVE HEALTH MEASURED and recorded in §2 with the raw attribute table so the numbers
are auditable. Verdict: usable, but not for production etcd/Longhorn state — 31% of
rated life consumed, 15 grown bad blocks, 187 erase failures, and 32 unexpected power
losses on a consumer drive with NO power-loss protection, in a single failure domain.
etcd is the worst possible tenant: it fsyncs constantly and unclean power loss without
PLP is exactly how a member corrupts.
6. NON-ECC RAM: RISK ACCEPTED by the operator, recorded as a CLOSED decision, not an open
item — do not re-litigate. Rationale: 3-node k3s homelab, not enterprise; corruption on
one node is outweighed by the other two and causes node ejection. That holds for the
dominant mode (etcd raft CRC + quorum; Longhorn 3 replicas + hard anti-affinity +
rebuild). Recorded residual: corruption introduced UPSTREAM of replication replicates
faithfully to all three replicas, so quorum sees no disagreement — low probability, and
the only mitigation is different hardware. Follow-up noted: Longhorn's
snapshotDataIntegrity is left at the chart default rather than set explicitly, and it
is the mechanism that operationalises the operator's own argument.
Serves: R-DNS-01, R-DNS-03, R-ENV-02, R-ENV-03, R-SEC-01. Task: T-7.1.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
26 lines
1.3 KiB
INI
26 lines
1.3 KiB
INI
[defaults]
|
|
inventory = inventory/hosts.ini
|
|
roles_path = roles
|
|
# Galaxy collections live in the repo (gitignored), installed from
|
|
# collections/requirements.yml. The devcontainer image does NOT ship them, so without
|
|
# this every play dies at parse time on community.general.modprobe.
|
|
collections_path = collections
|
|
host_key_checking = False
|
|
retry_files_enabled = False
|
|
# SSH key the k3s nodes authorize for the `k3s` user (see inventory ansible_user).
|
|
# Set here so node config plays (e.g. the rolling k3s config.yaml + Restart handler) connect
|
|
# without a per-run --private-key. Path is on the operator workstation / dev container ~/.ssh.
|
|
private_key_file = ~/.ssh/home_network
|
|
# ansible-core 2.20 removed community.general.yaml; use the builtin default with YAML result format.
|
|
stdout_callback = default
|
|
result_format = yaml
|
|
interpreter_python = auto_silent
|
|
# Vault passphrase file — kept OUTSIDE the repo (never committed). Decrypts the vaulted
|
|
# group_vars (e.g. the host become password) so plays run unattended. Create it once:
|
|
# install -m600 /dev/stdin ~/.config/home-k3s/ansible-vault-pass <<<'YOUR_VAULT_PASSPHRASE'
|
|
vault_password_file = ~/.config/home-k3s/ansible-vault-pass
|
|
|
|
[ssh_connection]
|
|
pipelining = True
|
|
ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=accept-new
|