feat(backup): MinIO S3 backup target on Proxmox host for Velero + CNPG
Stand up a single-node/single-drive MinIO S3 endpoint on the Proxmox host as the off-cluster backup target for Velero (R-STATE-03) and CloudNativePG (R-IDP-05). Ansible (author + executed): - roles/minio_backup_target + playbooks/backup-minio.yml — idempotent, additive only: ZFS dataset Storage/k3s-backups (zstd, quota=100G, /mnt/Storage/k3s-backups, no sharenfs); minio system user; pinned+checksummed MinIO/mc binaries (fetched on the controller since the host has no WAN, copied to host); hardened systemd unit bound to the host's cluster-reachable LAN IP (auto-discovered) on :9000; velero + cnpg buckets; a non-root access key scoped by IAM policy to only those two buckets. - inventory: new [proxmox_hosts] group ([email protected].2.102); overrides the cluster-wide ansible_user. Does not disturb existing k3s node entries. - group_vars/proxmox_hosts/vault.yml (ansible-vault): root + scoped MinIO creds. GitOps hand-off: - platform/backup/minio-credentials.sops.yaml (+ .example) — SOPS/age-encrypted endpoint + scoped key/secret for the Velero/CNPG wiring (WS-STORAGE/WS-SEC). Not yet wired into a kustomization (left for the consuming agents). Verified live: dataset props correct; minio active+enabled bound to 192.168.1.9:9000; S3 endpoint http://192.168.1.9:9000/minio/health/ready returns 200 from an in-cluster pod; buckets velero+cnpg present; scoped key created+policy attached; re-run is idempotent. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jakman85andClaude Opus 4.8 committed
1 parent
748e9ca55d
commit
ac32f01f7b
13 files changed
+609
No files matched your search
@@ -0,0 +1,29 @@
|
||||
# group_vars for the `proxmox_hosts` group (the Proxmox hypervisor host)
|
||||
|
||||
Target of `playbooks/backup-minio.yml` (role `minio_backup_target`). The role needs the MinIO
|
||||
credentials as **ansible-vault-encrypted** vars here — encrypted, so safe to commit; the passphrase
|
||||
that decrypts them stays out of the repo (`ansible.cfg` → `vault_password_file`).
|
||||
|
||||
## `vault.yml` (committed, encrypted)
|
||||
|
||||
```bash
|
||||
cd infra/ansible
|
||||
ansible-vault create inventory/group_vars/proxmox_hosts/vault.yml
|
||||
# contents:
|
||||
# minio_backup_target_root_user: "…" # MinIO root (host admin only)
|
||||
# minio_backup_target_root_password: "…"
|
||||
# minio_backup_target_access_key: "k3s-backup" # scoped key handed to Velero + CNPG
|
||||
# minio_backup_target_secret_key: "…"
|
||||
```
|
||||
|
||||
The **scoped** access key/secret is mirrored (SOPS-encrypted) for the GitOps side at
|
||||
`platform/backup/minio-credentials.sops.yaml` — the two MUST stay in sync (same key/secret), since
|
||||
the cluster authenticates to this MinIO with them. Rotating means updating BOTH and re-running the
|
||||
playbook (recreates the MinIO user) + reconciling Flux.
|
||||
|
||||
## Notes
|
||||
- This is the **Ansible-bootstrap-layer** secret (host credential), distinct from the platform's
|
||||
in-cluster **SOPS + age** secret (ADR-0007) — different layer, different tool, same values for the
|
||||
scoped key.
|
||||
- `vault.yml` is a vault blob (not parseable YAML) — excluded from the yamllint gate like the other
|
||||
`**/vault.yml`.
|
||||
@@ -0,0 +1,29 @@
|
||||
$ANSIBLE_VAULT;1.1;AES256
|
||||
30633837396565613861623931383238383836356238326436633738613935326461346138373432
|
||||
6234643535303037313337363463633765326239376238380a333331333136396239313733366163
|
||||
33353036366461393530633139636430313736366338313331353163616135303930323038636134
|
||||
6366353362313537330a373863623861663537353539376664306338643539346139396130396632
|
||||
30336231363731616334383863326638636238323565316364303530396636373163393765376438
|
||||
63373263623063656662353434623339316238663535386465653164326236616265616638656639
|
||||
30613466656439396534316339646139343334656165323965323965313361633933633038376663
|
||||
31653539653763623132313533396333643964373032313333613432383433626338376431323731
|
||||
63316163363361303439313266356361613737333066643931656564633038306236363439666435
|
||||
39343536656336343834663464356566373564636531343534343365376332343832343235663339
|
||||
66383561306533323938613738376632626263306337346532643063373166333838333033346365
|
||||
32383136363636643163613334366631653238613166303962656463363663643830343531363533
|
||||
39333630323765643963396562363537396564666462306163336366643739313035636633646661
|
||||
61653637376535393034383631303563363539633265333161363334616239323866353162636632
|
||||
65663830323465393138306361333662383934346438663562316235353035633064643861353334
|
||||
63646435653639316439366139316539653833656561646439323766633139306565383565656261
|
||||
61323230306635343236646438336165643737626132613962633930653461393131303731623438
|
||||
63313265653166643334396461353231343235386434363438623034323762653433303932326630
|
||||
62633464316139353433366338313837646162323231363135353964623035366132643332366662
|
||||
38393465326363616139376465383738626636393639363338653437323832613439313138353033
|
||||
39393765613735666331636230363463306462373238623138316263326339366330333332643833
|
||||
33363764303939326662343665643963333463356638613231363834386132666339393132356538
|
||||
38303362643337366430303165633432666632343536383366653836663135383336316232393936
|
||||
66626632386537333739313432313564633866383132353862626433626366633562346464613066
|
||||
35376332303132326438313132303264313763376163396137356132306131373335346266646532
|
||||
31666230616461386339366461653661323138353662373866316639626134326339643231613337
|
||||
30343361643166353162363866653061313966663435373332353562656164663962326562333038
|
||||
35363834656361393930
|
||||
@@ -37,3 +37,11 @@ localhost ansible_connection=local
|
||||
# with their ansible_host under k3s_secondary; this group only tags the egress role target.
|
||||
[vpn_egress_gateway]
|
||||
k3s-dev-3
|
||||
|
||||
# The Proxmox HYPERVISOR host itself (NOT a k3s node) — target of playbooks/backup-minio.yml,
|
||||
# which stands up the off-cluster MinIO S3 backup target for Velero + CNPG. Reached DIRECTLY
|
||||
# over SSH as root (its own 192.168.2.102:22, the pfSense NAT to the host's 192.168.1.9), so
|
||||
# it overrides the cluster-wide `ansible_user: k3s` from group_vars/all.yml. Distinct from the
|
||||
# `hypervisor` group above (that is `localhost` for host-prereqs.yml, a different machine).
|
||||
[proxmox_hosts]
|
||||
proxmox ansible_host=192.168.2.102 ansible_port=22 ansible_user=root ansible_ssh_private_key_file=~/.ssh/home_network
|
||||
@@ -0,0 +1,20 @@
|
||||
---
|
||||
# Stand up the off-cluster S3 backup target (MinIO) on the Proxmox host — the store Velero
|
||||
# (R-STATE-03) and CloudNativePG (R-IDP-05) back up into. Host infra, additive + idempotent:
|
||||
# ONE new ZFS dataset (Storage/k3s-backups), the minio user/service/binaries, and the
|
||||
# velero + cnpg buckets with a scoped non-root access key. It never touches existing
|
||||
# datasets, sharenfs, pvesm storage, VMs, or firewall.
|
||||
#
|
||||
# Credentials come from Ansible vault (inventory/group_vars/proxmox_hosts/vault.yml):
|
||||
# minio_backup_target_root_user/_root_password, minio_backup_target_access_key/_secret_key
|
||||
# The scoped key is ALSO staged for the GitOps side (SOPS) at
|
||||
# platform/backup/minio-credentials.sops.yaml
|
||||
#
|
||||
# Prereqs: ansible-galaxy collection install -r collections/requirements.yml (community.general.zfs)
|
||||
# Run: ansible-playbook playbooks/backup-minio.yml
|
||||
- name: Provision the MinIO S3 backup target on the Proxmox host
|
||||
hosts: proxmox_hosts
|
||||
become: true
|
||||
gather_facts: true # need ansible_facts.all_ipv4_addresses to discover the LAN bind addr
|
||||
roles:
|
||||
- role: minio_backup_target
|
||||
@@ -0,0 +1,46 @@
|
||||
# roles/minio_backup_target
|
||||
|
||||
Stands up a **single-node, single-drive MinIO** S3 endpoint on the **Proxmox host** to serve as the
|
||||
off-cluster **backup target** for the k3s platform's **Velero** (volume/resource DR, R-STATE-03) and
|
||||
**CloudNativePG** (Keycloak DB Barman object store, R-IDP-05). A backup target — not primary storage —
|
||||
so a single drive is intentional.
|
||||
|
||||
## What it does (all additive + idempotent)
|
||||
|
||||
1. **ZFS dataset** `Storage/k3s-backups` — `compression=zstd`, `quota=100G`,
|
||||
`mountpoint=/mnt/Storage/k3s-backups`, **no sharenfs**. Uses `community.general.zfs` so re-runs are
|
||||
no-ops and existing datasets are never touched.
|
||||
2. **`minio` system user/group** + a data dir (`…/k3s-backups/data`) it owns.
|
||||
3. **MinIO + `mc` binaries** (pinned, checksummed). The Proxmox host has **no outbound internet**, so
|
||||
the binaries are downloaded on the **Ansible controller** and copied to the host.
|
||||
4. **systemd unit** `minio.service` (Type=notify, hardened), bound to the host's
|
||||
**cluster-reachable `192.168.1.x` address** on port **9000** (console 9001) — auto-discovered from
|
||||
facts, assert-guarded.
|
||||
5. **Buckets** `velero` + `cnpg`, a **scoped non-root access key** limited by an IAM policy to just
|
||||
those two buckets, plus the retained root creds for host-side admin.
|
||||
|
||||
## Credentials
|
||||
|
||||
Supplied via the vaulted `inventory/group_vars/proxmox_hosts/vault.yml` (ansible-vault). Nothing is
|
||||
defaulted — a missing vault fails the play loudly:
|
||||
|
||||
| var | purpose |
|
||||
|---|---|
|
||||
| `minio_backup_target_root_user` / `_root_password` | MinIO root (host admin only) |
|
||||
| `minio_backup_target_access_key` / `_secret_key` | scoped key handed to Velero + CNPG |
|
||||
|
||||
The **same scoped key** is staged for the GitOps side, SOPS-encrypted, at
|
||||
`platform/backup/minio-credentials.sops.yaml`.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
cd infra/ansible
|
||||
ansible-galaxy collection install -r collections/requirements.yml # community.general (zfs)
|
||||
ansible-playbook playbooks/backup-minio.yml
|
||||
```
|
||||
|
||||
## Endpoint
|
||||
|
||||
`http://<host-192.168.1.x>:9000` — the S3 URL the cluster uses (path-style). Health:
|
||||
`GET /minio/health/ready` ⇒ 200. Verified reachable from an in-cluster pod at bring-up.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
# roles/minio_backup_target — a single-node, single-drive MinIO S3 target on the Proxmox
|
||||
# host, used ONLY as an off-cluster backup bucket store for Velero + CloudNativePG (ADR-0003,
|
||||
# R-STATE-03 / R-IDP-05). This is a BACKUP TARGET, not primary storage — single drive is fine.
|
||||
#
|
||||
# SAFETY: every task in this role is additive and idempotent. It creates ONE new ZFS dataset,
|
||||
# ONE system user, ONE systemd service and two buckets. It never touches existing datasets,
|
||||
# sharenfs, pvesm storage, VMs or firewall. See tasks/main.yml for the guards.
|
||||
|
||||
# ── MinIO / mc release pins (Renovate-friendly; both are AGPL community builds) ───────────────
|
||||
# The Proxmox host has NO outbound internet (PiHole-only DNS, no default route to WAN), so the
|
||||
# binaries are fetched on the ANSIBLE CONTROLLER and copied to the host (tasks/main.yml).
|
||||
minio_backup_target_minio_version: "RELEASE.2025-09-07T16-13-09Z"
|
||||
minio_backup_target_minio_sha256: "7c5bd8512c6e966455b1d198209358b2d191c77a83ab377c4073281065fb855f"
|
||||
minio_backup_target_minio_url: "https://dl.min.io/server/minio/release/linux-amd64/archive/minio.{{ minio_backup_target_minio_version }}"
|
||||
|
||||
minio_backup_target_mc_version: "RELEASE.2025-08-13T08-35-41Z"
|
||||
minio_backup_target_mc_sha256: "01f866e9c5f9b87c2b09116fa5d7c06695b106242d829a8bb32990c00312e891"
|
||||
minio_backup_target_mc_url: "https://dl.min.io/client/mc/release/linux-amd64/archive/mc.{{ minio_backup_target_mc_version }}"
|
||||
|
||||
# Controller-side cache the binaries land in before being copied to the host (delegate_to localhost).
|
||||
minio_backup_target_controller_cache: "/tmp/minio-backup-target-cache"
|
||||
|
||||
# ── ZFS dataset (backup drive) ────────────────────────────────────────────────────────────────
|
||||
# Host convention is Storage/<purpose> mounted at /mnt/Storage/<purpose>. NO sharenfs.
|
||||
minio_backup_target_zfs_dataset: "Storage/k3s-backups"
|
||||
minio_backup_target_mountpoint: "/mnt/Storage/k3s-backups"
|
||||
minio_backup_target_zfs_compression: "zstd"
|
||||
minio_backup_target_zfs_quota: "100G"
|
||||
# MinIO writes its drive (incl. .minio.sys) into a subdir of the dataset, keeping the dataset
|
||||
# root clean and owned by root.
|
||||
minio_backup_target_data_dir: "{{ minio_backup_target_mountpoint }}/data"
|
||||
|
||||
# ── Service identity ────────────────────────────────────────────────────────────────────────
|
||||
minio_backup_target_user: "minio"
|
||||
minio_backup_target_group: "minio"
|
||||
minio_backup_target_minio_bin: "/usr/local/bin/minio"
|
||||
minio_backup_target_mc_bin: "/usr/local/bin/mc"
|
||||
minio_backup_target_env_file: "/etc/default/minio"
|
||||
|
||||
# ── Network binding ──────────────────────────────────────────────────────────────────────────
|
||||
# MinIO's S3 API must bind the host address the k3s pods can reach (the 192.168.1.0/24 LAN;
|
||||
# nodes are .50/.51/.52). tasks/main.yml auto-discovers the host's address in this CIDR prefix
|
||||
# and asserts it exists; override minio_backup_target_bind_address to pin it explicitly.
|
||||
minio_backup_target_lan_prefix: "192.168.1."
|
||||
minio_backup_target_bind_address: "" # "" ⇒ auto-discover from facts (see tasks)
|
||||
minio_backup_target_api_port: 9000
|
||||
minio_backup_target_console_port: 9001
|
||||
|
||||
# ── Buckets + scoped (non-root) access key ────────────────────────────────────────────────────
|
||||
# Velero and CNPG each get a bucket; a single non-root user scoped to ONLY these two buckets is
|
||||
# handed to the cluster (SOPS). Root creds stay on the host for admin.
|
||||
minio_backup_target_buckets:
|
||||
- velero
|
||||
- cnpg
|
||||
minio_backup_target_policy_name: "k3s-backup-rw"
|
||||
|
||||
# Credentials come from the vaulted group_vars (inventory/group_vars/proxmox_hosts/vault.yml):
|
||||
# minio_backup_target_root_user, minio_backup_target_root_password,
|
||||
# minio_backup_target_access_key, minio_backup_target_secret_key
|
||||
# They are intentionally NOT defaulted here so a missing vault fails loudly rather than
|
||||
# silently provisioning guessable creds.
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
- name: Restart minio
|
||||
ansible.builtin.systemd:
|
||||
name: minio
|
||||
state: restarted
|
||||
daemon_reload: true
|
||||
@@ -0,0 +1,263 @@
|
||||
---
|
||||
# ── 0. Preconditions / safety asserts ─────────────────────────────────────────────────────────
|
||||
- name: Assert MinIO credentials are supplied (from vaulted group_vars)
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- minio_backup_target_root_user | default('') | length > 0
|
||||
- minio_backup_target_root_password | default('') | length >= 8
|
||||
- minio_backup_target_access_key | default('') | length > 0
|
||||
- minio_backup_target_secret_key | default('') | length >= 8
|
||||
fail_msg: >-
|
||||
MinIO credentials are missing. Provide minio_backup_target_root_user/_password and
|
||||
minio_backup_target_access_key/_secret_key via inventory/group_vars/proxmox_hosts/vault.yml.
|
||||
quiet: true
|
||||
|
||||
# Determine the host's 192.168.1.0/24 address (the S3 endpoint the cluster reaches). If the
|
||||
# operator pinned minio_backup_target_bind_address, honour it; else auto-discover and assert.
|
||||
- name: Discover the host LAN (cluster-reachable) IPv4 address
|
||||
ansible.builtin.set_fact:
|
||||
minio_backup_target_bind_address: >-
|
||||
{{ minio_backup_target_bind_address
|
||||
if (minio_backup_target_bind_address | length > 0)
|
||||
else (ansible_facts.all_ipv4_addresses
|
||||
| select('match', '^' ~ (minio_backup_target_lan_prefix | regex_escape))
|
||||
| list | first | default('')) }}
|
||||
|
||||
- name: Assert a cluster-reachable LAN address was found
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- minio_backup_target_bind_address | length > 0
|
||||
fail_msg: >-
|
||||
Could not find a {{ minio_backup_target_lan_prefix }}x address on this host
|
||||
(interfaces: {{ ansible_facts.all_ipv4_addresses | default([]) | join(', ') }}).
|
||||
The cluster egresses onto that LAN — refusing to bind MinIO to an unreachable address.
|
||||
Set minio_backup_target_bind_address explicitly if discovery is wrong.
|
||||
quiet: true
|
||||
|
||||
# ── 1. ZFS dataset (backup drive) — idempotent, never touches other datasets ───────────────────
|
||||
- name: Create/ensure the k3s-backups ZFS dataset (compression, quota, mountpoint; NO sharenfs)
|
||||
community.general.zfs:
|
||||
name: "{{ minio_backup_target_zfs_dataset }}"
|
||||
state: present
|
||||
extra_zfs_properties:
|
||||
compression: "{{ minio_backup_target_zfs_compression }}"
|
||||
quota: "{{ minio_backup_target_zfs_quota }}"
|
||||
mountpoint: "{{ minio_backup_target_mountpoint }}"
|
||||
|
||||
# ── 2. Service identity ───────────────────────────────────────────────────────────────────────
|
||||
- name: Create the minio system group
|
||||
ansible.builtin.group:
|
||||
name: "{{ minio_backup_target_group }}"
|
||||
system: true
|
||||
|
||||
- name: Create the minio system user (no login shell, home = data dir)
|
||||
ansible.builtin.user:
|
||||
name: "{{ minio_backup_target_user }}"
|
||||
group: "{{ minio_backup_target_group }}"
|
||||
system: true
|
||||
shell: /usr/sbin/nologin
|
||||
home: "{{ minio_backup_target_data_dir }}"
|
||||
create_home: false
|
||||
|
||||
- name: Ensure the MinIO data directory exists (owned by minio, inside the dataset)
|
||||
ansible.builtin.file:
|
||||
path: "{{ minio_backup_target_data_dir }}"
|
||||
state: directory
|
||||
owner: "{{ minio_backup_target_user }}"
|
||||
group: "{{ minio_backup_target_group }}"
|
||||
mode: "0750"
|
||||
|
||||
# ── 3. Binaries — fetched on the CONTROLLER (host has no internet), copied to the host ──────────
|
||||
- name: Ensure controller-side download cache exists
|
||||
ansible.builtin.file:
|
||||
path: "{{ minio_backup_target_controller_cache }}"
|
||||
state: directory
|
||||
mode: "0700"
|
||||
delegate_to: localhost
|
||||
become: false
|
||||
run_once: true
|
||||
|
||||
- name: Fetch MinIO server binary onto the controller (checksummed)
|
||||
ansible.builtin.get_url:
|
||||
url: "{{ minio_backup_target_minio_url }}"
|
||||
dest: "{{ minio_backup_target_controller_cache }}/minio.{{ minio_backup_target_minio_version }}"
|
||||
checksum: "sha256:{{ minio_backup_target_minio_sha256 }}"
|
||||
mode: "0755"
|
||||
delegate_to: localhost
|
||||
become: false
|
||||
run_once: true
|
||||
|
||||
- name: Fetch mc client binary onto the controller (checksummed)
|
||||
ansible.builtin.get_url:
|
||||
url: "{{ minio_backup_target_mc_url }}"
|
||||
dest: "{{ minio_backup_target_controller_cache }}/mc.{{ minio_backup_target_mc_version }}"
|
||||
checksum: "sha256:{{ minio_backup_target_mc_sha256 }}"
|
||||
mode: "0755"
|
||||
delegate_to: localhost
|
||||
become: false
|
||||
run_once: true
|
||||
|
||||
# Integrity is already guaranteed by the SHA256 verification get_url performed on the controller
|
||||
# above; copy transfers that verified artifact faithfully. (The copy module's own `checksum` arg is
|
||||
# SHA1, not SHA256 — deliberately not used here to avoid a false mismatch.)
|
||||
- name: Install MinIO server binary on the host
|
||||
ansible.builtin.copy:
|
||||
src: "{{ minio_backup_target_controller_cache }}/minio.{{ minio_backup_target_minio_version }}"
|
||||
dest: "{{ minio_backup_target_minio_bin }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
notify: Restart minio
|
||||
|
||||
- name: Install mc client binary on the host
|
||||
ansible.builtin.copy:
|
||||
src: "{{ minio_backup_target_controller_cache }}/mc.{{ minio_backup_target_mc_version }}"
|
||||
dest: "{{ minio_backup_target_mc_bin }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
|
||||
# ── 4. systemd service ────────────────────────────────────────────────────────────────────────
|
||||
- name: Write the MinIO environment file (root creds, bind address, data dir)
|
||||
ansible.builtin.template:
|
||||
src: minio.env.j2
|
||||
dest: "{{ minio_backup_target_env_file }}"
|
||||
owner: root
|
||||
group: "{{ minio_backup_target_group }}"
|
||||
mode: "0640"
|
||||
no_log: true
|
||||
notify: Restart minio
|
||||
|
||||
- name: Install the MinIO systemd unit
|
||||
ansible.builtin.template:
|
||||
src: minio.service.j2
|
||||
dest: /etc/systemd/system/minio.service
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
notify: Restart minio
|
||||
|
||||
- name: Enable + start MinIO (apply any pending unit/env changes now)
|
||||
ansible.builtin.systemd:
|
||||
name: minio
|
||||
enabled: true
|
||||
state: started
|
||||
daemon_reload: true
|
||||
|
||||
- name: Flush handlers so MinIO is (re)started before we configure buckets/users
|
||||
ansible.builtin.meta: flush_handlers
|
||||
|
||||
# ── 5. Readiness gate ─────────────────────────────────────────────────────────────────────────
|
||||
- name: Wait for the MinIO S3 health endpoint to report ready
|
||||
ansible.builtin.uri:
|
||||
url: "http://{{ minio_backup_target_bind_address }}:{{ minio_backup_target_api_port }}/minio/health/ready"
|
||||
status_code: 200
|
||||
register: minio_backup_target_health
|
||||
retries: 30
|
||||
delay: 2
|
||||
until: minio_backup_target_health.status == 200
|
||||
|
||||
# ── 6. Buckets + scoped non-root user (via mc, idempotent) ─────────────────────────────────────
|
||||
- name: Configure the local mc alias against the running MinIO
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- "{{ minio_backup_target_mc_bin }}"
|
||||
- alias
|
||||
- set
|
||||
- k3sbackup
|
||||
- "http://{{ minio_backup_target_bind_address }}:{{ minio_backup_target_api_port }}"
|
||||
- "{{ minio_backup_target_root_user }}"
|
||||
- "{{ minio_backup_target_root_password }}"
|
||||
no_log: true
|
||||
changed_when: false
|
||||
|
||||
- name: Create the Velero + CNPG buckets (idempotent)
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- "{{ minio_backup_target_mc_bin }}"
|
||||
- mb
|
||||
- --ignore-existing
|
||||
- "k3sbackup/{{ item }}"
|
||||
loop: "{{ minio_backup_target_buckets }}"
|
||||
register: minio_backup_target_mb
|
||||
changed_when: "'created successfully' in (minio_backup_target_mb.stdout | default(''))"
|
||||
|
||||
- name: Render the scoped bucket RW policy to the host
|
||||
ansible.builtin.template:
|
||||
src: bucket-policy.json.j2
|
||||
dest: "/etc/minio-{{ minio_backup_target_policy_name }}.json"
|
||||
owner: root
|
||||
group: "{{ minio_backup_target_group }}"
|
||||
mode: "0640"
|
||||
|
||||
- name: Check whether the scoped policy already exists
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- "{{ minio_backup_target_mc_bin }}"
|
||||
- admin
|
||||
- policy
|
||||
- info
|
||||
- k3sbackup
|
||||
- "{{ minio_backup_target_policy_name }}"
|
||||
register: minio_backup_target_policy_info
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Create the scoped bucket RW policy (velero + cnpg only)
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- "{{ minio_backup_target_mc_bin }}"
|
||||
- admin
|
||||
- policy
|
||||
- create
|
||||
- k3sbackup
|
||||
- "{{ minio_backup_target_policy_name }}"
|
||||
- "/etc/minio-{{ minio_backup_target_policy_name }}.json"
|
||||
when: minio_backup_target_policy_info.rc != 0
|
||||
changed_when: true
|
||||
|
||||
- name: Check whether the scoped access key already exists
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- "{{ minio_backup_target_mc_bin }}"
|
||||
- admin
|
||||
- user
|
||||
- info
|
||||
- k3sbackup
|
||||
- "{{ minio_backup_target_access_key }}"
|
||||
register: minio_backup_target_user_info
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
|
||||
- name: Create the scoped non-root access key
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- "{{ minio_backup_target_mc_bin }}"
|
||||
- admin
|
||||
- user
|
||||
- add
|
||||
- k3sbackup
|
||||
- "{{ minio_backup_target_access_key }}"
|
||||
- "{{ minio_backup_target_secret_key }}"
|
||||
when: minio_backup_target_user_info.rc != 0
|
||||
no_log: true
|
||||
changed_when: true
|
||||
|
||||
- name: Attach the scoped policy to the access key (idempotent)
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- "{{ minio_backup_target_mc_bin }}"
|
||||
- admin
|
||||
- policy
|
||||
- attach
|
||||
- k3sbackup
|
||||
- "{{ minio_backup_target_policy_name }}"
|
||||
- --user
|
||||
- "{{ minio_backup_target_access_key }}"
|
||||
register: minio_backup_target_attach
|
||||
changed_when:
|
||||
- minio_backup_target_attach.rc == 0
|
||||
- "'already in effect' not in (minio_backup_target_attach.stderr | default(''))"
|
||||
failed_when:
|
||||
- minio_backup_target_attach.rc != 0
|
||||
- "'already in effect' not in (minio_backup_target_attach.stderr | default(''))"
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": ["s3:*"],
|
||||
"Resource": [
|
||||
{% for b in minio_backup_target_buckets %}
|
||||
"arn:aws:s3:::{{ b }}",
|
||||
"arn:aws:s3:::{{ b }}/*"{{ "," if not loop.last else "" }}
|
||||
{% endfor %}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
# {{ ansible_managed }}
|
||||
# MinIO server environment (single-node, single-drive backup target for k3s Velero + CNPG).
|
||||
# Managed by roles/minio_backup_target — do not edit by hand.
|
||||
MINIO_ROOT_USER={{ minio_backup_target_root_user }}
|
||||
MINIO_ROOT_PASSWORD={{ minio_backup_target_root_password }}
|
||||
|
||||
# The S3 drive lives on the ZFS dataset {{ minio_backup_target_zfs_dataset }}.
|
||||
MINIO_VOLUMES="{{ minio_backup_target_data_dir }}"
|
||||
|
||||
# Bind the S3 API (and console) to the cluster-reachable LAN address only.
|
||||
MINIO_OPTS="--address {{ minio_backup_target_bind_address }}:{{ minio_backup_target_api_port }} --console-address {{ minio_backup_target_bind_address }}:{{ minio_backup_target_console_port }}"
|
||||
@@ -0,0 +1,28 @@
|
||||
{# {{ ansible_managed }} — MinIO systemd unit (roles/minio_backup_target) #}
|
||||
[Unit]
|
||||
Description=MinIO S3 backup target (k3s Velero + CNPG)
|
||||
Documentation=https://min.io/docs/minio/linux/index.html
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
AssertFileIsExecutable={{ minio_backup_target_minio_bin }}
|
||||
|
||||
[Service]
|
||||
Type=notify
|
||||
User={{ minio_backup_target_user }}
|
||||
Group={{ minio_backup_target_group }}
|
||||
EnvironmentFile={{ minio_backup_target_env_file }}
|
||||
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo 'MINIO_VOLUMES unset in {{ minio_backup_target_env_file }}'; exit 1; fi"
|
||||
ExecStart={{ minio_backup_target_minio_bin }} server $MINIO_OPTS $MINIO_VOLUMES
|
||||
Restart=always
|
||||
RestartSec=5s
|
||||
LimitNOFILE=65536
|
||||
TimeoutStopSec=30s
|
||||
# Hardening (backup target holds no other roles on this host).
|
||||
NoNewPrivileges=true
|
||||
ProtectSystem=full
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
ReadWritePaths={{ minio_backup_target_mountpoint }}
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,41 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: minio-backup-credentials
|
||||
namespace: backup
|
||||
type: Opaque
|
||||
stringData:
|
||||
#ENC[AES256_GCM,data:V1G4ViUPhrnkiRLFNZyYbSaEPFVLsZ74gB4zo1Y7AM2QP7R5lWVEvDO9VgLVatRZwt7cbeObMAYXeI873tZ9hOi2JNSvYuuXrscC3q8nMuJGswvIMvkM,iv:sGb4E/j5mhtkwY05YP/U7lCa9UHV/35dGCziymGUJHo=,tag:S5KnPKRaxJuGCMwPZkIFBw==,type:comment]
|
||||
S3_ENDPOINT: ENC[AES256_GCM,data:UwzKnjDVv3HoEeAPxlzkhUifL6hmebM=,iv:IWmiCYrvuIBez2i3qyDs6Yu4mQ0YuNYv46Rn3+ksm/k=,tag:dlefmBsY3TuLZPAPmaWs/g==,type:str]
|
||||
S3_REGION: ENC[AES256_GCM,data:GvGIw6qxx9M4,iv:uArcS3uwHKwjif00ICIc0Og0OF98OL10ArhT6teJzh4=,tag:2tn4XykslMtwYk7C4m8ndw==,type:str]
|
||||
S3_FORCE_PATH_STYLE: ENC[AES256_GCM,data:5cEjjA==,iv:k5R7iszw/kxYLSH8aqIAHmq8ePdL7xYYPQcutHloN7s=,tag:DJPE0500UP5ECYuBJM9nBg==,type:str]
|
||||
VELERO_BUCKET: ENC[AES256_GCM,data:Q1Oo4Amz,iv:XnAT/Nn8C9BmnE5e0d1LI0I0BVSUL7C+mI+s31FrMYI=,tag:oWDHC2jjKYpKNXA98wYqpQ==,type:str]
|
||||
CNPG_BUCKET: ENC[AES256_GCM,data:P3BUXg==,iv:BPuFgI9iL9lWZhQZ0k3skrBwyp1UDLyz0aIyq6/V7nI=,tag:R4n30UxwcCY1k+X/bu7pxw==,type:str]
|
||||
#ENC[AES256_GCM,data:A6h/rEzdn5Sp1BLDnxoHrYdEwiB9Eb/5NAxCMZRXvoVcnTfOvLHVA0ntno+9GwMueIhwN9WgG7fnCHhG7A==,iv:1+VfHF59PppwpLiYW4gUTwgGcvXMEzAnvowVvE/awFU=,tag:skLHPchOW2PZzjqqaJ2TSQ==,type:comment]
|
||||
AWS_ACCESS_KEY_ID: ENC[AES256_GCM,data:wNnfGYBspJDyYw==,iv:d9fK8E5ujPu9vXV1yRR5AgO3+j69EWfbWEoMt7Lq06w=,tag:0uuRSadwtodLol2Sw2ih9Q==,type:str]
|
||||
AWS_SECRET_ACCESS_KEY: ENC[AES256_GCM,data:IywofASX3m8sznW3StMf/SvCPBcvxCUlT4cIQvu4gF9RglHlFCujwi8amnm7ASfc,iv:JnyMBQQ7oLg8TAVx7RveZHYDo6a/bl3t1IEPRxZufI8=,tag:5rUUqd6ZByuLIlZxQV1dPQ==,type:str]
|
||||
#ENC[AES256_GCM,data:J5fQZE/UbhqpWo5OXzvGxCNTb5f6r+sh55wUHf3M0H4oRU2dyna7nJqFQzP1EQXfJTlxjZOpBkzZyppn,iv:bDGljEMmMiieCvfmVO8uhfw/O1rUslNvNAY6LfFFA4E=,tag:D9Gw78gtCigsWJnlmFzMdA==,type:comment]
|
||||
ACCESS_KEY_ID: ENC[AES256_GCM,data:TYn3AVIkjU4QhA==,iv:09W2JkJywD5aadOr+XkbZlR2sSz3glT+yknI4Z46C9o=,tag:QGwIp3seKuemzi4jysDRcg==,type:str]
|
||||
ACCESS_SECRET_KEY: ENC[AES256_GCM,data:lZ9fAqwPGwiKbTJYyvNGMOS9L37IB3/GcfpuZH3G2s0of3YII6vyPlAKZeGI6i2P,iv:Y0p8EDzZS3uUZ+BBHjjBgDmkyphwZ43X7VfZJboGS1s=,tag:rQ0EyRBEPNG41PGrqHKkhw==,type:str]
|
||||
#ENC[AES256_GCM,data:EbjHG+jG6OnhZzWUAamIomBYYa5jwogBWlAgOPyIdFivuNZGDSbkyZ0dsUNJJ8yhl7lkIjvi1VvGA/B2QXdCw0pP4xxCs84uLSgpHDWB9v4G5cjF+kThANvaMomKoQ==,iv:1JpFJPSTXVxKVkLBkaYZAYzRHCmqSAgNYGmcW4ajZxo=,tag:9vWPXJ3COQC4m2AHCWPOlg==,type:comment]
|
||||
cloud: ENC[AES256_GCM,data:RmDGXXK5ct97WO18YAYIUF3zPE7nb1zkuJV8pBZPkNEHV6y1Iup/RXAMg+l1rXcI7xdPSiMfSJ6FMudfxOzCzAJPRGxoHMWhzO5N9H+O30KOhHwYpeud2mL4xNZPzOvQgpszqJQzk82D5tnjGeE=,iv:wz16XKNzAWrOZv+Rx68MArqRJHZmRVX/613w7/3WXH4=,tag:5cxblz07Mepy9xOZ3pko2w==,type:str]
|
||||
sops:
|
||||
kms: []
|
||||
gcp_kms: []
|
||||
azure_kv: []
|
||||
hc_vault: []
|
||||
age:
|
||||
- recipient: age175l8mdpdfu7hq0aml3exs3gykarwjmuvr23sq56s3wjejnyk44wsp4mfzg
|
||||
enc: |
|
||||
-----BEGIN AGE ENCRYPTED FILE-----
|
||||
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA5Z1hkZTlrWmpsdzBuODlM
|
||||
NXQvTXhmb00rMWNqY1l4Ly8yVjVrNXFSMXdRCmxMNXlBTEJyVncrcHVrWXFoZHBI
|
||||
L3JLM3lLVHBJNFVHY3k0MVBjYk5USTgKLS0tIFJ4WWxFL0Flc2J5V3ZSU1JhSVE1
|
||||
TDFyWmhqQUdCOTUyZGVnRDQ4d2lSRnMKPBexDlO9fZK6K6Rpcs2RzYE4drYtivwk
|
||||
QsnSigjEPY35ixsPTMHTdTP0Almxc4U6lhT4gsIEqk3hqmJ3fcmzzA==
|
||||
-----END AGE ENCRYPTED FILE-----
|
||||
lastmodified: "2026-08-29T03:09:53Z"
|
||||
mac: ENC[AES256_GCM,data:y1Uxrj9yAuZCryfCviU0pPplcQ0ccxcJSuNov4CvZKwnIimHVIsr7sU1mw3ZqHV/arQ2SjXkfDHxAEhvL6WSAjEjnDNaCMm++mPrfKXoGZhDU3gjaeyI7W6Iexo/bfc/oR6hALRjp6g7nTK6Yqdi6UMRoZZ+1YGSkFkRJjygLF4=,iv:9scdNtJQk6TkB/SN2yeFdhmiVCtGd2qRsD3N+wv0L5o=,tag:dYFy6oKEUIr7BLstFrhsEA==,type:str]
|
||||
pgp: []
|
||||
encrypted_regex: ^(data|stringData)$
|
||||
version: 3.9.4
|
||||
@@ -0,0 +1,51 @@
|
||||
# =============================================================================
|
||||
# TEMPLATE — NOT APPLIED, NOT ENCRYPTED (.yaml.example is ignored by kustomize and never
|
||||
# auto-encrypted by SOPS). Reference scaffold + OPERATOR HAND-OFF.
|
||||
#
|
||||
# WHAT THIS IS: the scoped (non-root) MinIO S3 access key for the off-cluster backup target that
|
||||
# roles/minio_backup_target stands up on the Proxmox host (playbooks/backup-minio.yml). It is the
|
||||
# credential Velero (R-STATE-03) and CloudNativePG / Barman (R-IDP-05) use to write backups. The
|
||||
# SAME key/secret live (ansible-vault) in infra/ansible/inventory/group_vars/proxmox_hosts/vault.yml
|
||||
# as minio_backup_target_access_key/_secret_key — the two MUST stay in sync.
|
||||
#
|
||||
# ENDPOINT (not secret): http://192.168.1.9:9000 (path-style; region us-east-1 is a placeholder,
|
||||
# MinIO ignores it but S3 clients require one). Buckets: velero, cnpg.
|
||||
#
|
||||
# HAND-OFF STEPS (operator):
|
||||
# (a) Copy to: platform/backup/minio-credentials.sops.yaml
|
||||
# (b) Fill the access key + secret (== the vaulted scoped key on the Proxmox host).
|
||||
# (c) Encrypt: sops -e -i platform/backup/minio-credentials.sops.yaml
|
||||
# (d) Wire per consumer (WS-STORAGE / WS-SEC):
|
||||
# - Velero → credentials.existingSecret pointing at a Secret with the `cloud` INI key,
|
||||
# backupStorageLocation.config.s3Url=S3_ENDPOINT, s3ForcePathStyle="true",
|
||||
# bucket=velero (namespace: velero).
|
||||
# - CNPG → Barman objectStore endpointURL=S3_ENDPOINT, destinationPath=s3://cnpg,
|
||||
# s3Credentials.accessKeyId/secretAccessKey from ACCESS_KEY_ID/ACCESS_SECRET_KEY
|
||||
# (namespace: cnpg-system).
|
||||
# A Secret is namespace-scoped — replicate/rename this one into each consumer namespace as
|
||||
# that wiring requires.
|
||||
# =============================================================================
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: minio-backup-credentials
|
||||
namespace: backup
|
||||
type: Opaque
|
||||
stringData:
|
||||
# ── endpoint + buckets (not secret; single source of truth) ──
|
||||
S3_ENDPOINT: "http://192.168.1.9:9000"
|
||||
S3_REGION: "us-east-1"
|
||||
S3_FORCE_PATH_STYLE: "true"
|
||||
VELERO_BUCKET: "velero"
|
||||
CNPG_BUCKET: "cnpg"
|
||||
# ── scoped access key (velero + cnpg buckets only) ──
|
||||
AWS_ACCESS_KEY_ID: "REPLACE-ME"
|
||||
AWS_SECRET_ACCESS_KEY: "REPLACE-ME"
|
||||
# CNPG Barman object store consumes these discrete key names.
|
||||
ACCESS_KEY_ID: "REPLACE-ME"
|
||||
ACCESS_SECRET_KEY: "REPLACE-ME"
|
||||
# Velero's aws plugin reads an INI credentials file from the `cloud` key of its existingSecret.
|
||||
cloud: |
|
||||
[default]
|
||||
aws_access_key_id=REPLACE-ME
|
||||
aws_secret_access_key=REPLACE-ME
|
||||
Reference in new issue
Block a user