From 4297e5a71e399e77507c41fd5d7070c14557c7d6 Mon Sep 17 00:00:00 2001 From: Amritanshu Date: Fri, 16 Oct 2026 06:52:27 +0530 Subject: [PATCH] Run Postgres as a rootless Podman Quadlet, and move the 17 cluster across Replaces the docker_container playbook, which published 5432 on 0.0.0.0 and kept its data in /var/lib/postgresql. Postgres is now the base Quadlet unit, the same shape as Caddy: it knows nothing about its clients, and a service attaches itself by dropping a file into postgres.container.d/ via postgres/roles/attach. Nothing here changes when a service is added or removed. Pinned to 18, which is two decisions rather than one tag. Postgres 18 moved both the image's VOLUME (/var/lib/postgresql/data -> /var/lib/postgresql) and its default PGDATA (/var/lib/postgresql/data/pgdata -> /var/lib/postgresql/18/docker), and the 18 entrypoint refuses to start when it finds a cluster in the old layout. So postgres_major is the single knob: it picks the image tag and the PGDATA path together, and the mount is now the parent directory. That also settles the migration: a PGDATA copy cannot cross a major version, so monoco's 17.5 cluster comes across as a pg_dumpall. Three things that bit us and are worth keeping: - PGDATA and its version directory are created by the role, not left to the container. The entrypoint runs as container-root, which under keep-id is a subuid on the host, and it only chowns PGDATA and its contents -- so ~/podman-data/postgres/18 would have been stranded subuid-owned, and the host user could not even rm -rf it without podman unshare. - NetworkAlias= applies to every network the container joins, which is what a database hub wants, but it is one fixed value in a shared drop-in rather than per-service: two services writing their own would make Quadlet pass --network-alias twice. It is not in postgres.container itself because on a fresh host that unit has no Network= at all. - The credentials live in a 0600 file read with EnvironmentFile=, not in Environment= lines: the unit is world-readable in ~/.config. The play no longer uses become, and does not apply the rootless-Podman prerequisites -- those are the shared task file at caddy/roles/podman, and the three variables they need are duplicated at the top of vars/default.yml. migrate-from-monoco.md records the one-time 17.5 -> 18.6 move: the dump, transfer and restore commands, the checks that prove the restore is complete (per-database table counts against the source), and the verified result. The dead nginx/php-fpm templates in postgres/files/ are removed. --- postgres/files/info.php.j2 | 2 - postgres/files/nginx.conf.j2 | 19 - postgres/migrate-from-monoco.md | 315 ++++++++++++++++ postgres/playbook.yml | 61 ++-- postgres/readme.md | 337 ++++++++++++++++-- postgres/roles/attach/tasks/main.yaml | 93 +++++ postgres/roles/postgres/handlers/main.yaml | 31 ++ postgres/roles/postgres/tasks/main.yaml | 155 ++++++++ .../postgres/templates/postgres.container.j2 | 50 +++ .../roles/postgres/templates/postgres.env.j2 | 10 + postgres/vars/default.yml | 94 ++++- 11 files changed, 1079 insertions(+), 88 deletions(-) delete mode 100644 postgres/files/info.php.j2 delete mode 100644 postgres/files/nginx.conf.j2 create mode 100644 postgres/migrate-from-monoco.md create mode 100644 postgres/roles/attach/tasks/main.yaml create mode 100644 postgres/roles/postgres/handlers/main.yaml create mode 100644 postgres/roles/postgres/tasks/main.yaml create mode 100644 postgres/roles/postgres/templates/postgres.container.j2 create mode 100644 postgres/roles/postgres/templates/postgres.env.j2 diff --git a/postgres/files/info.php.j2 b/postgres/files/info.php.j2 deleted file mode 100644 index 61ace19..0000000 --- a/postgres/files/info.php.j2 +++ /dev/null @@ -1,2 +0,0 @@ - **Status: done — do not re-run against the live cluster.** Executed +> 2026-10-15, Postgres 17.5 → 18.6. Verified afterwards: 12 databases restored +> with per-database table counts *identical* to monoco, 13 databases and 3 roles +> (`postgres`, `oc_tanshu`, `zitadel`), `btree_gist`/`pg_trgm`/`unaccent` present, +> no listener on host `5432`, and every directory in the data root owned by +> `tanshu` rather than a subuid. +> +> The steps below are kept as the record of how it was done, and for the next +> major-version move. Running them again would now fail on the first +> `CREATE DATABASE` and leave a half-loaded cluster — and the "start over" block +> in step 5 *destroys* `~/podman-data/postgres/18`. + +One-time move of every database from `monoco`'s Docker Postgres to `emily`'s +rootless Quadlet Postgres, across a major-version upgrade. Run the commands from +the machine that holds the `monoco` and `emily` SSH aliases. + +## Why this is a dump and not a copy + +Two independent reasons, either of which is decisive: + +- **17 data files cannot be read by an 18 server.** A copy of `PGDATA` produces a + cluster that 18 refuses to open — it would have to go through `pg_upgrade` with + both versions present. +- **The 18 image moved PGDATA.** In 17 it was `/var/lib/postgresql/data/pgdata`; in + 18 it is `/var/lib/postgresql/18/docker`, and the entrypoint deliberately + refuses to start when it finds a cluster in the old location, to stop an image + bump from silently looking like data loss. + +A logical dump sidesteps both, and at this size it is also the fastest thing to +get right. `pg_upgrade` would avoid re-importing, but it needs the 17 data +directory copied to `emily` first *and* a purpose-built image with both versions +(`tianon/postgres-upgrade:17-to-18`), for a cluster of well under a gigabyte. Not +worth the extra failure modes. + +## What is being moved (measured on monoco) + +Source: Docker container `postgres`, `postgres:17-alpine`, server **17.5**, +`PGDATA=/var/lib/postgresql/data/pgdata`, ~995 MB on disk. + +| Database | Size | Database | Size | +|---|---|---|---| +| `nextcloud` | 320 MB | `hiwikijs` | 16 MB | +| `brewman_acc` | 174 MB | `tandoor` | 16 MB | +| `brewman_mhl` | 132 MB | `brewman_exp` | 16 MB | +| `brewman_hops` | 122 MB | `affine` | 11 MB | +| `brewman_hinchco` | 41 MB | `bitwarden` | 10 MB | +| `medusa-criollo` | 21 MB | `zitadel` | 9 MB | + +Roles: `postgres` (superuser), `oc_tanshu` (creates databases), `zitadel`. +No custom tablespaces. `template1` is stock; the `postgres` database has no user +tables. `medusa-criollo` and `zitadel` have no running container — nothing writes +them — so they come along for completeness. + +## Order of operations + +1. Deploy the 18 cluster on `emily` (`postgres/playbook.yml`) and leave it **empty**. +2. Dump on `monoco`. +3. Transfer. +4. Restore into `emily`. +5. Verify, then point the app playbooks at it. + +The target must be empty before the restore. `psql` will happily skip a +`CREATE DATABASE` for a database that already exists, so a second restore into a +populated cluster is not an error but is also not a clean overwrite. + +## 1. Pre-flight + +**The 18 cluster has to exist before any of this.** If it does not, deploy it: + +```bash +cd prepare-server && ansible-playbook -u tanshu ../postgres/playbook.yml +``` + +Then check it is up and empty: + +```bash +# The 18 cluster exists, is running, and holds nothing but the defaults +ssh emily 'systemctl --user is-active postgres.service + podman exec postgres psql -U postgres -tAc "select datname from pg_database order by 1"' +# expect: active, then just: postgres, template0, template1 + +# Room for the dump plus the restored data (~2 GB) +ssh emily 'df -h / | tail -1' +ssh monoco 'df -h / | tail -1' +``` + +If that says `inactive`, or `podman exec` answers `no container with name or ID +"postgres" found`, the unit has not been deployed on `emily` — stop and do that +first. The restore in step 5 would otherwise pipe the whole dump into a +non-existent container and leave almost no trace. + +### Optional: quiesce the writers + +`pg_dumpall` takes one snapshot per database, so each database is internally +consistent but the cluster is not a single point in time. Every app here owns its +own database and there are no cross-database references, so a live dump is fine +in practice. If you want zero doubt, stop the writers for the duration: + +```bash +WRITERS='nextcloud gitea affine bitwarden hiwikijs tandoor + brewman-acc brewman-hops brewman-mhl brewman-exp brewman-hinchco' + +ssh monoco "docker stop $WRITERS" # before the dump +ssh monoco "docker start $WRITERS" # after step 3 +``` + +That is real downtime for those sites, so take it or leave it deliberately. + +## 2. Dump on monoco + +```bash +ssh monoco 'set -euo pipefail + mkdir -p ~/pg-migrate && cd ~/pg-migrate + TS=$(date +%Y%m%d-%H%M) + docker exec -e PGPASSWORD=123456 postgres pg_dumpall -U postgres \ + | zstd -T0 -6 -o "pg17-cluster-$TS.sql.zst" + zstd -t "pg17-cluster-$TS.sql.zst" + sha256sum "pg17-cluster-$TS.sql.zst" | tee "pg17-cluster-$TS.sql.zst.sha256" + ls -lh' +``` + +It prints nothing while it runs — `pg_dump` buffers, so there is no progress bar +— and takes a couple of minutes for ~900 MB. `zstd -t` is an integrity check; +there is no point transferring a truncated dump. + +`pg_dumpall` is used rather than per-database `pg_dump` because it carries the +roles and `template1` in the same stream, and this cluster has three roles that +would otherwise have to be recreated by hand. + +## 3. Verify the dump before trusting it + +```bash +ssh monoco 'cd ~/pg-migrate && F=$(ls -t pg17-cluster-*.sql.zst | head -1) + echo "file: $F" + echo "databases connected : $(zstd -dc "$F" | grep -cE "^\\\\connect")" # expect 14 + echo "databases created : $(zstd -dc "$F" | grep -cE "^CREATE DATABASE")" # expect 12 + echo "role statements : $(zstd -dc "$F" | grep -cE "^(CREATE|ALTER) ROLE")" # expect 6 + zstd -dc "$F" | head -5' +``` + +The counts are the shape of this particular cluster: 12 databases get a +`CREATE DATABASE`, and `postgres` + `template1` are dumped in place (14 +`\connect` lines). The 6 role statements are 3 roles × (`CREATE` + `ALTER`). + +Also check which extensions the dump creates, because step 5 runs with +`ON_ERROR_STOP=1` and a missing extension aborts the whole restore: + +```bash +ssh emily 'cd ~/pg-migrate && zstd -dc pg17-cluster.sql.zst | grep -E "^CREATE EXTENSION" | sort | uniq -c' +``` + +This cluster needs `btree_gist` (×5), `pg_trgm` and `unaccent` — all contrib +modules, and the official image builds contrib in +(`make -C contrib install`), so they are present. An extension that is *not* +contrib — `pgvector`, `postgis`, `timescaledb` — would not be, and would have to +be installed into the image before the restore can succeed. (`docs/podman-compat-audit.md` +notes affine's indexer is disabled, which is why there is no `pgvector` here.) + +## 4. Transfer to emily + +```bash +# stream it straight across; no copy on the workstation and no second hop +ssh monoco 'cat ~/pg-migrate/pg17-cluster-*.sql.zst' \ + | ssh emily 'mkdir -p ~/pg-migrate && cat > ~/pg-migrate/pg17-cluster.sql.zst' +``` + +Then gate on the checksum — a truncated dump restores into a partial cluster that +looks fine until an app reads a missing table: + +```bash +a=$(ssh monoco 'sha256sum ~/pg-migrate/pg17-cluster-*.sql.zst | cut -d" " -f1') +b=$(ssh emily 'sha256sum ~/pg-migrate/pg17-cluster.sql.zst | cut -d" " -f1') +[ -n "$a" ] && [ "$a" = "$b" ] && echo "transfer OK: $a" \ + || { echo "MISMATCH — do not restore: $a vs $b"; false; } +``` + +`rsync` cannot copy remote-to-remote directly, so the pipe above is the shortest +path. If the link is slow or drops, do it in two hops instead and check the +checksum on both sides: + +```bash +rsync -avP monoco:~/pg-migrate/ ~/pg-migrate/ +rsync -avP ~/pg-migrate/ emily:~/pg-migrate/ +``` + +## 5. Restore into the 18 cluster on emily + +```bash +ssh emily 'cd ~/pg-migrate + # Fail here, loudly, if the unit was never deployed. Without this, the + # pipeline below feeds the dump to a podman exec that cannot find a container + # and the only trace is a line in restore.err that a grep for ERROR misses. + podman exec postgres pg_isready -U postgres -d postgres \ + || { echo "FAILED: postgres is not running on emily — deploy it first (step 1)."; exit 1; } + + set -o pipefail + zstd -dc pg17-cluster.sql.zst \ + | grep -v "^CREATE ROLE postgres;$" \ + | podman exec -i -e PGPASSWORD=123456 postgres \ + psql -U postgres -d postgres -v ON_ERROR_STOP=1 -q 2> restore.err \ + || { echo "FAILED: restore stopped on an error:"; tail -20 restore.err; exit 1; } + + echo "OK — restore clean. Databases now:" + podman exec -e PGPASSWORD=123456 postgres psql -U postgres -tAc \ + "select count(*) from pg_database where datallowconn and not datistemplate"' +``` + +**Expect `13`** — the `postgres` database plus the 12 restored ones. That count is +the positive proof the restore happened; an empty error log on its own proves +nothing, which is exactly how a missing container slipped through the first time. + +Three details are doing real work here: + +- **`pg_isready` first.** `podman exec` on a missing container writes + `Error: no container with name or ID "postgres" found` and exits 1 — but the + pipeline still "completes", and that message is invisible to a case-sensitive + `grep ERROR`. This is the check that turns that into an immediate, obvious stop. +- **`grep -v '^CREATE ROLE postgres;$'`.** That is the one statement expected to + fail: `pg_dumpall` emits it and initdb has already created the role. The + `ALTER ROLE postgres ... PASSWORD` that follows is kept and succeeds, so the + superuser ends up with monoco's password hash — the same `123456` anyway. + Removing that exact line, rather than tolerating the error, is what allows + `ON_ERROR_STOP=1`: any *unexpected* error now aborts the restore and is printed + instead of being skipped. +- **The database count at the end**, as above. + +The dump was made by `pg_dump` 17.5 and is replayed by `psql` 18: forward +restores are the supported direction, and source and target are both the +Alpine/musl image, so the `en_US.utf8` locale the databases were created with +resolves on both sides. + +If the restore does stop, fix the cause and start over — `ON_ERROR_STOP=1` can +leave the cluster half-loaded, and re-running on top of that adds "already +exists" noise rather than converging. + +```bash +# destroys everything in the emily cluster +ssh emily 'systemctl --user stop postgres.service + rm -rf ~/podman-data/postgres/18' +cd prepare-server && ansible-playbook -u tanshu ../postgres/playbook.yml +``` + +That `rm -rf` needs no `podman unshare` precisely because the role creates the +version directory as `tanshu` rather than letting the container strand it as a +subuid-owned directory. If it ever fails with permission denied, that is the +assumption breaking — check `ls -ln ~/podman-data/postgres/18`. + +## 6. Verify + +```bash +# server version, and the databases with their sizes — compare against the table above +ssh emily 'podman exec postgres postgres --version' +ssh emily 'podman exec -e PGPASSWORD=123456 postgres psql -U postgres -tAc \ + "select datname, pg_size_pretty(pg_database_size(datname)) from pg_database order by pg_database_size(datname) desc"' + +# all three roles made it +ssh emily 'podman exec -e PGPASSWORD=123456 postgres psql -U postgres -c "\du"' +``` + +Then the check that actually matters — the same per-database table count on both +sides. Feed the loop to the remote shell over stdin so the quoting stays sane: + +```bash +ssh monoco 'bash -s' <<'EOF' +EXEC="docker exec -e PGPASSWORD=123456 postgres" +for db in $($EXEC psql -U postgres -tAc "select datname from pg_database where datallowconn and not datistemplate and datname <> 'postgres' order by 1"); do + printf '%-16s %s\n' "$db" "$($EXEC psql -U postgres -d "$db" -tAc "select count(*) from information_schema.tables where table_schema not in ('pg_catalog','information_schema')")" +done +EOF + +ssh emily 'bash -s' <<'EOF' +EXEC="podman exec -e PGPASSWORD=123456 postgres" +for db in $($EXEC psql -U postgres -tAc "select datname from pg_database where datallowconn and not datistemplate and datname <> 'postgres' order by 1"); do + printf '%-16s %s\n' "$db" "$($EXEC psql -U postgres -d "$db" -tAc "select count(*) from information_schema.tables where table_schema not in ('pg_catalog','information_schema')")" +done +EOF +``` + +Save both outputs and diff them — identical table counts per database is the +signal that the restore was complete. Sizes will differ slightly and that is +normal (toast/fillfactor are not byte-reproducible). + +Also confirm ownership is what the playbook promises, because a wrong `keep-id` +shows up here and nowhere else: + +```bash +ssh emily 'ls -ln ~/podman-data/postgres/18/docker | head' +# PGDATA and its contents must be owned by tanshu:tanshu, not a 165536+ subuid +``` + +## 7. Afterwards + +- **Keep monoco's cluster.** Do not `docker rm postgres` and do not drop + databases: a stopped, intact 17 cluster is the fastest rollback there is. + Leave the dump in `~/pg-migrate` on both hosts for the same reason. +- **Stop the old writers before the new ones start** if both hosts could serve + the same app, or you will have two databases diverging behind the same site. +- **Apps connect unchanged.** `pg_dumpall` restores each database under its + original name with its original role grants, and the app playbooks' + `postgres_container: postgres` resolves over the attached app networks — same + container name, same `db` alias, same password. +- **The dump is not a backup regime.** It is a migration artifact. Per-database + `pg_dump` on a schedule is still owed (see the Backups section of + [readme.md](readme.md)). + +## Rollback + +There is nothing to reverse on `emily` beyond stopping the new container: +`systemctl --user stop postgres.service`. `monoco` was never modified — its +container was only read — so rolling back is starting the old stack again. + +The one exception is the quiesce step, if you took it: `docker start` the +`WRITERS` list above. diff --git a/postgres/playbook.yml b/postgres/playbook.yml index a22633b..bead677 100755 --- a/postgres/playbook.yml +++ b/postgres/playbook.yml @@ -1,37 +1,32 @@ -################################################# -# DO Community Playbooks: Docker -################################################# --- -- hosts: all - become: true +# Deploys PostgreSQL as a rootless Podman Quadlet unit. +# +# Connects as the container user and runs everything as that user: the Quadlet +# units belong to it, so the connection's PAM session is what makes +# `systemctl --user` work without any environment juggling. +# +# gather_facts is required: podman_user and the home directory come from the +# user_id / user_dir facts. +# +# The rootless-Podman prerequisites (installing podman, enabling lingering, the +# low-port sysctl, creating the Quadlet directory) are deliberately NOT applied +# here. They are the shared task file at caddy/roles/podman/tasks/main.yaml, and +# a host that has already run caddy/playbook.yml has them: +# +# - name: Rootless Podman prerequisites +# ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/podman/tasks/main.yaml" +# +# The variables those tasks need are duplicated at the top of vars/default.yml, +# so this play stands on its own. +# +# Run with: ansible-playbook -u tanshu postgres/playbook.yml +- name: Deploy PostgreSQL as a rootless Podman Quadlet + hosts: emily + gather_facts: true vars_files: - vars/default.yml - tasks: - - name: Pull default Postgresql image - docker_image: - name: "{{ postgres_image }}" - source: pull - force_source: yes - - - name: Ensure Nextcloud Directory exists - file: - path: "/var/lib/{{ host_directory }}" - state: directory - mode: 0755 - - - name: Create postgres container - docker_container: - name: "{{ postgres_name }}" - image: "{{ postgres_image }}" - state: started - restart_policy: "unless-stopped" - env: - POSTGRES_USER: "{{ pg_user }}" - POSTGRES_PASSWORD: "{{ pg_password }}" - POSTGRES_DB: 'postgres' - PGDATA: "{{ data_location }}/pgdata" - published_ports: - - 5432:5432 - volumes: - - "/var/lib/{{ host_directory }}/data:{{ data_location }}" + roles: + # The base postgres unit. It does NOT know which app networks it serves. + # Each service attaches itself via postgres/roles/attach. + - postgres diff --git a/postgres/readme.md b/postgres/readme.md index f6d7ecc..48bfa2a 100644 --- a/postgres/readme.md +++ b/postgres/readme.md @@ -1,46 +1,323 @@ -# Docker on Ubuntu 18.04 +# PostgreSQL (rootless Podman + Quadlet) -This playbook will install Docker an Ubuntu 18.04 machine, as explained in the guide on -[How to Use Ansible to Install and Set Up Docker on Ubuntu 18.04](https://www.digitalocean.com/community/tutorials/how-to-use-ansible-to-install-and-set-up-docker-on-ubuntu-18-04). -A number of containers will be created with the options specified in the `vars/default.yml` variable file. +Runs PostgreSQL as a **rootless Podman** container managed by **systemd user +units** (Quadlet). Replaces the `docker_container` playbook, which published +`5432` on `0.0.0.0` and kept its data in `/var/lib/postgresql`. + +**Postgres is the base unit and knows nothing about its clients.** It does not +list any app network and does not need editing when a service is added or +removed. Each service attaches *itself*. That mirrors [caddy](../caddy/readme.md), +which is the reverse proxy end of the same idea. + +## Layout + +``` +postgres/ +├── playbook.yml +├── vars/default.yml +└── roles/ + ├── postgres/ # the base postgres.container unit, env file, handlers + └── attach/ # includable task file: a service attaches itself to Postgres +``` + +Everything is installed under `{{ quadlet_dir }}` +(`~/.config/containers/systemd/`). State lives under `{{ host_directory }}` +(`~/podman-data/postgres/`). + +## How services attach themselves + +Postgres joins the app network of every service that needs a database. A service +joins by writing one Quadlet *drop-in* next to `postgres.container`: + +``` +~/.config/containers/systemd/ +├── postgres.container # base unit, no app networks +├── postgres.container.d/ # ← drop-ins, one per client service +│ ├── 10-network-aliases.conf # [Container] NetworkAlias=db (shared) +│ ├── 50-nextcloud.conf # [Container] Network=nextcloud.network +│ ├── 50-gitea.conf +│ └── 50-tandoor.conf +├── nextcloud.network +├── gitea.network +└── tandoor.network +``` + +A service playbook does both halves — its own network, then the attach: + +```yaml +- name: Register this service with Caddy + ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/register/tasks/main.yaml" + vars: + network_name: "{{ container_name }}" + +- name: Attach this service to Postgres + ansible.builtin.include_tasks: "{{ playbook_dir }}/../postgres/roles/attach/tasks/main.yaml" + vars: + network_name: "{{ container_name }}" +``` + +The play must also run the `podman` role (for `podman_user` and `quadlet_dir`) +and define a `Restart Postgres` handler — see +[roles/postgres/handlers/main.yaml](roles/postgres/handlers/main.yaml) for the +exact snippet. + +Order matters in one direction only: the `attach` include needs +`.network` to already exist, which is why Caddy's `register` comes first. +It fails with a readable message if it does not, because a drop-in naming a +missing `.network` file makes Quadlet reject the *whole* postgres unit with an +error that never mentions the drop-in. + +### Names: `postgres` and `db` + +Both resolve on every attached network. `postgres` comes from +`ContainerName=postgres`; `db` is added by `10-network-aliases.conf`. The alias +exists because the Docker-era playbooks are not consistent — nextcloud, gitea, +tandoor and outline connect to `db`, the rest to `postgres`. + +`NetworkAlias=` applies to *every* network the container joins, which is exactly +what a hub wants. It lives in one shared drop-in with a fixed value rather than +in each service's `50-*.conf`, for two reasons: + +- it is not per-service — if one service could set a different value, the shared + file would be rewritten on every run, and every service writes the same bytes; +- with the alias in each service's own file, two attached services would make + Quadlet pass `--network-alias db` twice. + +It is *not* in `postgres.container` itself for a duller reason: on a fresh host, +before anything has attached, the unit has no `Network=` at all, and asking +Podman for an alias on the default network is not a configuration worth +depending on. + +### One way to attach, and one rule + +[roles/attach](roles/attach/tasks/main.yaml) is the shared include, and is what +every service uses. A service may instead write the same two files from its own +role, but then it **must write identical bytes** on purpose: the drop-ins live in +one directory, so anything less means whichever play runs last rewrites the +other's file on every run. If you add a second form, make it match. + +Whichever form you use, **flush handlers before the app starts on a first +attach.** The restart that joins Postgres to the new network is a handler, so by +default it runs at the end of the play — after the app has already come up and +failed to resolve `db`. `tandoor/roles/postgres` is the working example. + +### Why drop-ins rather than listing the networks + +Listing app networks in `postgres.container` would mean editing this playbook +every time a service is added, and again on removal. Drop-ins survive +re-templating of the base unit, are the mechanism Quadlet documents for this, +and make removal trivial: delete `50-tandoor.conf`, `daemon-reload`, done. + +### The trade-off + +A new drop-in changes the generated unit, so Postgres is **recreated** to join +the network — a few seconds of downtime for *every* connected app, not just the +new one. Quadlet has no live equivalent of `docker network connect`. If that +matters, `podman network connect postgres` takes effect immediately, and +the drop-in makes it survive the next recreate; the `attach` role does not do +this by default because it would make the play non-idempotent. + +### Why not one shared database network + +Every app on a shared `postgres_net` could reach every other app on it. The +generated `.network` units set `Options=isolate=true`, so keeping Postgres +as the only common member of otherwise separate networks is what preserves the +per-app isolation Caddy's networks give you. + +## Who runs as whom + +`UserNS=keep-id:uid=70,gid=70`. `70` is the uid the *server* writes as: the +official Alpine entrypoint starts as root, chowns PGDATA to the `postgres` +account, and drops privileges, so `docker exec postgres id -u` reporting `0` is +misleading. `keep-id` maps that one uid onto `podman_user`, which means: + +- the whole data root — `~/podman-data/postgres/18/docker` and the version + directory above it — is owned by `podman_user`, never by a subuid; +- plain `tar`, `pg_dump`, `cp` and `rm -rf` work without `podman unshare` and + without `sudo`; +- the reverse — chowning the data to `podman_user` before first start — is also + correct, which is what makes bringing data across simple. + +PGDATA and its version directory are created by the role rather than left to the +container's own `mkdir -p`. The entrypoint runs as container-root — a subuid +under `keep-id` — and only chowns PGDATA and its contents, so the version +directory would be stranded as subuid-owned: readable but not removable by +`podman_user`. Creating the chain up front is what makes the bullets above true. + +Like Caddy, `podman_user` is derived from the connection +(`ansible_facts['user_id']`) and the play carries no `become`. This playbook does +not run the podman role, so it has no guard against connecting as `root`; the +prerequisites are applied separately (see Running). ## Settings -- `create_containers`: number of containers to create. -- `default_container_name`: default name for new containers. -- `default_container_image`: default image for new containers. -- `default_container_command`: default command to run on new containers. +| Variable | Purpose | +|---|---| +| `podman_user`, `podman_data_dir`, `quadlet_dir` | Rootless-Podman contract, duplicated from [caddy/vars/default.yml](../caddy/vars/default.yml) | +| `postgres_container` | Container / unit name, and the prefix of the drop-in directory | +| `postgres_major` | The one knob: pins both the image tag and the PGDATA path | +| `postgres_image` | `postgres:-alpine`, fully qualified on purpose (see below) | +| `postgres_uid` / `postgres_gid` | The uid the server writes as — `70` in the Alpine image | +| `host_directory` | Where state lives (`~/podman-data/postgres`) | +| `postgres_mount` / `pgdata` | The image's VOLUME, and PGDATA inside it (`/var/lib/postgresql`, `/var/lib/postgresql/18/docker`) | +| `postgres_env_file` | `0600` file holding the credentials, read via `EnvironmentFile=` | +| `postgres_dropin_dir` | Where services attach their networks | +| `pg_user` / `pg_password` / `pg_database` | Credentials and the default database | +| `postgres_timeout_start` / `postgres_timeout_stop` | initdb on first start, clean shutdown on stop | +### Image and PGDATA layout -## Running this Playbook +`docker.io/library/postgres:18-alpine` — fully qualified because rootless Podman +resolves short names against `unqualified-search-registries`, and a Quadlet unit +has no TTY to answer a short-name prompt. `-alpine` still floats within 18.x for +security patches. -Quick Steps: +**18 moved both the image's VOLUME and its default PGDATA**, which is why +`postgres_major` exists and why the mount is a *directory* rather than the data +directory itself: -### 1. Obtain the playbook -```shell -git clone https://github.com/do-community/ansible-playbooks.git -cd ansible-playbooks/docker_ubuntu1804 -``` +| | 17 | 18+ | +|---|---|---| +| `VOLUME` | `/var/lib/postgresql/data` | `/var/lib/postgresql` | +| `PGDATA` | `/var/lib/postgresql/data/pgdata` | `/var/lib/postgresql/18/docker` | -### 2. Customize Options +The 18 entrypoint *refuses to start* if it finds a cluster at the old +`/var/lib/postgresql/data` path — that check exists so that bumping the image tag +without upgrading the database fails loudly instead of appearing to be data loss. +Keeping the major version in the path is also what lets a later +`pg_upgrade --link` run without mount-boundary problems. -```shell -nano vars/default.yml -``` +`monoco` runs 17.5, so the data comes across as a **logical dump**, not a copy of +`PGDATA`: 17 data files are unreadable by an 18 server. The one-time move is +[step-by-step here](migrate-from-monoco.md). -```yml -#vars/default.yml ---- -create_containers: 4 -default_container_name: docker -default_container_image: ubuntu -default_container_command: sleep 1d -``` +### No published port -### 3. Run the Playbook +There is deliberately no `PublishPort`. The Docker-era playbook published +`5432:5432` on `0.0.0.0` with UFW inactive, so the database was reachable from +the internet; it needs no host port at all now that clients reach it over the +app networks. Verify with `ss -tlnp | grep 5432` — it should print nothing. + +### Credentials + +`pg_password` is still the shared `123456` from the app playbooks, so a migrated +app connects unchanged. It is written to `~/podman-data/postgres/postgres.env` +mode `0600` and read with `EnvironmentFile=` rather than as `Environment=` lines, +because the unit file itself is world-readable in `~/.config`. Note that +`podman inspect postgres` still prints the environment to anyone who can run +podman as that user; a real secret store is the next step, and is a +one-file change here plus the `postgres_password` vars in the app playbooks. + +## Running + +The play targets `emily` directly (`hosts: emily`): ```command -ansible-playbook -l [target] -i [inventory file] -u [remote user] playbook.yml +ansible-playbook -u tanshu postgres/playbook.yml ``` -For more information on how to run this Ansible setup, please check this guide: [How to Use Ansible to Install and Set Up Docker on Ubuntu 18.04](https://www.digitalocean.com/community/tutorials/how-to-use-ansible-to-install-and-set-up-docker-on-ubuntu-18-04). \ No newline at end of file +The inventory that defines `emily` lives in `prepare-server/`, whose `ansible.cfg` +names it — run from there, or pass an inventory explicitly. + +This play applies **only** the Postgres unit and its data directory. The +rootless-Podman prerequisites — installing podman, `loginctl enable-linger`, the +low-port sysctl and the Quadlet directory — are the shared task file at +[caddy/roles/podman/tasks/main.yaml](../caddy/roles/podman/tasks/main.yaml), and +are not run here. A host that has already run `caddy/playbook.yml` has them. To +apply them on their own: + +```yaml +- name: Rootless Podman prerequisites + ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/podman/tasks/main.yaml" +``` + +The three variables those tasks need are duplicated at the top of +[vars/default.yml](vars/default.yml) rather than imported, because a play cannot +include Caddy's vars file wholesale — it also sets `host_directory`. + +## Operational notes + +``` +systemctl --user status postgres +systemctl --user restart postgres +journalctl --user -u postgres -f +podman exec -it postgres psql -U postgres +podman exec postgres pg_isready -U postgres +podman inspect postgres --format '{{range $k,$v := .NetworkSettings.Networks}}{{$k}} {{end}}' +``` + +The play waits on `pg_isready` inside the container rather than trusting +systemd. `Restart=always` means the unit reports `active` even while the +container crash-loops, so a wrong `keep-id` (an entrypoint permission error, +over and over) would otherwise look like success. + +`systemctl --user enable postgres` is neither needed nor possible: the unit is +*generated*, and Quadlet applies `[Install] WantedBy=default.target` itself. + +### Debugging a unit that will not generate + +``` +/usr/lib/systemd/user-generators/podman-user-generator --user --dryrun +systemd-analyze --user --generators=true verify postgres.service +``` + +The most common cause is a `Network=` naming a `.network` file that does not +exist — which is why `attach` checks first. + +## Backups + +`--link postgres:db` does not exist in Podman, so every `*-backup.sh` that uses +it (`db-`, `bitwarden-`, `hiwikijs-`, `kakawa-`, `nextcloud-`, `tandoor-`) is +broken on this host. Dump in place instead — no helper container, no networking, +no `--user` juggling: + +```bash +podman exec -e PGPASSWORD="..." postgres \ + pg_dump -U postgres nextcloud | bzip2 > "$WORKDIR/$DB_DUMP_NAME" +``` + +Whole-cluster form, which is what a migration uses: + +```bash +podman exec -e PGPASSWORD="..." postgres pg_dumpall -U postgres > cluster.sql +``` + +Because the data is owned by `podman_user`, the `sudo tar` / `chown $(logname)` +gymnastics in the old scripts can go away too. The database is also the reason +root's crontab cannot drive it any more: root cannot `podman exec` into another +user's rootless containers, so anything scheduled belongs in a **systemd user +timer** next to the Quadlet units. + +## Bringing data from the Docker host + +The one-time move of monoco's cluster — 12 databases, ~900 MB, and a 17 → 18 +major-version jump — is written out command by command in +**[migrate-from-monoco.md](migrate-from-monoco.md)**: dump, transfer, restore, +verify, rollback. + +The short version of why it is a dump rather than a file copy: + +- **17 data files are unreadable by an 18 server**, so `PGDATA` cannot simply be + copied across the major version; +- **the 18 image moved PGDATA** to `/var/lib/postgresql/18/docker` and refuses to + start when it finds a cluster at the old path; +- for a cluster well under a gigabyte, a logical dump is both the safer and the + faster thing to get right, and it re-creates the three roles + (`postgres`, `oc_tanshu`, `zitadel`) in the same pass. + +`pg_dumpall` restores each database under its original name with its original +grants, so the app playbooks connect unchanged — same container name, same `db` +alias, same shared password. + +## Verification + +- [ ] `systemctl --user status postgres` is active, and stays active after a reboot +- [ ] `podman exec postgres pg_isready -U postgres` reports accepting connections +- [ ] `podman exec postgres psql -U postgres -c '\l'` lists the expected databases +- [ ] `ss -tlnp | grep 5432` on the host prints nothing +- [ ] `ls -ln ~/podman-data/postgres/18/docker` is owned by `tanshu`, not a `165536+` subuid +- [ ] `ls -ln ~/podman-data/postgres/18` is owned by `tanshu` too — if it is not, the + container created it as subuid and `rm -rf` will fight you +- [ ] From each client app's container: `getent hosts postgres` and `getent hosts db` +- [ ] Each app can read **and** write — that is the check that catches a wrong `keep-id` +- [ ] A `pg_dump` of every database completes and restores into a scratch database diff --git a/postgres/roles/attach/tasks/main.yaml b/postgres/roles/attach/tasks/main.yaml new file mode 100644 index 0000000..323ef31 --- /dev/null +++ b/postgres/roles/attach/tasks/main.yaml @@ -0,0 +1,93 @@ +--- +# Attach a service's network to Postgres. +# +# Postgres is the *base* unit and knows nothing about its clients, the same way +# Caddy knows nothing about its upstreams. A service that needs a database adds +# itself by dropping files next to postgres.container, and Quadlet merges them in. +# +# Include this from a service playbook: +# +# - name: Attach this service to Postgres +# ansible.builtin.include_tasks: "{{ playbook_dir }}/../postgres/roles/attach/tasks/main.yaml" +# vars: +# network_name: "{{ network_name }}" +# +# The including play must: +# +# 1. have created {{ network_name }}.network already — caddy/roles/register +# does that, and every service here that needs a database is also behind +# Caddy, so its register include comes first; +# 2. run the `podman` role (or include its task file and define the same four +# vars), for podman_user and quadlet_dir; +# 3. define a "Restart Postgres" handler — see +# roles/postgres/handlers/main.yaml for the exact snippet; and +# 4. flush handlers before starting the app, on a first attach. This include +# notifies that restart, so by default it runs at the end of the play — +# after the app has already started and failed to resolve `db`. A +# `meta: flush_handlers` immediately after this include is enough; see +# tandoor/roles/postgres for the working example. +# +# It runs as the connection user, which must be the rootless podman_user. +# +# A service may instead write these two files from its own role; the bytes have +# to match this file exactly, or the two will overwrite each other on every run. + +- name: "Check that the {{ network_name }} network unit exists" + ansible.builtin.stat: + path: "{{ quadlet_dir }}/{{ network_name }}.network" + register: postgres_attach_network + +# A drop-in naming a .network file that does not exist makes Quadlet fail to +# generate the whole postgres unit, with a message that does not mention the +# drop-in at all. Fail here instead, where the cause is obvious. +- name: "Fail if {{ network_name }} has no network unit" + ansible.builtin.assert: + that: postgres_attach_network.stat.exists + fail_msg: >- + {{ quadlet_dir }}/{{ network_name }}.network does not exist, so attaching + it to Postgres would break the postgres unit. Create the network first + (caddy/roles/register does it), then include this file. + +- name: Ensure the Postgres Quadlet drop-in directory exists + ansible.builtin.file: + path: "{{ quadlet_dir }}/{{ postgres_container | default('postgres') }}.container.d" + state: directory + owner: "{{ podman_user }}" + group: "{{ podman_user }}" + mode: '0755' + +- name: "Attach {{ network_name }} to Postgres" + ansible.builtin.copy: + dest: >- + {{ quadlet_dir }}/{{ postgres_container | default('postgres') }}.container.d/50-{{ network_name }}.conf + owner: "{{ podman_user }}" + group: "{{ podman_user }}" + mode: '0644' + content: | + # Added by the {{ network_name }} playbook. Do not edit by hand. + # + # Deleting this file removes the network from {{ postgres_container | default('postgres') }} on + # the next daemon-reload; nothing else changes. + [Container] + Network={{ network_name }}.network + notify: Restart Postgres + +# NetworkAlias= applies to every network the container joins, which is exactly +# right for a hub: every app in this repo reaches postgres as `db` (nextcloud, +# gitea, tandoor and outline use the alias; the rest use the container name). +# It is a fixed value in one shared file, not a per-service option, so the same +# bytes are written no matter how many services attach — two services writing +# their own aliases would make Quadlet pass --network-alias twice. +- name: "Give {{ postgres_container | default('postgres') }} the shared 'db' alias" + ansible.builtin.copy: + dest: >- + {{ quadlet_dir }}/{{ postgres_container | default('postgres') }}.container.d/10-network-aliases.conf + owner: "{{ podman_user }}" + group: "{{ podman_user }}" + mode: '0644' + content: | + # Shared container aliases for {{ postgres_container | default('postgres') }}. + # Managed by Ansible (the service playbooks that use it). Do not edit. + [Container] + NetworkAlias=db + notify: Restart Postgres diff --git a/postgres/roles/postgres/handlers/main.yaml b/postgres/roles/postgres/handlers/main.yaml new file mode 100644 index 0000000..a2f22e7 --- /dev/null +++ b/postgres/roles/postgres/handlers/main.yaml @@ -0,0 +1,31 @@ +--- +# All handlers run as the connection user ({{ podman_user }}), so systemctl +# --user works without setting XDG_RUNTIME_DIR. + +- name: Reload systemd user units + ansible.builtin.command: systemctl --user daemon-reload + changed_when: true + listen: Reload systemd user units + +- name: Restart Postgres + ansible.builtin.systemd_service: + name: "{{ postgres_unit }}" + scope: user + state: restarted + daemon_reload: true + listen: Restart Postgres + +# Reference implementation for service playbooks. A service that attaches its +# network with postgres/roles/attach must notify "Restart Postgres", and defines +# its own copy of this handler, e.g.: +# +# - name: Restart Postgres +# ansible.builtin.systemd_service: +# name: postgres.service +# scope: user +# state: restarted +# daemon_reload: true +# listen: Restart Postgres +# +# A new drop-in changes the generated unit, so Postgres has to be recreated to +# join the network — Quadlet has no live equivalent of `podman network connect`. diff --git a/postgres/roles/postgres/tasks/main.yaml b/postgres/roles/postgres/tasks/main.yaml new file mode 100644 index 0000000..688925a --- /dev/null +++ b/postgres/roles/postgres/tasks/main.yaml @@ -0,0 +1,155 @@ +--- +# Start PostgreSQL +# +# Everything here runs as {{ podman_user }} — the connection user. No become, +# no XDG_RUNTIME_DIR: SSH gave us a real PAM session, so systemd's user manager +# is reachable via the standard environment. + +- name: Ensure the Postgres data directory exists + ansible.builtin.file: + path: "{{ host_directory }}" + state: directory + owner: "{{ podman_user }}" + group: "{{ podman_user }}" + mode: '0755' + +# PGDATA and its version directory are created here rather than left to the +# container's own `mkdir -p`. The entrypoint runs as container-root, which under +# keep-id is a subuid on the host, and it only chowns PGDATA and its contents — +# so the version directory would be left owned by 165536 and the host user could +# not even `rm -rf` it without podman unshare. Creating the whole chain up front +# keeps every directory in the data root owned by podman_user. +# +# Empty PGDATA is exactly right: the 18 entrypoint reads that as "run initdb". +- name: Ensure the versioned data directory exists + ansible.builtin.file: + path: "{{ pgdata_host_dir | dirname }}" + state: directory + owner: "{{ podman_user }}" + group: "{{ podman_user }}" + mode: '0755' + +- name: Ensure PGDATA exists + ansible.builtin.file: + path: "{{ pgdata_host_dir }}" + state: directory + owner: "{{ podman_user }}" + group: "{{ podman_user }}" + mode: '0700' + +- name: Ensure the Quadlet drop-in directory exists + ansible.builtin.file: + path: "{{ postgres_dropin_dir }}" + state: directory + owner: "{{ podman_user }}" + group: "{{ podman_user }}" + mode: '0755' + +# The unit file is 0644 in ~/.config, so the password does not belong in it. +# This file is 0600 and is read by the container through EnvironmentFile=. +- name: Write the Postgres environment file + ansible.builtin.template: + src: postgres.env.j2 + dest: "{{ postgres_env_file }}" + owner: "{{ podman_user }}" + group: "{{ podman_user }}" + mode: '0600' + notify: Restart Postgres + +- name: Install the base Postgres Quadlet unit + ansible.builtin.template: + src: postgres.container.j2 + dest: "{{ quadlet_dir }}/{{ postgres_container }}.container" + owner: "{{ podman_user }}" + group: "{{ podman_user }}" + mode: '0644' + notify: + - Reload systemd user units + - Restart Postgres + +# The wait is part of the block on purpose. `Restart=always` means systemd +# reports the unit active even while the container crash-loops — a wrong +# keep-id shows up as an entrypoint permission error over and over — so +# is-active alone would call a broken database healthy. +- name: Start Postgres and wait for it to accept connections + block: + - name: Start Postgres + ansible.builtin.systemd_service: + name: "{{ postgres_unit }}" + scope: user + state: started + daemon_reload: true + + # pg_isready is the server's own answer, and it fails while initdb is still + # running on a fresh data directory (it exits 2, so the retry loop holds). + - name: Wait for Postgres to accept connections + ansible.builtin.command: >- + podman exec {{ postgres_container }} + pg_isready -U {{ pg_user }} -d {{ pg_database }} + register: postgres_ready + until: postgres_ready.rc == 0 + retries: 12 + delay: 5 + changed_when: false + rescue: + # systemd reports "control process exited with error code" and nothing + # else, which is useless for the usual failures (a bad uid mapping, an + # unwritable PGDATA, a missing image). Pull the real reason out. + - name: Collect Postgres diagnostics + ansible.builtin.command: "{{ item }}" + loop: + - "systemctl --user status {{ postgres_unit }} --no-pager" + - "journalctl --user -u {{ postgres_unit }} --no-pager -n 40" + register: postgres_diag + changed_when: false + failed_when: false + + - name: Show Postgres diagnostics + ansible.builtin.debug: + msg: "{{ postgres_diag.results | map(attribute='stdout') | list }}" + + - name: Fail + ansible.builtin.fail: + msg: "Postgres did not come up — see the diagnostics above." + +# Quadlet honours [Install] WantedBy= when it generates the unit, so +# `systemctl enable` is neither needed nor possible on a generated unit. +- name: Verify the Postgres unit is active + ansible.builtin.command: "systemctl --user is-active {{ postgres_unit }}" + register: postgres_active + changed_when: false + failed_when: postgres_active.stdout != 'active' + +- name: List the services attached to Postgres + ansible.builtin.find: + paths: "{{ postgres_dropin_dir }}" + patterns: "*.conf" + register: postgres_dropins + +- name: Report the networks Postgres is attached to + ansible.builtin.command: >- + podman inspect {{ postgres_container }} + --format '{% raw %}{{ range $k, $v := .NetworkSettings.Networks }}{{ $k }} {{ end }}{% endraw %}' + register: postgres_networks_now + changed_when: false + failed_when: false + +- name: Show attached services + ansible.builtin.debug: + msg: >- + Postgres networks: + {{ postgres_networks_now.stdout | default('(container not running)') }}. + Attached by: + {{ postgres_dropins.files | map(attribute='path') | map('basename') | join(', ') + | default('(nothing yet)', true) }}. + +# Say this out loud rather than let it be discovered by a failing app: with no +# drop-in, Postgres has no Network= at all, so it sits on the default Podman +# network and no app network can resolve it. +- name: Note that no service has attached yet + ansible.builtin.debug: + msg: >- + No service has attached to Postgres. It is on the default Podman network + only, and no app can resolve it until a service playbook includes + postgres/roles/attach (see readme.md). + when: postgres_dropins.matched == 0 diff --git a/postgres/roles/postgres/templates/postgres.container.j2 b/postgres/roles/postgres/templates/postgres.container.j2 new file mode 100644 index 0000000..b39b339 --- /dev/null +++ b/postgres/roles/postgres/templates/postgres.container.j2 @@ -0,0 +1,50 @@ +# PostgreSQL — rootless Podman Quadlet unit. +# Managed by Ansible (postgres/roles/postgres). Do not edit by hand. +# +# Postgres deliberately does NOT list the app networks it serves. Each service +# attaches itself by dropping a file into: +# +# {{ postgres_dropin_dir }}/ +# +# which Quadlet merges into this unit. Removing a service therefore never means +# editing this file. See readme.md, "How services attach themselves". + +[Unit] +Description=PostgreSQL database +Documentation=https://hub.docker.com/_/postgres +After=network-online.target +Wants=network-online.target + +[Container] +ContainerName={{ postgres_container }} +Image={{ postgres_image }} + +# Map the container's postgres account (uid {{ postgres_uid }}) onto +# {{ podman_user }} on the host, so PGDATA stays owned by {{ podman_user }} and +# never by a subuid. The official entrypoint starts as root, chowns PGDATA to +# uid {{ postgres_uid }}, then drops privileges — so everything it writes lands +# on {{ podman_user }}, and plain tar / pg_dump backups keep working. +UserNS=keep-id:uid={{ postgres_uid }},gid={{ postgres_gid }} + +# No PublishPort, on purpose. The Docker-era playbook published 5432 on +# 0.0.0.0 (and a UFW rule was never there to narrow it); here Postgres is +# reachable only from the app networks that attach themselves to it. +# +# The mount is the image's VOLUME, which in 18 is the *parent* directory — the +# major version is in the path instead ({{ pgdata }}). Mounting +# /var/lib/postgresql/data here instead would make the 18 entrypoint refuse to +# start, because that path is where it looks for an un-upgraded 17 cluster. +Volume={{ host_directory }}:{{ postgres_mount }} + +# POSTGRES_USER / POSTGRES_PASSWORD / POSTGRES_DB / PGDATA live in this 0600 +# file rather than in Environment= lines here: this unit is world-readable. +EnvironmentFile={{ postgres_env_file }} + +[Service] +Restart=always +# The first start runs initdb, and a database needs time to shut down cleanly. +TimeoutStartSec={{ postgres_timeout_start }} +TimeoutStopSec={{ postgres_timeout_stop }} + +[Install] +WantedBy=default.target diff --git a/postgres/roles/postgres/templates/postgres.env.j2 b/postgres/roles/postgres/templates/postgres.env.j2 new file mode 100644 index 0000000..e44b00a --- /dev/null +++ b/postgres/roles/postgres/templates/postgres.env.j2 @@ -0,0 +1,10 @@ +# Environment for postgres.container. +# Managed by Ansible (postgres/roles/postgres). Do not edit by hand. +# +# Mode 0600 on purpose: the Quadlet unit that reads this file is world-readable, +# so the password must not live there. Note that `podman inspect` still prints +# the resulting environment to anyone who can run podman as this user. +POSTGRES_USER={{ pg_user }} +POSTGRES_PASSWORD={{ pg_password }} +POSTGRES_DB={{ pg_database }} +PGDATA={{ pgdata }} diff --git a/postgres/vars/default.yml b/postgres/vars/default.yml index f8518a7..6a96014 100644 --- a/postgres/vars/default.yml +++ b/postgres/vars/default.yml @@ -1,8 +1,94 @@ --- -postgres_name: postgres -postgres_image: postgres:17-alpine +# --------------------------------------------------------------------------- +# Rootless Podman / Quadlet +# --------------------------------------------------------------------------- +# Duplicated on purpose from caddy/vars/default.yml: a play cannot include that +# file wholesale, because it also sets `host_directory` — to the Caddy data +# directory — and every playbook defines that for itself. Keep these three in +# sync with caddy/vars/default.yml. (Its fourth, podman_unprivileged_port_start, +# belongs to the podman tasks, which this play does not run: Postgres listens on +# 5432 and publishes nothing.) +# +# podman_user is derived from the connection: this play carries no become, so +# the Quadlet units belong to whoever Ansible connects as. Requires gather_facts +# (set in playbook.yml). +podman_user: "{{ ansible_facts['user_id'] }}" +podman_data_dir: "{{ ansible_facts['user_dir'] }}/podman-data" +quadlet_dir: "{{ ansible_facts['user_dir'] }}/.config/containers/systemd" + +# --------------------------------------------------------------------------- +# PostgreSQL +# --------------------------------------------------------------------------- +# The name the rest of the repo already uses for this container in its +# `postgres_container` vars, and the name every app resolves over the network. +postgres_container: postgres +postgres_unit: "{{ postgres_container }}.service" + +# The major version is the one knob, because the image tag and the PGDATA +# directory have to agree. +# +# 18 is deliberate. monoco runs 17.5 and the data is brought across with a +# logical dump — a PGDATA copy cannot cross a major version. See +# migrate-from-monoco.md for the one-time move. +# +# `-alpine` still floats within 18.x, which is what you want for security +# patches; pin a digest here if the image itself must be immutable. +postgres_major: '18' + +# Fully qualified on purpose: rootless Podman resolves short names against +# unqualified-search-registries, and there is no TTY to answer a prompt when +# the systemd unit starts. +postgres_image: "docker.io/library/postgres:{{ postgres_major }}-alpine" + +# The uid the server actually writes as — not the image's configured user. The +# Alpine image creates the "postgres" account at uid/gid 70 ("70 is the standard +# uid/gid for postgres in Alpine"), and the entrypoint starts as root, chowns +# PGDATA to it, then drops privileges. keep-id maps that uid onto podman_user, +# so PGDATA ends up owned by podman_user on the host and never by a subuid. That +# is what keeps backups working without podman unshare or sudo. +postgres_uid: 70 +postgres_gid: 70 + +host_directory: "{{ podman_data_dir }}/{{ postgres_container }}" + +# Postgres 18 moved both the image's VOLUME and its default PGDATA: +# +# 17 Volume /var/lib/postgresql/data PGDATA /var/lib/postgresql/data/pgdata +# 18+ Volume /var/lib/postgresql PGDATA /var/lib/postgresql//docker +# +# So the mount is now the parent directory and the version lives in the path. +# That is what the 18 entrypoint expects: it refuses to start — "there appears +# to be PostgreSQL data in ..." — if it finds a cluster in the old layout, and +# the versioned subdirectory is what makes a later `pg_upgrade --link` work +# without mount-boundary trouble. +postgres_mount: /var/lib/postgresql + +# Everything under the mount that identifies this cluster, kept as one value so +# the container path and the host path cannot drift apart. +postgres_versioned_subdir: "{{ postgres_major }}/docker" + +pgdata: "{{ postgres_mount }}/{{ postgres_versioned_subdir }}" +pgdata_host_dir: "{{ host_directory }}/{{ postgres_versioned_subdir }}" + +# Secrets do not go in the unit file: it is world-readable, and `podman inspect` +# prints the environment anyway. They go in a 0600 file the unit reads with +# EnvironmentFile=. +postgres_env_file: "{{ host_directory }}/postgres.env" + +# Quadlet drop-in directory for the Postgres unit. Services attach themselves +# here; see the "How services attach themselves" section of readme.md. +# Postgres itself never lists its app networks. +postgres_dropin_dir: "{{ quadlet_dir }}/{{ postgres_container }}.container.d" + +# Credentials. This is the placeholder the app playbooks already embed in their +# db_url / POSTGRES_PASSWORD, kept as-is so a migrated app connects unchanged. +# monoco's cluster uses the same one, so the dump restores and the apps connect +# with no password reset. Replace it here and in those playbooks together when +# the fleet moves to a real secret. pg_user: postgres pg_password: '123456' -data_location: /var/lib/postgresql/data +pg_database: postgres -host_directory: postgresql +# The first start runs initdb, and a database wants time to shut down cleanly. +postgres_timeout_start: 120 +postgres_timeout_stop: 60