Run Postgres as a rootless Podman Quadlet, and move the 17 cluster across

Replaces the docker_container playbook, which published 5432 on 0.0.0.0 and kept
its data in /var/lib/postgresql. Postgres is now the base Quadlet unit, the same
shape as Caddy: it knows nothing about its clients, and a service attaches itself
by dropping a file into postgres.container.d/ via postgres/roles/attach. Nothing
here changes when a service is added or removed.

Pinned to 18, which is two decisions rather than one tag. Postgres 18 moved both
the image's VOLUME (/var/lib/postgresql/data -> /var/lib/postgresql) and its
default PGDATA (/var/lib/postgresql/data/pgdata -> /var/lib/postgresql/18/docker),
and the 18 entrypoint refuses to start when it finds a cluster in the old layout.
So postgres_major is the single knob: it picks the image tag and the PGDATA path
together, and the mount is now the parent directory. That also settles the
migration: a PGDATA copy cannot cross a major version, so monoco's 17.5 cluster
comes across as a pg_dumpall.

Three things that bit us and are worth keeping:

- PGDATA and its version directory are created by the role, not left to the
  container. The entrypoint runs as container-root, which under keep-id is a
  subuid on the host, and it only chowns PGDATA and its contents -- so
  ~/podman-data/postgres/18 would have been stranded subuid-owned, and the host
  user could not even rm -rf it without podman unshare.

- NetworkAlias= applies to every network the container joins, which is what a
  database hub wants, but it is one fixed value in a shared drop-in rather than
  per-service: two services writing their own would make Quadlet pass
  --network-alias twice. It is not in postgres.container itself because on a
  fresh host that unit has no Network= at all.

- The credentials live in a 0600 file read with EnvironmentFile=, not in
  Environment= lines: the unit is world-readable in ~/.config.

The play no longer uses become, and does not apply the rootless-Podman
prerequisites -- those are the shared task file at caddy/roles/podman, and the
three variables they need are duplicated at the top of vars/default.yml.

migrate-from-monoco.md records the one-time 17.5 -> 18.6 move: the dump, transfer
and restore commands, the checks that prove the restore is complete (per-database
table counts against the source), and the verified result.

The dead nginx/php-fpm templates in postgres/files/ are removed.

postgres/roles/attach/tasks/main.yaml

@@ -0,0 +1,93 @@
+---
+# Attach a service's network to Postgres.
+#
+# Postgres is the *base* unit and knows nothing about its clients, the same way
+# Caddy knows nothing about its upstreams. A service that needs a database adds
+# itself by dropping files next to postgres.container, and Quadlet merges them in.
+#
+# Include this from a service playbook:
+#
+#   - name: Attach this service to Postgres
+#     ansible.builtin.include_tasks: "{{ playbook_dir }}/../postgres/roles/attach/tasks/main.yaml"
+#     vars:
+#       network_name: "{{ network_name }}"
+#
+# The including play must:
+#
+#   1. have created {{ network_name }}.network already — caddy/roles/register
+#      does that, and every service here that needs a database is also behind
+#      Caddy, so its register include comes first;
+#   2. run the `podman` role (or include its task file and define the same four
+#      vars), for podman_user and quadlet_dir;
+#   3. define a "Restart Postgres" handler — see
+#      roles/postgres/handlers/main.yaml for the exact snippet; and
+#   4. flush handlers before starting the app, on a first attach. This include
+#      notifies that restart, so by default it runs at the end of the play —
+#      after the app has already started and failed to resolve `db`. A
+#      `meta: flush_handlers` immediately after this include is enough; see
+#      tandoor/roles/postgres for the working example.
+#
+# It runs as the connection user, which must be the rootless podman_user.
+#
+# A service may instead write these two files from its own role; the bytes have
+# to match this file exactly, or the two will overwrite each other on every run.
+
+- name: "Check that the {{ network_name }} network unit exists"
+  ansible.builtin.stat:
+    path: "{{ quadlet_dir }}/{{ network_name }}.network"
+  register: postgres_attach_network
+
+# A drop-in naming a .network file that does not exist makes Quadlet fail to
+# generate the whole postgres unit, with a message that does not mention the
+# drop-in at all. Fail here instead, where the cause is obvious.
+- name: "Fail if {{ network_name }} has no network unit"
+  ansible.builtin.assert:
+    that: postgres_attach_network.stat.exists
+    fail_msg: >-
+      {{ quadlet_dir }}/{{ network_name }}.network does not exist, so attaching
+      it to Postgres would break the postgres unit. Create the network first
+      (caddy/roles/register does it), then include this file.
+
+- name: Ensure the Postgres Quadlet drop-in directory exists
+  ansible.builtin.file:
+    path: "{{ quadlet_dir }}/{{ postgres_container | default('postgres') }}.container.d"
+    state: directory
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0755'
+
+- name: "Attach {{ network_name }} to Postgres"
+  ansible.builtin.copy:
+    dest: >-
+      {{ quadlet_dir }}/{{ postgres_container | default('postgres') }}.container.d/50-{{ network_name }}.conf
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0644'
+    content: |
+      # Added by the {{ network_name }} playbook. Do not edit by hand.
+      #
+      # Deleting this file removes the network from {{ postgres_container | default('postgres') }} on
+      # the next daemon-reload; nothing else changes.
+      [Container]
+      Network={{ network_name }}.network
+  notify: Restart Postgres
+
+# NetworkAlias= applies to every network the container joins, which is exactly
+# right for a hub: every app in this repo reaches postgres as `db` (nextcloud,
+# gitea, tandoor and outline use the alias; the rest use the container name).
+# It is a fixed value in one shared file, not a per-service option, so the same
+# bytes are written no matter how many services attach — two services writing
+# their own aliases would make Quadlet pass --network-alias twice.
+- name: "Give {{ postgres_container | default('postgres') }} the shared 'db' alias"
+  ansible.builtin.copy:
+    dest: >-
+      {{ quadlet_dir }}/{{ postgres_container | default('postgres') }}.container.d/10-network-aliases.conf
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0644'
+    content: |
+      # Shared container aliases for {{ postgres_container | default('postgres') }}.
+      # Managed by Ansible (the service playbooks that use it). Do not edit.
+      [Container]
+      NetworkAlias=db
+  notify: Restart Postgres

postgres/roles/postgres/handlers/main.yaml

@@ -0,0 +1,31 @@
+---
+# All handlers run as the connection user ({{ podman_user }}), so systemctl
+# --user works without setting XDG_RUNTIME_DIR.
+
+- name: Reload systemd user units
+  ansible.builtin.command: systemctl --user daemon-reload
+  changed_when: true
+  listen: Reload systemd user units
+
+- name: Restart Postgres
+  ansible.builtin.systemd_service:
+    name: "{{ postgres_unit }}"
+    scope: user
+    state: restarted
+    daemon_reload: true
+  listen: Restart Postgres
+
+# Reference implementation for service playbooks. A service that attaches its
+# network with postgres/roles/attach must notify "Restart Postgres", and defines
+# its own copy of this handler, e.g.:
+#
+#   - name: Restart Postgres
+#     ansible.builtin.systemd_service:
+#       name: postgres.service
+#       scope: user
+#       state: restarted
+#       daemon_reload: true
+#     listen: Restart Postgres
+#
+# A new drop-in changes the generated unit, so Postgres has to be recreated to
+# join the network — Quadlet has no live equivalent of `podman network connect`.

postgres/roles/postgres/tasks/main.yaml

@@ -0,0 +1,155 @@
+---
+# Start PostgreSQL
+#
+# Everything here runs as {{ podman_user }} — the connection user. No become,
+# no XDG_RUNTIME_DIR: SSH gave us a real PAM session, so systemd's user manager
+# is reachable via the standard environment.
+
+- name: Ensure the Postgres data directory exists
+  ansible.builtin.file:
+    path: "{{ host_directory }}"
+    state: directory
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0755'
+
+# PGDATA and its version directory are created here rather than left to the
+# container's own `mkdir -p`. The entrypoint runs as container-root, which under
+# keep-id is a subuid on the host, and it only chowns PGDATA and its contents —
+# so the version directory would be left owned by 165536 and the host user could
+# not even `rm -rf` it without podman unshare. Creating the whole chain up front
+# keeps every directory in the data root owned by podman_user.
+#
+# Empty PGDATA is exactly right: the 18 entrypoint reads that as "run initdb".
+- name: Ensure the versioned data directory exists
+  ansible.builtin.file:
+    path: "{{ pgdata_host_dir | dirname }}"
+    state: directory
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0755'
+
+- name: Ensure PGDATA exists
+  ansible.builtin.file:
+    path: "{{ pgdata_host_dir }}"
+    state: directory
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0700'
+
+- name: Ensure the Quadlet drop-in directory exists
+  ansible.builtin.file:
+    path: "{{ postgres_dropin_dir }}"
+    state: directory
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0755'
+
+# The unit file is 0644 in ~/.config, so the password does not belong in it.
+# This file is 0600 and is read by the container through EnvironmentFile=.
+- name: Write the Postgres environment file
+  ansible.builtin.template:
+    src: postgres.env.j2
+    dest: "{{ postgres_env_file }}"
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0600'
+  notify: Restart Postgres
+
+- name: Install the base Postgres Quadlet unit
+  ansible.builtin.template:
+    src: postgres.container.j2
+    dest: "{{ quadlet_dir }}/{{ postgres_container }}.container"
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0644'
+  notify:
+    - Reload systemd user units
+    - Restart Postgres
+
+# The wait is part of the block on purpose. `Restart=always` means systemd
+# reports the unit active even while the container crash-loops — a wrong
+# keep-id shows up as an entrypoint permission error over and over — so
+# is-active alone would call a broken database healthy.
+- name: Start Postgres and wait for it to accept connections
+  block:
+    - name: Start Postgres
+      ansible.builtin.systemd_service:
+        name: "{{ postgres_unit }}"
+        scope: user
+        state: started
+        daemon_reload: true
+
+    # pg_isready is the server's own answer, and it fails while initdb is still
+    # running on a fresh data directory (it exits 2, so the retry loop holds).
+    - name: Wait for Postgres to accept connections
+      ansible.builtin.command: >-
+        podman exec {{ postgres_container }}
+        pg_isready -U {{ pg_user }} -d {{ pg_database }}
+      register: postgres_ready
+      until: postgres_ready.rc == 0
+      retries: 12
+      delay: 5
+      changed_when: false
+  rescue:
+    # systemd reports "control process exited with error code" and nothing
+    # else, which is useless for the usual failures (a bad uid mapping, an
+    # unwritable PGDATA, a missing image). Pull the real reason out.
+    - name: Collect Postgres diagnostics
+      ansible.builtin.command: "{{ item }}"
+      loop:
+        - "systemctl --user status {{ postgres_unit }} --no-pager"
+        - "journalctl --user -u {{ postgres_unit }} --no-pager -n 40"
+      register: postgres_diag
+      changed_when: false
+      failed_when: false
+
+    - name: Show Postgres diagnostics
+      ansible.builtin.debug:
+        msg: "{{ postgres_diag.results | map(attribute='stdout') | list }}"
+
+    - name: Fail
+      ansible.builtin.fail:
+        msg: "Postgres did not come up — see the diagnostics above."
+
+# Quadlet honours [Install] WantedBy= when it generates the unit, so
+# `systemctl enable` is neither needed nor possible on a generated unit.
+- name: Verify the Postgres unit is active
+  ansible.builtin.command: "systemctl --user is-active {{ postgres_unit }}"
+  register: postgres_active
+  changed_when: false
+  failed_when: postgres_active.stdout != 'active'
+
+- name: List the services attached to Postgres
+  ansible.builtin.find:
+    paths: "{{ postgres_dropin_dir }}"
+    patterns: "*.conf"
+  register: postgres_dropins
+
+- name: Report the networks Postgres is attached to
+  ansible.builtin.command: >-
+    podman inspect {{ postgres_container }}
+    --format '{% raw %}{{ range $k, $v := .NetworkSettings.Networks }}{{ $k }} {{ end }}{% endraw %}'
+  register: postgres_networks_now
+  changed_when: false
+  failed_when: false
+
+- name: Show attached services
+  ansible.builtin.debug:
+    msg: >-
+      Postgres networks:
+      {{ postgres_networks_now.stdout | default('(container not running)') }}.
+      Attached by:
+      {{ postgres_dropins.files | map(attribute='path') | map('basename') | join(', ')
+         | default('(nothing yet)', true) }}.
+
+# Say this out loud rather than let it be discovered by a failing app: with no
+# drop-in, Postgres has no Network= at all, so it sits on the default Podman
+# network and no app network can resolve it.
+- name: Note that no service has attached yet
+  ansible.builtin.debug:
+    msg: >-
+      No service has attached to Postgres. It is on the default Podman network
+      only, and no app can resolve it until a service playbook includes
+      postgres/roles/attach (see readme.md).
+  when: postgres_dropins.matched == 0

postgres/roles/postgres/templates/postgres.container.j2

@@ -0,0 +1,50 @@
+# PostgreSQL — rootless Podman Quadlet unit.
+# Managed by Ansible (postgres/roles/postgres). Do not edit by hand.
+#
+# Postgres deliberately does NOT list the app networks it serves. Each service
+# attaches itself by dropping a file into:
+#
+#   {{ postgres_dropin_dir }}/
+#
+# which Quadlet merges into this unit. Removing a service therefore never means
+# editing this file. See readme.md, "How services attach themselves".
+
+[Unit]
+Description=PostgreSQL database
+Documentation=https://hub.docker.com/_/postgres
+After=network-online.target
+Wants=network-online.target
+
+[Container]
+ContainerName={{ postgres_container }}
+Image={{ postgres_image }}
+
+# Map the container's postgres account (uid {{ postgres_uid }}) onto
+# {{ podman_user }} on the host, so PGDATA stays owned by {{ podman_user }} and
+# never by a subuid. The official entrypoint starts as root, chowns PGDATA to
+# uid {{ postgres_uid }}, then drops privileges — so everything it writes lands
+# on {{ podman_user }}, and plain tar / pg_dump backups keep working.
+UserNS=keep-id:uid={{ postgres_uid }},gid={{ postgres_gid }}
+
+# No PublishPort, on purpose. The Docker-era playbook published 5432 on
+# 0.0.0.0 (and a UFW rule was never there to narrow it); here Postgres is
+# reachable only from the app networks that attach themselves to it.
+#
+# The mount is the image's VOLUME, which in 18 is the *parent* directory — the
+# major version is in the path instead ({{ pgdata }}). Mounting
+# /var/lib/postgresql/data here instead would make the 18 entrypoint refuse to
+# start, because that path is where it looks for an un-upgraded 17 cluster.
+Volume={{ host_directory }}:{{ postgres_mount }}
+
+# POSTGRES_USER / POSTGRES_PASSWORD / POSTGRES_DB / PGDATA live in this 0600
+# file rather than in Environment= lines here: this unit is world-readable.
+EnvironmentFile={{ postgres_env_file }}
+
+[Service]
+Restart=always
+# The first start runs initdb, and a database needs time to shut down cleanly.
+TimeoutStartSec={{ postgres_timeout_start }}
+TimeoutStopSec={{ postgres_timeout_stop }}
+
+[Install]
+WantedBy=default.target

postgres/roles/postgres/templates/postgres.env.j2

@@ -0,0 +1,10 @@
+# Environment for postgres.container.
+# Managed by Ansible (postgres/roles/postgres). Do not edit by hand.
+#
+# Mode 0600 on purpose: the Quadlet unit that reads this file is world-readable,
+# so the password must not live there. Note that `podman inspect` still prints
+# the resulting environment to anyone who can run podman as this user.
+POSTGRES_USER={{ pg_user }}
+POSTGRES_PASSWORD={{ pg_password }}
+POSTGRES_DB={{ pg_database }}
+PGDATA={{ pgdata }}