Run Tandoor as a rootless Podman Quadlet

Replaces the docker_container role with a Quadlet systemd user unit, and moves
state from /var/lib/tandoor to ~/podman-data/tandoor. Tandoor's entrypoint
starts as root and drops privileges, but root is what ends up owning the data,
so keep-id maps container root onto the connecting user and everything stays
owned by tanshu rather than a subuid.

The play does not run the rootless-Podman prerequisites: they are the shared
task file at caddy/roles/podman, and a host that has run caddy/playbook.yml has
them. Both integrations go through the shared includes rather than local copies
-- caddy/roles/register for the network and the Caddy drop-in, postgres/roles/
attach for the database -- so this app writes the same drop-in bytes as every
other service and a new service never means editing Caddy or Postgres.

Three ordering details are load-bearing:

- The register include runs in pre_tasks, before the unit is started. Quadlet
  fails unit *generation*, not merely startup, when a referenced .network file
  does not exist yet, so starting first reports "unit not found".
- roles/postgres flushes handlers immediately after attaching. The restart that
  joins Postgres to this network is a handler, so by default it runs at the end
  of the play -- after the app has already started and failed to resolve `db`.
- The role waits for a real HTTP response from inside the container instead of
  trusting systemctl is-active. The entrypoint runs Django migrations before
  gunicorn answers, and until it does the image's own nginx returns 502, which
  Caddy relays verbatim -- complete with `server: nginx`, which reads as a stray
  reverse proxy that is not actually there.

Also drops the dead nginx.conf.j2 and the network role, both superseded by the
shared includes, and moves .env into the role's templates/ as tandoor.env.j2,
rendered at mode 0600 and read through EnvironmentFile=.

tandoor/roles/caddy/handlers/main.yaml

@@ -1,7 +1,31 @@
 ---
-# Handlers for Nginx role
-- name: Restart caddy container
-  docker_container:
-    name: "{{ caddy_container }}"
-    state: started
-    restart: true
+# Caddy is its own playbook (caddy/playbook.yml) and a rootless Quadlet unit
+# there. These handlers run as the connection user, so `systemctl --user` and
+# `podman exec` work without any XDG_RUNTIME_DIR juggling.
+
+# Notified by this role when the Caddyfile changes. A reload keeps existing
+# connections; a restart would briefly drop every hosted site.
+#
+# Note the config path: /etc/caddy/Caddyfile is the mount inside the container,
+# not {{ caddyfile_path }}, which is the host side of it.
+- name: Reload Caddy configuration
+  ansible.builtin.command: >-
+    podman exec {{ caddy_name }} caddy reload --config /etc/caddy/Caddyfile
+  changed_when: true
+  listen: "Reload Caddy"
+
+# Notified by caddy/roles/register when this service's network unit, or the
+# drop-in that attaches Caddy to it, is written. Quadlet has no live equivalent
+# of `docker network connect`, so the container has to be recreated — a brief
+# restart of every hosted site, not just this one. See caddy/readme.md,
+# "Trade-off worth knowing".
+#
+# Defined after the reload above so that when both are notified the cheap
+# config reload happens first.
+- name: Restart Caddy
+  ansible.builtin.systemd_service:
+    name: "{{ caddy_name }}.service"
+    scope: user
+    state: restarted
+    daemon_reload: true
+  listen: Restart Caddy

tandoor/roles/caddy/tasks/main.yaml

@@ -1,22 +1,22 @@
 ---
-- name: Read current Caddyfile
-  ansible.builtin.slurp:
-    path: "{{ caddyfile_path }}"
-  register: caddyfile_raw
+# Append this site's block to the shared Caddyfile.
+#
+# The Caddyfile is a bind mount into the Caddy container, so Caddy sees the
+# change as soon as the file is written; the handler only asks Caddy to re-read
+# it. That is a reload, not a restart, so the other hosted sites are untouched.
+#
+# blockinfile is idempotent on its own via the marker, so there is no need to
+# slurp the file and compare first — which also matters because slurp *fails*
+# when the Caddyfile does not exist yet, on a host where the Caddy playbook has
+# not run.
 
-- name: Decode Caddyfile content
-  set_fact:
-    caddyfile_content: "{{ caddyfile_raw['content'] | b64decode }}"
-
-- name: Check if snippet already exists
-  set_fact:
-    snippet_present: "{{ snippet_block in caddyfile_content }}"
-
-- name: Add snippet if missing
+- name: "Add the {{ host }} site block to the Caddyfile"
   ansible.builtin.blockinfile:
     path: "{{ caddyfile_path }}"
     marker: "# {mark} Ansible managed Caddy snippet for {{ host }}"
     block: "{{ snippet_block }}"
     create: yes
-  when: not snippet_present
-  notify: Restart caddy container
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0644'
+  notify: Reload Caddy

tandoor/roles/network/tasks/main.yaml

@@ -1,11 +0,0 @@
----
-- name: Ensure tandoor, caddy, postgres are on Docker network
-  docker_network:
-    name: "{{ docker_network }}"
-    state: present
-    connected:
-      - name: "{{ docker_container }}"
-      - name: "{{ postgres_container }}"
-        alias: db
-      - name: "{{ caddy_container }}"
-    appends: yes

tandoor/roles/postgres/handlers/main.yaml

@@ -0,0 +1,17 @@
+---
+# Postgres is its own playbook (postgres/playbook.yml) and a rootless Quadlet
+# unit there. This handler runs as the connection user, so `systemctl --user`
+# works without setting XDG_RUNTIME_DIR.
+#
+# Required by postgres/roles/attach, which notifies it whenever this app's
+# drop-in changes: a new drop-in changes the generated unit, so Postgres must be
+# recreated to join the network — Quadlet has no live equivalent of
+# `podman network connect`. That is a few seconds of downtime for every attached
+# app, not just this one. See postgres/readme.md, "The trade-off".
+- name: Restart Postgres
+  ansible.builtin.systemd_service:
+    name: "{{ postgres_unit }}"
+    scope: user
+    state: restarted
+    daemon_reload: true
+  listen: Restart Postgres

tandoor/roles/postgres/tasks/main.yaml

@@ -0,0 +1,26 @@
+---
+# Attach this app's network to the shared Postgres unit.
+#
+# Postgres knows nothing about its clients, exactly like Caddy: a service joins
+# by dropping 50-tandoor.conf and the shared 10-network-aliases.conf next to
+# postgres.container. The canonical way to do that is the shared include below —
+# it writes the same bytes every service writes, asserts that tandoor.network
+# already exists, and notifies the "Restart Postgres" handler defined in this
+# role.
+#
+# This runs in the roles phase, which is after the register include in
+# playbook.yml's pre_tasks has created tandoor.network.
+
+- name: Attach Tandoor to Postgres
+  ansible.builtin.include_tasks: "{{ playbook_dir }}/../postgres/roles/attach/tasks/main.yaml"
+
+# On a first attach both drop-ins are new, so Postgres has to be recreated to
+# join tandoor_net — and that restart is a *handler*, which by default runs at
+# the end of the play, after the tandoor role has already started the app. The
+# app would then come up before `db` resolves, fail its own is-active check, and
+# fail the play on a machine where nothing is actually wrong.
+#
+# Flushing applies the new drop-in before the app starts. On later runs the
+# drop-ins are unchanged, nothing is pending, and this is a no-op.
+- name: Apply the Postgres network change before starting Tandoor
+  ansible.builtin.meta: flush_handlers

tandoor/roles/tandoor/handlers/main.yaml

@@ -0,0 +1,16 @@
+---
+# All handlers run as the connection user ({{ podman_user }}), so systemctl
+# --user works without setting XDG_RUNTIME_DIR.
+
+- name: Reload systemd user units
+  ansible.builtin.command: systemctl --user daemon-reload
+  changed_when: true
+  listen: Reload systemd user units
+
+- name: Restart Tandoor
+  ansible.builtin.systemd_service:
+    name: "{{ unit_name }}"
+    scope: user
+    state: restarted
+    daemon_reload: true
+  listen: Restart Tandoor

tandoor/roles/tandoor/tasks/main.yaml

@@ -1,42 +1,121 @@
 ---
-# Tasks for a Tandoor.dev role
-- name: Pull image
-  docker_image:
-    name: "{{ docker_image }}"
-    source: pull
-    state: present
-    force_source: yes
+# Tandoor Recipes as a rootless Podman Quadlet unit.
+#
+# Everything here runs as {{ podman_user }} — the connection user. No become, no
+# XDG_RUNTIME_DIR: the SSH connection gave us a real PAM session, so systemd's
+# user manager is reachable through the standard environment.
 
-- name: Ensure Host Directory exists
-  file:
-      path: "/var/lib/{{ host_directory }}"
-      state: directory
-      mode: 0755
+- name: Ensure the Tandoor data directories exist
+  ansible.builtin.file:
+    path: "{{ host_directory }}/{{ item }}"
+    state: directory
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0755'
+  loop:
+    # mediafiles is the irreplaceable one; the other two are regenerated by the
+    # app and are bind-mounted anyway because the image expects them to exist.
+    - mediafiles
+    - staticfiles
+    - externalfiles
 
-- name: Upload the .env file
-  template:
-    src: "files/.env"
-    dest: "/var/lib/{{ host_directory }}/.env"
+# The app's secrets and database coordinates are consumed by systemd
+# (EnvironmentFile= in the Quadlet unit), not by Ansible, so this has to be
+# written before the unit starts. Mode 0600: it holds SECRET_KEY and the
+# postgres password.
+- name: Write the environment file
+  ansible.builtin.template:
+    src: tandoor.env.j2
+    dest: "{{ host_directory }}/.env"
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0600'
+  notify: Restart Tandoor
 
+- name: Install the Tandoor Quadlet unit
+  ansible.builtin.template:
+    src: tandoor.container.j2
+    dest: "{{ quadlet_dir }}/{{ container_name }}.container"
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0644'
+  notify:
+    - Reload systemd user units
+    - Restart Tandoor
 
-# - name: Ensure Docker network exists
-#   docker_network:
-#     name: "{{ docker_network }}"
-#     connected:
-#       - name: "{{ postgres_container }}"
-#         alias: db
-#       - name: "{{ caddy_container }}"
+- name: Start Tandoor
+  block:
+    - name: Start Tandoor
+      ansible.builtin.systemd_service:
+        name: "{{ unit_name }}"
+        scope: user
+        state: started
+        daemon_reload: true
+  rescue:
+    # systemd reports "control process exited with error code" and nothing
+    # else, which is useless for the usual failures (an unattached network, an
+    # unreachable database, a bad volume). Pull the real reason out.
+    - name: Collect Tandoor diagnostics
+      ansible.builtin.command: "{{ item }}"
+      loop:
+        - "systemctl --user status {{ unit_name }} --no-pager"
+        - "journalctl --user -u {{ unit_name }} --no-pager -n 40"
+        - "podman logs --tail 40 {{ container_name }}"
+      register: tandoor_diag
+      changed_when: false
+      failed_when: false
 
-- name: Create container
-  docker_container:
-    name: "{{ docker_container }}"
-    image: "{{ docker_image }}"
-    state: started
-    restart_policy: "unless-stopped"
-    env_file: "/var/lib/{{ host_directory }}/.env"
-    volumes:
-      - "/var/lib/{{ host_directory }}/staticfiles:/opt/recipes/staticfiles"
-      - "/var/lib/{{ host_directory }}/mediafiles:/opt/recipes/mediafiles"
-      - "/var/lib/{{ host_directory }}/externalfiles:/opt/recipes/externalfiles"
-    networks:
-      - name: "{{ docker_network }}"
+    - name: Show Tandoor diagnostics
+      ansible.builtin.debug:
+        msg: "{{ tandoor_diag.results | map(attribute='stdout') | list }}"
+
+    - name: Fail
+      ansible.builtin.fail:
+        msg: "Tandoor did not start — see the diagnostics above."
+
+# Quadlet honours [Install] WantedBy= when it generates the unit, so
+# `systemctl --user enable` is neither needed nor possible on a generated unit.
+- name: Verify Tandoor is running
+  ansible.builtin.command: "systemctl --user is-active {{ unit_name }}"
+  register: tandoor_active
+  changed_when: false
+  failed_when: tandoor_active.stdout != 'active'
+
+# `db` is the alias the environment file actually asks for; the container name
+# is checked too because it is what the Quadlet network gives us for free.
+- name: Confirm Tandoor can resolve its database
+  ansible.builtin.command: >-
+    podman exec {{ container_name }} getent hosts {{ item }}
+  loop:
+    - "{{ postgres_container }}"
+    - db
+  register: tandoor_db_lookup
+  changed_when: false
+  failed_when: false
+
+- name: Report database resolution
+  ansible.builtin.debug:
+    msg: >-
+      {{ item.item }} resolves as
+      {{ item.stdout | default('(no answer)') }}
+  loop: "{{ tandoor_db_lookup.results }}"
+
+# `is-active` says the container is running; it does not say the app can serve.
+# Tandoor's entrypoint runs Django migrations before gunicorn answers, and until
+# it does the image's own nginx returns 502 to Caddy — which Caddy relays as-is,
+# complete with `server: nginx`. That is exactly the confusing "nginx 502" a
+# first deploy shows after a play that reported success.
+#
+# python3 is used rather than curl or wget because this is a Django image: the
+# interpreter is guaranteed present, the others are not. A 502 raises HTTPError,
+# so the retry loop holds until a real page comes back.
+- name: Wait for Tandoor to serve HTTP
+  ansible.builtin.command: >-
+    podman exec {{ container_name }}
+    python3 -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1/', timeout=5)"
+  register: tandoor_http
+  until: tandoor_http.rc == 0
+  retries: 24
+  delay: 5
+  changed_when: false
+  failed_when: tandoor_http.rc != 0

tandoor/roles/tandoor/templates/tandoor.container.j2

@@ -0,0 +1,43 @@
+# Tandoor Recipes — rootless Podman Quadlet unit.
+# Managed by Ansible (tandoor/roles/tandoor). Do not edit by hand.
+#
+# This unit joins {{ network_name }}.network, which is created by
+# caddy/roles/register (the same include that attaches Caddy to it). The
+# postgres container is joined to that network by postgres/roles/attach, which
+# is what makes POSTGRES_HOST=db in the environment file resolve.
+
+[Unit]
+Description=Tandoor Recipes
+Documentation=https://tandoor.dev/
+After=network-online.target
+Wants=network-online.target
+
+[Container]
+ContainerName={{ container_name }}
+Image={{ image }}
+
+# Tandoor's entrypoint starts as root and drops privileges, but the uid that
+# actually owns the data is root's (see docs/podman-compat-audit.md §3.2).
+# Mapping container root onto {{ podman_user }} keeps {{ host_directory }}
+# readable and writable by both the container and the host user, and leaves no
+# subuid-owned files to trip over backups.
+UserNS=keep-id:uid=0,gid=0
+
+# SECRET_KEY and the postgres coordinates live here. Read by systemd, not by
+# Ansible, so the file has to exist before this unit starts.
+EnvironmentFile={{ host_directory }}/.env
+
+Network={{ network_name }}.network
+
+Volume={{ host_directory }}/staticfiles:/opt/recipes/staticfiles
+Volume={{ host_directory }}/mediafiles:/opt/recipes/mediafiles
+Volume={{ host_directory }}/externalfiles:/opt/recipes/externalfiles
+
+[Service]
+Restart=always
+# The entrypoint runs Django migrations on every start, so allow generously for
+# a large database before systemd gives up.
+TimeoutStartSec=600
+
+[Install]
+WantedBy=default.target

tandoor/roles/tandoor/templates/tandoor.env.j2

@@ -0,0 +1,8 @@
+SECRET_KEY=7ukvcoUnuyhPK6Nds3KbwG+3zeYuJGTDyi0ohFJqMwgjrdwBxn%
+DB_ENGINE=django.db.backends.postgresql
+POSTGRES_HOST=db
+POSTGRES_PORT=5432
+POSTGRES_USER=postgres
+POSTGRES_PASSWORD=123456
+POSTGRES_DB=tandoor
+ALLOWED_HOSTS={{ host }},localhost,127.0.0.1