Run Nextcloud as a rootless Podman Quadlet

Replaces the docker_container playbook, which bind-mounted /var/lib/nextcloud at
/var/www/html on monoco and ran cron.php from root's crontab. Nextcloud now runs
on emily behind the shared Caddy, with its database in the shared Postgres and
its cache in the shared Valkey, each attached the way every other service
attaches to them.

/var/www/html is the unusual part: it holds the code as well as the state --
version.php, core/, lib/ and apps/ alongside config/config.php, data/ and
custom_apps/ -- and the entrypoint's version check reads version.php from the
mount. That is why the migration copies the whole tree rather than only the three
irreplaceable directories (docs/backup-plan.md 2.1 lists the rest as regenerable,
and they are, but only from an image that is the same version or newer). It is
also what makes an image bump an in-place upgrade: a newer image rsyncs its code
in and runs `occ upgrade` on first start, and an older one refuses to start at
all, so the version on both hosts is worth checking before the copy.

UserNS=keep-id:uid=33,gid=33. 33 is www-data, the uid the server actually writes
as -- `docker exec nextcloud id -u` reports 0, because the entrypoint drops
privileges after starting (docs/podman-compat-audit.md 3.2). 33 is also what the
data is owned by on monoco, so the copy needs no chown pass: extracting as the
connection user is already the ownership the container expects. One file is
created by container-root and never chowned, the entrypoint's
nextcloud-init-sync.lock, so the role pre-creates it -- the entrypoint truncates
that file rather than recreating it, so its owner then never changes, and the
webroot ends up with no subuid-owned file in it.

The Valkey attach is not optional. config.php carries memcache.locking pointed at
Redis, written by the image from REDIS_HOST, and the same variable rewrites PHP's
session-handler config on every start -- so an unresolvable `redis` does not just
make things slower, it fails uploads at write time. Hence the `redis` alias
rather than the container name, and hence roles/valkey flushing handlers before
the app starts, the same as roles/postgres.

cron.php moves from root's crontab to a systemd user timer, because root cannot
`podman exec` into another user's rootless containers and there is no container
user to own a crontab (docs/podman-migration.md 8b). Unlike the Quadlet units
this pair is plain and is genuinely enabled, which is what brings it back after a
reboot; journalctl --user -u nextcloud-cron shows every run, which the crontab
never did.

Readiness is three gates, because no one of them is sufficient: is-active (which
Restart=always satisfies even while crash-looping), `occ status` reporting
installed, and status.php served over loopback by Apache rather than by the PHP
CLI. The second is followed by a wait for maintenance mode to clear rather than
an assert: maintenance mode is on for the whole of a first-start `occ upgrade`,
so asserting it off would fail a healthy deploy halfway through. The case that
never clears is a config.php copied with `maintenance => true`, which
docs/backup-plan.md 3.4 tells an operator to set before a consistent dump, and
which is invisible from outside because status.php is exempt from maintenance
mode -- so the play names it and prints the command that clears it.

Register and attach ordering is the same as the other service playbooks: register
runs in pre_tasks, because the unit references nextcloud.network and Quadlet
fails unit generation when that file does not exist, and both attaches flush
handlers before the app starts, which is also what pulls Caddy onto the network.

The Caddy snippet is unchanged, HSTS header included; only caddyfile_path moves,
from /var/lib/caddy/conf/Caddyfile to ~/podman-data/caddy/conf/Caddyfile. The
Docker-era roles/network is deleted rather than ported: it existed to attach an
already-created container to a network with the `redis` alias, and
podman_network has no connected:/appends: equivalent
(docs/podman-compat-audit.md 3.3). Nothing is published host-side; Caddy reaches
nextcloud:80 over nextcloud_net.

The rootless-Podman prerequisites are not applied here: they are the shared task
file at caddy/roles/podman, and a host that has run caddy/playbook.yml has them.
valkey/playbook.yml has to have been run first, or the first Restart Valkey
handler fails against a unit that does not exist yet.

nextcloud/roles/nextcloud/handlers/main.yaml

@@ -0,0 +1,21 @@
+---
+# All handlers run as the connection user ({{ podman_user }}), so systemctl
+# --user works without setting XDG_RUNTIME_DIR.
+
+- name: Reload systemd user units
+  ansible.builtin.command: systemctl --user daemon-reload
+  changed_when: true
+  listen: Reload systemd user units
+
+# A changed environment file or unit file changes the generated unit, so
+# Nextcloud is recreated. Nothing is lost by that: config.php, data/ and
+# custom_apps/ are on the bind mount and the database is in Postgres. It is still
+# a few seconds of downtime for the site, and the entrypoint re-runs its version
+# check — including `occ upgrade` if the image has moved on — on the way back up.
+- name: Restart Nextcloud
+  ansible.builtin.systemd_service:
+    name: "{{ unit_name }}"
+    scope: user
+    state: restarted
+    daemon_reload: true
+  listen: Restart Nextcloud

nextcloud/roles/nextcloud/tasks/main.yaml

@@ -1,36 +1,267 @@
 ---
-# Tasks for Nextcloud role
-- name: Pull Nextcloud image
-  docker_image:
-    name: "{{ container_image }}"
-    source: pull
-    force_source: yes
+# Nextcloud as a rootless Podman Quadlet unit.
+#
+# Everything here runs as {{ podman_user }} — the connection user. No become, no
+# XDG_RUNTIME_DIR: the SSH connection gave us a real PAM session, so systemd's
+# user manager is reachable through the standard environment.
 
-- name: Ensure Nextcloud Directory exists
-  file:
-      path: "/var/lib/{{ host_directory }}"
-      state: directory
-      mode: 0755
+- name: Ensure the Nextcloud webroot directory exists
+  ansible.builtin.file:
+    path: "{{ host_directory }}"
+    state: directory
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0755'
 
-- name: Create Nextcloud container
-  docker_container:
-    name: "{{ container_name }}"
-    image: "{{ container_image }}"
+# The entrypoint takes an flock on this path using a root-owned shell redirect
+# (`) 9> /var/www/html/nextcloud-init-sync.lock`, in the nextcloud/docker
+# entrypoint). Creating it here means it belongs to {{ podman_user }} and stays
+# that way: the redirect *truncates* the file, it does not recreate it, so its
+# owner never changes.
+#
+# Left to the container, the file would be created by container-root — a subuid
+# under keep-id — and would be the one thing in the webroot not owned by
+# {{ podman_user }}.
+#
+# `force: false` leaves an existing file completely alone, which is what makes
+# this idempotent and what stops it failing on a file a previous run stranded as
+# a subuid (a host user cannot chown a file it does not own).
+- name: Pre-create the entrypoint's init lock file
+  ansible.builtin.copy:
+    dest: "{{ host_directory }}/nextcloud-init-sync.lock"
+    content: ""
+    force: false
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0644'
+
+# The app's database and cache coordinates are consumed by systemd
+# (EnvironmentFile= in the Quadlet unit), not by Ansible, so this has to be
+# written before the unit starts. Mode 0600: POSTGRES_PASSWORD is in it.
+- name: Write the environment file
+  ansible.builtin.template:
+    src: nextcloud.env.j2
+    dest: "{{ env_file }}"
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0600'
+  notify: Restart Nextcloud
+
+- name: Install the Nextcloud Quadlet unit
+  ansible.builtin.template:
+    src: nextcloud.container.j2
+    dest: "{{ quadlet_dir }}/{{ container_name }}.container"
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0644'
+  notify:
+    - Reload systemd user units
+    - Restart Nextcloud
+
+- name: Start Nextcloud
+  block:
+    - name: Start Nextcloud
+      ansible.builtin.systemd_service:
+        name: "{{ unit_name }}"
+        scope: user
+        state: started
+        daemon_reload: true
+  rescue:
+    # systemd reports "control process exited with error code" and nothing else,
+    # which is useless for the usual failures (an unattached network, an
+    # unreachable database, an unwritable webroot). Pull the real reason out.
+    - name: Collect Nextcloud diagnostics
+      ansible.builtin.command: "{{ item }}"
+      loop:
+        - "systemctl --user status {{ unit_name }} --no-pager"
+        - "journalctl --user -u {{ unit_name }} --no-pager -n 40"
+        - "podman logs --tail 40 {{ container_name }}"
+      register: nextcloud_diag
+      changed_when: false
+      failed_when: false
+
+    - name: Show Nextcloud diagnostics
+      ansible.builtin.debug:
+        msg: "{{ nextcloud_diag.results | map(attribute='stdout') | list }}"
+
+    - name: Fail
+      ansible.builtin.fail:
+        msg: "Nextcloud did not start — see the diagnostics above."
+
+# Quadlet honours [Install] WantedBy= when it generates the unit, so
+# `systemctl --user enable` is neither needed nor possible on a generated unit.
+- name: Verify Nextcloud is running
+  ansible.builtin.command: "systemctl --user is-active {{ unit_name }}"
+  register: nextcloud_active
+  changed_when: false
+  failed_when: nextcloud_active.stdout != 'active'
+
+# `is-active` says the container is running; it does not say the app can serve.
+# The entrypoint rsyncs the image's code into the webroot and, when the copied
+# version.php is older than the image, runs `occ upgrade` against
+# {{ pg_database }} before Apache answers — and `Restart=always` means systemd
+# reports an active unit even while that crash-loops.
+#
+# occ is the app's own answer and is always present. It has to run as www-data
+# (uid {{ app_uid }}): occ refuses to run as a user other than the one that owns
+# config.php, which under keep-id is what the mount is owned by.
+#
+# The retry budget of the first wait matches the unit's TimeoutStartSec, because
+# a first run after an image bump really can spend minutes in the database
+# upgrade.
+#
+# The status JSON is matched after stripping spaces rather than with a regex, so
+# both the compact form and a pretty-printed one match, and there is no `\s` to
+# escape (an invalid escape in a Jinja string literal is only a warning today and
+# may not stay one).
+- name: Wait for Nextcloud to become ready
+  block:
+    - name: Wait for Nextcloud to report itself installed
+      ansible.builtin.command: >-
+        podman exec -u www-data {{ container_name }}
+        php occ status --output=json
+      register: nextcloud_status
+      until: >-
+        nextcloud_status.rc == 0
+        and ((nextcloud_status.stdout | replace(' ', '')) is search('"installed":true'))
+      retries: 120
+      delay: 5
+      changed_when: false
+
+    # Maintenance mode is ON for the whole of `occ upgrade`, which the entrypoint
+    # runs on the first start after an image bump — so waiting is not optional
+    # here: asserting immediately would fail a perfectly healthy deploy halfway
+    # through the upgrade.
+    #
+    # It can also be left ON in the config.php copied from monoco, because
+    # docs/backup-plan.md §3.4 tells an operator to set it before a consistent
+    # backup. That case never resolves, which is why the rescue below names it
+    # instead of leaving a bare timeout: `status.php` is exempt from maintenance
+    # mode, so the app looks up while every real request gets a 503.
+    - name: Wait for maintenance mode to be off
+      ansible.builtin.command: >-
+        podman exec -u www-data {{ container_name }}
+        php occ status --output=json
+      register: nextcloud_ready
+      until: >-
+        nextcloud_ready.rc == 0
+        and ((nextcloud_ready.stdout | replace(' ', '')) is search('"installed":true'))
+        and not ((nextcloud_ready.stdout | replace(' ', '')) is search('"maintenance":true'))
+      retries: 120
+      delay: 5
+      changed_when: false
+  rescue:
+    - name: Collect Nextcloud readiness diagnostics
+      ansible.builtin.command: "{{ item }}"
+      loop:
+        - "podman exec -u www-data {{ container_name }} php occ status --output=json"
+        - "journalctl --user -u {{ unit_name }} --no-pager -n 40"
+        - "podman logs --tail 40 {{ container_name }}"
+      register: nextcloud_ready_diag
+      changed_when: false
+      failed_when: false
+
+    - name: Fail with the reason
+      ansible.builtin.fail:
+        msg: >-
+          Nextcloud never became ready. If the status above shows
+          "maintenance": true, the instance is in maintenance mode — either left
+          on in the copied config.php or still mid-upgrade — and because
+          status.php is exempt from maintenance mode the site looks healthy while
+          every real request gets a 503. Clear it with:
+          podman exec -u www-data {{ container_name }} php occ maintenance:mode --off
+          Otherwise, see the diagnostics above.
+
+- name: Report Nextcloud status
+  ansible.builtin.debug:
+    msg: "{{ nextcloud_ready.stdout }}"
+
+# The check above runs the PHP CLI, which does not prove Apache is serving. This
+# goes through the image's own front end, over loopback, and asserts the JSON
+# status.php returns says the instance is installed.
+- name: Wait for Apache to serve status.php
+  ansible.builtin.command: >-
+    podman exec {{ container_name }} php -r
+    '$b=@file_get_contents("http://127.0.0.1/status.php"); if ($b === false) exit(1);
+    $j=json_decode($b, true); exit(!empty($j["installed"]) ? 0 : 1);'
+  register: nextcloud_http
+  until: nextcloud_http.rc == 0
+  retries: 24
+  delay: 5
+  changed_when: false
+
+# The database and the cache arrive over the networks that roles/postgres and
+# roles/valkey attached them to. Check the names config.php actually asks for,
+# plus the container names, because a wrong alias is the failure that looks like
+# a broken app rather than a broken network.
+- name: Confirm Nextcloud can resolve the database and the cache
+  ansible.builtin.command: >-
+    podman exec {{ container_name }} getent hosts {{ item }}
+  loop:
+    - "{{ db_host }}"
+    - "{{ postgres_container }}"
+    - "{{ redis_host }}"
+    - "{{ valkey_container }}"
+  register: nextcloud_lookup
+  changed_when: false
+  failed_when: false
+
+- name: Report database and cache resolution
+  ansible.builtin.debug:
+    msg: >-
+      {{ item.item }} resolves as {{ item.stdout | default('(no answer)') }}
+  loop: "{{ nextcloud_lookup.results }}"
+
+# ---------------------------------------------------------------------------
+# Background jobs
+# ---------------------------------------------------------------------------
+# The Docker-era playbook added a *root* crontab entry running `php cron.php`
+# every five minutes. Root cannot `podman exec` into another user's rootless
+# containers (docs/podman-migration.md §8(b)), so it becomes a systemd user timer
+# next to the Quadlet units — which is also what makes it survive a reboot and
+# report its failures in the journal.
+
+- name: Ensure the systemd user unit directory exists
+  ansible.builtin.file:
+    path: "{{ user_unit_dir }}"
+    state: directory
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0755'
+
+- name: Install the Nextcloud background-job service
+  ansible.builtin.template:
+    src: nextcloud-cron.service.j2
+    dest: "{{ user_unit_dir }}/{{ cron_unit }}.service"
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0644'
+
+- name: Install the Nextcloud background-job timer
+  ansible.builtin.template:
+    src: nextcloud-cron.timer.j2
+    dest: "{{ user_unit_dir }}/{{ cron_unit }}.timer"
+    owner: "{{ podman_user }}"
+    group: "{{ podman_user }}"
+    mode: '0644'
+
+# Unlike the Quadlet units, a plain .timer is not generated, so it does need
+# enabling — and `enabled: true` is what brings it back after a reboot, the same
+# way `[Install] WantedBy=default.target` does for the container units.
+- name: Enable and start the Nextcloud background-job timer
+  ansible.builtin.systemd_service:
+    name: "{{ cron_unit }}.timer"
+    scope: user
+    enabled: true
     state: started
-    restart_policy: "unless-stopped"
-    env:
-      POSTGRES_DB: "{{ db_name }}"
-      POSTGRES_USER: "{{ db_user }}"
-      POSTGRES_PASSWORD: "{{ db_pass }}"
-      POSTGRES_HOST: "{{ db_host }}"
-      REDIS_HOST: "redis"
-    networks:
-      - name: "{{ docker_network }}"
-    volumes:
-      - "/var/lib/{{ host_directory }}:/var/www/html"
+    daemon_reload: true
 
-- name: Add cron job
-  cron:
-    name: Nextcloud cron
-    minute: "*/5"
-    job: "docker exec -u www-data {{ container_name }} php cron.php"
+- name: Report the background-job timer
+  ansible.builtin.command: "systemctl --user list-timers {{ cron_unit }}.timer --no-pager"
+  register: nextcloud_timer
+  changed_when: false
+  failed_when: false
+
+- name: Show the background-job timer
+  ansible.builtin.debug:
+    msg: "{{ nextcloud_timer.stdout_lines | default(['(no output)']) }}"

nextcloud/roles/nextcloud/templates/nextcloud-cron.service.j2

@@ -0,0 +1,29 @@
+# Nextcloud background jobs — systemd user service.
+# Managed by Ansible (nextcloud/roles/nextcloud). Do not edit by hand.
+#
+# This replaces the Docker-era root crontab entry:
+#
+#   */5 * * * * docker exec -u www-data nextcloud php cron.php
+#
+# Root cannot `podman exec` into another user's rootless containers, and a
+# container user's cron is not a thing. A user service started by the timer next
+# to it is the idiomatic replacement, and it is what comes back after a reboot
+# with the rest of the stack (docs/podman-migration.md §8(b), §10).
+#
+# No XDG_RUNTIME_DIR juggling is needed: systemd's *user* manager sets it for
+# every unit it starts, which is also why the whole playbook runs as the
+# connection user rather than through become.
+
+[Unit]
+Description=Nextcloud background jobs (cron.php)
+After={{ unit_name }}
+
+[Service]
+Type=oneshot
+# `-u www-data` matters: cron.php must run as the user that owns config.php, and
+# under keep-id that is uid {{ app_uid }}.
+ExecStart=/usr/bin/podman exec -u www-data {{ container_name }} php {{ webroot }}/cron.php
+# A long-running job must not pile up behind itself; the next timer tick would
+# start a second copy. Generous, because a first run after an upgrade can have a
+# lot of queued work.
+TimeoutStartSec=900

nextcloud/roles/nextcloud/templates/nextcloud-cron.timer.j2

@@ -0,0 +1,19 @@
+# Nextcloud background jobs — systemd user timer.
+# Managed by Ansible (nextcloud/roles/nextcloud). Do not edit by hand.
+#
+# Fires nextcloud-cron.service every five minutes, which is what Nextcloud's own
+# installation guide asks for and what the Docker-era root crontab did.
+
+[Unit]
+Description=Run Nextcloud background jobs every five minutes
+
+[Timer]
+OnCalendar={{ cron_schedule }}
+# Catch up once if a slot was missed while the host was down, so a reboot does
+# not delay the first run by five minutes. It is not a queue: one catch-up run,
+# never one per missed slot.
+Persistent=true
+Unit={{ cron_unit }}.service
+
+[Install]
+WantedBy=timers.target

nextcloud/roles/nextcloud/templates/nextcloud.container.j2

@@ -0,0 +1,56 @@
+# Nextcloud — rootless Podman Quadlet unit.
+# Managed by Ansible (nextcloud/roles/nextcloud). Do not edit by hand.
+#
+# This unit joins {{ network_name }}.network, which is created by
+# caddy/roles/register (the same include that attaches Caddy to it). The
+# postgres container is joined to that network by postgres/roles/attach and the
+# valkey container by valkey/roles/attach, which is what makes config.php's `db`
+# and `redis` hosts resolve.
+
+[Unit]
+Description=Nextcloud
+Documentation=https://github.com/nextcloud/docker
+After=network-online.target
+Wants=network-online.target
+
+[Container]
+ContainerName={{ container_name }}
+Image={{ image }}
+
+# Map the container's www-data account (uid {{ app_uid }}) onto {{ podman_user }}
+# on the host. The entrypoint starts as container-root — a subuid under keep-id —
+# but it rsyncs the code in with `--chown www-data:www-data` and Apache drops to
+# www-data, so everything it writes lands on {{ podman_user }} and no subuid-owned
+# files appear to break backups.
+#
+# 33 is also the uid the data on monoco is owned by, so the copied tree needs no
+# chown pass: extracting it as {{ podman_user }} is already the right ownership.
+UserNS=keep-id:uid={{ app_uid }},gid={{ app_gid }}
+
+# POSTGRES_* and REDIS_HOST live here rather than as Environment= lines above:
+# this unit is world-readable in ~/.config, and POSTGRES_PASSWORD is in it. Read
+# by systemd, not by Ansible, so the file has to exist before this unit starts.
+#
+# These are not an install step any more — config.php is copied from monoco, not
+# generated. REDIS_HOST is still load-bearing on every start, because the
+# entrypoint rewrites PHP's session-handler config from it.
+EnvironmentFile={{ env_file }}
+
+Network={{ network_name }}.network
+
+# The image's VOLUME and Apache document root. This is the whole installation:
+# the code (version.php, core/, lib/, apps/), config/config.php, data/ and
+# custom_apps/. The Docker-era playbook bind-mounted exactly this, and the
+# entrypoint's version check reads version.php from here — which is why the
+# migration copies the tree rather than only the state. See readme.md.
+Volume={{ host_directory }}:{{ webroot }}
+
+[Service]
+Restart=always
+# The entrypoint rsyncs the image's code into the webroot and, if the copied
+# version.php is older than the image, runs `occ upgrade` before Apache answers.
+# On a 320 MB database that is minutes, not seconds.
+TimeoutStartSec=600
+
+[Install]
+WantedBy=default.target

nextcloud/roles/nextcloud/templates/nextcloud.env.j2

@@ -0,0 +1,19 @@
+# Environment for {{ container_name }}.container.
+# Managed by Ansible (nextcloud/roles/nextcloud). Do not edit by hand.
+#
+# Mode 0600 on purpose: the Quadlet unit that reads this file is world-readable,
+# and POSTGRES_PASSWORD is in it. Note that `podman inspect` still prints the
+# resulting environment to anyone who can run podman as this user.
+#
+# config.php is copied from monoco, not generated, so the POSTGRES_* values are
+# not an install step any more — they only have to agree with what config.php
+# already carries, which is exactly what the Docker-era container passed.
+#
+# REDIS_HOST is different: the entrypoint reads it on *every* start and rewrites
+# PHP's session handler from it (/usr/local/etc/php/conf.d/redis-session.ini), so
+# a wrong value here shows up as failed logins rather than as a missing cache.
+POSTGRES_DB={{ pg_database }}
+POSTGRES_USER={{ pg_user }}
+POSTGRES_PASSWORD={{ pg_password }}
+POSTGRES_HOST={{ db_host }}
+REDIS_HOST={{ redis_host }}