diff --git a/nextcloud/playbook.yml b/nextcloud/playbook.yml index 0736653..a39bcce 100755 --- a/nextcloud/playbook.yml +++ b/nextcloud/playbook.yml @@ -1,35 +1,21 @@ --- -# Nextcloud as a rootless Podman Quadlet unit, behind the shared Caddy unit -# (caddy/playbook.yml), with its database in the shared Postgres unit -# (postgres/playbook.yml) and its cache in the shared Valkey unit -# (valkey/playbook.yml). +# Nextcloud as a rootless Podman Quadlet unit, behind the shared Caddy, with its +# database in the shared Postgres and its cache in the shared Valkey. Each of +# those is its own playbook and has to have been run first: caddy, postgres, +# valkey, then this. # -# Standalone and idempotent: the first run on a host creates the container and -# leaves a fresh instance ready for its setup wizard, and every run after that -# converges the unit, the environment and the image — Quadlet recreates the -# container when any of them change. Nothing here depends on where the instance -# came from; only `hosts` is host-specific. +# Idempotent and standalone: the first run creates and starts the container (a +# fresh webroot then serves the setup wizard), and later runs converge it — +# Quadlet recreates the container when the unit or its environment changes. # -# Connects as the container user and runs everything as that user: the Quadlet -# units belong to it, so the connection's PAM session is what makes -# `systemctl --user` work without any environment juggling. -# -# gather_facts is required: podman_user and the home directory come from the +# Runs as the connection user with no become: the units belong to that user, so +# the SSH session's own PAM session is what makes `systemctl --user` work. +# gather_facts is required — podman_user and the home directory come from the # user_id / user_dir facts. # -# The rootless-Podman prerequisites (installing podman, enabling lingering, the -# low-port sysctl, creating the Quadlet directory) are deliberately NOT applied -# here. They are the shared task file at caddy/roles/podman/tasks/main.yaml, and -# a host that has already run caddy/playbook.yml has them: -# -# - name: Rootless Podman prerequisites -# ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/podman/tasks/main.yaml" -# -# The three shared services it attaches to have to exist first, in this order: -# caddy, postgres, valkey. -# -# The variables this play needs are duplicated at the top of vars/default.yml, so -# it stands on its own. +# The rootless-Podman prerequisites (podman, lingering, the low-port sysctl, the +# Quadlet directory) are not applied here; they are the shared task file at +# caddy/roles/podman, and a host that has run caddy/playbook.yml has them. # # Run with: ansible-playbook -u tanshu nextcloud/playbook.yml - name: Deploy Nextcloud as a rootless Podman Quadlet diff --git a/nextcloud/readme.md b/nextcloud/readme.md index f4af119..4f45a9e 100644 --- a/nextcloud/readme.md +++ b/nextcloud/readme.md @@ -9,36 +9,24 @@ Standalone and idempotent, and host-agnostic apart from `hosts:` in [playbook.yml](playbook.yml) and `host` in [vars/default.yml](vars/default.yml): - **First run** creates the webroot, the environment file, the unit and the - background-job timer, starts the container, and leaves a fresh instance ready - for its [setup wizard](#first-run) — or installs it unattended, if you set the - two optional credentials first. -- **Every run after that** converges the same things, and Quadlet recreates the - container when the unit, the environment or the image changed. There is no - update step to remember and nothing to copy in by hand. - -Nextcloud is the one service here whose webroot is *also* its code directory: the -image's `VOLUME`, `config/config.php`, `data/`, `custom_apps/` and every PHP file -live under one mount. That single fact drives the ownership story and the backup -guidance below — see [What is state, and what is not](#what-is-state-and-what-is-not). + background-job timer, creates the database if the cluster does not have it, and + starts the container. A fresh webroot then serves the setup wizard — or installs + itself unattended, if you set the two optional credentials first. +- **Every run after that** converges the same things. Quadlet recreates the + container when the unit or the environment changes, so there is no update step + to remember. ## Deploy order -Three shared services have to exist on the host first, each of them standalone: +Three shared services have to exist first, each standalone: -1. [caddy/playbook.yml](../caddy/playbook.yml) — the reverse proxy, which owns the +1. [caddy/playbook.yml](../caddy/playbook.yml) — the reverse proxy. It owns the Caddyfile this site appends to, supplies the `caddy.service` handlers, and is the one play that applies the rootless-Podman prerequisites. -2. [postgres/playbook.yml](../postgres/playbook.yml) — the cluster. **This play - attaches to it and creates its own `nextcloud` database in it**, so the cluster +2. [postgres/playbook.yml](../postgres/playbook.yml) — the cluster. This playbook + attaches to it and creates its own `nextcloud` database in it, so the cluster can be brand new and empty. -3. [valkey/playbook.yml](../valkey/playbook.yml) — the cache. It has to exist - before this playbook runs: the attach include only writes drop-ins next to - `valkey.container`, so a `Restart Valkey` handler against an undeployed unit - fails. - -Then this playbook. `config.php` connects to the hosts `db` and `redis`; until it -has attached `nextcloud_net` to Postgres and to Valkey, neither name resolves on -that network, so running this playbook is what makes the app reachable. +3. [valkey/playbook.yml](../valkey/playbook.yml) — the cache. ## Layout @@ -49,31 +37,24 @@ nextcloud/ └── roles/ ├── postgres/ # the database, and this app's network on the cluster ├── valkey/ # this app's network on the cache - ├── nextcloud/ # webroot, env file, Quadlet unit, session config, - │ # background-job timer, start + verify - └── caddy/ # appends this site's block to the shared Caddyfile + ├── nextcloud/ # webroot, env file, unit, session config, cron timer + └── caddy/ # the site block in the shared Caddyfile ``` -Quadlet units and state: - | On the host | What | |---|---| -| `~/.config/containers/systemd/nextcloud.container` | the generated unit (Ansible-templated) | -| `~/.config/containers/systemd/nextcloud.network` | the app network, written by `caddy/roles/register` | -| `~/.config/containers/systemd/caddy.container.d/50-nextcloud.conf` | attaches Caddy to that network | -| `~/.config/containers/systemd/postgres.container.d/50-nextcloud.conf` | attaches Postgres to it | -| `~/.config/containers/systemd/postgres.container.d/10-network-aliases.conf` | the shared `db` alias | -| `~/.config/containers/systemd/valkey.container.d/50-nextcloud.conf` | attaches Valkey to it | -| `~/.config/containers/systemd/valkey.container.d/10-network-aliases.conf` | the shared `redis` alias | -| `~/.config/systemd/user/nextcloud-cron.{service,timer}` | the background-job timer (not Quadlet) | -| `~/podman-data/nextcloud/` | the whole `/var/www/html`: code, `config/config.php`, `data/`, `custom_apps/` | -| `~/podman-data/nextcloud/nextcloud.env` | `POSTGRES_*` + `REDIS_HOST`, mode `0600`, read by systemd | -| `~/podman-data/nextcloud/php-conf.d/redis-session.ini` | PHP's session handler, mounted into the image | -| `~/podman-data/nextcloud/nextcloud-init-sync.lock` | the entrypoint's init lock, pre-created so it stays host-owned | +| `~/.config/containers/systemd/nextcloud.container` | the generated unit | +| `~/.config/containers/systemd/nextcloud.network` | the app network | +| `~/.config/containers/systemd/*.container.d/50-nextcloud.conf` | attaches Caddy, Postgres and Valkey to it | +| `~/.config/containers/systemd/*.container.d/10-network-aliases.conf` | the shared `db` and `redis` aliases | +| `~/.config/systemd/user/nextcloud-cron.{service,timer}` | the background-job timer | +| `~/podman-data/nextcloud/` | the whole `/var/www/html` | +| `~/podman-data/nextcloud/nextcloud.env` | `POSTGRES_*` + `REDIS_HOST`, mode `0600` | +| `~/podman-data/nextcloud/redis-session.ini` | mounted into the image; see [below](#the-php-session-file) | ## What is state, and what is not -The webroot is about 1.2 GB on a working instance, and most of it is not worth +The webroot is about 1.2 GB on a working instance and most of it is not worth backing up: | Path | Keep? | Why | @@ -84,112 +65,44 @@ backing up: | `themes/` | if themed | only exists when a theme was installed | | `version.php`, `core/`, `lib/`, `apps/`, `3rdparty/`, `dist/` | regenerable | shipped by the image, rsynced into the webroot on start | -The database is separate and is not in this directory at all. A backup that -matters is `config/`, `data/`, `custom_apps/` **and** a `pg_dump` of the -`nextcloud` database taken together — files without the database restore to an -instance that does not know about its own contents, and the reverse is just as -useless. Nextcloud's own guidance is `occ maintenance:mode --on` around the pair; -the play waits for maintenance mode to clear on the next run, so a forgotten -`--on` is reported rather than silently 503ing. +The database is separate. A backup that matters is `config/`, `data/`, +`custom_apps/` **and** a `pg_dump` of the `nextcloud` database taken together; +either half alone restores to an instance that does not know its own contents. -## How it registers with Caddy +## How it wires itself up -Caddy is the base unit and knows nothing about its upstreams. This playbook calls -the shared include in -[caddy/roles/register](../caddy/roles/register/tasks/main.yaml), which writes the -network unit **and** the drop-in that pulls it into Caddy: +**Caddy.** `caddy/roles/register` writes `nextcloud.network` and a drop-in that +attaches Caddy to it. This runs in `pre_tasks`, because the unit references +`nextcloud.network` and Quadlet fails *unit generation* — `systemctl --user start` +then reports "unit not found" — when that file does not exist. The site block goes +into the shared Caddyfile with `blockinfile`, marker and all, so re-runs replace +it in place. -```yaml -- name: Register Nextcloud's network with Caddy - ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/register/tasks/main.yaml" -``` +**Postgres.** `postgres/roles/attach` writes two drop-ins next to +`postgres.container`: `Network=nextcloud.network` and the shared `NetworkAlias=db`. +`roles/postgres` wraps it to define the `Restart Postgres` handler, to flush +handlers afterwards (a new drop-in means Postgres must be *recreated* to join the +network, and that restart would otherwise happen after the app started), and to +create the `nextcloud` database if it is missing. A new drop-in is a few seconds +of downtime for every attached app; see +[postgres/readme.md](../postgres/readme.md). -It runs in `pre_tasks`, **before** the app unit is started, because -`nextcloud.container` references `nextcloud.network`: Quadlet fails *unit -generation* — so `systemctl --user start` reports "unit not found" — when a -referenced `.network` file does not exist yet. Both attaches below depend on it -for the same reason. - -`network_name` is `nextcloud`, so the *file* is `nextcloud.network` and the -*Podman network* is `nextcloud_net`. - -The site block is appended to the shared Caddyfile by -[roles/caddy](roles/caddy/tasks/main.yaml) with `blockinfile`, exactly as the -other services do — plus the HSTS header Nextcloud asks for. There is no -request-body limit to set: Caddy streams request bodies and imposes none by -default. - -## How it attaches to Postgres - -Postgres knows nothing about its clients, and this playbook uses the shared -include in [postgres/roles/attach](../postgres/roles/attach/tasks/main.yaml) -rather than writing the drop-ins itself. That writes two files, both mergeable -into `postgres.container` by Quadlet: - -| File | Content | -|---|---| -| `postgres.container.d/50-nextcloud.conf` | `Network=nextcloud.network` | -| `postgres.container.d/10-network-aliases.conf` | `NetworkAlias=db` | - -`db` is the alias `config.php` connects to; the `postgres` container name resolves -too, and the play checks both. - -[roles/postgres](roles/postgres/tasks/main.yaml) wraps that include for three -reasons: - -- it defines the `Restart Postgres` handler the include notifies; -- it **flushes handlers** immediately afterwards. On a first attach the drop-in is - new, so Postgres has to be *recreated* to join `nextcloud_net` — and that restart - is a handler, which by default runs at the end of the play, after the `nextcloud` - role has already started the app. The app would then come up before `db` - resolves, fail its readiness check, and fail the play on a machine where nothing - is wrong. The flush also fires the `Restart Caddy` queued by the register - pre_task, which is what attaches Caddy to the network. On later runs nothing is - pending and it is a no-op; -- it **creates the `nextcloud` database** if the cluster does not have it yet. - Postgres is a shared unit that ships one default database, and nothing else in - this repo knows what this app's database is called, so without that task a - first run on a fresh host would reach the setup wizard only to fail on - "database nextcloud does not exist". It is a `createdb` guarded by a lookup, so - it is a no-op on every run after the first. - -Note the cost stated in [postgres/readme.md](../postgres/readme.md): a new drop-in -recreates Postgres, which is a few seconds of downtime for **every** attached app, -not just this one. - -## How it attaches to Valkey - -The same idea at the cache end, via -[valkey/roles/attach](../valkey/roles/attach/tasks/main.yaml): - -| File | Content | -|---|---| -| `valkey.container.d/50-nextcloud.conf` | `Network=nextcloud.network` | -| `valkey.container.d/10-network-aliases.conf` | `NetworkAlias=redis` | - -`redis` is not a name this playbook chose: the image writes the Redis host into -`config.php` and into PHP's session config from `REDIS_HOST`, which is `redis` -here. Renaming it would mean editing a live instance's configuration for nothing. - -It is also **not optional**. Go through the setup wizard and Nextcloud puts -`memcache.locking` on Redis; the image's own install does too when the unattended -credentials are set. An unresolvable `redis` therefore does not merely make things -slower — uploads fail, and they fail at write time rather than at startup. That is -why `roles/valkey` flushes handlers before the app starts (the same reason as -Postgres) and why `REDIS_HOST` is in the environment file at all. - -The cost of the attach is a recreate of Valkey: no keys are lost (RDB persistence -is on by default, the stop is graceful, and the new container loads the snapshot -back off the bind mount), but for the seconds it is down, PHP sessions and -`memcache.locking` are unreachable. See [valkey/readme.md](../valkey/readme.md), -"The trade-off". +**Valkey.** The same, via +[valkey/roles/attach](../valkey/roles/attach/tasks/main.yaml), with +`NetworkAlias=redis`. `redis` is not a name this playbook chose: `REDIS_HOST` is +what the image writes PHP's session config from, and Redis is where the setup +wizard puts `memcache.locking`. An unresolvable `redis` therefore breaks uploads +at write time rather than at startup, which is why the attach is flushed before +the app starts as well. The recreate costs a few seconds of cache availability; +the keys survive it (see [valkey/readme.md](../valkey/readme.md)). ## Background jobs -Nextcloud expects `cron.php` every five minutes. It cannot be a crontab entry: -root cannot `podman exec` into another user's rootless containers, and there is no -container user to own a crontab. The role installs a **systemd user** pair next to -the Quadlet units instead: +Nextcloud expects `cron.php` every five minutes. **It cannot be a crontab entry:** +root cannot `podman exec` into another user's rootless containers, and a container +has no crontab of its own. A root crontab would fail; a *user* crontab would need +`XDG_RUNTIME_DIR` faked into every line and a cron daemon present. The role +installs a systemd user pair instead, next to the Quadlet units: | Unit | What it does | |---|---| @@ -197,326 +110,181 @@ the Quadlet units instead: | `~/.config/systemd/user/nextcloud-cron.timer` | `OnCalendar=*:0/5`, `Persistent=true` | Unlike the Quadlet units these are plain units, so they really are `enable`d — -that is what brings the timer back after a reboot. `journalctl --user -u -nextcloud-cron` shows every run and every failure. - -The background-job *mode* is left alone. Both this timer and AJAX polling run -jobs, and job locking keeps them from colliding; switching AJAX off is -`occ background:cron`, a deliberate change rather than something this play -imposes. +that is what brings the timer back after a reboot — and +`journalctl --user -u nextcloud-cron` shows every run. ## Settings | Variable | Value / purpose | |---|---| -| `podman_user` | **Derived** from the connection (`ansible_facts['user_id']`), not set by hand | -| `podman_data_dir` | `~/podman-data` | -| `quadlet_dir` | `~/.config/containers/systemd` | -| `host` | `cloud.hopsngrains.com` — the hostname Caddy serves, and the trusted domain for an unattended install | -| `container_name` | `nextcloud`, and the Quadlet unit / container name | -| `image` | `docker.io/library/nextcloud:latest` — fully qualified on purpose | +| `podman_user` | **Derived** from the connection (`ansible_facts['user_id']`) | +| `podman_data_dir` / `quadlet_dir` | `~/podman-data`, `~/.config/containers/systemd` | +| `host` | `cloud.hopsngrains.com` — what Caddy serves, and the trusted domain for an unattended install | +| `container_name` | `nextcloud`, and the unit / container name | +| `image` | `docker.io/library/nextcloud:latest`, fully qualified on purpose | | `network_name` | `nextcloud` → `nextcloud.network` / `nextcloud_net` | | `host_directory` / `webroot` | `~/podman-data/nextcloud` → `/var/www/html` | | `app_uid` / `app_gid` | `33` — `www-data`, the uid the webroot is owned by | -| `container_port` | `80`, the port inside the container that Caddy proxies to | +| `container_port` | `80`, what Caddy proxies to | | `env_file` | `~/podman-data/nextcloud/nextcloud.env`, mode `0600` | -| `php_conf_dir` / `php_session_ini` | PHP's session config on the host, and the path in the image it is mounted over | -| `postgres_container` / `postgres_unit` | The shared database unit this app attaches to | -| `pg_user` / `pg_password` / `pg_database` | Credentials, and the database the play creates and the app uses — `nextcloud` | -| `db_host` | `db` — the alias `config.php` connects to | -| `valkey_container` / `valkey_unit` | The shared cache unit this app attaches to | -| `redis_host` | `redis` — the alias `config.php` and PHP sessions connect to | +| `pg_*` / `db_host` | The database, its credentials, and the `db` alias | +| `valkey_container` / `redis_host` | The cache unit, and the `redis` alias | | `nextcloud_admin_user` / `nextcloud_admin_password` | Empty by default; set both to install unattended | -| `cron_unit` / `cron_schedule` / `user_unit_dir` | The background-job timer | -| `caddy_name` / `caddyfile_path` | The shared Caddy unit and its Caddyfile | +| `cron_*` / `user_unit_dir` | The background-job timer | +| `caddy_*` | The shared Caddy unit and its Caddyfile | -The three `podman_*` / `quadlet_dir` vars are **duplicated** from -[caddy/vars/default.yml](../caddy/vars/default.yml), as in every service -playbook. That is a deliberate trade: a play cannot include the Caddy vars file -wholesale, because it also sets `host_directory` (to the *Caddy* data directory) -and every service defines that for itself. Keep them in sync. Caddy's fourth, -`podman_unprivileged_port_start`, belongs to the podman tasks this play does not -run — Nextcloud publishes nothing host-side. - -Like Caddy, `podman_user` is derived from the connection and the play carries no -`become`. Because the podman tasks are not run here, there is **no guard against -connecting as `root`**; the prerequisites, including that check, live in -`caddy/playbook.yml`. +The three `podman_*` / `quadlet_dir` vars are duplicated from +[caddy/vars/default.yml](../caddy/vars/default.yml) because a play cannot include +that file wholesale — it also sets `host_directory`. Keep them in sync. Like +Caddy, the play carries no `become`, so there is no guard against connecting as +`root`; that check lives in `caddy/playbook.yml`. `UserNS=keep-id:uid=33,gid=33`. `33` is `www-data`, the uid the webroot is owned -by: the entrypoint rsyncs the code in with `--chown www-data:www-data` and Apache -runs its workers as that account. `keep-id` maps that one uid onto the connection -user, so the whole webroot stays owned by `tanshu` and plain `tar` and `rm -rf` -keep working on it. - -Two paths the entrypoint writes fall outside that arrangement, and both are -handled explicitly rather than left to chance — each is somewhere a naive -deployment ends up with a file owned by a `165536+` subuid, or with no running -container at all: - -- **`/var/www/html/nextcloud-init-sync.lock`**, which the entrypoint creates with - a root-owned shell redirect. The role **pre-creates it** as `podman_user`: the - redirect truncates the file rather than recreating it, so its owner never - changes afterwards. -- **`/usr/local/etc/php/conf.d/redis-session.ini`**, in the image, which is not - writable at all. It has its own section below. +by: the entrypoint rsyncs the code in as that uid and Apache runs its workers as +it. `keep-id` maps that one uid onto the connection user, so the webroot stays +owned by `tanshu` and plain `tar` and `rm -rf` keep working on it. `POSTGRES_PASSWORD` is committed in plaintext, like the other playbooks' shared `123456`. It lives in `nextcloud.env` at mode `0600` rather than in the -world-readable unit file; a real secret store is the next step, and it is a -one-file change here plus the `pg_password` vars in the app playbooks. +world-readable unit file. ## The PHP session file -The webroot is not the only thing the entrypoint writes. With `REDIS_HOST` set — -and it is, because PHP sessions should survive a redeploy — the entrypoint -generates that session configuration on **every** start: +The entrypoint generates PHP's session config on every start, with a redirect: ```sh /entrypoint.sh:121 } > /usr/local/etc/php/conf.d/redis-session.ini ``` -That directory belongs to the image's root, created by the php base image with no -`chown`: - -```dockerfile -RUN set -eux; \ - mkdir -p "$PHP_INI_DIR/conf.d"; \ - ... -``` - -so the uid this container runs as has no write bit on it, and the redirect fails: +That directory belongs to the image's root — the php base image creates it with +`mkdir -p "$PHP_INI_DIR/conf.d"` and never chowns it — so the write is refused, +the entrypoint exits `2` before Apache is reached, `Restart=always` retries it, +systemd hits its start limit, and the unit ends up **`failed`** with the only +explanation in the journal: ``` cannot create /usr/local/etc/php/conf.d/redis-session.ini: Permission denied ``` -The entrypoint exits `2` before Apache is ever reached, `Restart=always` retries -it, systemd hits its start limit, and the unit ends up **`failed`** — with the -whole thing looking like a network or database problem, because that message only -appears in the journal. +The role creates `~/podman-data/nextcloud/redis-session.ini` and the unit binds it +over that exact path. A bind-mounted file replaces the directory entry, so the +directory's ownership stops mattering and the write lands on a file owned by +`podman_user`. Apache's own runtime directories need none of this — the same base +image chowns `/var/run/apache2`, `/var/lock/apache2` and `/var/log/apache2` to +`www-data` and chmods them `1777`. -The fix is a mount: `nextcloud.container` binds -`~/podman-data/nextcloud/php-conf.d/redis-session.ini` over that exact path. A -bind-mounted file replaces the directory entry, so the directory's ownership stops -mattering and the write lands on a file owned by `podman_user`. The role also -templates the file with the five lines the entrypoint would have written, so a -start where the write is somehow refused still gets working Redis sessions instead -of a crash loop. - -Apache's *own* runtime directories need none of this — the same base image chowns -`/var/run/apache2`, `/var/lock/apache2` and `/var/log/apache2` to `www-data` and -chmods them `1777`: - -```dockerfile - for dir in "$APACHE_LOCK_DIR" "$APACHE_RUN_DIR" "$APACHE_LOG_DIR" \ - "$APACHE_RUN_DIR/socks"; do - rm -rvf "$dir"; mkdir -p "$dir"; \ - chown "$APACHE_RUN_USER:$APACHE_RUN_GROUP" "$dir"; \ - chmod 1777 "$dir"; \ - done -``` - -so the session file is the only write that was ever going to fail. - -## First run +## First run and updates ```command ansible-playbook -u tanshu nextcloud/playbook.yml ``` -The play targets `emily` directly, so the inventory must contain it — the one in -`prepare-server/` does, and its `ansible.cfg` names it. +A fresh webroot is a valid state, and the play succeeds: the image initialises the +code, writes no `config.php` (it has no credentials to install with), and serves +the setup wizard. The readiness gate is Nextcloud's own `status.php` inside the +container — it answers only once the entrypoint has finished and Apache is up, and +it reports `installed` either way, so the same gate covers a wizard that has not +been completed yet and an instance that is being updated. -On a host with no webroot yet, this is what happens, in order: the webroot and the -PHP session directory are created; the `0600` environment file, the session config -and the Quadlet unit are written; the unit is started; anything the run queued (a -new image, a changed unit or environment) is applied *before* verification; and -then the play waits for Apache to answer `status.php` inside the container. +Three things about the wizard: -At that point there is a running container and **no instance yet** — the image -initialises the code, writes no `config.php` (it has no admin credentials to -install with), and serves the setup wizard. The play says so and succeeds: - -``` -The container is running, but no instance is installed in -/home/tanshu/podman-data/nextcloud yet. Open https://cloud.hopsngrains.com and -finish the setup wizard with the database credentials in .../nextcloud.env — -database nextcloud, user postgres, host db. ... -``` - -Three things to know about that wizard: - -- **DNS first.** `host` must resolve to this host before Caddy can obtain a - certificate for it, and before the wizard is reachable at all. -- **The database already exists.** [roles/postgres](roles/postgres/tasks/main.yaml) - created it, or verified it, before the container started. Point the wizard at - host `db`, database `nextcloud`, user `postgres`, and the password in - `nextcloud.env`. +- **DNS first.** `host` must resolve here before Caddy can get a certificate, and + before the wizard is reachable at all. +- **The database already exists** — `roles/postgres` created it, or verified it, + before the container started. Point the wizard at host `db`, database + `nextcloud`, user `postgres`, and the password in `nextcloud.env`. - **`trusted_domains`.** The wizard installs with `localhost` as the only trusted - domain unless you tell it otherwise, so opening the finished instance on `host` - can answer "Access through untrusted domain". Add it before or after: + domain, so the finished instance can answer "Access through untrusted domain" on + `host`. Fix it with + `podman exec -u www-data nextcloud php occ config:system:set trusted_domains 1 --value=cloud.hopsngrains.com`. - ```bash - podman exec -u www-data nextcloud php occ config:system:set trusted_domains 1 --value=cloud.hopsngrains.com - ``` +To skip the wizard, set `nextcloud_admin_user` and `nextcloud_admin_password` +before the first run. The entrypoint then runs `occ maintenance:install` itself and +sets `trusted_domains` from `host` in the same pass. Both are inert once +`config.php` exists. -To skip the wizard entirely, set `nextcloud_admin_user` and -`nextcloud_admin_password` in [vars/default.yml](vars/default.yml) before the first -run. The entrypoint then runs `occ maintenance:install` itself and sets -`trusted_domains` from `host` in the same pass, and the play reports the installed -version instead of the wizard note. Both variables are inert once `config.php` -exists, so they can stay set (or be removed) afterwards. +**Updates** are a re-run: a changed unit or environment is re-templated and Quadlet +recreates the container. An image bump is two commands, because a `latest` tag +moving is not something the unit can see: -## Updates +```bash +podman pull docker.io/library/nextcloud:latest && systemctl --user restart nextcloud +``` -Nothing special is required: re-run the playbook. What happens depends on what -changed. - -| Changed | What the play does | -|---|---| -| The unit, the environment file or the session config | re-templates it, and Quadlet recreates the container | -| A new image behind the tag | pulls it, notices the ID changed, and recreates the container | -| Nothing | nothing — no restart, no churn | - -The image is pulled on every run precisely because a Quadlet unit alone cannot see -a tag move: the unit names `nextcloud:latest`, `podman run` reuses whatever image -is local, and nothing fetches a newer one. The role reads the image ID either side -of the pull, so an unchanged tag reports no change and does not restart a running -site. A pull that fails (a host with no route to the registry) is reported and the -run continues with the local image. - -**Version drift is the one irreversible step.** The entrypoint compares -`version.php` in the webroot against the image. A newer image rsyncs its code in -and runs `occ upgrade` before Apache answers — and it will only step **one major -version at a time**, so several majors behind means several runs with an -intermediate image. An image *older* than the installed version makes the -container exit with "downgrading is not supported". So: pin the tag if you want -no surprises, and take a database dump before a run that is going to upgrade. - -`occ upgrade` sets maintenance mode for its duration, and the play **waits for it -to clear** rather than asserting it is off — so a long upgrade is tolerated, and an -instance left in maintenance mode (a backup that set `--on` and never finished) is -reported with the one-line command to clear it. That distinction matters because -`status.php` is exempt from maintenance mode: a site in that state looks healthy -to a probe while returning `503` to every real request. +The entrypoint then compares `version.php` in the webroot against the image, +rsyncs the new code in, and runs `occ upgrade` — **one major version at a time**, +so several majors behind means several runs through intermediate images. An image +*older* than the installed version makes the container exit with "downgrading is +not supported". Take a database dump before a run that is going to upgrade. ## Verification ```bash systemctl --user status nextcloud journalctl --user -u nextcloud -f -podman logs nextcloud - -# What the playbook checks for you: -podman exec -u www-data nextcloud php occ status # installed, version, maintenance -podman exec nextcloud getent hosts db # config.php's database host -podman exec nextcloud getent hosts redis # config.php's cache host -podman exec nextcloud getent hosts postgres valkey # the container names - -# The background job, and that it is actually scheduled: +podman exec -u www-data nextcloud php occ status +podman exec nextcloud getent hosts db redis systemctl --user list-timers nextcloud-cron.timer -systemctl --user start nextcloud-cron.service # run one by hand -journalctl --user -u nextcloud-cron -n 20 - -# The database this play created, and its size: -podman exec -e PGPASSWORD=123456 postgres psql -U postgres -l | grep nextcloud - -# The site, end to end: curl -sSI https://cloud.hopsngrains.com ``` -Then open it **in a browser** and do two things a `curl` cannot: log in, and -**upload a file**. Uploading is the check that matters — it exercises the uid-33 -mapping (a wrong `keep-id` shows up as a permissions error), the Redis file -locking in `config.php`, and PHP sessions, which the image configures from -`REDIS_HOST`. - -After the upload, confirm on the host that the new file is owned by `tanshu`: +Then open it in a browser, log in, and **upload a file**: that exercises the uid-33 +mapping (a wrong `keep-id` shows up as a permissions error), the Redis file locking +in `config.php`, and PHP sessions. Confirm the new file is owned by `tanshu`: ```bash find ~/podman-data/nextcloud/data -newermt "-10 minutes" -type f ! -user tanshu -# empty = the write path lands on the right uid ``` -Then reboot the host and confirm both the container and the timer return -unattended — without `loginctl enable-linger tanshu` neither will, and the failure -is silent. +Finally reboot and confirm the container and the timer come back unattended — +without `loginctl enable-linger tanshu` neither will, and the failure is silent. ### Debugging a unit that will not start A unit that has hit systemd's start limit reports only a state, and `Restart=always` -means the interesting output is in the journal, not in `podman logs` (the container -may already be gone — Quadlet passes `--rm`). The play collects all of this itself -when a start fails; by hand: +means the reason is in the journal rather than in `podman logs` (the container may +already be gone — Quadlet passes `--rm`). The play collects this itself when a +start fails; by hand: ```bash systemctl --user status nextcloud.service --no-pager -l | head -25 -systemctl --user cat nextcloud.service # what systemd is running journalctl --user -u nextcloud.service --no-pager -n 60 podman ps -a --filter name=nextcloud -podman logs --tail 40 nextcloud ``` -Read it in this order: - -1. **`journalctl`** — the entrypoint's own output. `cannot create ...: Permission - denied` for a path in `/usr/local/etc/php/conf.d` is the session-file problem - above; anything about `db`/`redis` is the network attach; `downgrading is not - supported` is version drift. -2. **`podman ps -a`** — a container left behind under this name makes `podman run` - refuse to start. -3. **`systemctl --user cat`** — the generated unit, if a drop-in or a `Network=` - is in question (the usual failure is a `.network` file that does not exist). - -A `failed` unit needs its start-limit counter cleared before it will try again: +`journalctl` is the one that matters: the entrypoint's own output is there. A +`failed` unit needs its start-limit counter cleared before it will try again: ```bash systemctl --user reset-failed nextcloud.service -systemctl --user start nextcloud.service ``` ## Notes and gotchas -- **`REDIS_HOST` makes the entrypoint write into the image, which it is not - allowed to do.** `/usr/local/etc/php/conf.d` belongs to the image's root, so the - crash is a `Permission denied` on `redis-session.ini`, the unit ends up `failed` - via systemd's start limit, and nothing about it looks like a permissions - problem. The unit mounts a host-owned file over that path; see - ["The PHP session file"](#the-php-session-file). - **`activating` is not a healthy state.** A crash-looping container spends its - time `activating` between restarts, so the play's wait retries only while the - state can still change on its own and otherwise hands over to a rescue that has - the logs. A bare `is-active` check would report either state as a failure with no - explanation. -- **A new drop-in restarts Postgres for everyone** (a few seconds without a - database for every attached app), **registering restarts all of Caddy** (a brief - restart of every hosted site) and **attaching restarts Valkey** (seconds without - sessions or locks, though the keys survive the graceful stop). See the - trade-offs in [postgres/readme.md](../postgres/readme.md), - [caddy/readme.md](../caddy/readme.md) and [valkey/readme.md](../valkey/readme.md). + time `activating` between restarts, so the play waits on `status.php` rather than + on `is-active`, and reports the journal on failure. +- **The webroot holds the code**, which is the image's design and not this + playbook's: an image bump edits `~/podman-data/nextcloud`. It is also why the + table above calls `apps/`, `core/` and `lib/` regenerable. +- **`occ` must run as `www-data`** — it refuses any other user, and under + `keep-id:uid=33` that is the user the host sees as `tanshu`. +- **A new drop-in restarts Postgres for everyone**, registering restarts all of + Caddy, and attaching restarts Valkey. See the trade-offs in + [postgres/readme.md](../postgres/readme.md), [caddy/readme.md](../caddy/readme.md) + and [valkey/readme.md](../valkey/readme.md). - **If the Caddyfile does not exist yet**, this playbook creates it containing only - the site block. That is a valid Caddyfile; the Caddy playbook's seeded - global-options block is then not added, because its seed only fires on a missing - or zero-length file. Deploy Caddy first on a fresh host. + the site block. That is valid; the Caddy playbook's global-options seed only + fires on a missing or empty file. Deploy Caddy first on a fresh host. - **Client IPs.** Nextcloud uses the client address for brute-force protection and - for the `X-Forwarded-For` it logs, so the reverse proxy has to be trusted — see - [caddy/readme.md](../caddy/readme.md), "Client IPs", which also covers whether - the original address survives the rootless port forward at all. -- **The webroot holds the code.** That is the image's design, not this playbook's: - an image bump edits `~/podman-data/nextcloud`. It is also why `apps/`, `core/` - and `lib/` are listed as regenerable above — they are, but only from an image - that is the same version or newer. -- **`occ` must run as `www-data`.** It refuses to run as any other user, and under - `keep-id:uid=33` that is the same user the host sees as `tanshu`. Every - `podman exec` in this file therefore carries `-u www-data`; the plain form works - for `getent`, which does not care. -- **The backup scripts are not this playbook's business.** Whatever dumps the - database and archives the webroot has to run as a **systemd user timer** next to - the Quadlet units: root cannot `podman exec` into another user's rootless - containers. Dump in place rather than through a helper container — `--link` does - not exist in Podman: + for the `X-Forwarded-For` it logs, so the proxy has to be trusted — see + [caddy/readme.md](../caddy/readme.md), "Client IPs", including whether the + original address survives the rootless port forward. +- **Backups are not this playbook's business.** Whatever dumps the database and + archives the webroot has to run as a **systemd user timer** for the same reason + `cron.php` does, and should dump in place rather than through a helper container + (`--link` does not exist in Podman): ```bash podman exec -e PGPASSWORD="..." postgres pg_dump -U postgres nextcloud | bzip2 > nextcloud.sql.bz2 @@ -524,9 +292,6 @@ systemctl --user start nextcloud.service ## Not done here -- Rotating the secrets that are committed in plaintext — `pg_password` is the - fleet's shared `123456`, and it lives in `nextcloud.env` at mode `0600` rather - than in the world-readable unit file. -- Switching the background-job mode to `cron` (`occ background:cron`). -- Pinning the image to a digest, so that a `latest` bump is always deliberate. +- Rotating the secrets that are committed in plaintext. +- Pinning the image to a digest, so a `latest` bump is deliberate. - A backup timer for the webroot and the database. diff --git a/nextcloud/roles/nextcloud/tasks/main.yaml b/nextcloud/roles/nextcloud/tasks/main.yaml index de1e638..06905da 100644 --- a/nextcloud/roles/nextcloud/tasks/main.yaml +++ b/nextcloud/roles/nextcloud/tasks/main.yaml @@ -1,20 +1,16 @@ --- # Nextcloud as a rootless Podman Quadlet unit. # -# Two paths, and the same play serves both: +# Everything runs as the connection user: the units belong to it, so +# `systemctl --user` works through the SSH session's own PAM session — no become, +# no XDG_RUNTIME_DIR juggling. Requires gather_facts (set in playbook.yml). # -# - a fresh webroot, where the image initialises itself and Apache serves the -# setup wizard (or installs unattended, if the admin credentials below are -# set); and -# - an existing one, where an instance is already installed and this run is an -# update. -# -# Nothing here is tied to a particular host, and nothing has to be copied in by -# hand. Everything runs as {{ podman_user }} — the connection user. No become, no -# XDG_RUNTIME_DIR: the SSH connection gave us a real PAM session, so systemd's -# user manager is reachable through the standard environment. +# A fresh webroot is a valid state. The image initialises the code and serves the +# setup wizard, and this play succeeds. Set nextcloud_admin_user and +# nextcloud_admin_password in vars/default.yml to have it installed unattended +# instead. -- name: Ensure the Nextcloud webroot directory exists +- name: Ensure the webroot exists ansible.builtin.file: path: "{{ host_directory }}" state: directory @@ -22,31 +18,25 @@ group: "{{ podman_user }}" mode: '0755' -# The entrypoint takes an flock on this path using a root-owned shell redirect -# (`) 9> /var/www/html/nextcloud-init-sync.lock`, in the nextcloud/docker -# entrypoint). Creating it here means it belongs to {{ podman_user }} and stays -# that way: the redirect *truncates* the file, it does not recreate it, so its -# owner never changes. -# -# Left to the container, the file would be created by the entrypoint's root phase -# — a subuid under keep-id — and would be the one thing in the webroot not owned -# by {{ podman_user }}. -# -# `force: false` leaves an existing file completely alone, which is what makes -# this idempotent and what stops it failing on a file a previous run stranded as -# a subuid (a host user cannot chown a file it does not own). -- name: Pre-create the entrypoint's init lock file - ansible.builtin.copy: - dest: "{{ host_directory }}/nextcloud-init-sync.lock" - content: "" - force: false +# The entrypoint generates PHP's session config with a redirect into +# /usr/local/etc/php/conf.d, a directory the image creates as root and never +# chowns. That write is refused, the entrypoint exits 2, and the unit crash-loops +# into systemd's start limit. Bind-mounting a file over that path gives the write +# somewhere it is allowed; the file has to exist first, or the mount lands as a +# directory. +- name: Create the file the entrypoint writes PHP's session config to + ansible.builtin.file: + path: "{{ host_directory }}/redis-session.ini" + state: touch owner: "{{ podman_user }}" group: "{{ podman_user }}" mode: '0644' + access_time: preserve + modification_time: preserve -# The app's database and cache coordinates are consumed by systemd -# (EnvironmentFile= in the Quadlet unit), not by Ansible, so this has to be -# written before the unit starts. Mode 0600: POSTGRES_PASSWORD is in it. +# Read by the entrypoint, not by Ansible, so it has to exist before the unit +# starts. Mode 0600: POSTGRES_PASSWORD is in it, and the unit file is +# world-readable. - name: Write the environment file ansible.builtin.template: src: nextcloud.env.j2 @@ -56,73 +46,7 @@ mode: '0600' notify: Restart Nextcloud -# An update in a Quadlet deployment is "pull, then recreate": the unit references -# a tag, `podman run` reuses whatever image is local, and nothing fetches a newer -# `latest` on its own. The ID is read either side of the pull so an unchanged tag -# reports no change and does not restart a running site on every run of this play. -# -# The pull is allowed to fail: a host that already has the image and no route to a -# registry should still converge, and the start below reports the real problem if -# there is no image at all. -- name: Read the current image ID - ansible.builtin.command: >- - podman image inspect --format '{% raw %}{{ .Id }}{% endraw %}' {{ image }} - register: nextcloud_image_before - changed_when: false - failed_when: false - -- name: Pull {{ image }} - ansible.builtin.command: "podman pull {{ image }}" - register: nextcloud_image_pull - changed_when: false - failed_when: false - -- name: Warn if the image could not be pulled - ansible.builtin.debug: - msg: >- - Could not pull {{ image }} ({{ nextcloud_image_pull.stderr | default('') | trim }}). - Continuing with the image already on this host, if there is one. - when: nextcloud_image_pull.rc != 0 - -- name: Note whether the pull changed the image - ansible.builtin.command: >- - podman image inspect --format '{% raw %}{{ .Id }}{% endraw %}' {{ image }} - register: nextcloud_image_after - changed_when: >- - (nextcloud_image_before.stdout | default('')) | length > 0 - and nextcloud_image_before.stdout != nextcloud_image_after.stdout - notify: Restart Nextcloud - -# PHP's session-handler config, and the one path in the image the entrypoint has -# to write to. With REDIS_HOST set it generates this file on every start, in a -# directory the php base image creates as root with no chown: -# -# RUN mkdir -p "$PHP_INI_DIR/conf.d" -# -# The uid this container runs as therefore cannot create a file there, and the -# entrypoint exits before Apache is reached. Mounting a host-owned file over the -# path gives that write somewhere it is allowed to land — a bind-mounted file -# replaces the directory entry, so the directory's ownership stops mattering — and -# the template keeps the image's own content in it, so a refused write is not -# fatal either. See readme.md, "The PHP session file". -- name: Ensure the PHP session config directory exists - ansible.builtin.file: - path: "{{ php_conf_dir }}" - state: directory - owner: "{{ podman_user }}" - group: "{{ podman_user }}" - mode: '0755' - -- name: Write the PHP session handler config - ansible.builtin.template: - src: redis-session.ini.j2 - dest: "{{ php_session_ini }}" - owner: "{{ podman_user }}" - group: "{{ podman_user }}" - mode: '0644' - notify: Restart Nextcloud - -- name: Install the Nextcloud Quadlet unit +- name: Install the Quadlet unit ansible.builtin.template: src: nextcloud.container.j2 dest: "{{ quadlet_dir }}/{{ container_name }}.container" @@ -133,7 +57,9 @@ - Reload systemd user units - Restart Nextcloud -- name: Start Nextcloud and wait for the unit to become active +# Quadlet applies [Install] WantedBy= when it generates the unit, so a generated +# unit is never `systemctl enable`d. +- name: Start Nextcloud and wait for it to serve block: - name: Start Nextcloud ansible.builtin.systemd_service: @@ -142,162 +68,52 @@ state: started daemon_reload: true - # Quadlet honours [Install] WantedBy= when it generates the unit, so - # `systemctl --user enable` is neither needed nor possible on a generated - # unit. - # - # This waits rather than checking once, because `systemctl start` returns - # while the unit is still `activating` and `is-active` exits 3 for that state. - # - # `activating` is not evidence of a healthy start: a container that crash-loops - # spends its time `activating` between `Restart=always` attempts. What this - # wait buys is a prompt, bounded stop — it retries only while the state can - # still change on its own, and a terminal `failed` or `inactive` falls through - # to the rescue below, which has the logs. - - name: Wait for the Nextcloud unit to become active - ansible.builtin.command: "systemctl --user is-active {{ unit_name }}" - register: nextcloud_active - until: >- - nextcloud_active.stdout == 'active' - or nextcloud_active.stdout not in ['activating', 'reloading'] + # `systemctl start` returns while the unit is still `activating`, and a + # crash-looping container is `activating` most of the time as well, so + # is-active proves nothing. status.php is Nextcloud's own endpoint: it answers + # only once the entrypoint has finished (including any `occ upgrade`) and + # Apache is up, it reports `installed` either way — so one gate fits both a + # fresh webroot and an update — and `maintenance` is in it too, because a + # maintenance-mode 503 on every page is otherwise invisible to a probe. + - name: Wait for Nextcloud to serve status.php + ansible.builtin.command: >- + podman exec {{ container_name }} php -r + '$b=@file_get_contents("http://127.0.0.1/status.php"); if ($b === false) exit(1); + $j=json_decode($b, true); if (!is_array($j) || !array_key_exists("installed", $j)) exit(1); + exit(empty($j["maintenance"]) ? 0 : 1);' + register: nextcloud_serving + until: nextcloud_serving.rc == 0 retries: 60 delay: 5 changed_when: false - failed_when: nextcloud_active.stdout != 'active' rescue: - # systemd reports "control process exited with error code" and nothing else, - # which is useless for the usual failures (an unattached network, an - # unreachable database, an unwritable webroot, a path in the image the - # entrypoint cannot write). Pull the real reason out. - # - # `systemctl cat` settles what systemd thinks it is running, and `podman ps -a` - # catches a container left behind under this name by an interrupted run, which - # makes `podman run` refuse to start and leaves `podman logs` showing the old - # container and explaining nothing. - - name: Collect Nextcloud diagnostics + # systemd reports "control process exited with error code" and nothing else. + # The reason is in the entrypoint's own output. + - name: Collect diagnostics ansible.builtin.command: "{{ item }}" loop: - "systemctl --user status {{ unit_name }} --no-pager -l" - - "systemctl --user cat {{ unit_name }}" - "journalctl --user -u {{ unit_name }} --no-pager -n 60" - - "podman ps -a --filter name={{ container_name }}" - "podman logs --tail 60 {{ container_name }}" register: nextcloud_diag changed_when: false failed_when: false - - name: Show Nextcloud diagnostics + - name: Show them ansible.builtin.debug: msg: "{{ nextcloud_diag.results | map(attribute='stdout') | list }}" - name: Fail ansible.builtin.fail: msg: >- - Nextcloud did not come up — see the diagnostics above. If `podman ps -a` - shows a container already using this name, it was left behind by an - interrupted run; clear it with: - systemctl --user reset-failed {{ unit_name }} ; - systemctl --user stop {{ unit_name }} ; podman rm -f {{ container_name }} - -# Apply anything queued above — a new image, a changed unit, a changed environment -# file — *before* verifying, so that the checks below describe what will keep -# running rather than what is about to be replaced. A restart here is picked up by -# the waits that follow. -- name: Apply queued changes before verifying - ansible.builtin.meta: flush_handlers - -# The universal readiness gate, and the one thing `is-active` cannot tell us: -# Apache answering over loopback. status.php is Nextcloud's own endpoint and -# reports `installed` either way, so this passes for a setup wizard that has not -# been completed yet — which is the normal state after a first run. -# -# The probe runs inside the container with the image's own PHP, so it depends on -# nothing the play has to install. -- name: Wait for Apache to serve status.php - ansible.builtin.command: >- - podman exec {{ container_name }} php -r - '$b=@file_get_contents("http://127.0.0.1/status.php"); if ($b === false) exit(1); - $j=json_decode($b, true); exit(isset($j["installed"]) ? 0 : 1);' - register: nextcloud_http - until: nextcloud_http.rc == 0 - retries: 60 - delay: 5 - changed_when: false - -# config.php is what the installer writes, so its presence is what separates "an -# installed instance" from "the image has initialised a webroot for the wizard". -# Reading it beats parsing `occ status`, whose output carries a warning line on an -# uninstalled instance and whose exit code is not a reliable signal. -- name: Check whether an instance is installed - ansible.builtin.stat: - path: "{{ host_directory }}/config/config.php" - register: nextcloud_config - -# An uninstalled instance is a successful run, not a failure: the container is up -# and serving the wizard, and this says what is left for a human (or for the two -# optional variables) to do. -- name: Note that the setup wizard still has to be finished - ansible.builtin.debug: - msg: >- - The container is running, but no instance is installed in - {{ host_directory }} yet. Open https://{{ host }} and finish the setup - wizard with the database credentials in {{ env_file }} — database - {{ pg_database }}, user {{ pg_user }}, host {{ db_host }}. To install - unattended instead, set nextcloud_admin_user and nextcloud_admin_password - in vars/default.yml and run this play again. - when: not nextcloud_config.stat.exists - -# The update path. `occ upgrade` sets maintenance mode for the length of a -# database upgrade, which the entrypoint runs itself when the image is newer than -# the installed version — so this waits for it to clear rather than asserting it -# is off, and only asks once an instance exists. -- name: Wait for maintenance mode to be off - ansible.builtin.command: >- - podman exec -u www-data {{ container_name }} - php occ status --output=json - register: nextcloud_status - until: >- - nextcloud_status.rc == 0 - and ('"maintenance":true' not in (nextcloud_status.stdout | replace(' ', ''))) - retries: 120 - delay: 5 - changed_when: false - when: nextcloud_config.stat.exists - -- name: Report the instance status - ansible.builtin.debug: - msg: "{{ nextcloud_status.stdout_lines | default(['(not installed yet)']) }}" - -# The database and the cache arrive over the networks that roles/postgres and -# roles/valkey attached them to. Check the names the app actually asks for, plus -# the container names, because a wrong alias is the failure that looks like a -# broken app rather than a broken network. -- name: Confirm Nextcloud can resolve the database and the cache - ansible.builtin.command: >- - podman exec {{ container_name }} getent hosts {{ item }} - loop: - - "{{ db_host }}" - - "{{ postgres_container }}" - - "{{ redis_host }}" - - "{{ valkey_container }}" - register: nextcloud_lookup - changed_when: false - failed_when: false - -- name: Report database and cache resolution - ansible.builtin.debug: - msg: >- - {{ item.item }} resolves as {{ item.stdout | default('(no answer)') }} - loop: "{{ nextcloud_lookup.results }}" - -# --------------------------------------------------------------------------- -# Background jobs -# --------------------------------------------------------------------------- -# Nextcloud expects cron.php every five minutes. It cannot be a crontab entry: -# root cannot `podman exec` into another user's rootless containers, and there is -# no container user to own a crontab. A systemd user timer next to the Quadlet -# units is the equivalent, and it is what comes back after a reboot. + Nextcloud is not serving — see the diagnostics above. A "Permission + denied" under /usr/local/etc/php/conf.d, anything about `db` or `redis`, + and "downgrading is not supported" each point somewhere different. +# Nextcloud wants `cron.php` every five minutes, and it cannot be a crontab entry: +# root cannot `podman exec` into another user's rootless containers, and a +# container has no crontab of its own. A systemd user timer is the equivalent, and +# it comes back after a reboot with the rest of the stack. - name: Ensure the systemd user unit directory exists ansible.builtin.file: path: "{{ user_unit_dir }}" @@ -306,7 +122,7 @@ group: "{{ podman_user }}" mode: '0755' -- name: Install the Nextcloud background-job service +- name: Install the background-job service ansible.builtin.template: src: nextcloud-cron.service.j2 dest: "{{ user_unit_dir }}/{{ cron_unit }}.service" @@ -314,7 +130,7 @@ group: "{{ podman_user }}" mode: '0644' -- name: Install the Nextcloud background-job timer +- name: Install the background-job timer ansible.builtin.template: src: nextcloud-cron.timer.j2 dest: "{{ user_unit_dir }}/{{ cron_unit }}.timer" @@ -322,23 +138,12 @@ group: "{{ podman_user }}" mode: '0644' -# Unlike the Quadlet units, a plain .timer is not generated, so it does need -# enabling — and `enabled: true` is what brings it back after a reboot, the same -# way `[Install] WantedBy=default.target` does for the container units. -- name: Enable and start the Nextcloud background-job timer +# A plain .timer is not generated, so unlike the Quadlet units it does have to be +# enabled — that is what brings it back after a reboot. +- name: Enable and start the background-job timer ansible.builtin.systemd_service: name: "{{ cron_unit }}.timer" scope: user enabled: true state: started daemon_reload: true - -- name: Report the background-job timer - ansible.builtin.command: "systemctl --user list-timers {{ cron_unit }}.timer --no-pager" - register: nextcloud_timer - changed_when: false - failed_when: false - -- name: Show the background-job timer - ansible.builtin.debug: - msg: "{{ nextcloud_timer.stdout_lines | default(['(no output)']) }}" diff --git a/nextcloud/roles/nextcloud/templates/nextcloud.container.j2 b/nextcloud/roles/nextcloud/templates/nextcloud.container.j2 index 0eab465..c3a59cc 100644 --- a/nextcloud/roles/nextcloud/templates/nextcloud.container.j2 +++ b/nextcloud/roles/nextcloud/templates/nextcloud.container.j2 @@ -1,11 +1,9 @@ # Nextcloud — rootless Podman Quadlet unit. # Managed by Ansible (nextcloud/roles/nextcloud). Do not edit by hand. # -# This unit joins {{ network_name }}.network, which is created by -# caddy/roles/register (the same include that attaches Caddy to it). The -# postgres container is joined to that network by postgres/roles/attach and the -# valkey container by valkey/roles/attach, which is what makes config.php's `db` -# and `redis` hosts resolve. +# The unit joins {{ network_name }}.network, which caddy/roles/register creates and +# attaches Caddy to. Postgres and Valkey are joined to it by their attach roles, +# which is what makes the `db` and `redis` hosts in config.php resolve. [Unit] Description=Nextcloud @@ -17,48 +15,32 @@ Wants=network-online.target ContainerName={{ container_name }} Image={{ image }} -# Map the container's www-data account (uid {{ app_uid }}) onto {{ podman_user }} -# on the host, so the webroot stays owned by {{ podman_user }} and no subuid-owned -# files appear to break backups. The entrypoint rsyncs the code in with -# `--chown www-data:www-data` and Apache runs its workers as that account. +# Maps the container's www-data (uid {{ app_uid }}) onto {{ podman_user }}, so the +# webroot stays owned by {{ podman_user }} and no subuid-owned files appear. The +# entrypoint rsyncs the code in with --chown www-data:www-data and Apache runs its +# workers as that account. UserNS=keep-id:uid={{ app_uid }},gid={{ app_gid }} -# POSTGRES_* and REDIS_HOST live here rather than as Environment= lines above: -# this unit is world-readable in ~/.config, and POSTGRES_PASSWORD is in it. Read -# by systemd, not by Ansible, so the file has to exist before this unit starts. -# -# POSTGRES_* are what the entrypoint installs with on a fresh webroot; -# REDIS_HOST is load-bearing on every start, because the entrypoint rewrites -# PHP's session-handler config from it. +# The app's environment, read by systemd rather than by Ansible. Mode 0600, +# because POSTGRES_PASSWORD is in it and this unit file is world-readable. EnvironmentFile={{ env_file }} Network={{ network_name }}.network -# The image's VOLUME and Apache document root. This is the whole installation: the -# code (version.php, core/, lib/, apps/), config/config.php, data/ and -# custom_apps/. The entrypoint's own version check reads version.php from here, so -# an image bump rsyncs new code into this directory and upgrades the database in -# place — see readme.md, "First run and updates". +# The image's VOLUME and Apache document root: the code, config.php, data/ and +# custom_apps/. The version check reads version.php from here, so an image bump +# rsyncs new code in and upgrades the database in place. Volume={{ host_directory }}:{{ webroot }} -# PHP's session-handler config. The entrypoint writes this file itself on every -# start, from REDIS_HOST, but it cannot create it: `/usr/local/etc/php/conf.d` is -# created by root in the php base image with no chown, so the uid this container -# runs as has no write bit on the directory. Mounting a host-owned file over the -# path is what makes that write land somewhere allowed — and because the mounted -# file replaces the directory entry, the directory's ownership never comes into it. -# -# This is the one place the entrypoint writes outside the webroot. Apache's own -# runtime directories (/var/run/apache2, /var/lock/apache2, /var/log/apache2) do -# not need the same treatment: the php base image chowns them to www-data and -# chmods them 1777. -Volume={{ php_session_ini }}:{{ php_session_ini_mount }} +# PHP's session config, which the entrypoint generates on every start. Without +# this mount it cannot: /usr/local/etc/php/conf.d belongs to the image's root. +# See the file task in roles/nextcloud for the failure this prevents. +Volume={{ host_directory }}/redis-session.ini:/usr/local/etc/php/conf.d/redis-session.ini [Service] Restart=always -# A first start initialises the webroot; a start after an image bump rsyncs the -# new code in and runs `occ upgrade` before Apache answers. Both can take minutes -# on a large instance. +# A first start initialises the webroot; a start after an image bump rsyncs code +# and runs `occ upgrade`. Both can take minutes. TimeoutStartSec=600 [Install] diff --git a/nextcloud/roles/nextcloud/templates/nextcloud.env.j2 b/nextcloud/roles/nextcloud/templates/nextcloud.env.j2 index b809654..fe86646 100644 --- a/nextcloud/roles/nextcloud/templates/nextcloud.env.j2 +++ b/nextcloud/roles/nextcloud/templates/nextcloud.env.j2 @@ -1,17 +1,11 @@ # Environment for {{ container_name }}.container. # Managed by Ansible (nextcloud/roles/nextcloud). Do not edit by hand. # -# Mode 0600 on purpose: the Quadlet unit that reads this file is world-readable, -# and POSTGRES_PASSWORD is in it. Note that `podman inspect` still prints the -# resulting environment to anyone who can run podman as this user. +# Mode 0600: this file is read by systemd, and POSTGRES_PASSWORD is in it. # -# POSTGRES_* are what the entrypoint uses to write config.php when it installs a -# fresh instance; on an instance that is already installed they only have to agree -# with what config.php already carries, and nothing rewrites it. -# -# REDIS_HOST is live on *every* start: the entrypoint regenerates PHP's session -# handler from it ({{ php_session_ini_mount }}), so a wrong value here shows up as -# failed logins rather than as a missing cache. +# POSTGRES_* are what the entrypoint installs with on a fresh webroot; on an +# installed instance they only have to agree with config.php. REDIS_HOST is live on +# every start — the entrypoint regenerates PHP's session config from it. POSTGRES_DB={{ pg_database }} POSTGRES_USER={{ pg_user }} POSTGRES_PASSWORD={{ pg_password }} @@ -19,8 +13,7 @@ POSTGRES_HOST={{ db_host }} REDIS_HOST={{ redis_host }} {% if nextcloud_admin_user | length > 0 and nextcloud_admin_password | length > 0 %} # Consumed only while there is no config.php, by the entrypoint's unattended -# install. They are set in vars/default.yml; remove them from there to go back to -# the web wizard on a fresh host. +# install. Empty by default, which leaves the setup wizard in charge. NEXTCLOUD_ADMIN_USER={{ nextcloud_admin_user }} NEXTCLOUD_ADMIN_PASSWORD={{ nextcloud_admin_password }} NEXTCLOUD_TRUSTED_DOMAINS={{ host }} diff --git a/nextcloud/roles/nextcloud/templates/redis-session.ini.j2 b/nextcloud/roles/nextcloud/templates/redis-session.ini.j2 deleted file mode 100644 index ad1f1ae..0000000 --- a/nextcloud/roles/nextcloud/templates/redis-session.ini.j2 +++ /dev/null @@ -1,18 +0,0 @@ -; PHP session handler for Redis. -; Managed by Ansible (nextcloud/roles/nextcloud). Do not edit by hand. -; -; The nextcloud image writes this file itself on every start — it is how -; REDIS_HOST reaches PHP — and this file exists because that write has nowhere to -; land on its own: the image's /usr/local/etc/php/conf.d belongs to root and is -; not writable by the uid this container runs as, which makes the entrypoint exit -; before Apache ever starts. See readme.md, "The PHP session file". -; -; The contents are what the entrypoint generates for a Redis host with no -; password, so a start in which the entrypoint's own write is refused still gets -; working Redis sessions — and the entrypoint overwrites this file with its own -; version whenever it can. -session.save_handler = redis -session.save_path = "tcp://{{ redis_host }}:6379" -redis.session.locking_enabled = 1 -redis.session.lock_retries = -1 -redis.session.lock_wait_time = 10000 diff --git a/nextcloud/roles/postgres/tasks/main.yaml b/nextcloud/roles/postgres/tasks/main.yaml index 3be9784..9843ec5 100644 --- a/nextcloud/roles/postgres/tasks/main.yaml +++ b/nextcloud/roles/postgres/tasks/main.yaml @@ -1,41 +1,21 @@ --- -# Attach this app's network to the shared Postgres unit. -# -# Postgres knows nothing about its clients, exactly like Caddy: a service joins -# by dropping 50-nextcloud.conf and the shared 10-network-aliases.conf next to -# postgres.container. The canonical way to do that is the shared include below — -# it writes the same bytes every service writes, asserts that nextcloud.network -# already exists, and notifies the "Restart Postgres" handler defined in this -# role. -# -# This runs in the roles phase, which is after the register include in -# playbook.yml's pre_tasks has created nextcloud.network. +# Attach this app's network to the shared Postgres unit, and make sure the +# database exists. - name: Attach Nextcloud to Postgres ansible.builtin.include_tasks: "{{ playbook_dir }}/../postgres/roles/attach/tasks/main.yaml" -# On a first attach both drop-ins are new, so Postgres has to be recreated to -# join nextcloud_net — and that restart is a *handler*, which by default runs at -# the end of the play, after the nextcloud role has already started the app. The -# app would then come up before `db` resolves, fail its own readiness check, and -# fail the play on a machine where nothing is actually wrong. -# -# Flushing applies the new drop-in before the app starts. It also flushes the -# "Restart Caddy" that the register pre_task queued, which is what attaches Caddy -# to nextcloud_net — also wanted before the app is expected to serve. On later -# runs nothing is pending and this is a no-op. -- name: Apply the Postgres network change before starting Nextcloud +# The restart that joins Postgres to nextcloud_net is a handler, so by default it +# would run at the end of the play — after the app has started and failed to +# resolve `db`. Flushing first also applies the Caddy restart queued by the +# register pre_task. Nothing is pending on later runs. +- name: Apply the network change before starting Nextcloud ansible.builtin.meta: flush_handlers -# The database itself. Postgres is a shared unit that ships one default database, -# and nothing else in this repo knows what this app's database is called — so -# without this, a first run on a fresh host reaches the setup wizard (or the -# unattended install) only to fail on "database {{ pg_database }} does not exist". -# Later runs find it and do nothing. -# -# psql runs inside the Postgres container rather than over the app network, so the -# only thing this needs is a cluster that answers. The retry loop is there because -# the flush above may have just restarted it. +# Postgres ships one default database and only this app knows the name of its own, +# so on a fresh cluster the installer would otherwise fail with "database +# nextcloud does not exist". The retry is for the restart the flush above may have +# just done. - name: Check whether the {{ pg_database }} database exists ansible.builtin.command: >- podman exec -e PGPASSWORD={{ pg_password }} {{ postgres_container }} @@ -52,10 +32,9 @@ ansible.builtin.assert: that: nextcloud_db.rc == 0 fail_msg: >- - The shared Postgres container ({{ postgres_container }}) did not answer. - Run postgres/playbook.yml on this host first: this playbook attaches to that - cluster and creates its own database in it. Postgres said: - {{ nextcloud_db.stderr | default('') | trim }} + {{ postgres_container }} did not answer. Run postgres/playbook.yml on this + host first — this play attaches to that cluster and creates its own database + in it. - name: Create the {{ pg_database }} database ansible.builtin.command: >- diff --git a/nextcloud/vars/default.yml b/nextcloud/vars/default.yml index 5778c4f..b2df7dd 100644 --- a/nextcloud/vars/default.yml +++ b/nextcloud/vars/default.yml @@ -2,16 +2,10 @@ # --------------------------------------------------------------------------- # Rootless Podman / Quadlet # --------------------------------------------------------------------------- -# Duplicated on purpose from caddy/vars/default.yml: a play cannot include that -# file wholesale, because it also sets `host_directory` — to the Caddy data -# directory — and every playbook defines that for itself. Keep these three in -# sync with caddy/vars/default.yml. (Its fourth, podman_unprivileged_port_start, -# belongs to the podman tasks, which this play does not run: Nextcloud listens on -# 80 inside the container and publishes nothing host-side.) -# -# podman_user is derived from the connection: this play carries no become, so the -# Quadlet units belong to whoever Ansible connects as. Requires gather_facts (set -# in playbook.yml). +# Duplicated from caddy/vars/default.yml (which also sets `host_directory` for +# Caddy, so it cannot be included wholesale). Keep the three in sync. Its fourth, +# podman_unprivileged_port_start, is not needed here: Nextcloud publishes nothing +# host-side. podman_user: "{{ ansible_facts['user_id'] }}" podman_data_dir: "{{ ansible_facts['user_dir'] }}/podman-data" quadlet_dir: "{{ ansible_facts['user_dir'] }}/.config/containers/systemd" @@ -19,118 +13,80 @@ quadlet_dir: "{{ ansible_facts['user_dir'] }}/.config/containers/systemd" # --------------------------------------------------------------------------- # Nextcloud # --------------------------------------------------------------------------- -# The one site-specific value: the hostname Caddy serves this instance on and the -# hostname that goes into trusted_domains when the install is unattended. +# The site's hostname: what Caddy serves, and the trusted domain for an unattended +# install. host: cloud.hopsngrains.com container_name: nextcloud unit_name: "{{ container_name }}.service" -# Fully qualified on purpose: rootless Podman resolves short names against -# unqualified-search-registries, and there is no TTY to answer a prompt when the -# systemd unit starts. +# Fully qualified: rootless Podman resolves short names against +# unqualified-search-registries, and a unit has no TTY to answer the prompt. image: docker.io/library/nextcloud:latest -# The Quadlet file is .network. caddy/roles/register pins the -# Podman network name to _net (so: nextcloud.network, -# nextcloud_net). +# caddy/roles/register pins the Podman network name to _net, so the +# file is nextcloud.network and the network is nextcloud_net. network_name: "{{ container_name }}" -# The image's VOLUME and Apache document root — and, unlike the other services -# here, it holds the *code* as well as the state: config.php, data/, -# custom_apps/ and every app's PHP file live under it. See readme.md, "What is -# state, and what is not". +# The image's VOLUME and Apache document root, and — unlike the other services +# here — the code as well as the state: config.php, data/, custom_apps/ and every +# PHP file live under it. See readme.md, "What is state, and what is not". host_directory: "{{ podman_data_dir }}/{{ container_name }}" webroot: /var/www/html -# The uid the *server* writes as: 33 is www-data, which owns the webroot, because -# the entrypoint rsyncs the code in with --chown www-data:www-data and Apache runs -# its workers as that account. `docker exec nextcloud id -u` reports 0 in the -# upstream image, because the entrypoint drops privileges after starting — do not -# derive the mapping from that number (docs/podman-compat-audit.md §3.2). +# www-data. The entrypoint rsyncs the code in as this uid and Apache runs its +# workers as it, so this is the one uid to map onto the connection user. app_uid: 33 app_gid: 33 -# The port Apache listens on inside the container, and the one Caddy proxies to. +# Apache's port inside the container, and the one Caddy proxies to. container_port: 80 -# PHP's session-handler config, and the one path in the image the entrypoint has -# to write to. It generates the file from REDIS_HOST on every start, but its -# target directory (`/usr/local/etc/php/conf.d`) is created by root in the php -# base image with no chown, so the uid this container runs as cannot create a file -# inside it. Mounting a host-owned file over that exact path is what gives the -# write somewhere it is allowed to land, and the template keeps the image's own -# content in it so a refused write is not fatal either. See readme.md, "The PHP -# session file". -php_conf_dir: "{{ host_directory }}/php-conf.d" -php_session_ini: "{{ php_conf_dir }}/redis-session.ini" -php_session_ini_mount: /usr/local/etc/php/conf.d/redis-session.ini - -# POSTGRES_* and REDIS_HOST are read by the entrypoint on every start — REDIS_HOST -# in particular rewrites PHP's session handler config — and POSTGRES_PASSWORD is in -# there, so it goes in this 0600 file rather than in the world-readable unit. env_file: "{{ host_directory }}/nextcloud.env" -# The shared Postgres unit this app attaches itself to (postgres/playbook.yml). -# roles/postgres owns this app's drop-ins there and the "Restart Postgres" -# handler that applies them. +# --------------------------------------------------------------------------- +# Shared services (each its own playbook) +# --------------------------------------------------------------------------- +# Postgres, and the alias its attach role adds. postgres_container: postgres postgres_unit: "{{ postgres_container }}.service" +db_host: db -# The database this instance uses, and the credentials for it. They have to match -# whatever the shared Postgres unit was initialised with (postgres/vars/default.yml) -# — this play attaches to a database, it does not create one, so the database has -# to exist before the first run. +# The database this play creates if it is missing, and the credentials in +# postgres/vars/default.yml that the cluster was initialised with. pg_user: postgres pg_password: '123456' pg_database: nextcloud -# `db` is the alias postgres/roles/attach adds to the shared Postgres unit, and -# the host this instance's config.php connects to. -db_host: db - -# The shared Valkey unit this app attaches itself to (valkey/playbook.yml), and -# the alias it is reached by. REDIS_HOST is not cosmetic: config.php's -# `memcache.locking` is Redis, so an unresolvable `redis` breaks uploads rather -# than merely slowing them down. +# Valkey, and the alias its attach role adds. REDIS_HOST is not cosmetic: +# config.php puts `memcache.locking` on Redis, so an unresolvable `redis` breaks +# uploads rather than merely slowing them down. valkey_container: valkey valkey_unit: "{{ valkey_container }}.service" redis_host: redis -# The shared Caddy unit and its Caddyfile, both owned by caddy/playbook.yml. caddy_name: caddy caddyfile_path: "{{ podman_data_dir }}/{{ caddy_name }}/conf/Caddyfile" # --------------------------------------------------------------------------- # First-run install (optional) # --------------------------------------------------------------------------- -# Leave both empty to install through the web wizard at https://{{ host }} — the -# play reports that this is what is left to do and succeeds. Set both to have the -# image install unattended on a fresh webroot instead: it runs -# `occ maintenance:install` with these credentials and sets trusted_domains to -# `host` in the same pass, so the instance is usable immediately. -# -# They are only consumed while there is no config.php, so they are inert on an -# existing instance, and the environment template omits them entirely when they -# are empty (an empty NEXTCLOUD_ADMIN_USER would be treated as a value and fail -# the install). +# Empty by default: a fresh webroot is initialised by the image and the setup +# wizard takes over. Set both to have the image install unattended instead, with +# trusted_domains set from `host`. Inert once config.php exists. nextcloud_admin_user: "" nextcloud_admin_password: "" # --------------------------------------------------------------------------- # Background jobs # --------------------------------------------------------------------------- -# Nextcloud wants `cron.php` every five minutes. It cannot be a crontab entry: -# root cannot `podman exec` into another user's rootless containers, and there is -# no container user to own a crontab. A systemd user timer next to the Quadlet -# units is the equivalent. cron_unit: "{{ container_name }}-cron" cron_schedule: "*:0/5" user_unit_dir: "{{ ansible_facts['user_dir'] }}/.config/systemd/user" -# The site block. Caddy terminates TLS, and Nextcloud asks for HSTS. There is no -# request-body limit to carry over: Caddy streams bodies and imposes none by -# default. +# --------------------------------------------------------------------------- +# Caddy site block +# --------------------------------------------------------------------------- snippet_block: | {{ host }} { reverse_proxy {{ container_name }}:{{ container_port }}