Mount PHP's session config, the one path the entrypoint cannot write

The real first run of this play did not fail on a transition. The journal shows
the entrypoint dying on every attempt:

    nextcloud[302063]: => Configuring PHP session handler...
    nextcloud[302063]: /entrypoint.sh: 121: cannot create
      /usr/local/etc/php/conf.d/redis-session.ini: Permission denied

With REDIS_HOST set, the entrypoint generates PHP's session-handler config on
every start, and the target directory belongs to the image's root: the php base
image does `mkdir -p "$PHP_INI_DIR/conf.d"` with no chown. The uid this container
runs as has no write bit on it, the entrypoint exits 2 before Apache is reached,
Restart=always retries that, and systemd's start limit turns it into a `failed`
unit. Nothing in that shape says "permissions": the symptom is a unit state and a
message that only exists in journalctl.

The container now mounts a host-owned file over that exact path. A bind-mounted
file replaces the directory entry, so the directory's ownership stops mattering
and the write lands on podman_user. The role templates the five lines the
entrypoint would have written, so a refused write is no longer fatal either --
PHP still gets Redis sessions instead of a crash loop.

This also corrects a reading of the previous run that this play got wrong and
documented wrongly: `activating` is not evidence of a healthy start. A
crash-looping container spends its time `activating` between restarts, so the
earlier "reports a healthy start as a failure" comment and commit message were
describing a start that was in fact failing. The wait is still right -- it stops
promptly at a terminal state and hands over to a rescue that has the logs -- but
its comment now says what it actually buys.

Apache's own runtime directories need none of this, and the php base image is why:
it chowns /var/run/apache2, /var/lock/apache2 and /var/log/apache2 to www-data and
chmods them 1777, so only the session file was ever going to bite. The readme
gains a section on it, the diagnostic commands the play now collects, and a note
that the uid mapping is not to be derived from a `docker exec id -u` of 0.
This commit is contained in:
tanshu committed 2026-10-16 11:45:37 +05:30
1 parent 90b1149413
commit 1038037e45
5 files changed
+209 -31

No files matched your search

+110 -13
View File
@@ -70,6 +70,8 @@ Quadlet units and state:
| `~/.config/systemd/user/nextcloud-cron.{service,timer}` | the background-job timer (not Quadlet) |
| `~/podman-data/nextcloud/` | the whole `/var/www/html`: code, `config/config.php`, `data/`, `custom_apps/` |
| `~/podman-data/nextcloud/nextcloud.env` | `POSTGRES_*` + `REDIS_HOST`, mode `0600`, read by systemd |
| `~/podman-data/nextcloud/php-conf.d/redis-session.ini` | PHP's session handler, mounted into the image at `/usr/local/etc/php/conf.d/` |
| `~/podman-data/nextcloud/nextcloud-init-sync.lock` | the entrypoint's init lock, pre-created so it stays host-owned |
## What is state, and what is not
@@ -271,20 +273,77 @@ Like Caddy, `podman_user` is derived from the connection and the play carries no
connecting as `root`**; the prerequisites, including that check, live in
`caddy/playbook.yml`.
`UserNS=keep-id:uid=33,gid=33` — `33` is the uid the server writes as. The
entrypoint starts as container-root, but it rsyncs the code in with
`--chown www-data:www-data` and Apache then drops to `www-data`, so everything it
writes lands on `podman_user` and no subuid-owned files appear to break backups
([docs/podman-compat-audit.md §3.2](../docs/podman-compat-audit.md)). `33` is
also what the data is owned by on monoco, which is what makes the copy below need
no `chown` pass.
`UserNS=keep-id:uid=33,gid=33` — `33` is the uid the server writes as. Whatever
the entrypoint's *own* starting uid under this mapping, the writes that matter land
on `podman_user`: the code is rsynced with `--chown www-data:www-data`, and Apache's
workers run as `www-data`
([docs/podman-compat-audit.md §3.2](../docs/podman-compat-audit.md)). `33` is also
what the data is owned by on monoco, which is what makes the copy below need no
`chown` pass.
One file is created by container-root and never chowned: the entrypoint takes an
`flock` on `/var/www/html/nextcloud-init-sync.lock` using a root-owned shell
redirect. The role **pre-creates it** as `podman_user`, because the redirect
truncates the file rather than recreating it — so its owner never changes. Left
to the container it would be the one file in the webroot owned by a `165536+`
subuid and not removable by the host user.
Two paths the entrypoint writes fall outside that arrangement, and both are handled
explicitly rather than left to chance — each one is somewhere a naive deployment
ends up with a file owned by a `165536+` subuid, or with no deployment at all:
- **`/var/www/html/nextcloud-init-sync.lock`**, which the entrypoint creates with a
root-owned shell redirect. The role **pre-creates it** as `podman_user`: the
redirect truncates the file rather than recreating it, so its owner never changes
afterwards.
- **`/usr/local/etc/php/conf.d/redis-session.ini`**, in the image, which is not
writable at all — it is covered in its own section below.
### The PHP session file
The webroot is not the only thing the entrypoint writes. With `REDIS_HOST` set —
which it is, because `config.php` puts `memcache.locking` in Redis and PHP
sessions should survive a redeploy — the entrypoint generates that session
configuration on **every** start:
```sh
/entrypoint.sh:121
} > /usr/local/etc/php/conf.d/redis-session.ini
```
That directory belongs to the image's root, created by the php base image with no
`chown`:
```dockerfile
RUN set -eux; \
mkdir -p "$PHP_INI_DIR/conf.d"; \
...
```
so the uid this container runs as has no write bit on it, and the redirect fails:
```
cannot create /usr/local/etc/php/conf.d/redis-session.ini: Permission denied
```
The entrypoint exits `2` before Apache is ever reached, `Restart=always` retries
it, systemd hits its start limit, and the unit ends up **`failed`** — with the
whole thing looking like a network or database problem, because that message only
appears in the journal.
The fix is a mount: `nextcloud.container` binds
`~/podman-data/nextcloud/php-conf.d/redis-session.ini` over that exact path. A
bind-mounted file replaces the directory entry, so the directory's ownership stops
mattering and the write lands on a file owned by `podman_user`. The role also
templates the file with the five lines the entrypoint would have written, so a
start where the write is somehow refused still gets working Redis sessions instead
of a crash loop.
Apache's *own* runtime directories need none of this — the same base image chowns
`/var/run/apache2`, `/var/lock/apache2` and `/var/log/apache2` to `www-data` and
chmods them `1777`:
```dockerfile
for dir in "$APACHE_LOCK_DIR" "$APACHE_RUN_DIR" "$APACHE_LOG_DIR" \
"$APACHE_RUN_DIR/socks"; do
rm -rvf "$dir"; mkdir -p "$dir"; \
chown "$APACHE_RUN_USER:$APACHE_RUN_GROUP" "$dir"; \
chmod 1777 "$dir"; \
done
```
`POSTGRES_PASSWORD` is the fleet's shared `123456`, which matches
`pg_password` in [postgres/vars/default.yml](../postgres/vars/default.yml) and
@@ -548,8 +607,46 @@ An empty `trusted_proxies` means every client looks like the proxy; see
[caddy/readme.md](../caddy/readme.md), "Client IPs", which also explains why the
real address may not survive the rootless port forward at all.
### Debugging a unit that will not start
A unit that has hit systemd's start limit reports only a state, and `Restart=always`
means the interesting output is in the journal, not in `podman logs` (the container
may already be gone — Quadlet passes `--rm`). The play collects all of this itself
when a start fails; by hand:
```bash
ssh emily 'systemctl --user status nextcloud.service --no-pager -l | head -25
systemctl --user cat nextcloud.service # what systemd is running
journalctl --user -u nextcloud.service --no-pager -n 60
podman ps -a --filter name=nextcloud
podman logs --tail 40 nextcloud'
```
Read it in this order:
1. **`journalctl`** — the entrypoint's own output. `cannot create ...: Permission
denied` for a path in `/usr/local/etc/php/conf.d` is the session-file problem
above; anything about `db`/`redis` is the network attach; `downgrading is not
supported` is version drift.
2. **`podman ps -a`** — a container left behind under this name makes `podman run`
refuse to start. Quadlet passes `--replace`, so this is rare, and it means an
interrupted run.
3. **`systemctl --user cat`** — the generated unit, if a drop-in or a `Network=`
is in question (the usual failure is a `.network` file that does not exist).
The first real run of this playbook failed at a bare `is-active` check with no
diagnostics at all, which is why the wait now lives inside the start block whose
rescue collects the above.
## Notes and gotchas
- **`REDIS_HOST` makes the entrypoint write into the image, which it is not
allowed to do.** `/usr/local/etc/php/conf.d` belongs to the image's root, so the
crash is a `Permission denied` on `redis-session.ini`, the unit ends up `failed`
via systemd's start limit, and nothing about it looks like a permissions
problem. The unit mounts a host-owned file over that path; see
["The PHP session file"](#the-php-session-file) above, including why a refused
write is no longer fatal.
- **Maintenance mode travels with `config.php`.** The backup plan recommends
`occ maintenance:mode --on` before a consistent dump
([docs/backup-plan.md §3.4](../docs/backup-plan.md)); if monoco was left that