tanshu/playbooks
1
0 Fork 0
Code Issues Pull Requests Releases Wiki Activity

TriliumNext: User=0 for keep-id, and our own health check

Browse Source 
The first run of this playbook against emily failed, and all three reasons are
worth recording because none of them is visible from the files alone.

keep-id runs the container *process* as the uid it maps, so
UserNS=keep-id:uid=1000,gid=1000 started the container as `node` rather than root
-- and the image's start-docker.sh is written to start as root, ending with
`exec su -c "node ./main.mjs" node`. `su` from a non-root uid asks for a password
the node account does not have, so the container died with "su: Authentication
failure" and Restart=always turned that into a crash loop. User=0 forces the
process back to root and changes nothing about the mapping: node is still tanshu,
so the app is still what writes the data directory and it stays tanshu's.
nextcloud.container carries the same pair for the same reason.

The health check had to be defined rather than inherited. `latest` floated between
the two pulls -- monoco runs v0.95.0 (built 2025-06-15) and emily pulled v0.106.0
(built 2026-09-25) -- and v0.106.0 ships no HEALTHCHECK where v0.95.0 had
docker_healthcheck.cjs, so `podman healthcheck run` failed with "no defined
healthcheck". The unit now defines HealthCmd= against /api/health-check, the
endpoint the app advertises itself in the healthcheck-url file it writes into the
data directory, and the image has curl for it (but not wget).

The verify and health-check steps are folded in with the start, because they fail
the same way and want the same evidence. `systemctl start` reports success as soon
as systemd holds the unit, so a container that dies a second later surfaced only
as "activating" from `is-active`; the reason -- that "su: Authentication failure"
-- was in the journal and nowhere the play looked. The rescue now dumps status,
journal and podman logs for any of the three, which is how the su failure was
found in the first place.

The version float becomes a documented risk rather than a hypothetical: the
cutover upgrades the app as well as the runtime, and the first start migrates
document.db forward. That is one-way for the copy on emily, so the readme compares
digests before the first start, keeps monoco's own tree as the rollback, and notes
that pinning a tag here would decouple the two changes.

Verified on emily rather than by inspection: the rendered unit was run first as a
throwaway Quadlet unit against a scratch volume, then the playbook itself ran to
ok=22/changed=6/failed=0 with the mount assertion passing, `healthcheck run`
returning 0, podman reporting healthy, and the app's files landing in
~/podman-data/trilium owned by tanshu. The data directory was emptied afterwards
and the unit left stopped, ready for the copy from monoco, which has not run yet.
This commit is contained in:
tanshu committed 2026-10-16 16:31:16 +05:30
1 parent 17c64cf2f8
commit 3d4ccdd6d1
4 files changed
+161 -64
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

trilium/readme.md
+81 -28
View File
@@ -52,10 +52,23 @@ misspelling survives in exactly one place, deliberately: the **source path**
`/var/lib/trillium` in the copy commands below, which is what the old host
actually has.
The image's data directory is `/home/node/trilium-data`, and the image ships its
own `HEALTHCHECK` (`docker_healthcheck.cjs`), which the Docker-era container
reported healthy with. The unit therefore defines no `HealthCmd=` — unlike
[audiobookshelf/](../audiobookshelf/readme.md), whose image has none.
Two things about the image had to be settled against `emily` rather than read off
`monoco`, because `latest` floated by far more than a patch release between the
two pulls:
| | monoco (running) | emily (pulled) |
|---|---|---|
| version | v0.95.0, built 2025-06-15 | **v0.106.0**, built 2026-09-25 |
| `HEALTHCHECK` | ships one (`docker_healthcheck.cjs`) | **ships none** |
So the unit defines its own `HealthCmd=`, probing the endpoint the app advertises
in the `healthcheck-url` file it writes into the data directory, and the cutover
is **also an app upgrade**: TriliumNext migrates `document.db` forward on first
start. That migration is one-way for the copy on emily, which is why the digests
below are compared *before* the first start and monoco's own tree is left alone.
The image also runs as root and drops to `node` with `su`, which is what `User=0`
exists for — see [Settings](#settings).
## Ordering
@@ -69,7 +82,10 @@ this app has no database.
This playbook does **not** move the data. Copy it first (see
["Migrating the data"](#migrating-the-data-from-monoco)) so the first start opens
the real `document.db`, rather than creating an empty document beside it.
the real `document.db`, rather than creating an empty document beside it. If it
does start against an empty directory — which the play warns about — that empty
document and its log have to be cleared before the copy, and the copy's steps say
how.
## Layout
@@ -92,6 +108,7 @@ Quadlet units and state:
| `~/podman-data/trilium/document.db` | the document, with `-wal` / `-shm` beside it |
| `~/podman-data/trilium/config.ini` | the app's settings — `port`, `trustedReverseProxy`, `noBackup` |
| `~/podman-data/trilium/session_secret.txt` | the session signing secret |
| `~/podman-data/trilium/healthcheck-url` | the app's own health endpoint, written on every start. `HealthCmd=` in the unit probes it |
| `~/podman-data/trilium/log/` | one log file per day, 91 MB of the 106 MB tree |
| `~/podman-data/trilium/backup/` | Trilium's own zipped `document.db` backups |
| `~/podman-data/trilium/tmp/` | scratch, empty at rest |
@@ -163,6 +180,7 @@ port-floor work Caddy itself needs.
| `container_data_dir` | `/home/node/trilium-data` — the image's data directory |
| `log_dir` / `tmp_dir` / `backup_dir` | The three app-managed subdirectories, created at 0700 |
| `container_port` | `8080`, matching `config.ini`'s `port` |
| `health_cmd` | The unit's `HealthCmd=`, since the image ships none |
| `caddy_name` / `caddyfile_path` | The shared Caddy unit and its Caddyfile |
The three `podman_*` / `quadlet_dir` vars are **duplicated** from
@@ -178,12 +196,23 @@ Like Caddy, `podman_user` is derived from the connection and the play carries no
connecting as `root`**; the prerequisites, including that check, live in
`caddy/playbook.yml`.
`UserNS=keep-id:uid=1000,gid=1000` — the entrypoint starts as container-root and
drops to the image's `node` user, which is uid 1000, and uid 1000 is what owns
every file in the Docker-era tree on monoco
([docs/podman-compat-audit.md §3.2](../docs/podman-compat-audit.md)). Mapping
container uid 1000 onto `podman_user` keeps `~/podman-data/trilium` owned by
`tanshu` and leaves no subuid-owned files to trip over backups.
Two lines decide who the container is, and **both are needed** — this is the one
thing that actually broke the first deployment of this playbook, and it is the
same pair [nextcloud/](../nextcloud/readme.md) needs:
- **`UserNS=keep-id:uid=1000,gid=1000`** — `1000` is `node`, the user the app runs
as, and it is also what owns every file in the Docker-era tree on monoco
([docs/podman-compat-audit.md §3.2](../docs/podman-compat-audit.md)). `keep-id`
maps that one uid onto `podman_user`, so `~/podman-data/trilium` stays owned by
`tanshu` and plain `tar`, `rm -rf` and the backup scripts keep working on it.
- **`User=0`** — `keep-id` also runs the container *process* as that mapped uid,
i.e. as `node`, while the image's `start-docker.sh` is written to start as root:
it ends with `exec su -c "node ./main.mjs" node`, and `su` from a non-root uid
asks for a password the `node` account does not have. Without `User=0` the
container dies with `su: Authentication failure` and `Restart=always` turns that
into a crash loop. Forcing the process back to root restores what the script
expects and changes nothing about the mapping above: `node` is still `tanshu`,
so the app is still what writes the data directory and it stays `tanshu`'s.
**There is no environment file here**, unlike `bitwarden/` and `tandoor/`. This
deployment holds no secrets Ansible has to pass in: `config.ini` and
@@ -243,6 +272,11 @@ ssh monoco 'docker stop trillium'
# 3. Create the destination — this is the mount target, /home/node/trilium-data.
ssh emily 'mkdir -p ~/podman-data/trilium'
# If this playbook has already run against that directory — it warns when it
# does — the app has left a document of its own there: an empty one, plus its
# log and healthcheck-url. Clear it, or step 5's digests will not match:
# ssh emily 'systemctl --user stop trilium.service; find ~/podman-data/trilium -mindepth 1 -delete'
# 4. Copy. `sudo` on the source side is not optional here: log/, tmp/ and
# backup/ are 0700 and unreadable as tanshu, and backup/ is state.
#
@@ -317,7 +351,7 @@ journalctl --user -u trilium -f
podman logs trilium
# What the playbook checks for you:
podman healthcheck run trilium # the image's own probe
podman healthcheck run trilium # the unit's own HealthCmd=, not the image's
# The unit's mount is real and not the container's writable layer:
podman inspect trilium --format '{{range .Mounts}}{{.Source}} -> {{.Destination}}{{"\n"}}{{end}}'
@@ -358,24 +392,41 @@ Then reboot the host and confirm the unit returns unattended — without
`/root/trilium-data` and predates TriliumNext's schema migrations; pointed at
this `document.db` it is at best a failure to start. The unit pins
`docker.io/triliumnext/notes:latest`.
- **The image's own health check is used.** There is no `HealthCmd=` in the unit
and there does not need to be: the image ships one
(`exec gosu node node /usr/src/app/docker_healthcheck.cjs`), which is why the
Docker-era container showed `healthy`. `is-active` alone cannot tell a serving
server from a crash-looping one under `Restart=always`, which is why the play
runs the healthcheck rather than trusting it.
- **The unit's health check is its own.** Unlike the v0.95.0 on monoco, the
v0.106.0 image ships no `HEALTHCHECK`, so `podman healthcheck run` would fail
with "no defined healthcheck"; the unit probes the endpoint the app advertises
in `healthcheck-url` instead. `is-active` alone cannot tell a serving server
from a crash-looping one under `Restart=always`, which is why the play runs the
healthcheck rather than trusting it.
- **`keep-id` runs the container as the mapped uid, and this image needs root.**
The full explanation is in [Settings](#settings); the short version is that
without `User=0` the image's own `su node` cannot work and the unit crash-loops.
The three failures this playbook has actually produced, and what each means:
| In the journal | Cause |
|---|---|
| `Password:` then `su: Authentication failure` | the container process is not root, so the image's privilege drop fails — `User=0` is missing |
| `no defined healthcheck` from `podman healthcheck run` | the image ships none and the unit's `HealthCmd=` is missing |
| `DB not initialized, please visit setup page` | the app is serving a **new, empty** document — the copy from monoco has not run (or was wiped) |
- **The mount check is a mount check, not a file check.** Audiobookshelf can
assert its database appeared on the host, because the app creates it. Here the
migration puts `document.db` on the host *before* the app runs, so its presence
proves nothing — the play asserts against `podman inspect` instead. That is the
one failure that is otherwise silent: with a missing `Volume=` line the app
runs perfectly and loses the document on the next recreate.
- **`latest` floats.** The image measured for this playbook is **v0.95.0**
(`org.opencontainers.image.version`), built 2025-06-15, revision `b9ae2265`.
Pin a digest if reproducible deploys matter —
`podman image inspect docker.io/triliumnext/notes:latest --format '{{.Digest}}'`
— since an upgrade can run schema migrations on `document.db` on first start.
Take Trilium's own backup (or stop-and-copy) before upgrading.
proves nothing — the play asserts against `podman inspect` instead, and checks
the *source* as well as the destination, because a destination-only check is
satisfied by an anonymous volume mounted where the bind mount should be. That is
the one failure that is otherwise silent: the app runs perfectly and loses the
document on the next recreate.
- **`latest` floats, and it moved here.** monoco runs **v0.95.0** (built
2025-06-15, revision `b9ae2265`) and the pull on emily is **v0.106.0** (built
2026-09-25, revision `c0fdceba`) — so this cutover upgrades the app as well as
the runtime, and the first start migrates `document.db` forward. That is
one-way *for the copy on emily*; monoco's own tree is untouched and remains the
rollback. Pin a version tag or a digest in `vars/default.yml` if the migration
should change only the runtime and leave the upgrade to a separate, deliberate
step (`podman image inspect docker.io/triliumnext/notes:latest --format
'{{.Digest}}'`). Take Trilium's own backup, or a stop-and-copy, before any
upgrade.
- **`trustedReverseProxy=false` in `config.ini`.** Trilium does not trust
`X-Forwarded-For`, so its own logs see the proxy rather than the client. That
is the existing behaviour, carried across unchanged; set it to `true` (or to
@@ -403,7 +454,9 @@ Then reboot the host and confirm the unit returns unattended — without
- **DNS.** `notes.tanshu.com` must resolve to `emily` before Caddy can obtain a
certificate for it. The certificates and ACME account key from monoco's
`/var/lib/caddy/data` are **not** part of this migration — Caddy's own playbook
and readme cover that.
and readme cover that. Until the name moves, emily's Caddy retries the ACME
order and logs the failures; that is expected while monoco still holds the name,
and the site keeps being served from monoco.
- **Root's crontab cannot `podman exec`** into another user's rootless
containers, so anything scheduled belongs in a **systemd user timer** next to
the Quadlet units. Trilium's own `backup/` mechanism runs *inside* the app