playbooks/wikijs

Wiki.js (rootless Podman + Quadlet)

Runs Wiki.js on emily as a rootless Podman container managed by a systemd user unit (Quadlet), behind the shared rootless Caddy, with its data in the shared rootless Postgres. Replaces the docker_container deployment on monoco.

The name is split on purpose: the container, unit, Podman network and Postgres database are all hiwikijs, as they have been since the Docker-era playbook, and the Caddyfile upstream is hiwikijs:3000. Only the directory is wikijs — the playbook's name. Renaming it would touch the Caddyfile, the network name and the database for no gain.

The database is the whole wiki. Every page, every revision, every user and every uploaded file lives in the hiwikijs database. The image's declared /wiki/data/content volume is empty on monoco, /wiki/data/uploads is empty too, and the 49 uploaded assets are rows in the assetData table (5.8 MB of the 16 MB database). So this is the one service in the fleet where "copy the old data" is a verification rather than a transfer — the commands that prove it are in Migrating from monoco.

This corrects the expectation set by docs/podman-compat-audit.md §3.1 and docs/backup-plan.md §7.6, which treat the anonymous volume as Wiki.js's store. What those documents are right about is the bug: ~/bin/hiwikijs-backup.sh archives /var/lib/hiwikijs, which is an empty decoy, so the wiki had no usable file backup. The Volume= mounts below fix the mechanism — nothing may live in the container's writable layer — but the bytes it was meant to capture are in Postgres, and they already moved.

Ordering

Both shared services are already migrated and running on emily:

1. caddy/playbook.yml — the reverse proxy, which owns the Caddyfile this site appends to, supplies the caddy*.service handlers, and is the one play that applies the rootless-Podman prerequisites.
2. postgres/playbook.yml — deployed, and its cluster brought across from monoco. The hiwikijs database is already restored (16 MB, one of the 12 databases in postgres/migrate-from-monoco.md).
3. This playbook.

Unlike the other services there is no file copy to do first, and there is no ordering constraint between this playbook and the data: the app is stateless on disk. Deploy it and point DNS at emily.

wiki.hinchco.in still resolves to monoco. Until the record is repointed, Caddy on emily cannot obtain a certificate for the site, and the old host is still what answers. That DNS change — not this playbook — is what performs the cutover.

Layout

wikijs/
├── playbook.yml
├── vars/default.yml
└── roles/
    ├── postgres/   # attaches this app's network to Postgres (+ its handler)
    ├── wikijs/     # data dirs, the 0600 env file, the Quadlet unit, start + verify
    └── caddy/      # appends this site's block to the shared Caddyfile

Quadlet units and state on the host:

Path	What
~/.config/containers/systemd/hiwikijs.container	the generated unit (Ansible-templated)
~/.config/containers/systemd/hiwikijs.network	the app network, written by caddy/roles/register
~/.config/containers/systemd/caddy.container.d/50-hiwikijs.conf	attaches Caddy to that network
~/.config/containers/systemd/postgres.container.d/50-hiwikijs.conf	attaches Postgres to it
~/.config/containers/systemd/postgres.container.d/10-network-aliases.conf	the shared db alias DB_HOST asks for
~/podman-data/hiwikijs/content/	the image's /wiki/data/content (empty in practice)
~/podman-data/hiwikijs/uploads/	/wiki/data/uploads, the local storage target
~/podman-data/hiwikijs/hiwikijs.env	DB_*, mode 0600, read by systemd

Note the network-name indirection, which is the same everywhere in this repo: the file is hiwikijs.network and the Podman network is hiwikijs_net, pinned by NetworkName= in the network unit. The Docker-era container used hiwikijs_net, so nothing about the network name changes.

What is state, and what is not

Where	Keep?	Why
Postgres hiwikijs	required	32 tables: pages (80), pageHistory, users (2), comments, navigation, settings, and assetData
assetData table	required	49 rows, 5.8 MB — the actual bytes of every uploaded image. A file-level backup of the wiki would miss all of them
~/podman-data/hiwikijs/content/	empty	the image's declared VOLUME. Empty on monoco, and Wiki.js writes nothing to it in this configuration
~/podman-data/hiwikijs/uploads/	empty	the default local storage target. The wiki is configured to store assets in the database instead
hiwikijs.env	regenerated	the template reproduces the Docker-era .env exactly
/var/lib/hiwikijs on monoco	no	a decoy: an empty directory holding one root-owned 90-byte .env, never mounted into the container

How it registers with Caddy

Caddy is the base unit and knows nothing about its upstreams. This playbook calls the shared include in caddy/roles/register, which writes the network unit and the drop-in that pulls it into Caddy:

- name: Register Wiki.js's network with Caddy
  ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/register/tasks/main.yaml"

It runs in pre_tasks, before the app unit is started, because hiwikijs.container references hiwikijs.network: Quadlet fails unit generation — so systemctl --user start reports "unit not found" — when a referenced .network file does not exist yet. The Postgres attach below depends on it for the same reason.

The site block itself is appended to the shared Caddyfile by roles/caddy with blockinfile, giving exactly the block the Docker-era Caddyfile had:

# BEGIN Ansible managed Caddy snippet for wiki.hinchco.in
wiki.hinchco.in {
	reverse_proxy hiwikijs:3000
}
# END Ansible managed Caddy snippet for wiki.hinchco.in

caddyfile_path changed from /var/lib/caddy/conf/Caddyfile to ~/podman-data/caddy/conf/Caddyfile; the marker and the upstream are unchanged, so running this playbook against the old host's Caddyfile would be a no-op.

The old roles/caddy slurped the Caddyfile and compared strings before appending. That is gone: blockinfile is idempotent through its marker, and slurp fails when the file does not exist yet, which is the case on a host where the Caddy playbook has not run.

How it attaches to Postgres

Same idea at the other end: Postgres knows nothing about its clients, and this playbook uses the shared include in postgres/roles/attach rather than writing those drop-ins itself. That writes two files, both merged into postgres.container by Quadlet:

File	Content
postgres.container.d/50-hiwikijs.conf	Network=hiwikijs.network
postgres.container.d/10-network-aliases.conf	NetworkAlias=db

DB_HOST=db is what the Docker-era container used — it was --link postgres:db — and 10-network-aliases.conf is the Quadlet equivalent. postgres, the container name, resolves too, and the play checks both.

roles/postgres wraps that include for two reasons:

• it defines the Restart Postgres handler the include notifies; and
• it flushes handlers immediately afterwards. On a first attach the drop-in is new, so Postgres has to be recreated to join hiwikijs_net — and that restart is a handler, which by default runs at the end of the play, after the wikijs role has already started the app. The app would then come up before db resolves, fail its own HTTP probe, and fail the play on a machine where nothing is wrong. Flushing applies the drop-in first; on later runs nothing is pending and it is a no-op.

flush_handlers flushes everything pending, which here includes the Restart Caddy that the register include notified in pre_tasks. That is correct rather than incidental: Caddy's drop-in is new in the same run, so Caddy has to be recreated to join the network before it can proxy to hiwikijs.

Note the cost stated in postgres/readme.md: a new drop-in recreates Postgres, which is a few seconds of downtime for every attached app, not just this one.

Settings

Variable	Value / purpose
podman_user	Derived from the connection (ansible_facts['user_id']), not set by hand
podman_data_dir	~/podman-data
quadlet_dir	~/.config/containers/systemd
host	wiki.hinchco.in — the site block Caddy serves
container_name	hiwikijs, and the Quadlet unit / container name
image	ghcr.io/requarks/wiki:latest — registry + org + repo, so no short-name lookup
network_name	hiwikijs → hiwikijs.network / hiwikijs_net
wikijs_uid / wikijs_gid	1000 / 1000 — the image's node account, mapped onto podman_user by keep-id
content_dir	~/podman-data/hiwikijs/content → the image's /wiki/data/content
uploads_dir	~/podman-data/hiwikijs/uploads → /wiki/data/uploads
env_file	~/podman-data/hiwikijs/hiwikijs.env, mode 0600
container_port	3000, the port inside the container that Caddy proxies to
postgres_container / postgres_unit	The shared database unit this app attaches to
pg_user / pg_password / pg_database	Credentials, and the database name — hiwikijs
db_host	db, the shared NetworkAlias
caddy_name / caddyfile_path	The shared Caddy unit and its Caddyfile

The three podman_* / quadlet_dir vars are duplicated from caddy/vars/default.yml, as in every service playbook. That is a deliberate trade: a play cannot include the Caddy vars file wholesale, because it also sets host_directory (to the Caddy data directory) and every service defines that for itself. Keep them in sync.

Like Caddy, podman_user is derived from the connection and the play carries no become. Because the podman tasks are not run here, there is no guard against connecting as root; the prerequisites, including that check, live in caddy/playbook.yml.

keep-id:uid=1000, not uid=0

Wiki.js is the odd one out among the converted services. Vaultwarden, Tandoor, Caddy and Postgres all run as container-root and use keep-id:uid=0,gid=0. This image sets USER node, and node is uid/gid 1000:

docker exec hiwikijs id   →  uid=1000(node) gid=1000(node)

/usr/local/bin/docker-entrypoint.sh is a bare exec "$@" — it chowns nothing, so whatever uid the image starts as is the uid that writes the bind mounts. Mapping uid 1000 onto podman_user is what keeps ~/podman-data/hiwikijs owned by tanshu and leaves no subuid-owned files to trip over backups (docs/podman-migration.md §4.2 lists hiwikijs under exactly this mapping).

tanshu is uid 1001, not 1000 — docs/podman-migration.md §4.1 is about precisely this trap, and most uid-1000 data on monoco belongs to ubuntu. Nothing in this playbook assumes the app can read a file just because tanshu owns it; the two data directories are created as tanshu and the container writes as 1000, which keep-id maps onto tanshu. The mapping is what makes that true, not the ownership.

Running

ansible-playbook -u tanshu wikijs/playbook.yml

The play targets emily directly, so the inventory must contain it — the one in prepare-server/ does, and its ansible.cfg names it.

This play applies only this app's units and data. Its pre_tasks run Caddy's register include and nothing else. The rootless-Podman prerequisites — installing podman, loginctl enable-linger, the low-port sysctl and the Quadlet directory — are the shared task file at caddy/roles/podman/tasks/main.yaml, and are not run here. A host that has already run caddy/playbook.yml has them.

systemctl --user enable hiwikijs is neither needed nor possible: the unit is generated, and Quadlet applies [Install] WantedBy=default.target itself.

Migrating from monoco

There is nothing to copy. The database already moved on 2026-10-15, and the app has no file state. What follows is how to confirm that, and what to do in the unlikely case the confirmation fails.

The commands are meant to be run from the machine that holds the monoco and emily SSH aliases. rsync cannot copy remote-to-remote, so anything that does move is streamed through a pipe.

1. Prove the file state is empty

The image declares VOLUME /wiki/data/content, which Docker turned into an anonymous volume under /var/lib/docker/volumes/<hash>/_data. The running container already has it mounted, so no hash and no sudo are needed to look:

ssh monoco 'bash -s' <<'EOF'
docker exec hiwikijs sh -c '
  echo "--- /wiki/data/content (the image VOLUME) ---"; ls -la /wiki/data/content
  echo "--- /wiki/data/uploads (local storage target) ---"; ls -la /wiki/data/uploads
  echo "--- sizes ---"; du -sh /wiki/data/content /wiki/data/uploads'
EOF

Expected — both listings show only . and .., and both du lines say 4.0K:

--- /wiki/data/content (the image VOLUME) ---
total 8
drwxr-xr-x    2 node     node          4096 Mar 24  2025 .
drwxr-xr-x    1 node     node          4096 Jul 14  2025 ..
--- /wiki/data/uploads (local storage target) ---
total 8
drwxr-xr-x    2 node     node          4096 Jul 14  2025 .
drwxr-xr-x    1 node     node          4096 Jul 14  2025 ..
--- sizes ---
4.0K	/wiki/data/content
4.0K	/wiki/data/uploads

/var/lib/hiwikijs on monoco is a decoy and is worth looking at once so nobody copies it later. It holds a single root-owned .env and is not mounted into the container — the container's environment came from env_file: at create time, which is the Docker-side counterpart of the Quadlet unit's EnvironmentFile=, and which the role's template reproduces:

ssh monoco 'ls -la /var/lib/hiwikijs; cat /var/lib/hiwikijs/.env'

2. Prove the data is in the database, and already on emily

This is the real check, and it is the whole migration. Run the same block on both hosts and compare — the only difference is docker exec against podman exec:

ssh monoco 'bash -s' <<'EOF'
EXEC="docker exec -e PGPASSWORD=123456 postgres"
$EXEC psql -U postgres -d hiwikijs -tAc 'select count(*) from pages'                   | sed 's/^/pages  : /'
$EXEC psql -U postgres -d hiwikijs -tAc 'select count(*) from users'                   | sed 's/^/users  : /'
$EXEC psql -U postgres -d hiwikijs -tAc 'select count(*), pg_size_pretty(sum(octet_length(data))::bigint) from "assetData"' | sed 's/^/assets : /'
$EXEC psql -U postgres -tAc "select pg_size_pretty(pg_database_size('hiwikijs'))"      | sed 's/^/size   : /'
echo "newest edit: $($EXEC psql -U postgres -d hiwikijs -tAc 'select max("updatedAt") from pages')"
EOF

ssh emily 'bash -s' <<'EOF'
EXEC="podman exec -e PGPASSWORD=123456 postgres"
$EXEC psql -U postgres -d hiwikijs -tAc 'select count(*) from pages'                   | sed 's/^/pages  : /'
$EXEC psql -U postgres -d hiwikijs -tAc 'select count(*) from users'                   | sed 's/^/users  : /'
$EXEC psql -U postgres -d hiwikijs -tAc 'select count(*), pg_size_pretty(sum(octet_length(data))::bigint) from "assetData"' | sed 's/^/assets : /'
$EXEC psql -U postgres -tAc "select pg_size_pretty(pg_database_size('hiwikijs'))"      | sed 's/^/size   : /'
echo "newest edit: $($EXEC psql -U postgres -d hiwikijs -tAc 'select max("updatedAt") from pages')"
EOF

The assetData identifier is quoted because the table is genuinely camelCase — created "assetData" by Wiki.js's migrations, so unquoted assetdata does not find it.

Both hosts print the same thing, which is what makes this a verified copy rather than an assumption:

pages  : 80
users  : 2
assets : 49|5979 kB
size   : 16 MB
newest edit: 2024-02-03T03:23:57.645Z

The newest edit line is the one that matters for cutover. The dump was taken on 2026-10-15; if the newest updatedAt on monoco is later than on emily, the wiki was edited after the dump and those revisions exist only on the old host. On this wiki it is not: the newest revision is from February 2024, identical on both hosts, so the copy is current and the wiki is effectively read-only.

3. If the numbers differ: refresh the database

Only needed if step 2 shows drift (a later updatedAt on monoco, or different counts). Stop both apps first — nothing may write while the dump is taken, and drop database needs the connections gone:

ssh monoco 'docker stop hiwikijs'
ssh emily  'systemctl --user stop hiwikijs.service'

# Dump just this database, stream it across, and keep a copy on emily.
set -o pipefail
ssh monoco 'docker exec -e PGPASSWORD=123456 postgres pg_dump -U postgres -Fc hiwikijs' \
  | ssh emily 'cat > ~/hiwikijs-refresh.dump'

# Confirm the archive is readable before dropping anything.
ssh emily 'podman exec -i -e PGPASSWORD=123456 postgres pg_restore --list < ~/hiwikijs-refresh.dump | head -5'

ssh emily 'bash -s' <<'EOF'
set -e
EXEC="podman exec -e PGPASSWORD=123456 postgres"
$EXEC psql -U postgres -d postgres -v ON_ERROR_STOP=1 -c 'drop database if exists hiwikijs'
$EXEC psql -U postgres -d postgres -v ON_ERROR_STOP=1 -c 'create database hiwikijs owner postgres'
podman exec -i -e PGPASSWORD=123456 postgres pg_restore --exit-on-error -U postgres -d hiwikijs < ~/hiwikijs-refresh.dump
EOF

# Re-run step 2 — the counts must now match.

-Fc plus pg_restore rather than piping SQL into psql: it is the form postgres/readme.md recommends for per-database work, --exit-on-error stops on the first problem instead of half-loading, and pg_restore --list catches a truncated transfer before the old database is dropped.

Do not re-run postgres/migrate-from-monoco.md. That is a whole-cluster pg_dumpall into an empty cluster; against the live one it fails on the first CREATE DATABASE, and its "start over" block destroys ~/podman-data/postgres/18.

4. Repoint DNS, then stop the old container

The record, not the playbook, is the cutover:

# Before: monoco. After the change: emily.
dig +short wiki.hinchco.in
# 13.202.6.2        ← monoco

# Change the A/AAAA record to emily's address, then confirm it has propagated
# before Caddy on emily can complete an ACME challenge for the site.
dig +short wiki.hinchco.in
# 35.154.14.59      ← emily

Then retire the old instance so the two copies cannot diverge:

ssh monoco 'docker stop hiwikijs'

Rolling back means rolling back both halves, and the DNS change is the deciding one: docker start hiwikijs on monoco only produces a working site once the record points back at 13.202.6.2, and monoco's own Postgres 17 cluster has to still be intact for the wiki to have any content. That cluster is deliberately left alone as the rollback — do not drop databases on it, exactly as postgres/migrate-from-monoco.md §7 says.

Two loose ends when the old host is retired:

• Remove the wiki.hinchco.in block from monoco's Caddyfile. Monoco's Caddy keeps serving other sites, and once DNS points away it can no longer answer the ACME challenge for this one — which shows up as renewal errors in docker logs caddy forever. It does not affect the other sites.
• ~/bin/hiwikijs-backup.sh is not a backup. It archives /var/lib/hiwikijs, the empty decoy. The database is the wiki; anything scheduled belongs in a systemd user timer doing the podman exec … pg_dump form from postgres/readme.md, "Backups".

5. Deploy

cd prepare-server && ansible-playbook -u tanshu ../wikijs/playbook.yml

Verification

systemctl --user status hiwikijs
journalctl --user -u hiwikijs -f
podman logs hiwikijs

# What the playbook checks for you:
podman exec hiwikijs curl -fsS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:3000/
podman exec hiwikijs getent hosts db

# No anonymous volume was created: the mount is the real directory.
podman inspect hiwikijs --format '{{range .Mounts}}{{.Type}} {{.Source}} -> {{.Destination}}{{println}}{{end}}'
# expect: bind /home/tanshu/podman-data/hiwikijs/content -> /wiki/data/content
#         bind /home/tanshu/podman-data/hiwikijs/uploads -> /wiki/data/uploads

# Nothing is published host-side, exactly like the Docker-era container. Podman
# lists the image's EXPOSE'd ports with a null host binding, which is Docker's
# `PortBindings: {}` — the port numbers appear, the host side does not.
podman inspect hiwikijs --format '{{json .NetworkSettings.Ports}}'
# expect: {"3000/tcp":null,"3443/tcp":null} — null means "not published"

# Nothing may end up owned by root or by a subuid.
find ~/podman-data/hiwikijs ! -user tanshu     # empty = correct

# The site, end to end (after DNS points here).
curl -sSI https://wiki.hinchco.in

Then log in in a browser, open a page, and confirm an image renders. The image is the check that matters: it is served from the assetData table, so it is the one that proves the database came across whole. A curl of / cannot make that check, and neither can a table count.

Also reboot the host and confirm the unit returns unattended — without loginctl enable-linger tanshu it will not, and the failure is silent.

Notes and gotchas

• wiki.tanshu.com is not this service. The Caddyfile on monoco has a wiki.tanshu.com block pointing at affine:3010; that comes from affine/vars/default.yml, which uses the hostname for a different app. This playbook owns wiki.hinchco.in only. The two are unrelated, and the similar name is worth knowing before "fixing" either.
• There is no HEALTHCHECK in the image. docker inspect hiwikijs reports .Config.Healthcheck as null, so the podman healthcheck run form the Vaultwarden role uses is unavailable here. The play probes HTTP with the image's own curl instead, which is the next best thing: it says the app answers, not merely that the process exists.
• getent is there, despite the Alpine base. The image ships /usr/bin/getent (musl's libc-utils), so the database-resolution check the other roles use works unchanged. It exits 2 with no output when a name does not resolve, which is what the play's assert keys off — worth knowing, because a bare getent failure is silent.
• latest floats. Neither the old playbook nor this one pins a version or a digest, so an image bump is a normal podman pull away and may run new schema migrations against the 16 MB database on first start. Take a pg_dump first when that matters.
• The 49 assets have no files. assetData holds their bytes, so the wiki is self-contained — but this is why uploads/ is empty, and why a filesystem-level backup of this service would produce an archive that restores to a wiki with no images. Back up the database.
• Client IPs. The app sees Caddy's address on the app network, not the visitor's. Whether the original address would survive the rootless port forward is moot here: Caddy terminates the connection and proxies over hiwikijs_net either way. Nothing in Wiki.js uses the client address for security.
• Registering restarts all of Caddy, and attaching restarts Postgres for everyone — a brief restart of every hosted site, and a few seconds of downtime for every attached app. See the trade-offs in caddy/readme.md and postgres/readme.md.
• If the Caddyfile does not exist yet, this playbook creates it containing only the site block. That is a valid Caddyfile; the Caddy playbook's seeded global-options block is then not added, because its seed only fires on a missing or zero-length file. Deploy Caddy first on a fresh host.
• The old /var/lib/hiwikijs directory is not migrated and should not be: it holds one .env whose contents the role's template reproduces. Nothing in the Docker-era container ever mounted it.

Not done here

• Rotating the shared 123456 Postgres password — it lives in hiwikijs.env at mode 0600 rather than in the world-readable unit file, but it is still the fleet-wide placeholder committed in this repo.
• Replacing ~/bin/hiwikijs-backup.sh with a systemd user timer that dumps hiwikijs, and removing the entry from root's crontab — root cannot podman exec into another user's rootless containers.
• Upgrading Wiki.js to 3.x, which is a separate migration with its own schema changes and not something an image tag bump should do unattended.