mozimo.in/ansible/README.md

Deploying mozimo

Rootless podman + Quadlet on the host emily (aarch64, Ubuntu 26.04), fronted by that host's shared Caddy. Nothing here runs as root.

./deploy.sh                                   # build the pushed commit + deploy
ansible-playbook playbook.yml -e image_tag=<commit> [--check --diff]

How it fits together

Piece	Where
Quadlet units	~/.config/containers/systemd/{mozimo,vince}.container, mozimo.network
Service data	~/podman-data/mozimo/ (PocketBase db + files, .env), ~/podman-data/vince/
Shared Caddy config	~/podman-data/caddy/conf/Caddyfile — one marked block per service
Caddy's network membership	~/.config/containers/systemd/caddy.container.d/50-mozimo.conf

Quadlet units are not read by systemd directly: the podman user generator turns them into user services on daemon-reload, and [Install] WantedBy=default.target brings them up at boot. systemctl --user is-enabled mozimo reports generated — that is normal, there is no unit to enable.

Nothing is published on the host. Caddy joins mozimo_net (through the drop-in) and proxies to the containers by name:

mozimo.in        → mozimo:3000   (plus /_/* and /api/* → mozimo:8090)
www.mozimo.in    → 301 mozimo.in
cms.mozimo.in    → mozimo:8090
stats.mozimo.in  → vince:8000
mozimo.in/stats/* → vince:8000 (prefix stripped)

The user id mapping matters: the app image runs as uid 1000 (node) and Vince as root, so both units use UserNS=keep-id: to map that uid onto tanshu. Without it, PocketBase and Vince would write files owned by a subuid, and backups, rsync and tar would all trip over them.

Services register themselves

This host's convention (also used by its tandoor, postgres and caddy playbooks) is that a service owns its own registration rather than editing a central list:

1. it ships a <name>.network Quadlet with NetworkName=<name>_net;
2. it drops caddy.container.d/50-<name>.conf containing [Container] Network=<name>.network, which Quadlet merges into Caddy;
3. it appends a marked block to Caddy's Caddyfile and reloads it.

Removing a service is the reverse: delete the drop-in, delete the Caddyfile block, systemctl --user daemon-reload, and the network is no longer Caddy's.

Migrating monoco → emily

monoco is the current host: rootful Docker, data in /var/lib/mozimo, its own Caddy, DNS for mozimo.in / www / cms / stats all pointing at 13.202.6.2. emily is 35.154.14.59.

Certificates are not copied. emily's Caddy is shared with other sites, and its ACME storage already holds their certificates — overwriting it would break them. After the DNS switch emily issues fresh certificates for mozimo.in; until the DNS switch it retries, which is why the Caddy role can be skipped first (--skip-tags caddy).

0. Compare the deployed environment with the repo's copy

The playbook uploads ansible/files/.env (gitignored) over whatever is on the host, so anything that has drifted on monoco would be silently dropped:

ssh monoco 'sudo -n cat /var/lib/mozimo/.env' > /tmp/monoco.env
diff /tmp/monoco.env ansible/files/.env

1. Rehearse against emily without touching DNS

cd ansible
ansible-playbook playbook.yml -e image_tag=$(git rev-parse --short HEAD) --check --diff   # what would change
ansible-playbook playbook.yml -e image_tag=$(git rev-parse --short HEAD) --skip-tags caddy

This creates the network, the units and a fresh, empty PocketBase database on emily. Nothing public changes yet.

2. Copy the data (the cutover window)

Stop both sides first. PocketBase keeps its database in SQLite, so a copy taken underneath a running writer can be inconsistent — and on emily the rehearsal left an empty database that this snapshot replaces:

# emily: stop its copy and drop the empty database the rehearsal created
ssh emily 'systemctl --user stop mozimo vince; rm -rf ~/podman-data/mozimo/pb_data ~/podman-data/vince'

# monoco: the site is down from here until DNS + emily are live
ssh monoco 'docker stop mozimo vince'

# PocketBase database + files (23 MB)
ssh monoco 'sudo -n tar -C /var/lib/mozimo -cf - pb_data' \
  | ssh emily 'mkdir -p ~/podman-data/mozimo && tar -C ~/podman-data/mozimo -xf -'

# Vince analytics history (119 MB), contents dropped directly into its data dir
ssh monoco 'sudo -n tar -C /var/lib/mozimo -cf - vince_data' \
  | ssh emily 'mkdir -p ~/podman-data/vince && tar -C ~/podman-data/vince -xf - --strip-components=1'

The extraction runs as tanshu, so everything lands owned by tanshu — which is exactly what the keep-id mapping expects, whatever it was owned by on monoco. Resumable equivalents, if the link is unreliable:

rsync -a --info=progress2 --rsync-path="sudo -n rsync" \
  monoco:/var/lib/mozimo/pb_data/   emily:podman-data/mozimo/pb_data/
rsync -a --info=progress2 --rsync-path="sudo -n rsync" \
  monoco:/var/lib/mozimo/vince_data/ emily:podman-data/vince/

3. Point DNS at emily and deploy the Caddy block

mozimo.in, www.mozimo.in, cms.mozimo.in, stats.mozimo.in → 35.154.14.59. Then:

cd ansible
ansible-playbook playbook.yml -e image_tag=$(git rev-parse --short HEAD)

# the play restarts a service whose unit or .env changed; starting is explicit
# here because step 2 stopped both, and `start` is a no-op if already running
ssh emily 'systemctl --user start mozimo vince'

The first request after the reload makes Caddy issue certificates for the four names.

4. Verify, then retire monoco

ssh emily 'systemctl --user status mozimo vince caddy --no-pager | head -30'
ssh emily 'podman ps --format "{{.Names}} {{.Status}}"'
curl -sI https://mozimo.in | head -3
curl -s https://cms.mozimo.in/api/health | head -3      # PocketBase
curl -sI https://stats.mozimo.in | head -3

Check a product page renders live prices and that the CMS admin login works with the same superuser credentials. Only then:

ssh monoco 'docker rm -f mozimo vince'     # or leave stopped for a few days

Rollback at any point before that: point DNS back at 13.202.6.2 and ssh monoco 'docker start mozimo vince'. On emily, systemctl --user stop mozimo vince and delete the mozimo.in block from its Caddyfile.