tanshu
e68f8ab00d
The fleet on emily is rootless Podman with Quadlet units under ~/.config/containers/systemd and state in ~/podman-data, all behind one shared Caddy unit. This brings the menus in line instead of leaving them the last Docker deployment. The build moves with them: podman reads a Containerfile as readily as a Dockerfile, so the name now says which toolchain is real. The app is one .container and one .network file. Two details carry it. UserNS=keep-id maps the image's own node account onto the user that owns the units, so data/ and site.env stay owned by tanshu: plain tar backups, no subuids. provision-site-env.sh writes site.env with the same mapping rather than --user, which under rootless Podman would have produced a subuid-owned file the app could not read. And the Site registers its own network with Caddy by writing a drop-in beside caddy.container, so Caddy never lists its upstreams and never changes when a Site is added. The unit keeps the Docker-era container name for the scripts and muscle memory that look it up, while the Caddyfile proxies to a dotless NetworkAlias, because a container name with dots in it is a liability in DNS. The unit keeps podman's Pull=missing, so a restart or a reboot never depends on the registry being reachable, while the play pulls explicitly at deploy time. That is also where a missing `podman login` for the private registry now fails, with the instruction, rather than as a unit that will not start. copy-site-data.sh moves a Site's data across, and is not a plain rsync: /var/lib/menu.* on monoco belongs to uid 1000 while the ssh user there is 1001, so the read needs sudo; monoco cannot ssh to emily, so the workstation relays the tar; and the extraction runs as tanshu on emily, which is what makes the files land owned by the user keep-id maps the app onto. It proves the copy by comparing a SHA-256 of the whole tree against the source and checking every file's owner. Not yet done: the copies themselves and the DNS move, which need a human.
9.8 KiB
The menu deployment
Lives here, beside the app it deploys. One run deploys a Site: it starts the
app's container as a rootless Podman Quadlet unit on emily and points the
Site's web address at it through the shared Caddy unit that
ansible-playbooks/caddy owns. There is no older system to keep serving, so
there is no staged rollout and nothing to retire — deploying an earlier tag is
the rollback.
This is the same shape as the sibling services in ansible-playbooks (gitea,
nextcloud, tandoor, bitwarden, valkey, postgres, trillium): rootless Podman, one
Quadlet .container / .network per service in ~/.config/containers/systemd/,
state under ~/podman-data/, and Caddy reaching each app by name on a Podman
network. Nothing here talks to Docker.
The pieces
ansible/
├── hosts # one entry per Site, all on emily
├── playbook.yml # connects as tanshu, no become
├── vars/{default,hopsngrains,greatbear,hngmohali}.yml
└── roles/
├── register/ # writes menu-<site>.network and the Caddy drop-in
├── menu-app/ # the Quadlet unit, start + health + verify
└── caddy/ # appends the site block, reloads Caddy
Two things are worth internalising, because they are the whole design:
Caddy knows nothing about its upstreams. A service registers itself by
writing two files into the Quadlet directory: its own <name>.network unit and a
drop-in beside caddy.container that pulls that network into Caddy. This play
does that in pre_tasks, before the app unit starts, because Quadlet fails unit
generation when a unit references a .network file that does not exist yet.
The container's uid is mapped onto tanshu. UserNS=keep-id:uid=1000,gid=1000
maps the image's own node account onto the user that owns the units, so every
file the app writes is owned by tanshu on the host: plain tar/rsync
backups, no subuids, no podman unshare. scripts/provision-site-env.sh uses
the same mapping for the same reason.
What lives where
| Thing | Where |
|---|---|
| Name, tagline, icon, categories, menus | on the server under ~/podman-data/menu-<site>/data — the admin edits them |
| Admin password hash, session secret | ~/podman-data/menu-<site>/site.env (untracked, mode 600) |
| Theme colours | config/<site>.site.json in the repo, baked into the image |
| The unit that runs | ~/.config/containers/systemd/menu-<site>.container |
| The network and Caddy drop-in | menu-<site>.network, caddy.container.d/50-menu-<site>.conf |
| Which image tag runs | -e tag=<tag> |
| Which Site is deployed | the inventory entry in hosts |
| The web address it serves | host: in vars/<site>.yml |
<site> is the inventory name (hopsngrains, greatbear, hngmohali). The
container itself keeps the Docker-era name menu-app.<domain>, because
provision-site-env.sh and muscle memory look it up that way; the Caddyfile
proxies to the dotless menu-<site> alias, because a container name with dots in
it is a liability in DNS.
What the host needs
emily is already set up — it runs Caddy, Postgres, Tandoor and Valkey the same
way. If you are standing up a fresh host, run ansible-playbooks/caddy first: it
installs podman, enables lingering, lowers the privileged-port floor so rootless
Caddy can bind 80/443, creates the Quadlet directory, and starts the Caddy unit.
Those prerequisites deliberately are not duplicated here, so a host that has
not run that playbook will fail at the first systemctl --user — see
ansible-playbooks/caddy/readme.md.
The image is in the private registry, so log in once as tanshu on emily,
before the first deploy:
podman login registry.tanshu.com
The play pulls the image itself, so a missing login fails there with that
instruction rather than as a container that will not start. The unit keeps
podman's default Pull=missing on purpose, so a restart or a reboot never
depends on the registry being reachable.
ssh on some machines complains about
/etc/ssh/ssh_config.d/20-systemd-ssh-proxy.conf being owned by nobody; that is
a local file-permission bug whose fix is sudo chown -R root:root /etc/ssh/ssh_config.d, not something this playbook works around.
Deploying
From the repo root:
./deploy.sh 2026-09-22 # checks, podman build+push, then deploys
SKIP_DEPLOY=1 ./deploy.sh 2026-09-22 # build and push only
deploy.sh runs make build-production TAG=<tag>, which builds a multi-arch
podman manifest (linux/amd64,linux/arm64) and pushes latest and the tag to
registry.tanshu.com/menu-admin. Then it runs this playbook against every Site,
or the one named by SITE.
By hand:
make build-production TAG=2026-09-22
cd ansible
ansible-playbook --inventory=hosts playbook.yml --limit hopsngrains -e tag=2026-09-22
Every Site runs on the same box, so hosts has one entry per Site pointing at
the same ansible_host with its own var_file. The play carries no become: the
Quadlet units belong to ansible_user (tanshu), and running as that user is
what gives us a real PAM session, which is what makes systemctl --user work.
Connecting as anyone else would write units into the wrong home.
Adding a Site is one line in hosts, a copy of vars/hopsngrains.yml, and a
config/<site>.site.json. The image is shared, so the config has to be in it: a
new Site needs a new tag.
Migrating a Site's data from monoco
The menus still serve from Docker containers on monoco. A Site moves across
with its data, before it is ever pointed at from DNS. Run this from a machine
that can ssh to both hosts — the monoco and emily aliases this repo's own
tooling uses:
scripts/copy-site-data.sh hopsngrains
scripts/copy-site-data.sh greatbear
scripts/copy-site-data.sh hngmohali
For one Site it stops the old container, streams /var/lib/menu.<domain> across,
starts the old container again so monoco keeps serving until DNS moves, and
then proves the copy: the SHA-256 of the whole tree is compared against the
source, and every file is checked to belong to tanshu. --live skips the stop;
MONOCO=, EMILY=, SRC_ROOT= and DEST_ROOT= override the defaults.
Two things make the copy look different from a plain rsync:
sudoon monoco./var/lib/menu.<domain>belongs to uid 1000 (theubuntuaccount) and the ssh user there is 1001, so it is not readable without it. monoco'stanshuhas passwordless sudo.- The workstation relays it. monoco cannot ssh to emily, so the tar streams
through the machine running the script; nothing large is written locally.
Extracting as
tanshuon emily is what makes the files land owned by the userkeep-idmaps the app onto — as root they would arrive owned by uid 1000, which on emily is nobody.
By hand, that is:
SITE=hopsngrains
HOST=menu.hopsngrains.com
ssh monoco "docker stop menu-app.$HOST"
ssh -n monoco "sudo -n tar -C /var/lib/$HOST -cf - ." \
| ssh emily "mkdir -p ~/podman-data/menu-$SITE && tar -C ~/podman-data/menu-$SITE -xf -"
ssh monoco "docker start menu-app.$HOST"
# Same checks the script runs.
ssh -n monoco "sudo -n sh -c 'cd /var/lib/$HOST && find . -type f -exec sha256sum {} +' | sort | sha256sum"
ssh -n emily "cd ~/podman-data/menu-$SITE && find . -type f -exec sha256sum {} + | sort | sha256sum"
ssh -n emily "find ~/podman-data/menu-$SITE ! -user tanshu"
The old layout put everything under /var/lib/menu.<domain>/, with the content
one level down in data/; the new one is ~/podman-data/menu-<site>/ with the
same data/ inside it and site.env beside it. Nothing inside changes.
site.env carries over so the admin keeps working; without it the app starts
with the admin disabled and the playbook says so rather than failing.
Then deploy that Site and check it on a Host: override before touching DNS.
monoco keeps serving until the cutover; only then stop its container — and
leave it in place, not removed, so docker start is the rollback.
The admin password
The app starts with the admin disabled while ~/podman-data/menu-<site>/site.env
is missing, and the playbook says so rather than failing. Create it once per Site;
run it again with --force to reset the password (a fresh session secret signs
everyone out). There is no minimum password length.
scp scripts/provision-site-env.sh emily:/tmp/
ssh -t emily bash /tmp/provision-site-env.sh ~/podman-data/menu-hopsngrains
systemctl --user restart menu-hopsngrains.service
The -t matters: the script prompts for the password, and without a terminal on
the remote side the prompt cannot be read, so it would sit there looking frozen.
With no image argument it borrows the one the running container uses. It has to
run as tanshu, the user whose rootless Podman owns the container.
Checking it landed
npm run check-site -- https://menu.hopsngrains.com
Read-only, exits non-zero on failure. Then from a phone: the landing page, a
Section page, the page counter, tap-to-zoom, and signing into /admin to publish
something.
Rolling back
Deploy the previous tag:
./deploy.sh <previous-tag>
podman logs menu-app.<site> and journalctl --user -u menu-<site> are where a
container that will not start says why. To go all the way back to the Docker
deployment, start the container that was left on monoco and point Caddy at it
again.