The first deploy put all three Sites up serving and permanently unhealthy: every
healthcheck run died with `/bin/sh: syntax error: unterminated quoted string`.
The unit file carried the closing quote, but podman's stored config did not.
Quadlet's parser strips a trailing double quote, so `node -e "fetch(...)"`
reached the container as an unterminated shell string. The sibling apps never hit
this because their HealthCmd is a bare `curl`.
The check is now `wget -q -O /dev/null http://127.0.0.1:<port>/healthz` — busybox
wget, no quoting, non-zero on anything but a 2xx — and the Dockerfile's own
HEALTHCHECK says the same thing. The vars comment, the unit template and the
deployment readme all say why it has to stay quote-free.
Found from the running container rather than the play output: `podman
healthcheck run` returned 1, `.State.Health.Log` held the sh error, and the app
log showed it listening on 127.0.0.1:3000 the whole time.
menu-admin was the odd one out: it built from the working directory with a date
tag and a multi-arch manifest, deployed with `-e tag=`, and kept the Docker-era
container name. The other two apps build from `git archive HEAD` and tag with the
commit, so what runs on the host is exactly what git knows about; build only for
the architecture emily actually runs; and use one slug for the container, its
network and its data directory.
The build is now their three lines: `podman build --platform linux/arm64` from
`git archive HEAD`, tagged `:<commit>` and `:latest`, pushed with `podman push`
rather than a manifest. `deploy.sh` pushes the commit first, then refuses a dirty
tracked tree or a HEAD that is not origin/main — which is what makes the commit
tag trustworthy — takes the same `--force` and `--no-push`, and ends in the same
`-u tanshu ... -e image_tag=<commit>`. Its own lint/typecheck/test gate stays,
but behind `--checks`, so a plain deploy behaves as theirs does.
The registry credential moves into vars/default.yml and the role logs in, pulls
and logs out around it, as their roles do. The image is only pulled at deploy
time, so the unit keeps podman's default `Pull=missing` and a reboot never needs
the registry. This is the same plaintext credential the other two repos carry.
One real bug goes with it: the healthcheck gate ran before the handler that
restarts a changed unit, so a run that only changed the unit checked the stale
container. `meta: flush_handlers` before the gate is the fix the sibling repos
already made.
The container, network and data directory are all menu-<site> now. The dotted
Docker-era name and the NetworkAlias it needed are gone, so `podman logs
menu-<site>` and `systemctl --user status menu-<site>` are the same two commands
across all three apps. The Dockerfile keeps its name for the same reason: podman
reads it as readily as a Containerfile, and the siblings use that name too.
The fleet on emily is rootless Podman with Quadlet units under
~/.config/containers/systemd and state in ~/podman-data, all behind one shared
Caddy unit. This brings the menus in line instead of leaving them the last
Docker deployment. The build moves with them: podman reads a Containerfile as
readily as a Dockerfile, so the name now says which toolchain is real.
The app is one .container and one .network file. Two details carry it.
UserNS=keep-id maps the image's own node account onto the user that owns the
units, so data/ and site.env stay owned by tanshu: plain tar backups, no
subuids. provision-site-env.sh writes site.env with the same mapping rather
than --user, which under rootless Podman would have produced a subuid-owned
file the app could not read. And the Site registers its own network with Caddy
by writing a drop-in beside caddy.container, so Caddy never lists its upstreams
and never changes when a Site is added. The unit keeps the Docker-era container
name for the scripts and muscle memory that look it up, while the Caddyfile
proxies to a dotless NetworkAlias, because a container name with dots in it is
a liability in DNS.
The unit keeps podman's Pull=missing, so a restart or a reboot never depends on
the registry being reachable, while the play pulls explicitly at deploy time.
That is also where a missing `podman login` for the private registry now fails,
with the instruction, rather than as a unit that will not start.
copy-site-data.sh moves a Site's data across, and is not a plain rsync:
/var/lib/menu.* on monoco belongs to uid 1000 while the ssh user there is 1001,
so the read needs sudo; monoco cannot ssh to emily, so the workstation relays
the tar; and the extraction runs as tanshu on emily, which is what makes the
files land owned by the user keep-id maps the app onto. It proves the copy by
comparing a SHA-256 of the whole tree against the source and checking every
file's owner.
Not yet done: the copies themselves and the DNS move, which need a human.
verify reports integrity problems and repair rebuilds derivatives from the
Originals, which is what makes the file layout trustworthy without a database.
The image runs TypeScript on Node 24 with no build step, writes only under
MENU_ROOT, and the provision script writes the untracked site.env using the
image's own hasher so the server needs no Node.
tanshu