Make the healthcheck a command Quadlet can pass through

The first deploy put all three Sites up serving and permanently unhealthy: every
healthcheck run died with `/bin/sh: syntax error: unterminated quoted string`.
The unit file carried the closing quote, but podman's stored config did not.
Quadlet's parser strips a trailing double quote, so `node -e "fetch(...)"`
reached the container as an unterminated shell string. The sibling apps never hit
this because their HealthCmd is a bare `curl`.

The check is now `wget -q -O /dev/null http://127.0.0.1:<port>/healthz` — busybox
wget, no quoting, non-zero on anything but a 2xx — and the Dockerfile's own
HEALTHCHECK says the same thing. The vars comment, the unit template and the
deployment readme all say why it has to stay quote-free.

Found from the running container rather than the play output: `podman
healthcheck run` returned 1, `.State.Health.Log` held the sh error, and the app
log showed it listening on 127.0.0.1:3000 the whole time.
This commit is contained in:
tanshu committed 2026-10-16 11:22:48 +05:30
1 parent 537a4f1e46
commit 8c48cb2535
4 files changed
+25 -7

No files matched your search

+5 -1
View File
@@ -33,7 +33,11 @@ EXPOSE 3000
# Quadlet unit declares the same check again as HealthCmd= (see
# ansible/roles/menu-app/templates/menu-app.container.j2). Keeping it here means
# the image still healthchecks itself when run directly.
#
# The check is deliberately quote-free: a Quadlet HealthCmd that ends in a double
# quote loses it, so the unit would run an unterminated shell string. wget is
# busybox's and exits non-zero on anything but a 2xx.
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s \
CMD node -e "fetch('http://127.0.0.1:'+(process.env.PORT||3000)+'/healthz').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"
CMD wget -q -O /dev/null http://127.0.0.1:${PORT:-3000}/healthz
CMD ["node", "src/server.ts"]
+9
View File
@@ -40,6 +40,15 @@ file the app writes is owned by `tanshu` on the host: plain `tar`/`rsync`
backups, no subuids, no `podman unshare`. `scripts/provision-site-env.sh` uses
the same mapping for the same reason.
**A Quadlet `HealthCmd` must not contain quotes.** Quadlet's parser strips a
trailing double quote, so a check written as `node -e "fetch(…)"` reaches podman
as an unterminated shell string: every run fails with `/bin/sh: syntax error:
unterminated quoted string`, and the container serves while being permanently
unhealthy. That is why the check is `wget -q -O /dev/null …` in
`vars/default.yml` — quote-free, and the same reason the sibling apps use a bare
`curl`. Nothing about it is visible in `podman ps` beyond the word `unhealthy`;
`podman inspect <container> --format '{{json .State.Health.Log}}'` shows it.
## What lives where
| Thing | Where |
@@ -40,7 +40,8 @@ Network={{ network_name }}.network
# podman builds an OCI image by default and drops the Dockerfile's HEALTHCHECK
# instruction, so the same check is declared here and the play gates on it with
# `podman healthcheck run`. node is in the image.
# `podman healthcheck run`. wget is busybox's; see vars/default.yml for why the
# command must not contain quotes.
HealthCmd={{ health_cmd }}
HealthInterval=30s
HealthTimeout=5s
+9 -5
View File
@@ -48,11 +48,15 @@ site_port: 3000
# podman builds an OCI image by default and drops the Dockerfile's HEALTHCHECK
# instruction, so the same check is declared on the Quadlet unit and the play
# gates on `podman healthcheck run`. node is in the image, and the sh -c podman
# wraps it in handles the quotes.
health_cmd: >-
node -e "fetch('http://127.0.0.1:{{ site_port }}/healthz').then(r =>
process.exit(r.ok ? 0 : 1)).catch(() => process.exit(1))"
# gates on `podman healthcheck run`.
#
# The command must not contain quotes. Quadlet's parser strips a trailing double
# quote from HealthCmd, so `node -e "fetch(...)"` reaches podman as an
# unterminated string and every check dies with "/bin/sh: syntax error:
# unterminated quoted string" — the container serves, and is permanently
# unhealthy. wget is busybox's, needs no quoting, and exits non-zero on anything
# but a 2xx.
health_cmd: 'wget -q -O /dev/null http://127.0.0.1:{{ site_port }}/healthz'
# ---------------------------------------------------------------------------
# Shared Caddy unit (caddy/playbook.yml in the ansible-playbooks repo)