From 8c48cb2535067601a376830ed09fd6ffa4331d29 Mon Sep 17 00:00:00 2001 From: Amritanshu Date: Fri, 16 Oct 2026 11:22:48 +0530 Subject: [PATCH] Make the healthcheck a command Quadlet can pass through MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first deploy put all three Sites up serving and permanently unhealthy: every healthcheck run died with `/bin/sh: syntax error: unterminated quoted string`. The unit file carried the closing quote, but podman's stored config did not. Quadlet's parser strips a trailing double quote, so `node -e "fetch(...)"` reached the container as an unterminated shell string. The sibling apps never hit this because their HealthCmd is a bare `curl`. The check is now `wget -q -O /dev/null http://127.0.0.1:/healthz` — busybox wget, no quoting, non-zero on anything but a 2xx — and the Dockerfile's own HEALTHCHECK says the same thing. The vars comment, the unit template and the deployment readme all say why it has to stay quote-free. Found from the running container rather than the play output: `podman healthcheck run` returned 1, `.State.Health.Log` held the sh error, and the app log showed it listening on 127.0.0.1:3000 the whole time. --- Dockerfile | 6 +++++- ansible/readme.md | 9 +++++++++ .../roles/menu-app/templates/menu-app.container.j2 | 3 ++- ansible/vars/default.yml | 14 +++++++++----- 4 files changed, 25 insertions(+), 7 deletions(-) diff --git a/Dockerfile b/Dockerfile index 825a5d8..1aae31a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -33,7 +33,11 @@ EXPOSE 3000 # Quadlet unit declares the same check again as HealthCmd= (see # ansible/roles/menu-app/templates/menu-app.container.j2). Keeping it here means # the image still healthchecks itself when run directly. +# +# The check is deliberately quote-free: a Quadlet HealthCmd that ends in a double +# quote loses it, so the unit would run an unterminated shell string. wget is +# busybox's and exits non-zero on anything but a 2xx. HEALTHCHECK --interval=30s --timeout=5s --start-period=10s \ - CMD node -e "fetch('http://127.0.0.1:'+(process.env.PORT||3000)+'/healthz').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))" + CMD wget -q -O /dev/null http://127.0.0.1:${PORT:-3000}/healthz CMD ["node", "src/server.ts"] diff --git a/ansible/readme.md b/ansible/readme.md index 0e35726..5dfb649 100644 --- a/ansible/readme.md +++ b/ansible/readme.md @@ -40,6 +40,15 @@ file the app writes is owned by `tanshu` on the host: plain `tar`/`rsync` backups, no subuids, no `podman unshare`. `scripts/provision-site-env.sh` uses the same mapping for the same reason. +**A Quadlet `HealthCmd` must not contain quotes.** Quadlet's parser strips a +trailing double quote, so a check written as `node -e "fetch(…)"` reaches podman +as an unterminated shell string: every run fails with `/bin/sh: syntax error: +unterminated quoted string`, and the container serves while being permanently +unhealthy. That is why the check is `wget -q -O /dev/null …` in +`vars/default.yml` — quote-free, and the same reason the sibling apps use a bare +`curl`. Nothing about it is visible in `podman ps` beyond the word `unhealthy`; +`podman inspect --format '{{json .State.Health.Log}}'` shows it. + ## What lives where | Thing | Where | diff --git a/ansible/roles/menu-app/templates/menu-app.container.j2 b/ansible/roles/menu-app/templates/menu-app.container.j2 index f5ab0e6..d585a41 100644 --- a/ansible/roles/menu-app/templates/menu-app.container.j2 +++ b/ansible/roles/menu-app/templates/menu-app.container.j2 @@ -40,7 +40,8 @@ Network={{ network_name }}.network # podman builds an OCI image by default and drops the Dockerfile's HEALTHCHECK # instruction, so the same check is declared here and the play gates on it with -# `podman healthcheck run`. node is in the image. +# `podman healthcheck run`. wget is busybox's; see vars/default.yml for why the +# command must not contain quotes. HealthCmd={{ health_cmd }} HealthInterval=30s HealthTimeout=5s diff --git a/ansible/vars/default.yml b/ansible/vars/default.yml index 7a2902a..032d70e 100644 --- a/ansible/vars/default.yml +++ b/ansible/vars/default.yml @@ -48,11 +48,15 @@ site_port: 3000 # podman builds an OCI image by default and drops the Dockerfile's HEALTHCHECK # instruction, so the same check is declared on the Quadlet unit and the play -# gates on `podman healthcheck run`. node is in the image, and the sh -c podman -# wraps it in handles the quotes. -health_cmd: >- - node -e "fetch('http://127.0.0.1:{{ site_port }}/healthz').then(r => - process.exit(r.ok ? 0 : 1)).catch(() => process.exit(1))" +# gates on `podman healthcheck run`. +# +# The command must not contain quotes. Quadlet's parser strips a trailing double +# quote from HealthCmd, so `node -e "fetch(...)"` reaches podman as an +# unterminated string and every check dies with "/bin/sh: syntax error: +# unterminated quoted string" — the container serves, and is permanently +# unhealthy. wget is busybox's, needs no quoting, and exits non-zero on anything +# but a 2xx. +health_cmd: 'wget -q -O /dev/null http://127.0.0.1:{{ site_port }}/healthz' # --------------------------------------------------------------------------- # Shared Caddy unit (caddy/playbook.yml in the ansible-playbooks repo)