Match the deploy system to hops and greatbear

menu-admin was the odd one out: it built from the working directory with a date
tag and a multi-arch manifest, deployed with `-e tag=`, and kept the Docker-era
container name. The other two apps build from `git archive HEAD` and tag with the
commit, so what runs on the host is exactly what git knows about; build only for
the architecture emily actually runs; and use one slug for the container, its
network and its data directory.

The build is now their three lines: `podman build --platform linux/arm64` from
`git archive HEAD`, tagged `:<commit>` and `:latest`, pushed with `podman push`
rather than a manifest. `deploy.sh` pushes the commit first, then refuses a dirty
tracked tree or a HEAD that is not origin/main — which is what makes the commit
tag trustworthy — takes the same `--force` and `--no-push`, and ends in the same
`-u tanshu ... -e image_tag=<commit>`. Its own lint/typecheck/test gate stays,
but behind `--checks`, so a plain deploy behaves as theirs does.

The registry credential moves into vars/default.yml and the role logs in, pulls
and logs out around it, as their roles do. The image is only pulled at deploy
time, so the unit keeps podman's default `Pull=missing` and a reboot never needs
the registry. This is the same plaintext credential the other two repos carry.

One real bug goes with it: the healthcheck gate ran before the handler that
restarts a changed unit, so a run that only changed the unit checked the stale
container. `meta: flush_handlers` before the gate is the fix the sibling repos
already made.

The container, network and data directory are all menu-<site> now. The dotted
Docker-era name and the NetworkAlias it needed are gone, so `podman logs
menu-<site>` and `systemctl --user status menu-<site>` are the same two commands
across all three apps. The Dockerfile keeps its name for the same reason: podman
reads it as readily as a Containerfile, and the siblings use that name too.

ansible/roles/caddy/tasks/main.yaml

@@ -1,50 +1,25 @@
 ---
 # Append this Site's block to the shared Caddyfile.
 #
-# Caddy is its own playbook (ansible-playbooks/caddy/playbook.yml) and a rootless
-# Quadlet unit there, so this role does not manage the container — it only points
-# it at the app. The Caddyfile is a bind mount into that container, so Caddy sees
-# a written change as soon as the file lands; the handler only asks Caddy to
-# re-read it. That is a reload, not a restart, so the other hosted sites are
-# untouched.
+# The Caddyfile is a bind mount into the Caddy container, so Caddy sees the
+# change as soon as the file is written; the handler only asks Caddy to re-read
+# it. That is a reload, not a restart, so the other hosted sites are untouched.
 #
 # blockinfile is idempotent on its own via the marker, so there is no need to
 # slurp the file and compare first — which also matters because slurp *fails*
-# when the Caddyfile does not exist yet, on a host where the Caddy playbook has
+# when the Caddyfile does not exist yet, on a host where caddy/playbook.yml has
 # not run.
+#
+# The marker is the same one the retired nginx-menu playbook wrote per hostname,
+# so a Site cut over from it has its block replaced in place rather than a second
+# block for one address appended, which would make Caddy refuse the whole config.
 
-- name: Check the Caddyfile exists
-  ansible.builtin.stat:
-    path: '{{ caddyfile_path }}'
-  register: menu_caddyfile
-
-- name: Fail when Caddy is not where we expect it
-  ansible.builtin.fail:
-    msg: >-
-      {{ caddyfile_path }} is missing. Deploy the Caddy playbook
-      (ansible-playbooks/caddy/playbook.yml), which owns the Caddy unit and its
-      config, before this one.
-  when: not menu_caddyfile.stat.exists
-
-# The Site's web address may still be claimed by an older snippet: the retired
-# nginx-menu playbook wrote one per hostname. Two blocks for one address make
-# Caddy refuse the whole config, and a refused reload leaves the old proxy in
-# place, which is a 502 once the old container is gone.
-- name: Drop any older snippet claiming this web address
+- name: 'Add the {{ host }} site block to the Caddyfile'
   ansible.builtin.blockinfile:
     path: '{{ caddyfile_path }}'
     marker: '# {mark} Ansible managed Caddy snippet for {{ host }}'
-    state: absent
-  notify: Reload Caddy
-
-# blockinfile replaces whatever sits between its markers, so redeploying updates
-# the snippet in place rather than appending a second one.
-- name: Point the Site's web address at the app
-  ansible.builtin.blockinfile:
-    path: '{{ caddyfile_path }}'
-    marker: '# {mark} ANSIBLE MANAGED: menu app ({{ site }})'
-    block: '{{ caddy_snippet }}'
-    create: false
+    block: '{{ snippet_block }}'
+    create: true
     owner: '{{ podman_user }}'
     group: '{{ podman_user }}'
     mode: '0644'

ansible/roles/menu-app/tasks/main.yaml

@@ -2,29 +2,28 @@
 # The menu app as a rootless Podman Quadlet unit.
 #
 # Everything here runs as {{ podman_user }} — the connection user. No become, no
-# XDG_RUNTIME_DIR: the SSH connection gave us a real PAM session, so systemd's
+# XDG_RUNTIME_DIR: the SSH connection gives a real PAM session, so systemd's
 # user manager is reachable through the standard environment.
 
-# The pull is explicit, rather than left to the Quadlet unit, for two reasons.
-# The image lives in a private registry, so a failure here is where a missing
-# `podman login` should show up — not two tasks later as "unit not found". And
-# the unit keeps podman's default `Pull=missing`, so it never reaches for the
-# registry on a restart or after a reboot: that is what lets a box come back up
-# when the registry is down. Pulling here is what keeps a redeployed tag current.
-- name: Pull the app image
-  ansible.builtin.command: 'podman pull {{ image_name }}:{{ image_tag }}'
-  register: menu_pull
+# The registry is private, so the pull below has to be authenticated. Podman
+# keeps the credential in the user's own auth file; it is removed again at the
+# end of this role, so nothing but the image is left behind. Skipped when no
+# credentials are configured.
+- name: Log in to the private registry
+  ansible.builtin.command:
+    cmd: 'podman login --username {{ registry_username }} --password-stdin {{ registry }}'
+    stdin: '{{ registry_password }}'
+  no_log: true
   changed_when: false
-  failed_when: false
+  when:
+    - registry_username | length > 0
+    - registry_password | length > 0
 
-- name: Fail when the image cannot be pulled
-  ansible.builtin.fail:
-    msg: >-
-      podman could not pull {{ image_name }}:{{ image_tag }}.
-      {{ menu_pull.stderr | default('') | trim }}
-      The registry is private: run `podman login registry.tanshu.com` once as
-      {{ podman_user }} on this host.
-  when: menu_pull.rc != 0
+# Pulled here rather than left to the unit: Quadlet only pulls when the image is
+# missing, and at that point the service has no registry credential to offer.
+- name: Pull the menu app image
+  ansible.builtin.command: 'podman pull {{ image }}'
+  changed_when: false
 
 # The data directory is created here rather than left to the image. Under
 # keep-id the image's own root phase maps to a subuid, so anything it tries to
@@ -32,7 +31,7 @@
 # owned by {{ podman_user }}.
 - name: Ensure the app root exists
   ansible.builtin.file:
-    path: '{{ app_root }}'
+    path: '{{ host_directory }}'
     state: directory
     owner: '{{ podman_user }}'
     group: '{{ podman_user }}'
@@ -42,7 +41,7 @@
 # data/ (originals/, drafts/, published/, site.json, sections.json).
 - name: Ensure the data directory exists
   ansible.builtin.file:
-    path: '{{ app_root }}/data'
+    path: '{{ host_directory }}/data'
     state: directory
     owner: '{{ podman_user }}'
     group: '{{ podman_user }}'
@@ -50,21 +49,29 @@
 
 - name: Look for the untracked site.env
   ansible.builtin.stat:
-    path: '{{ app_root }}/site.env'
+    path: '{{ host_directory }}/site.env'
   register: menu_site_env
 
 - name: Warn when the admin has no password yet
   ansible.builtin.debug:
     msg: >-
-      {{ app_root }}/site.env is missing, so the admin is disabled: the public
-      menu still serves. Run scripts/provision-site-env.sh {{ app_root }} on this
-      host, then re-run this playbook.
+      {{ host_directory }}/site.env is missing, so the admin is disabled: the
+      public menu still serves. Run scripts/provision-site-env.sh
+      {{ host_directory }} on this host, then re-run this playbook.
   when: not menu_site_env.stat.exists
 
+- name: Keep site.env readable only by the app user
+  ansible.builtin.file:
+    path: '{{ host_directory }}/site.env'
+    owner: '{{ podman_user }}'
+    group: '{{ podman_user }}'
+    mode: '0600'
+  when: menu_site_env.stat.exists
+
 - name: Install the menu app Quadlet unit
   ansible.builtin.template:
     src: menu-app.container.j2
-    dest: '{{ quadlet_dir }}/{{ network_name }}.container'
+    dest: '{{ quadlet_dir }}/{{ container_name }}.container'
     owner: '{{ podman_user }}'
     group: '{{ podman_user }}'
     mode: '0644'
@@ -82,14 +89,14 @@
         daemon_reload: true
   rescue:
     # systemd reports "control process exited with error code" and nothing else,
-    # which is useless for the usual failures (a missing network unit, an
-    # unwritable /data, a bad uid mapping). Pull the real reason out.
+    # which is useless for the usual failures (a missing image, an unattached
+    # network, an unwritable data directory). Pull the real reason out.
     - name: Collect diagnostics
       ansible.builtin.command: '{{ item }}'
       loop:
         - 'systemctl --user status {{ unit_name }} --no-pager'
-        - 'journalctl --user -u {{ unit_name }} --no-pager -n 40'
-        - 'podman logs --tail 40 {{ container_name }}'
+        - 'journalctl --user -u {{ unit_name }} --no-pager -n 60'
+        - 'podman logs --tail 60 {{ container_name }}'
       register: menu_diagnostics
       changed_when: false
       failed_when: false
@@ -102,9 +109,18 @@
       ansible.builtin.fail:
         msg: 'The menu app did not start — see the diagnostics above.'
 
+# The unit template notifies "Restart the menu app", but handlers otherwise run
+# at the very end of the play — after the healthcheck gate below. Flush them
+# here so the gate tests the container that has the current HealthCmd; without
+# this, a run that only changes the unit would check the stale container and
+# fail. On a run where nothing changed, no handler is pending and this is a
+# no-op.
+- name: Apply pending unit changes before verifying
+  ansible.builtin.meta: flush_handlers
+
 # Quadlet honours [Install] WantedBy= when it generates the unit, so
 # `systemctl --user enable` is neither needed nor possible on a generated unit.
-- name: Verify the menu app is running
+- name: Verify the menu app unit is active
   ansible.builtin.command: 'systemctl --user is-active {{ unit_name }}'
   register: menu_active
   changed_when: false
@@ -112,13 +128,13 @@
 
 # `is-active` says the container is running; it does not say the app can serve.
 # Restart=always means even a crash-looping container is reported active — which
-# is exactly what a fatal integrity problem in the data directory produces. The
-# unit declares the health check, so run that once, synchronously.
-- name: Wait for the app to pass its health check
+# is what a fatal integrity problem in the data directory produces. The unit
+# declares the health check, so run that once, synchronously.
+- name: Wait for the container healthcheck to pass
   ansible.builtin.command: 'podman healthcheck run {{ container_name }}'
   register: menu_health
   until: menu_health.rc == 0
-  retries: 24
+  retries: 36
   delay: 5
   changed_when: false
   failed_when: false
@@ -130,8 +146,8 @@
       ansible.builtin.command: '{{ item }}'
       loop:
         - 'systemctl --user status {{ unit_name }} --no-pager'
-        - 'journalctl --user -u {{ unit_name }} --no-pager -n 40'
-        - 'podman logs --tail 40 {{ container_name }}'
+        - 'journalctl --user -u {{ unit_name }} --no-pager -n 60'
+        - 'podman logs --tail 60 {{ container_name }}'
       register: menu_health_diagnostics
       changed_when: false
       failed_when: false
@@ -143,16 +159,8 @@
     - name: Fail
       ansible.builtin.fail:
         msg: >-
-          {{ container_name }} did not pass its health check within two minutes.
-          Its logs are above, and: podman logs {{ container_name }}
-
-- name: Keep site.env readable only by the app user
-  ansible.builtin.file:
-    path: '{{ app_root }}/site.env'
-    owner: '{{ podman_user }}'
-    group: '{{ podman_user }}'
-    mode: '0600'
-  when: menu_site_env.stat.exists
+          {{ container_name }} did not pass its health check within three
+          minutes. Its logs are above, and: podman logs {{ container_name }}
 
 - name: Check the data directory for integrity problems
   # Reads the same directory the container writes, as the same user, so a
@@ -176,3 +184,27 @@
   ansible.builtin.debug:
     msg: '{{ menu_verify.stdout_lines | default([]) }}'
   when: menu_verify.rc != 0
+
+- name: Show the networks the container is attached to
+  ansible.builtin.command: >-
+    podman inspect {{ container_name }}
+    --format '{% raw %}{{ range $k, $v := .NetworkSettings.Networks }}{{ $k }} {{ end }}{% endraw %}'
+  register: menu_networks_now
+  changed_when: false
+  failed_when: false
+
+- name: Report the network
+  ansible.builtin.debug:
+    msg: >-
+      {{ container_name }} is on
+      {{ menu_networks_now.stdout | default('(container not running)') }}
+
+# Removes the credential from the user's auth file. The image is already local
+# and the unit only pulls when it is missing, so this does not break restarts.
+- name: Log out of the private registry
+  ansible.builtin.command: 'podman logout {{ registry }}'
+  changed_when: false
+  failed_when: false
+  when:
+    - registry_username | length > 0
+    - registry_password | length > 0

ansible/roles/menu-app/templates/menu-app.container.j2

@@ -1,49 +1,46 @@
 # {{ container_name }} — rootless Podman Quadlet unit.
 # Managed by Ansible (menu-admin/roles/menu-app). Do not edit by hand.
 #
-# Joins {{ network_name }}.network, written by roles/register — the same include
-# that attaches Caddy to it. Caddy reaches the app by container name, so nothing
-# is published on the host and the app is not reachable from outside the Podman
-# network at all.
+# This unit joins {{ network_name }}.network, which is created by
+# roles/register — the same include that attaches the shared Caddy unit to it,
+# so Caddy can resolve {{ container_name }} by name.
 
 [Unit]
-Description=Menu app for {{ host }}
+Description=Menu app for {{ host }} ({{ site }})
 After=network-online.target
 Wants=network-online.target
 
 [Container]
 ContainerName={{ container_name }}
-Image={{ image_name }}:{{ image_tag }}
+Image={{ image }}
 
 # keep-id maps the image's own account — `node`, uid/gid {{ app_uid }} — onto
-# {{ podman_user }}, which is what keeps every file the app writes owned by
+# {{ podman_user }}, which keeps every file the app writes owned by
 # {{ podman_user }} on the host: plain tar/rsync backups, no subuids, no
-# `podman unshare`. The same mapping is why scripts/provision-site-env.sh writes
-# site.env with --userns=keep-id.
+# `podman unshare`. scripts/provision-site-env.sh writes site.env with the same
+# mapping rather than --user, which under rootless Podman would have produced a
+# subuid-owned file the app could not read.
 UserNS=keep-id:uid={{ app_uid }},gid={{ app_gid }}
 
-# The alias is deliberately dotless. ContainerName keeps the Docker-era name so
-# operators and scripts find it, but a name with dots in it is a liability in
-# DNS, so {{ network_name }} is what the Caddyfile proxies to. Both resolve from
-# inside the network.
-NetworkAlias={{ network_name }}
-
 Environment=MENU_ROOT=/data
 Environment=MENU_SITE_CONFIG=/app/config/{{ site }}.site.json
 Environment=SECURE_COOKIES=true
 Environment=LOG_LEVEL=info
-Environment=PORT={{ container_port }}
+Environment=PORT={{ site_port }}
 
 # Everything the app writes: data/ (originals, drafts, published, site.json,
 # sections.json) and site.env. The Site's own config is baked into the image and
 # read from /app.
-Volume={{ app_root }}:/data
+Volume={{ host_directory }}:/data
 
 Network={{ network_name }}.network
 
-# podman builds an OCI image by default and drops the Containerfile's
-# HEALTHCHECK instruction, so the same check is declared here and the play gates
-# on it with `podman healthcheck run`. node is in the image.
+# No PublishPort on purpose: Caddy reaches the app over {{ network_name }}_net,
+# so nothing is exposed on the host.
+
+# podman builds an OCI image by default and drops the Dockerfile's HEALTHCHECK
+# instruction, so the same check is declared here and the play gates on it with
+# `podman healthcheck run`. node is in the image.
 HealthCmd={{ health_cmd }}
 HealthInterval=30s
 HealthTimeout=5s

ansible/roles/register/tasks/main.yaml

@@ -1,18 +1,26 @@
 ---
-# Register this Site's logical network with Caddy.
+# Register this Site's network with the shared Caddy unit.
 #
-# Copied from ansible-playbooks/caddy/roles/register/tasks/main.yaml: this repo
-# carries its own deployment, so it cannot include that file by path. Keep the
-# two in step.
+# Caddy is the base unit (caddy/playbook.yml in the ansible-playbooks repo) and
+# knows nothing about its upstreams. A service adds itself by writing two files
+# into the Quadlet directory:
 #
-# Caddy is the *base* unit and knows nothing about its upstreams. A service adds
-# itself by writing two files into the Quadlet directory:
-#
-#   1. its own <name>.network unit, and
+#   1. its own menu-<site>.network unit, and
 #   2. a drop-in next to caddy.container that pulls that network into Caddy.
 #
-# Caddy itself is owned by ansible-playbooks/caddy/playbook.yml; this play only
-# registers with it. The including play defines a "Restart Caddy" handler.
+# This mirrors ansible-playbooks/caddy/roles/register/tasks/main.yaml, vendored
+# so this repo's deploy.sh stands on its own. Keep the bytes in sync with it.
+#
+# Included from playbook.yml's pre_tasks, before the app unit is generated. Runs
+# as the connection user, which must be {{ podman_user }}.
+
+- name: Ensure the Quadlet directory exists
+  ansible.builtin.file:
+    path: '{{ quadlet_dir }}'
+    state: directory
+    owner: '{{ podman_user }}'
+    group: '{{ podman_user }}'
+    mode: '0755'
 
 - name: 'Create the {{ network_name }} network unit'
   ansible.builtin.copy:
@@ -28,13 +36,21 @@
       # create "systemd-{{ network_name }}".
       #
       # isolate blocks traffic between separate Podman networks, so a container
-      # on this network cannot reach one that is only on another. Caddy and any
-      # shared service still reach it, because they are members of it.
+      # on this network cannot reach one that is only on another. Caddy still
+      # reaches it, because it is a member of it.
       [Network]
       NetworkName={{ network_name }}_net
       Options=isolate=true
   notify: Restart Caddy
 
+- name: Ensure the Caddy Quadlet drop-in directory exists
+  ansible.builtin.file:
+    path: '{{ quadlet_dir }}/{{ caddy_name }}.container.d'
+    state: directory
+    owner: '{{ podman_user }}'
+    group: '{{ podman_user }}'
+    mode: '0755'
+
 - name: 'Register {{ network_name }} with Caddy'
   ansible.builtin.copy:
     dest: '{{ quadlet_dir }}/{{ caddy_name }}.container.d/50-{{ network_name }}.conf'
@@ -42,7 +58,7 @@
     group: '{{ podman_user }}'
     mode: '0644'
     content: |
-      # Added by the {{ network_name }} playbook. Do not edit by hand.
+      # Added by the menu-admin playbook. Do not edit by hand.
       #
       # Quadlet merges this into caddy.container. Deleting this file removes
       # the network from Caddy on the next daemon-reload.