Add the Yamtrack playbook
This commit is contained in:
1 parent
26a0654fea
commit
af2e7394b8
11 files changed
+1023
No files matched your search
Executable
+54
@@ -0,0 +1,54 @@
|
|||||||
|
---
|
||||||
|
# Yamtrack as a rootless Podman Quadlet unit, behind the shared Caddy unit
|
||||||
|
# (caddy/playbook.yml), with its cache in the shared Valkey unit
|
||||||
|
# (valkey/playbook.yml). The database stays SQLite, inside the app's own data
|
||||||
|
# directory — see readme.md, "The database stays SQLite".
|
||||||
|
#
|
||||||
|
# Connects as the container user and runs everything as that user: the Quadlet
|
||||||
|
# units belong to it, so the connection's PAM session is what makes
|
||||||
|
# `systemctl --user` work without any environment juggling.
|
||||||
|
#
|
||||||
|
# gather_facts is required: podman_user and the home directory come from the
|
||||||
|
# user_id / user_dir facts.
|
||||||
|
#
|
||||||
|
# The rootless-Podman prerequisites (installing podman, enabling lingering, the
|
||||||
|
# low-port sysctl, creating the Quadlet directory) are deliberately NOT applied
|
||||||
|
# here. They are the shared task file at caddy/roles/podman/tasks/main.yaml, and
|
||||||
|
# a host that has already run caddy/playbook.yml has them:
|
||||||
|
#
|
||||||
|
# - name: Rootless Podman prerequisites
|
||||||
|
# ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/podman/tasks/main.yaml"
|
||||||
|
#
|
||||||
|
# The variables this play needs are duplicated at the top of vars/default.yml, so
|
||||||
|
# it stands on its own.
|
||||||
|
#
|
||||||
|
# Run with: ansible-playbook -u tanshu yamtrack/playbook.yml
|
||||||
|
- name: Deploy Yamtrack as a rootless Podman Quadlet
|
||||||
|
hosts: emily
|
||||||
|
gather_facts: true
|
||||||
|
vars_files:
|
||||||
|
- vars/default.yml
|
||||||
|
|
||||||
|
pre_tasks:
|
||||||
|
# This must happen before the app unit is started, not after: the generated
|
||||||
|
# unit references {{ network_name }}.network, and Quadlet fails *unit
|
||||||
|
# generation* (so `systemctl --user start` reports "unit not found") when a
|
||||||
|
# referenced .network file does not exist yet. The same include writes the
|
||||||
|
# Caddy-side drop-in that attaches Caddy to this network, and the Valkey
|
||||||
|
# attach below needs the network unit to exist as well.
|
||||||
|
#
|
||||||
|
# network_name is a play var, so the include can see it directly.
|
||||||
|
- name: Register Yamtrack's network with Caddy
|
||||||
|
ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/register/tasks/main.yaml"
|
||||||
|
|
||||||
|
roles:
|
||||||
|
# Attaches this app's network to the shared Valkey unit, which is what makes
|
||||||
|
# REDIS_URL's host resolve, and owns the "Restart Valkey" handler that
|
||||||
|
# valkey/roles/attach notifies. Runs first so the cache is reachable before
|
||||||
|
# the app starts.
|
||||||
|
- valkey
|
||||||
|
# The app: data directory, the 0600 environment file, the Quadlet unit,
|
||||||
|
# start + verify.
|
||||||
|
- yamtrack
|
||||||
|
# Appends this site's block to the shared Caddyfile and reloads Caddy.
|
||||||
|
- caddy
|
||||||
@@ -0,0 +1,518 @@
|
|||||||
|
# Yamtrack (rootless Podman + Quadlet)
|
||||||
|
|
||||||
|
Runs [Yamtrack](https://github.com/FuzzyGrim/Yamtrack) on `emily` as a
|
||||||
|
**rootless Podman** container managed by a **systemd user unit** (Quadlet),
|
||||||
|
behind the shared [rootless Caddy](../caddy/readme.md), with its cache and Celery
|
||||||
|
broker in the shared [rootless Valkey](../valkey/readme.md). Replaces the
|
||||||
|
`docker_container` deployment on `monoco`.
|
||||||
|
|
||||||
|
The database stays **SQLite**, inside the app's own data directory
|
||||||
|
(`~/podman-data/yamtrack/data/db.sqlite3`). That is deliberate, not an
|
||||||
|
oversight — see [The database stays SQLite](#the-database-stays-sqlite).
|
||||||
|
|
||||||
|
Everything else about the old container is reproduced rather than redesigned:
|
||||||
|
the same image tag, the same four environment variables, the same upstream
|
||||||
|
(`yamtrack:8000`), the same network name (`yamtrack_net`) and the same
|
||||||
|
`SECRET`.
|
||||||
|
|
||||||
|
## Ordering
|
||||||
|
|
||||||
|
Both shared services are already migrated and running on `emily`:
|
||||||
|
|
||||||
|
1. [caddy/playbook.yml](../caddy/playbook.yml) — the reverse proxy, which owns
|
||||||
|
the Caddyfile this site appends to, supplies the `caddy.service` handlers, and
|
||||||
|
is the one play that applies the rootless-Podman prerequisites.
|
||||||
|
2. [valkey/playbook.yml](../valkey/playbook.yml) — the shared cache and Celery
|
||||||
|
broker. **Yamtrack hard-requires Redis**: its cache backend is `django_redis`
|
||||||
|
with no fallback, Celery worker and beat need a broker, and several metadata
|
||||||
|
providers talk to Redis directly. The old container was given
|
||||||
|
`redis://valkey:6379` and that is exactly what this playbook sets.
|
||||||
|
3. This playbook — **run it after the file copy below**, not before: the
|
||||||
|
entrypoint migrates the schema on first start, so an empty
|
||||||
|
`~/podman-data/yamtrack/data` must not be what the app wakes up to.
|
||||||
|
|
||||||
|
[postgres/playbook.yml](../postgres/playbook.yml) is **not** part of this
|
||||||
|
deployment: there is no `yamtrack` database in the restored cluster, because the
|
||||||
|
app never used Postgres.
|
||||||
|
|
||||||
|
This playbook attaches `yamtrack_net` to Valkey. Until it has, `valkey` does not
|
||||||
|
resolve on that network, so running this playbook is what makes the app
|
||||||
|
reachable — there is no separate step.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
yamtrack/
|
||||||
|
├── playbook.yml
|
||||||
|
├── vars/default.yml
|
||||||
|
└── roles/
|
||||||
|
├── valkey/ # attaches this app's network to the shared cache (+ its handler)
|
||||||
|
├── yamtrack/ # data dir, the 0600 env file, the Quadlet unit, start + verify
|
||||||
|
└── caddy/ # appends this site's block to the shared Caddyfile
|
||||||
|
```
|
||||||
|
|
||||||
|
Quadlet units and state:
|
||||||
|
|
||||||
|
| On the host | What |
|
||||||
|
|---|---|
|
||||||
|
| `~/.config/containers/systemd/yamtrack.container` | the generated unit (Ansible-templated) |
|
||||||
|
| `~/.config/containers/systemd/yamtrack.network` | the app network, written by `caddy/roles/register` |
|
||||||
|
| `~/.config/containers/systemd/caddy.container.d/50-yamtrack.conf` | attaches Caddy to that network |
|
||||||
|
| `~/.config/containers/systemd/valkey.container.d/50-yamtrack.conf` | attaches Valkey to it |
|
||||||
|
| `~/.config/containers/systemd/valkey.container.d/10-network-aliases.conf` | the shared `redis` alias |
|
||||||
|
| `~/podman-data/yamtrack/data/` | the image's `/yamtrack/db`: `db.sqlite3` (+ `-wal`/`-shm`) |
|
||||||
|
| `~/podman-data/yamtrack/yamtrack.env` | `SECRET`, `TZ`, `URLS`, `REDIS_URL`, `PUID`/`PGID`, mode `0600`, read by systemd |
|
||||||
|
|
||||||
|
Two Docker-era pieces are **deleted rather than ported**: `roles/network`, which
|
||||||
|
existed to attach an already-created container to a network
|
||||||
|
(`docker_network`'s `connected:`/`appends:`, which `podman_network` has no
|
||||||
|
equivalent for — [docs/podman-compat-audit.md §3.3](../docs/podman-compat-audit.md)),
|
||||||
|
and `files/nginx.conf.j2`, which was a copy-pasted *Vaultwarden* nginx template
|
||||||
|
that this playbook never used. Nothing has to be connected after the fact now:
|
||||||
|
the app's own unit names its `Network=`, and Caddy and Valkey learn about the
|
||||||
|
network from drop-ins written in the same run.
|
||||||
|
|
||||||
|
## How it registers with Caddy
|
||||||
|
|
||||||
|
Caddy is the base unit and knows nothing about its upstreams. This playbook calls
|
||||||
|
the shared include in
|
||||||
|
[caddy/roles/register](../caddy/roles/register/tasks/main.yaml), which writes the
|
||||||
|
network unit **and** the drop-in that pulls it into Caddy:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- name: Register Yamtrack's network with Caddy
|
||||||
|
ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/register/tasks/main.yaml"
|
||||||
|
```
|
||||||
|
|
||||||
|
It runs in `pre_tasks`, **before** the app unit is started, because
|
||||||
|
`yamtrack.container` references `yamtrack.network`: Quadlet fails *unit
|
||||||
|
generation* — so `systemctl --user start` reports "unit not found" — when a
|
||||||
|
referenced `.network` file does not exist yet. The Valkey attach below depends on
|
||||||
|
it for the same reason.
|
||||||
|
|
||||||
|
`network_name` is `yamtrack`, so the *file* is `yamtrack.network` and the *Podman
|
||||||
|
network* is `yamtrack_net` (`NetworkName=` in the network unit pins it, keeping
|
||||||
|
the Docker-era name).
|
||||||
|
|
||||||
|
The site block itself is appended to the shared Caddyfile by
|
||||||
|
[roles/caddy](roles/caddy/tasks/main.yaml) with `blockinfile` — the same bytes
|
||||||
|
the Docker-era Caddyfile on `monoco` already carries for this host:
|
||||||
|
|
||||||
|
```
|
||||||
|
track.tanshu.com {
|
||||||
|
reverse_proxy yamtrack:8000
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`8000` is the port the image's own nginx listens on inside the container (it
|
||||||
|
proxies to gunicorn); the image `EXPOSE`s exactly that, and
|
||||||
|
`vars/default.yml`'s `container_port` is where it is written down. The old
|
||||||
|
`roles/caddy` slurped and compared the Caddyfile before editing it, which
|
||||||
|
*fails* on a host where Caddy has not run yet; `blockinfile` is idempotent on
|
||||||
|
its own marker, so that whole dance is gone.
|
||||||
|
|
||||||
|
## How it attaches to Valkey
|
||||||
|
|
||||||
|
Same idea at the other end: Valkey knows nothing about its clients, and this
|
||||||
|
playbook uses the shared include in
|
||||||
|
[valkey/roles/attach](../valkey/roles/attach/tasks/main.yaml) rather than writing
|
||||||
|
those drop-ins itself. That writes two files, both mergeable into
|
||||||
|
`valkey.container` by Quadlet:
|
||||||
|
|
||||||
|
| File | Content |
|
||||||
|
|---|---|
|
||||||
|
| `valkey.container.d/50-yamtrack.conf` | `Network=yamtrack.network` |
|
||||||
|
| `valkey.container.d/10-network-aliases.conf` | `NetworkAlias=redis` |
|
||||||
|
|
||||||
|
`REDIS_URL` uses the host `valkey`, which is `ContainerName=valkey` and is what
|
||||||
|
the Docker-era container was given. The `redis` alias in the shared file is for
|
||||||
|
the *other* cache clients (Nextcloud writes `REDIS_HOST=redis` into its
|
||||||
|
`config.php`), not for this app; the play checks that `valkey` resolves.
|
||||||
|
|
||||||
|
[roles/valkey](roles/valkey/tasks/main.yaml) wraps that include for two reasons:
|
||||||
|
|
||||||
|
- it defines the `Restart Valkey` handler the include notifies; and
|
||||||
|
- it **flushes handlers** immediately afterwards. On a first attach the drop-in
|
||||||
|
is new, so Valkey has to be *recreated* to join `yamtrack_net` — and that
|
||||||
|
restart is a handler, which by default runs at the end of the play, after the
|
||||||
|
`yamtrack` role has already started the app. The app would then come up before
|
||||||
|
`valkey` resolves, fail its own health check, and fail the play on a machine
|
||||||
|
where nothing is wrong. Flushing applies the drop-in first; on later runs
|
||||||
|
nothing is pending and it is a no-op.
|
||||||
|
|
||||||
|
Unlike Postgres, the cost here is **not** downtime for a database: Valkey is a
|
||||||
|
cache, so every attached app simply starts with a cold cache and the first
|
||||||
|
requests after the restart are slower. See
|
||||||
|
[valkey/readme.md](../valkey/readme.md), "The trade-off". Attached means *every*
|
||||||
|
cache client, not just Yamtrack.
|
||||||
|
|
||||||
|
## Who runs as whom
|
||||||
|
|
||||||
|
`UserNS=keep-id:uid=0,gid=0` — container root is mapped onto `podman_user`, the
|
||||||
|
same choice Caddy, Vaultwarden and Tandoor make. That is what keeps
|
||||||
|
`~/podman-data/yamtrack` owned by `podman_user` rather than by a subuid, so
|
||||||
|
plain `tar`, `cp` and `rm -rf` work there without `podman unshare` or `sudo`,
|
||||||
|
and extracting the old data as `podman_user` is already correct.
|
||||||
|
|
||||||
|
**uid 0 is required here, not a convenience.** `--userns=keep-id` does two
|
||||||
|
things: it maps one container uid onto the connecting user, *and it runs the
|
||||||
|
container as that uid*. Yamtrack's entrypoint begins by rewriting `/etc/group`
|
||||||
|
and `/etc/passwd` (`groupmod`/`usermod`, applying `PUID`/`PGID`) — files owned by
|
||||||
|
container-root in the image. With `keep-id:uid=1000` the entrypoint runs as
|
||||||
|
`abc`, cannot lock `/etc/group`, and the unit dies with:
|
||||||
|
|
||||||
|
```
|
||||||
|
groupmod: /etc/group.5: Permission denied
|
||||||
|
groupmod: cannot lock /etc/group; try again later.
|
||||||
|
yamtrack.service: Main process exited, code=exited, status=10/n/a
|
||||||
|
```
|
||||||
|
|
||||||
|
before supervisord is ever reached. `manage.py migrate` has *already* run and
|
||||||
|
succeeded at that point, because the bind mount is writable as `abc` — which
|
||||||
|
makes the failure look like a database problem when it is not. Docker on
|
||||||
|
`monoco` ran the same entrypoint as real root, which is why it never surfaced
|
||||||
|
there.
|
||||||
|
|
||||||
|
`PUID`/`PGID` are therefore pinned to the same `0` in the environment file. The
|
||||||
|
entrypoint's own default is `1000`, and under this mapping container uid `1000`
|
||||||
|
is a subuid on the host, so its `chown -R abc:abc db` would leave the database
|
||||||
|
owned by e.g. `165536` — unreadable by a plain `tar` backup. With both pinned to
|
||||||
|
`0`, `abc` resolves to container root, which `keep-id` maps onto `podman_user`.
|
||||||
|
The entrypoint still chowns `db` on every start, so the copy only has to be
|
||||||
|
*readable* by `podman_user`.
|
||||||
|
|
||||||
|
The one visible side effect is this warning from Celery, once per start, in
|
||||||
|
`podman logs yamtrack`:
|
||||||
|
|
||||||
|
```
|
||||||
|
SecurityWarning: You're running the worker with superuser privileges
|
||||||
|
```
|
||||||
|
|
||||||
|
That is Celery spotting uid 0 *inside the container*, where uid 0 is an
|
||||||
|
unprivileged user on the host. Expected, and harmless in a rootless unit.
|
||||||
|
|
||||||
|
Like Caddy, `podman_user` is derived from the connection
|
||||||
|
(`ansible_facts['user_id']`) and the play carries no `become`. Because the podman
|
||||||
|
tasks are not run here, there is **no guard against connecting as `root`**; the
|
||||||
|
prerequisites, including that check, live in `caddy/playbook.yml`.
|
||||||
|
|
||||||
|
## The database stays SQLite
|
||||||
|
|
||||||
|
Yamtrack supports Postgres (`DB_HOST`, `DB_NAME`, `DB_USER`, `DB_PASSWORD`,
|
||||||
|
`DB_PORT`), and this repo now has a shared Postgres unit. It is still not used
|
||||||
|
here, on purpose:
|
||||||
|
|
||||||
|
- **The old data is SQLite.** 2.3 MB in one `db.sqlite3` file, owned by uid
|
||||||
|
`1000` ([docs/backup-plan.md §2.1](../docs/backup-plan.md)). Moving it to
|
||||||
|
Postgres is a *data conversion* — `dumpdata`/`loaddata` or a
|
||||||
|
`sqlite`→`postgres` copy — with its own failure modes, and it buys nothing
|
||||||
|
this deployment needs.
|
||||||
|
- **There is no `yamtrack` database in the restored cluster.** The twelve
|
||||||
|
databases brought across in
|
||||||
|
[postgres/migrate-from-monoco.md](../postgres/migrate-from-monoco.md) do not
|
||||||
|
include one, which is the same fact seen from the other side.
|
||||||
|
- **A single file is not a worse fit here than a shared server.** The app is a
|
||||||
|
one-user media tracker; the whole database is smaller than a single query
|
||||||
|
result in some of the other services.
|
||||||
|
|
||||||
|
The consequence to be deliberate about: **the entrypoint runs
|
||||||
|
`manage.py migrate` on every start**, so an image bump with `latest` can migrate
|
||||||
|
the schema, and that migration is not reversible by rolling the image back. The
|
||||||
|
copy in [Migrating the file data](#migrating-the-file-data-from-monoco) is the
|
||||||
|
rollback, which is one more reason to take it before the first start and to keep
|
||||||
|
`monoco` intact until this deployment has been lived with.
|
||||||
|
|
||||||
|
Moving to Postgres later is a normal, separate change: point the app at it,
|
||||||
|
convert the data, and add a `postgres` role here exactly like Nextcloud's or
|
||||||
|
Tandoor's (`postgres/roles/attach` plus a handler and a flush).
|
||||||
|
|
||||||
|
## Settings
|
||||||
|
|
||||||
|
| Variable | Value / purpose |
|
||||||
|
|---|---|
|
||||||
|
| `podman_user` | **Derived** from the connection (`ansible_facts['user_id']`), not set by hand |
|
||||||
|
| `podman_data_dir` | `~/podman-data` |
|
||||||
|
| `quadlet_dir` | `~/.config/containers/systemd` |
|
||||||
|
| `host` | `track.tanshu.com` — the site block Caddy serves |
|
||||||
|
| `container_name` / `unit_name` | `yamtrack`, and the Quadlet unit / container name |
|
||||||
|
| `image` | `ghcr.io/fuzzygrim/yamtrack:latest` — fully qualified on purpose |
|
||||||
|
| `network_name` | `yamtrack` → `yamtrack.network` / `yamtrack_net` |
|
||||||
|
| `host_directory` | `~/podman-data/yamtrack` |
|
||||||
|
| `data_dir` | `~/podman-data/yamtrack/data` → the image's `/yamtrack/db` |
|
||||||
|
| `container_port` | `8000`, the port inside the container that Caddy proxies to |
|
||||||
|
| `env_file` | `~/podman-data/yamtrack/yamtrack.env`, mode `0600` |
|
||||||
|
| `app_uid` / `app_gid` | `0` — container root, mapped by `keep-id` onto `podman_user`, and the entrypoint's `PUID`/`PGID` |
|
||||||
|
| `yamtrack_tz` | `Asia/Kolkata`, the `TZ` the old container was given |
|
||||||
|
| `yamtrack_secret` | The Django `SECRET`, carried over verbatim |
|
||||||
|
| `urls` | `https://track.tanshu.com`, the old container's `URLS` (CSRF origins) |
|
||||||
|
| `valkey_container` / `valkey_unit` | The shared cache unit this app attaches to |
|
||||||
|
| `redis_url` | `redis://valkey:6379`, unchanged from the Docker-era container |
|
||||||
|
| `caddy_name` / `caddyfile_path` | The shared Caddy unit and its Caddyfile |
|
||||||
|
|
||||||
|
The three `podman_*` / `quadlet_dir` vars are **duplicated** from
|
||||||
|
[caddy/vars/default.yml](../caddy/vars/default.yml), as in every service
|
||||||
|
playbook. That is a deliberate trade: a play cannot include the Caddy vars file
|
||||||
|
wholesale, because it also sets `host_directory` (to the *Caddy* data directory)
|
||||||
|
and every service defines that for itself. Keep them in sync. Caddy's fourth,
|
||||||
|
`podman_unprivileged_port_start`, belongs to the podman tasks this play does not
|
||||||
|
run — Yamtrack publishes nothing host-side.
|
||||||
|
|
||||||
|
`SECRET`, `TZ` and `REDIS_URL` are written to `yamtrack.env` at mode `0600` and
|
||||||
|
read by systemd via `EnvironmentFile=`, not by Ansible. `SECRET` is the same
|
||||||
|
value `monoco` uses, so existing sessions and password-reset tokens keep working
|
||||||
|
— rotating it logs every user out. It is committed in plaintext as before;
|
||||||
|
flagged as a follow-up, not changed here.
|
||||||
|
|
||||||
|
`ALLOWED_HOSTS` is deliberately absent: Yamtrack's default is `*`, which is what
|
||||||
|
the Docker-era container relied on. Setting `URLS` — as the old playbook did — is
|
||||||
|
what fills in the CSRF trusted origins.
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
```command
|
||||||
|
ansible-playbook -u tanshu yamtrack/playbook.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
The play targets `emily` directly, so the inventory must contain it — the one in
|
||||||
|
`prepare-server/` does, and its `ansible.cfg` names it.
|
||||||
|
|
||||||
|
This play applies **only** Yamtrack's own units and data. Its `pre_tasks` run
|
||||||
|
Caddy's `register` include and nothing else. The rootless-Podman prerequisites —
|
||||||
|
installing podman, `loginctl enable-linger`, the low-port sysctl and the Quadlet
|
||||||
|
directory — are the shared task file at
|
||||||
|
[caddy/roles/podman/tasks/main.yaml](../caddy/roles/podman/tasks/main.yaml), and
|
||||||
|
are **not** run here. A host that has already run `caddy/playbook.yml` has them.
|
||||||
|
Neither is Valkey deployed here: run `valkey/playbook.yml` first, or the flush in
|
||||||
|
`roles/valkey` fails on a `valkey.service` that does not exist.
|
||||||
|
|
||||||
|
`systemctl --user enable yamtrack` is neither needed nor possible: the unit is
|
||||||
|
*generated*, and Quadlet applies `[Install] WantedBy=default.target` itself.
|
||||||
|
|
||||||
|
## Migrating the file data from monoco
|
||||||
|
|
||||||
|
There is no database to restore from Postgres for this app: the whole of its
|
||||||
|
state is `db.sqlite3`, and the playbook does not move it because the file is on
|
||||||
|
the old host while the playbook runs on the new one. Run this from the machine
|
||||||
|
that holds the `monoco` and `emily` SSH aliases.
|
||||||
|
|
||||||
|
**Copy before deploying.** The entrypoint migrates the schema on first start, so
|
||||||
|
a first run against an empty directory creates a new, empty database and the
|
||||||
|
copy would then be overwriting a live one.
|
||||||
|
|
||||||
|
If a first run has *already* happened — the unit crash-looped, or a deploy was
|
||||||
|
tried before the copy — that directory now holds a fresh database with every
|
||||||
|
migration applied and no rows. Delete it before copying rather than copying on
|
||||||
|
top of it, so no `-wal`/`-shm` from the empty database can mix with the real
|
||||||
|
one:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Prove it is the empty one before removing it: 0 users, 0 items, 0 episodes.
|
||||||
|
ssh emily 'podman exec -i yamtrack python3 -' <<'PY'
|
||||||
|
import sqlite3
|
||||||
|
c = sqlite3.connect("/yamtrack/db/db.sqlite3")
|
||||||
|
for t in ("users_user", "app_item", "app_episode"):
|
||||||
|
print(t, c.execute("select count(*) from " + t).fetchone()[0])
|
||||||
|
PY
|
||||||
|
|
||||||
|
ssh emily 'systemctl --user stop yamtrack.service; rm -rf ~/podman-data/yamtrack/data'
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Look before you copy: size, contents, and whether it is readable without
|
||||||
|
# sudo. `ls -ln` shows numeric owners — the old data is uid 1000, which is
|
||||||
|
# `ubuntu` on monoco and the container's `abc` inside it.
|
||||||
|
ssh monoco 'du -sh /var/lib/yamtrack/data; ls -ln /var/lib/yamtrack/data'
|
||||||
|
ssh monoco 'find /var/lib/yamtrack/data -type f ! -readable' # empty = no sudo needed
|
||||||
|
|
||||||
|
# 2. Stop the old container so nothing writes while you copy. Leave it stopped:
|
||||||
|
# two live copies of one SQLite database would diverge.
|
||||||
|
#
|
||||||
|
# This is also what makes the copy *consistent*. SQLite runs in WAL mode here,
|
||||||
|
# and a plain tar of a live database can produce a file that will not open —
|
||||||
|
# the main file and the WAL are copied at different instants
|
||||||
|
# (docs/backup-plan.md §3.1). A clean stop checkpoints the WAL into
|
||||||
|
# db.sqlite3 and removes the sidecars, so the copy is a single frozen file.
|
||||||
|
ssh monoco 'docker stop yamtrack'
|
||||||
|
|
||||||
|
# 3. Create the destination.
|
||||||
|
ssh emily 'mkdir -p ~/podman-data/yamtrack/data'
|
||||||
|
|
||||||
|
# 4. Copy. tar streams it across the workstation; rsync cannot copy
|
||||||
|
# remote-to-remote in one command.
|
||||||
|
#
|
||||||
|
# The -wal/-shm sidecars are copied rather than filtered out on purpose: if
|
||||||
|
# anything is still in the WAL, it is part of the database. After a clean stop
|
||||||
|
# they are usually already gone, and copying "nothing" costs nothing.
|
||||||
|
#
|
||||||
|
# `set -o pipefail` matters here: without it the pipeline reports only the
|
||||||
|
# *receiving* tar's status, so a read error or a dropped stream on the source
|
||||||
|
# side exits 0 and leaves a silently short copy.
|
||||||
|
set -o pipefail
|
||||||
|
ssh monoco 'tar -C /var/lib/yamtrack/data -cf - .' \
|
||||||
|
| ssh emily 'tar --no-same-owner -C ~/podman-data/yamtrack/data -xf -'
|
||||||
|
|
||||||
|
# If step 1 listed unreadable files, read them as root instead:
|
||||||
|
# ssh monoco 'sudo tar -C /var/lib/yamtrack/data -cf - .' \
|
||||||
|
# | ssh emily 'tar --no-same-owner -C ~/podman-data/yamtrack/data -xf -'
|
||||||
|
|
||||||
|
# 5. Verify by content. Both sides are frozen by now (the old container is
|
||||||
|
# stopped and nothing writes on emily yet), so identical digests mean a
|
||||||
|
# byte-faithful copy.
|
||||||
|
ssh monoco 'cd /var/lib/yamtrack/data && find . -type f -print0 | sort -z | xargs -0 -r sha256sum | sha256sum'
|
||||||
|
ssh emily 'cd ~/podman-data/yamtrack/data && find . -type f -print0 | sort -z | xargs -0 -r sha256sum | sha256sum'
|
||||||
|
|
||||||
|
# 6. Nothing may end up owned by root or by a subuid.
|
||||||
|
ssh emily 'find ~/podman-data/yamtrack ! -user tanshu' # empty = correct
|
||||||
|
|
||||||
|
# 7. Deploy.
|
||||||
|
cd prepare-server && ansible-playbook -u tanshu ../yamtrack/playbook.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
Ownership needs no `chown` pass: the app runs as uid `1000` in-container, which
|
||||||
|
`keep-id` maps to `tanshu`, so files extracted as `tanshu` are already right, and
|
||||||
|
the entrypoint's own `chown abc:abc db` runs on every start anyway. That is also
|
||||||
|
why `--no-same-owner` is explicit on the receiving side — the source is owned by
|
||||||
|
uid `1000`, and restoring that ownership would be wrong here.
|
||||||
|
|
||||||
|
If the link is slow or drops, do it in two hops instead: resumable, and it shows
|
||||||
|
progress.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
rsync -aP monoco:/var/lib/yamtrack/data/ ~/yamtrack-copy/
|
||||||
|
rsync -aP ~/yamtrack-copy/ emily:~/podman-data/yamtrack/data/
|
||||||
|
```
|
||||||
|
|
||||||
|
If stopping the app is unacceptable — for instance while rehearsing this
|
||||||
|
deployment before the DNS move — ask SQLite for a consistent snapshot instead of
|
||||||
|
copying the files while they are hot:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh monoco 'docker exec -i yamtrack python3 -' <<'PY'
|
||||||
|
import sqlite3
|
||||||
|
sqlite3.connect("/yamtrack/db/db.sqlite3").execute(
|
||||||
|
"VACUUM INTO '/yamtrack/db/pre-migration.sqlite3'"
|
||||||
|
)
|
||||||
|
print("wrote /yamtrack/db/pre-migration.sqlite3")
|
||||||
|
PY
|
||||||
|
|
||||||
|
# then copy it as the database, and delete the snapshot on monoco afterwards
|
||||||
|
ssh monoco 'cat /var/lib/yamtrack/data/pre-migration.sqlite3' \
|
||||||
|
| ssh emily 'cat > ~/podman-data/yamtrack/data/db.sqlite3'
|
||||||
|
```
|
||||||
|
|
||||||
|
`VACUUM INTO` needs SQLite 3.27+; the image's Python 3.12 has far newer. This
|
||||||
|
still has to be followed by stopping the container before DNS moves: it shortens
|
||||||
|
the window, it does not remove the cutover.
|
||||||
|
|
||||||
|
Rolling back means rolling back both halves: `ssh monoco 'docker start yamtrack'`
|
||||||
|
is only meaningful while `monoco`'s data and reverse proxy are intact, and the
|
||||||
|
DNS record has to point back. `monoco` was never modified — its container was
|
||||||
|
only stopped — so the rollback is starting it again.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl --user status yamtrack
|
||||||
|
journalctl --user -u yamtrack -f
|
||||||
|
podman logs yamtrack
|
||||||
|
|
||||||
|
# `podman ps` shows (healthy) once the image's own probe passes.
|
||||||
|
podman ps --filter name=yamtrack
|
||||||
|
podman healthcheck run yamtrack
|
||||||
|
|
||||||
|
# When it does not pass, the health view names the failing check — cache,
|
||||||
|
# database, Redis or Celery — instead of just "unhealthy":
|
||||||
|
podman exec yamtrack wget -qO- http://127.0.0.1:8000/health/
|
||||||
|
|
||||||
|
# What the playbook checks for you:
|
||||||
|
podman exec yamtrack getent hosts valkey
|
||||||
|
|
||||||
|
# The file really is the migrated database, untouched by a first-start migrate:
|
||||||
|
podman exec yamtrack python3 -c "import sqlite3; print(sqlite3.connect('/yamtrack/db/db.sqlite3').execute('pragma integrity_check').fetchone()[0])"
|
||||||
|
|
||||||
|
# Row counts compared with monoco — equal numbers mean the library came across.
|
||||||
|
# (1517 items and 1312 episodes when this was written. Use `docker exec` on
|
||||||
|
# monoco, `podman exec` on emily.)
|
||||||
|
ssh monoco 'docker exec yamtrack python3 -c "import sqlite3; print(sqlite3.connect(\"/yamtrack/db/db.sqlite3\").execute(\"select count(*) from app_item\").fetchone()[0])"'
|
||||||
|
ssh emily 'podman exec yamtrack python3 -c "import sqlite3; print(sqlite3.connect(\"/yamtrack/db/db.sqlite3\").execute(\"select count(*) from app_item\").fetchone()[0])"'
|
||||||
|
|
||||||
|
# The site, end to end:
|
||||||
|
curl -sSI https://track.tanshu.com
|
||||||
|
```
|
||||||
|
|
||||||
|
Then log in **in a browser** and confirm the library, an episode's history and
|
||||||
|
one metadata lookup feel right — the health check proves the process, not the
|
||||||
|
data. Back it up with a **write**: mark an episode watched, then confirm on the
|
||||||
|
host that `db.sqlite3` changed and is owned by `tanshu`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh emily 'ls -ln ~/podman-data/yamtrack/data'
|
||||||
|
```
|
||||||
|
|
||||||
|
A write landing as root or as a `165536+` subuid is the `keep-id` check, and a
|
||||||
|
read-only smoke test will not catch it.
|
||||||
|
|
||||||
|
Then reboot the host and confirm the unit returns unattended — without
|
||||||
|
`loginctl enable-linger tanshu` it will not, and the failure is silent.
|
||||||
|
|
||||||
|
## Notes and gotchas
|
||||||
|
|
||||||
|
- **`SECRET` is committed in plaintext**, as it was before. It is the same value
|
||||||
|
`monoco` used, so sessions survive the move; changing it is a separate,
|
||||||
|
user-visible decision (everyone is logged out, pending password-reset links
|
||||||
|
break).
|
||||||
|
- **`latest` floats, and the entrypoint migrates.** Neither this playbook nor the
|
||||||
|
old one pins a digest or a version, so an image bump can run new Django
|
||||||
|
migrations against the SQLite file on first start. Rolling the image back does
|
||||||
|
not roll the schema back; the copy taken in
|
||||||
|
[Migrating the file data](#migrating-the-file-data-from-monoco) is the
|
||||||
|
rollback. This is not hypothetical: `monoco` is running an image pulled in
|
||||||
|
April (`v0.25.2`, 200 applied migrations) while `emily` pulled a newer
|
||||||
|
`latest`, so the copied database gets 12 migrations on its first start under
|
||||||
|
the new unit.
|
||||||
|
- **The entrypoint needs root, so `keep-id`'s uid matters.** See
|
||||||
|
[Who runs as whom](#who-runs-as-whom) — `keep-id:uid=1000` produces a crash
|
||||||
|
loop whose log looks like a database or permission problem and is neither.
|
||||||
|
After that failure the data directory contains a **fresh, empty** database
|
||||||
|
(0 users, 0 items) with every migration applied, which will happily serve an
|
||||||
|
empty library if the copy has not landed. Check
|
||||||
|
`select count(*) from app_item` before pointing DNS anywhere.
|
||||||
|
- **Yamtrack requires Redis.** There is no "cache disabled" mode: the cache
|
||||||
|
backend is `django_redis` and Celery needs the broker. If `valkey` is down, the
|
||||||
|
site does not degrade — it fails.
|
||||||
|
- **`staticfiles/` is not on the host.** Unlike Tandoor, this image bakes its
|
||||||
|
collected static files in, so `~/podman-data/yamtrack/data` is the whole of
|
||||||
|
what has to be backed up. The old container's `/yamtrack` showed
|
||||||
|
`staticfiles/` owned by `abc`; that is image content, not state.
|
||||||
|
- **Client IPs.** Yamtrack's nginx sits behind Caddy, and whether the real client
|
||||||
|
address survives the rootless port forward depends on the forwarder. See
|
||||||
|
[caddy/readme.md](../caddy/readme.md), "Client IPs", before relying on it.
|
||||||
|
- **Registering restarts all of Caddy**, and attaching restarts Valkey for every
|
||||||
|
cache client — a brief restart of every hosted site, and a cold cache
|
||||||
|
everywhere. See the trade-offs in [caddy/readme.md](../caddy/readme.md) and
|
||||||
|
[valkey/readme.md](../valkey/readme.md).
|
||||||
|
- **If the Caddyfile does not exist yet**, this playbook creates it containing
|
||||||
|
only the site block. That is a valid Caddyfile; the Caddy playbook's seeded
|
||||||
|
global-options block is then not added, because its seed only fires on a
|
||||||
|
missing or zero-length file. Deploy Caddy first on a fresh host.
|
||||||
|
- **DNS.** `track.tanshu.com` currently points at `monoco` (`13.202.6.2`) and
|
||||||
|
must be repointed at `emily` (`35.154.14.59`) before Caddy can obtain a
|
||||||
|
certificate for it. Until then the site answers from the old host.
|
||||||
|
- **Do not restart the container on `monoco`** after the cutover. It would keep
|
||||||
|
writing to the old database, and two live copies behind one hostname diverge
|
||||||
|
silently.
|
||||||
|
- **There is no `yamtrack-backup.sh`.** `~/bin` on `monoco` has scripts for
|
||||||
|
Nextcloud, Gitea, Bitwarden, Tandoor and others, but none for this app, and
|
||||||
|
root's crontab has only the Nextcloud entry — so nothing has ever been
|
||||||
|
scheduled ([docs/backup-plan.md §11](../docs/backup-plan.md)). Nothing has to
|
||||||
|
be ported here, but the 2.3 MB file that *is* this service has no backup
|
||||||
|
either: a `systemd --user` timer doing `podman exec yamtrack python3 -c "…"
|
||||||
|
"VACUUM INTO …"` next to the Quadlet units is the shape that fits this host,
|
||||||
|
because root cannot `podman exec` into another user's rootless containers.
|
||||||
|
|
||||||
|
## Not done here
|
||||||
|
|
||||||
|
- Rotating `SECRET` and the other secrets that are committed in plaintext.
|
||||||
|
- Moving the database to the shared Postgres unit (see
|
||||||
|
[The database stays SQLite](#the-database-stays-sqlite)).
|
||||||
|
- Replacing the missing backup with a systemd user timer (see above).
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
---
|
||||||
|
# Caddy is its own playbook (caddy/playbook.yml) and a rootless Quadlet unit
|
||||||
|
# there. These handlers run as the connection user, so `systemctl --user` and
|
||||||
|
# `podman exec` work without any XDG_RUNTIME_DIR juggling.
|
||||||
|
|
||||||
|
# Notified by this role when the Caddyfile changes. A reload keeps existing
|
||||||
|
# connections; a restart would briefly drop every hosted site.
|
||||||
|
#
|
||||||
|
# Note the config path: /etc/caddy/Caddyfile is the mount inside the container,
|
||||||
|
# not {{ caddyfile_path }}, which is the host side of it.
|
||||||
|
- name: Reload Caddy configuration
|
||||||
|
ansible.builtin.command: >-
|
||||||
|
podman exec {{ caddy_name }} caddy reload --config /etc/caddy/Caddyfile
|
||||||
|
changed_when: true
|
||||||
|
listen: "Reload Caddy"
|
||||||
|
|
||||||
|
# Notified by caddy/roles/register when this service's network unit, or the
|
||||||
|
# drop-in that attaches Caddy to it, is written. Quadlet has no live equivalent
|
||||||
|
# of `docker network connect`, so the container has to be recreated — a brief
|
||||||
|
# restart of every hosted site, not just this one. See caddy/readme.md,
|
||||||
|
# "Trade-off worth knowing".
|
||||||
|
#
|
||||||
|
# Defined after the reload above so that when both are notified the cheap
|
||||||
|
# config reload happens first.
|
||||||
|
- name: Restart Caddy
|
||||||
|
ansible.builtin.systemd_service:
|
||||||
|
name: "{{ caddy_name }}.service"
|
||||||
|
scope: user
|
||||||
|
state: restarted
|
||||||
|
daemon_reload: true
|
||||||
|
listen: Restart Caddy
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
---
|
||||||
|
# Append this site's block to the shared Caddyfile.
|
||||||
|
#
|
||||||
|
# The Caddyfile is a bind mount into the Caddy container, so Caddy sees the
|
||||||
|
# change as soon as the file is written; the handler only asks Caddy to re-read
|
||||||
|
# it. That is a reload, not a restart, so the other hosted sites are untouched.
|
||||||
|
#
|
||||||
|
# blockinfile is idempotent on its own via the marker, so there is no need to
|
||||||
|
# slurp the file and compare first — which also matters because slurp *fails*
|
||||||
|
# when the Caddyfile does not exist yet, on a host where the Caddy playbook has
|
||||||
|
# not run. (The Docker-era version of this role did exactly that.)
|
||||||
|
|
||||||
|
- name: "Add the {{ host }} site block to the Caddyfile"
|
||||||
|
ansible.builtin.blockinfile:
|
||||||
|
path: "{{ caddyfile_path }}"
|
||||||
|
marker: "# {mark} Ansible managed Caddy snippet for {{ host }}"
|
||||||
|
block: "{{ snippet_block }}"
|
||||||
|
create: yes
|
||||||
|
owner: "{{ podman_user }}"
|
||||||
|
group: "{{ podman_user }}"
|
||||||
|
mode: '0644'
|
||||||
|
notify: Reload Caddy
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
---
|
||||||
|
# Valkey is its own playbook (valkey/playbook.yml) and a rootless Quadlet unit
|
||||||
|
# there. This handler runs as the connection user, so `systemctl --user` works
|
||||||
|
# without setting XDG_RUNTIME_DIR.
|
||||||
|
#
|
||||||
|
# Required by valkey/roles/attach, which notifies it whenever this app's drop-in
|
||||||
|
# changes: a new drop-in changes the generated unit, so Valkey must be recreated
|
||||||
|
# to join the network — Quadlet has no live equivalent of
|
||||||
|
# `podman network connect`.
|
||||||
|
#
|
||||||
|
# The cost is a cold cache for every attached app, not lost data — but note that
|
||||||
|
# attached means every cache client, not just Yamtrack. See valkey/readme.md,
|
||||||
|
# "The trade-off".
|
||||||
|
- name: Restart Valkey
|
||||||
|
ansible.builtin.systemd_service:
|
||||||
|
name: "{{ valkey_unit }}"
|
||||||
|
scope: user
|
||||||
|
state: restarted
|
||||||
|
daemon_reload: true
|
||||||
|
listen: Restart Valkey
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
---
|
||||||
|
# Attach this app's network to the shared Valkey unit.
|
||||||
|
#
|
||||||
|
# Valkey knows nothing about its clients, exactly like Caddy and Postgres: a
|
||||||
|
# service joins by dropping 50-yamtrack.conf and the shared
|
||||||
|
# 10-network-aliases.conf next to valkey.container. The canonical way to do that
|
||||||
|
# is the shared include below — it writes the same bytes every service writes,
|
||||||
|
# asserts that yamtrack.network already exists, and notifies the "Restart Valkey"
|
||||||
|
# handler defined in this role.
|
||||||
|
#
|
||||||
|
# `valkey` is the host REDIS_URL asks for — the container name, not the shared
|
||||||
|
# `redis` alias, which is there for the clients whose config.php says `redis`.
|
||||||
|
# The Docker-era container was given `redis://valkey:6379` too, so the URL is
|
||||||
|
# unchanged.
|
||||||
|
#
|
||||||
|
# This runs in the roles phase, which is after the register include in
|
||||||
|
# playbook.yml's pre_tasks has created yamtrack.network.
|
||||||
|
|
||||||
|
- name: Attach Yamtrack to Valkey
|
||||||
|
ansible.builtin.include_tasks: "{{ playbook_dir }}/../valkey/roles/attach/tasks/main.yaml"
|
||||||
|
|
||||||
|
# On a first attach the drop-in is new, so Valkey has to be recreated to join
|
||||||
|
# yamtrack_net — and that restart is a *handler*, which by default runs at the
|
||||||
|
# end of the play, after the yamtrack role has already started the app. The app
|
||||||
|
# would then come up before `valkey` resolves, fail its own checks, and fail the
|
||||||
|
# play on a machine where nothing is actually wrong.
|
||||||
|
#
|
||||||
|
# Flushing applies the new drop-in before the app starts. On later runs the
|
||||||
|
# drop-in is unchanged, nothing is pending, and this is a no-op.
|
||||||
|
- name: Apply the Valkey network change before starting Yamtrack
|
||||||
|
ansible.builtin.meta: flush_handlers
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
---
|
||||||
|
# All handlers run as the connection user ({{ podman_user }}), so systemctl
|
||||||
|
# --user works without setting XDG_RUNTIME_DIR.
|
||||||
|
|
||||||
|
- name: Reload systemd user units
|
||||||
|
ansible.builtin.command: systemctl --user daemon-reload
|
||||||
|
changed_when: true
|
||||||
|
listen: Reload systemd user units
|
||||||
|
|
||||||
|
# A changed environment file or unit file changes the generated unit, so
|
||||||
|
# Yamtrack is recreated. The database is a bind mount and the rest of /yamtrack
|
||||||
|
# is image content, so a recreate loses nothing — but it is a few seconds of
|
||||||
|
# downtime for the one site.
|
||||||
|
- name: Restart Yamtrack
|
||||||
|
ansible.builtin.systemd_service:
|
||||||
|
name: "{{ unit_name }}"
|
||||||
|
scope: user
|
||||||
|
state: restarted
|
||||||
|
daemon_reload: true
|
||||||
|
listen: Restart Yamtrack
|
||||||
@@ -0,0 +1,121 @@
|
|||||||
|
---
|
||||||
|
# Yamtrack as a rootless Podman Quadlet unit.
|
||||||
|
#
|
||||||
|
# Everything here runs as {{ podman_user }} — the connection user. No become, no
|
||||||
|
# XDG_RUNTIME_DIR: the SSH connection gave us a real PAM session, so systemd's
|
||||||
|
# user manager is reachable through the standard environment.
|
||||||
|
|
||||||
|
# /yamtrack/db is the image's state directory: db.sqlite3 plus its -wal/-shm
|
||||||
|
# sidecars, and nothing else worth keeping — staticfiles/ is built into the
|
||||||
|
# image and the code is read-only.
|
||||||
|
- name: Ensure the Yamtrack data directory exists
|
||||||
|
ansible.builtin.file:
|
||||||
|
path: "{{ data_dir }}"
|
||||||
|
state: directory
|
||||||
|
owner: "{{ podman_user }}"
|
||||||
|
group: "{{ podman_user }}"
|
||||||
|
mode: '0755'
|
||||||
|
|
||||||
|
# The app's secrets and cache coordinates are consumed by systemd
|
||||||
|
# (EnvironmentFile= in the Quadlet unit), not by Ansible, so this has to be
|
||||||
|
# written before the unit starts. Mode 0600: it holds SECRET.
|
||||||
|
- name: Write the environment file
|
||||||
|
ansible.builtin.template:
|
||||||
|
src: yamtrack.env.j2
|
||||||
|
dest: "{{ env_file }}"
|
||||||
|
owner: "{{ podman_user }}"
|
||||||
|
group: "{{ podman_user }}"
|
||||||
|
mode: '0600'
|
||||||
|
notify: Restart Yamtrack
|
||||||
|
|
||||||
|
- name: Install the Yamtrack Quadlet unit
|
||||||
|
ansible.builtin.template:
|
||||||
|
src: yamtrack.container.j2
|
||||||
|
dest: "{{ quadlet_dir }}/{{ container_name }}.container"
|
||||||
|
owner: "{{ podman_user }}"
|
||||||
|
group: "{{ podman_user }}"
|
||||||
|
mode: '0644'
|
||||||
|
notify:
|
||||||
|
- Reload systemd user units
|
||||||
|
- Restart Yamtrack
|
||||||
|
|
||||||
|
- name: Start Yamtrack
|
||||||
|
block:
|
||||||
|
- name: Start Yamtrack
|
||||||
|
ansible.builtin.systemd_service:
|
||||||
|
name: "{{ unit_name }}"
|
||||||
|
scope: user
|
||||||
|
state: started
|
||||||
|
daemon_reload: true
|
||||||
|
rescue:
|
||||||
|
# systemd reports "control process exited with error code" and nothing
|
||||||
|
# else, which is useless for the usual failures (an unattached network, an
|
||||||
|
# unreachable cache, a bad volume). Pull the real reason out.
|
||||||
|
- name: Collect Yamtrack diagnostics
|
||||||
|
ansible.builtin.command: "{{ item }}"
|
||||||
|
loop:
|
||||||
|
- "systemctl --user status {{ unit_name }} --no-pager"
|
||||||
|
- "journalctl --user -u {{ unit_name }} --no-pager -n 40"
|
||||||
|
- "podman logs --tail 40 {{ container_name }}"
|
||||||
|
register: yamtrack_diag
|
||||||
|
changed_when: false
|
||||||
|
failed_when: false
|
||||||
|
|
||||||
|
- name: Show Yamtrack diagnostics
|
||||||
|
ansible.builtin.debug:
|
||||||
|
msg: "{{ yamtrack_diag.results | map(attribute='stdout') | list }}"
|
||||||
|
|
||||||
|
- name: Fail
|
||||||
|
ansible.builtin.fail:
|
||||||
|
msg: "Yamtrack did not start — see the diagnostics above."
|
||||||
|
|
||||||
|
# Quadlet honours [Install] WantedBy= when it generates the unit, so
|
||||||
|
# `systemctl --user enable` is neither needed nor possible on a generated unit.
|
||||||
|
#
|
||||||
|
# `is-active` says the container is running; it does not say the app can serve.
|
||||||
|
# The entrypoint runs migrations before supervisord starts nginx, gunicorn,
|
||||||
|
# celery and beat, and `Restart=always` means even a crash-looping container is
|
||||||
|
# reported active — so the checks below are the ones that matter.
|
||||||
|
- name: Verify Yamtrack is running
|
||||||
|
ansible.builtin.command: "systemctl --user is-active {{ unit_name }}"
|
||||||
|
register: yamtrack_active
|
||||||
|
changed_when: false
|
||||||
|
failed_when: yamtrack_active.stdout != 'active'
|
||||||
|
|
||||||
|
# `valkey` is the host the environment file actually asks for; it resolves from
|
||||||
|
# ContainerName= on the shared unit, which valkey/roles/attach joined to this
|
||||||
|
# app's network.
|
||||||
|
- name: Confirm Yamtrack can resolve its cache
|
||||||
|
ansible.builtin.command: >-
|
||||||
|
podman exec {{ container_name }} getent hosts {{ valkey_container }}
|
||||||
|
register: yamtrack_cache_lookup
|
||||||
|
changed_when: false
|
||||||
|
failed_when: false
|
||||||
|
|
||||||
|
- name: Report cache resolution
|
||||||
|
ansible.builtin.debug:
|
||||||
|
msg: >-
|
||||||
|
{{ valkey_container }} resolves as
|
||||||
|
{{ yamtrack_cache_lookup.stdout | default('(no answer)') }}
|
||||||
|
|
||||||
|
# `is-active` says the container is running; the image's own HEALTHCHECK is what
|
||||||
|
# says the app works. It wgets http://127.0.0.1:8000/health/, and that view
|
||||||
|
# checks all four things this deployment can get wrong — the cache, the database,
|
||||||
|
# Redis and Celery — so a green result here is the app being genuinely ready.
|
||||||
|
#
|
||||||
|
# Running the image's check rather than one written here means depending on
|
||||||
|
# nothing the play has to know: no curl or wget wrapper of our own, and no
|
||||||
|
# hardcoded path that a later image tag could move. `podman exec <c> wget -qO-
|
||||||
|
# http://127.0.0.1:{{ container_port }}/health/` prints the per-check breakdown
|
||||||
|
# when this fails; see readme.md, "Verification".
|
||||||
|
#
|
||||||
|
# The start period (30s) plus the entrypoint's migrations are why this allows
|
||||||
|
# three minutes rather than one.
|
||||||
|
- name: Wait for Yamtrack to pass its own health check
|
||||||
|
ansible.builtin.command: "podman healthcheck run {{ container_name }}"
|
||||||
|
register: yamtrack_health
|
||||||
|
until: yamtrack_health.rc == 0
|
||||||
|
retries: 36
|
||||||
|
delay: 5
|
||||||
|
changed_when: false
|
||||||
|
failed_when: yamtrack_health.rc != 0
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# Yamtrack — rootless Podman Quadlet unit.
|
||||||
|
# Managed by Ansible (yamtrack/roles/yamtrack). Do not edit by hand.
|
||||||
|
#
|
||||||
|
# This unit joins {{ network_name }}.network, which is created by
|
||||||
|
# caddy/roles/register (the same include that attaches Caddy to it). The shared
|
||||||
|
# valkey container is joined to that network by valkey/roles/attach, which is
|
||||||
|
# what makes the host in REDIS_URL resolve.
|
||||||
|
|
||||||
|
[Unit]
|
||||||
|
Description=Yamtrack
|
||||||
|
Documentation=https://github.com/FuzzyGrim/Yamtrack
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Container]
|
||||||
|
ContainerName={{ container_name }}
|
||||||
|
Image={{ image }}
|
||||||
|
|
||||||
|
# keep-id maps *container root* onto {{ podman_user }} on the host, so
|
||||||
|
# {{ data_dir }} stays owned by {{ podman_user }} and never by a subuid, and a
|
||||||
|
# plain tar backup keeps working without `podman unshare` or `sudo`.
|
||||||
|
#
|
||||||
|
# uid 0 is not incidental. `--userns=keep-id` also decides which user the
|
||||||
|
# container runs as — the *mapped* one — and the image's entrypoint needs real
|
||||||
|
# root for its first steps: `groupmod` and `usermod` rewrite /etc/group and
|
||||||
|
# /etc/passwd, which are owned by container-root in the image. With
|
||||||
|
# `keep-id:uid=1000` the entrypoint runs as `abc`, cannot lock /etc/group, and
|
||||||
|
# exits 10 before supervisord is ever reached. The entrypoint's PUID/PGID are
|
||||||
|
# pinned to the same 0 in the environment file, so `abc` resolves to container
|
||||||
|
# root and everything the app writes lands on {{ podman_user }}.
|
||||||
|
#
|
||||||
|
# See readme.md, "Who runs as whom".
|
||||||
|
UserNS=keep-id:uid={{ app_uid }},gid={{ app_gid }}
|
||||||
|
|
||||||
|
# SECRET, TZ, URLS and REDIS_URL live here rather than as Environment= lines
|
||||||
|
# above: this unit is world-readable in ~/.config, and SECRET signs sessions and
|
||||||
|
# password-reset tokens. Read by systemd, not by Ansible, so the file has to
|
||||||
|
# exist before this unit starts.
|
||||||
|
EnvironmentFile={{ env_file }}
|
||||||
|
|
||||||
|
Network={{ network_name }}.network
|
||||||
|
|
||||||
|
# The image's state directory: db.sqlite3 and its -wal/-shm sidecars. The rest
|
||||||
|
# of /yamtrack is image content (staticfiles/ is built in, the code is
|
||||||
|
# read-only), so this is the whole of what has to survive.
|
||||||
|
Volume={{ data_dir }}:/yamtrack/db
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Restart=always
|
||||||
|
# The entrypoint runs `manage.py migrate --noinput` before supervisord starts
|
||||||
|
# nginx, gunicorn, celery and celery beat, so allow for a slow first start on a
|
||||||
|
# large database.
|
||||||
|
TimeoutStartSec=300
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# Environment for {{ container_name }}.container.
|
||||||
|
# Managed by Ansible (yamtrack/roles/yamtrack). Do not edit by hand.
|
||||||
|
#
|
||||||
|
# Mode 0600 on purpose: the Quadlet unit that reads this file is world-readable,
|
||||||
|
# and SECRET signs sessions and password-reset tokens. Note that `podman inspect`
|
||||||
|
# still prints the resulting environment to anyone who can run podman as this
|
||||||
|
# user.
|
||||||
|
#
|
||||||
|
# The first four are the variables the Docker-era container was given, so this
|
||||||
|
# deployment behaves identically. ALLOWED_HOSTS is deliberately absent: its
|
||||||
|
# default is `*`, which is what the old container relied on. PUID/PGID are new —
|
||||||
|
# the Docker-era container ran as real root, and under rootless keep-id the
|
||||||
|
# entrypoint needs to be told which uid is the mapped one. See readme.md, "Who
|
||||||
|
# runs as whom".
|
||||||
|
|
||||||
|
# The Django SECRET_KEY. Carried over verbatim so existing sessions stay valid.
|
||||||
|
SECRET={{ yamtrack_secret }}
|
||||||
|
|
||||||
|
# TIME_ZONE for the app and for celery beat's schedules.
|
||||||
|
TZ={{ yamtrack_tz }}
|
||||||
|
|
||||||
|
# CSRF trusted origins. The old container set `URLS=https://{{ host }}`.
|
||||||
|
URLS={{ urls }}
|
||||||
|
|
||||||
|
# The shared Valkey unit, attached to this app's network by
|
||||||
|
# yamtrack/roles/valkey. Yamtrack requires Redis: it is the cache backend and
|
||||||
|
# the Celery broker, with no fallback.
|
||||||
|
REDIS_URL={{ redis_url }}
|
||||||
|
|
||||||
|
# The entrypoint runs `groupmod`/`usermod` to make the image's `abc` account
|
||||||
|
# match these, then chowns the data directory to it. Its own defaults are 1000,
|
||||||
|
# which under this unit's keep-id:uid=0 is a *subuid* on the host — the database
|
||||||
|
# would come out owned by e.g. 165536, invisible to a plain tar backup. Pinning
|
||||||
|
# both to 0 makes `abc` resolve to container root, which keep-id maps onto
|
||||||
|
# podman_user.
|
||||||
|
PUID={{ app_uid }}
|
||||||
|
PGID={{ app_gid }}
|
||||||
@@ -0,0 +1,113 @@
|
|||||||
|
---
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Rootless Podman / Quadlet
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Duplicated on purpose from caddy/vars/default.yml: a play cannot include that
|
||||||
|
# file wholesale, because it also sets `host_directory` — to the Caddy data
|
||||||
|
# directory — and every playbook defines that for itself. Keep these three in
|
||||||
|
# sync with caddy/vars/default.yml. (Its fourth, podman_unprivileged_port_start,
|
||||||
|
# belongs to the podman tasks, which this play does not run: Yamtrack listens on
|
||||||
|
# 8000 inside the container and publishes nothing host-side.)
|
||||||
|
#
|
||||||
|
# podman_user is derived from the connection: this play carries no become, so
|
||||||
|
# the Quadlet units belong to whoever Ansible connects as. Requires gather_facts
|
||||||
|
# (set in playbook.yml).
|
||||||
|
podman_user: "{{ ansible_facts['user_id'] }}"
|
||||||
|
podman_data_dir: "{{ ansible_facts['user_dir'] }}/podman-data"
|
||||||
|
quadlet_dir: "{{ ansible_facts['user_dir'] }}/.config/containers/systemd"
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Yamtrack
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
host: track.tanshu.com
|
||||||
|
|
||||||
|
container_name: yamtrack
|
||||||
|
unit_name: "{{ container_name }}.service"
|
||||||
|
|
||||||
|
# Fully qualified on purpose: rootless Podman resolves short names against
|
||||||
|
# unqualified-search-registries, and there is no TTY to answer a prompt when
|
||||||
|
# the systemd unit starts. ghcr.io was already qualified; the tag is explicit
|
||||||
|
# for the same reason. Nothing is pinned — see readme.md, "Notes and gotchas".
|
||||||
|
image: ghcr.io/fuzzygrim/yamtrack:latest
|
||||||
|
|
||||||
|
# The Quadlet file is <network_name>.network. caddy/roles/register pins the
|
||||||
|
# Podman network name to <network_name>_net (so: yamtrack.network, yamtrack_net,
|
||||||
|
# which is the name the Docker-era deployment used).
|
||||||
|
network_name: "{{ container_name }}"
|
||||||
|
|
||||||
|
host_directory: "{{ podman_data_dir }}/{{ container_name }}"
|
||||||
|
|
||||||
|
# The image's bind mount: /yamtrack/db holds db.sqlite3 and its WAL sidecars.
|
||||||
|
# Nothing else on /yamtrack is state — staticfiles/ is baked into the image and
|
||||||
|
# the code is read-only — so this is the only directory worth keeping.
|
||||||
|
data_dir: "{{ host_directory }}/data"
|
||||||
|
|
||||||
|
# The port inside the container that Caddy proxies to: the image's own nginx
|
||||||
|
# listens there and forwards to gunicorn. Reproduces the Docker-era Caddyfile
|
||||||
|
# entry (`reverse_proxy yamtrack:8000`) exactly.
|
||||||
|
container_port: 8000
|
||||||
|
|
||||||
|
# SECRET, TZ and REDIS_URL live here rather than as Environment= lines in the
|
||||||
|
# unit file: that file is world-readable in ~/.config. Read by systemd through
|
||||||
|
# EnvironmentFile=, not by Ansible, so it has to be written before the unit
|
||||||
|
# starts.
|
||||||
|
env_file: "{{ host_directory }}/{{ container_name }}.env"
|
||||||
|
|
||||||
|
# The uid this unit maps onto podman_user, and the PUID/PGID the image's
|
||||||
|
# entrypoint is given. It is 0 — container root — and that is deliberate:
|
||||||
|
#
|
||||||
|
# 1. `--userns=keep-id` does two things: it maps one container uid onto the
|
||||||
|
# connecting user, and it runs the container *as that uid*. The entrypoint's
|
||||||
|
# first act is `groupmod`/`usermod`, which rewrite /etc/group and
|
||||||
|
# /etc/passwd — files owned by container-root in the image. With
|
||||||
|
# `keep-id:uid=1000` those run as `abc` and die on "cannot lock /etc/group"
|
||||||
|
# (the unit exits 10) before supervisord is ever reached. Root is required.
|
||||||
|
# 2. Mapping container root onto podman_user is what keeps
|
||||||
|
# ~/podman-data/yamtrack owned by podman_user rather than by a subuid. It is
|
||||||
|
# the same choice caddy, bitwarden and tandoor make.
|
||||||
|
#
|
||||||
|
# It is also rendered as PUID/PGID in the environment file: the entrypoint's own
|
||||||
|
# default is 1000, which under this mapping is a subuid on the host, so its
|
||||||
|
# `chown -R abc:abc db` would strand the database on an unmapped uid. See
|
||||||
|
# readme.md, "Who runs as whom".
|
||||||
|
app_uid: 0
|
||||||
|
app_gid: 0
|
||||||
|
|
||||||
|
yamtrack_tz: "Asia/Kolkata"
|
||||||
|
|
||||||
|
# The Django SECRET_KEY, as the image calls it (`SECRET`). Carried over verbatim
|
||||||
|
# from the Docker-era playbook so existing sessions and password-reset tokens
|
||||||
|
# stay valid — rotating it logs every user out. Committed in plaintext as
|
||||||
|
# before; flagged as a follow-up, not changed here.
|
||||||
|
yamtrack_secret: "c14c5970eb728fbf8b2325fd3004069d"
|
||||||
|
|
||||||
|
# What the old container passed as URLS, which Yamtrack turns into its CSRF
|
||||||
|
# trusted origins. ALLOWED_HOSTS is deliberately not set: its default is `*`,
|
||||||
|
# which is what the Docker-era container relied on.
|
||||||
|
urls: "https://{{ host }}"
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Valkey (the shared cache / Celery broker)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Yamtrack hard-requires Redis: its cache backend is django_redis with no
|
||||||
|
# fallback, the Celery worker and beat need a broker, and several metadata
|
||||||
|
# providers talk to Redis directly. It is a *separate* playbook, deployed before
|
||||||
|
# this one: yamtrack/roles/valkey only attaches its network to that shared unit.
|
||||||
|
#
|
||||||
|
# The Docker-era container was given `redis://valkey:6379`, and valkey is what
|
||||||
|
# the container is called, so the URL is unchanged.
|
||||||
|
valkey_container: valkey
|
||||||
|
valkey_unit: "{{ valkey_container }}.service"
|
||||||
|
redis_url: "redis://{{ valkey_container }}:6379"
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Caddy
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# The shared Caddy unit and its Caddyfile, both owned by caddy/playbook.yml.
|
||||||
|
caddy_name: caddy
|
||||||
|
caddyfile_path: "{{ podman_data_dir }}/{{ caddy_name }}/conf/Caddyfile"
|
||||||
|
|
||||||
|
snippet_block: |
|
||||||
|
{{ host }} {
|
||||||
|
reverse_proxy {{ container_name }}:{{ container_port }}
|
||||||
|
}
|
||||||
Reference in new issue
Block a user