Add the Yamtrack playbook
This commit is contained in:
1 parent
26a0654fea
commit
af2e7394b8
11 files changed
+1023
No files matched your search
@@ -0,0 +1,518 @@
|
||||
# Yamtrack (rootless Podman + Quadlet)
|
||||
|
||||
Runs [Yamtrack](https://github.com/FuzzyGrim/Yamtrack) on `emily` as a
|
||||
**rootless Podman** container managed by a **systemd user unit** (Quadlet),
|
||||
behind the shared [rootless Caddy](../caddy/readme.md), with its cache and Celery
|
||||
broker in the shared [rootless Valkey](../valkey/readme.md). Replaces the
|
||||
`docker_container` deployment on `monoco`.
|
||||
|
||||
The database stays **SQLite**, inside the app's own data directory
|
||||
(`~/podman-data/yamtrack/data/db.sqlite3`). That is deliberate, not an
|
||||
oversight — see [The database stays SQLite](#the-database-stays-sqlite).
|
||||
|
||||
Everything else about the old container is reproduced rather than redesigned:
|
||||
the same image tag, the same four environment variables, the same upstream
|
||||
(`yamtrack:8000`), the same network name (`yamtrack_net`) and the same
|
||||
`SECRET`.
|
||||
|
||||
## Ordering
|
||||
|
||||
Both shared services are already migrated and running on `emily`:
|
||||
|
||||
1. [caddy/playbook.yml](../caddy/playbook.yml) — the reverse proxy, which owns
|
||||
the Caddyfile this site appends to, supplies the `caddy.service` handlers, and
|
||||
is the one play that applies the rootless-Podman prerequisites.
|
||||
2. [valkey/playbook.yml](../valkey/playbook.yml) — the shared cache and Celery
|
||||
broker. **Yamtrack hard-requires Redis**: its cache backend is `django_redis`
|
||||
with no fallback, Celery worker and beat need a broker, and several metadata
|
||||
providers talk to Redis directly. The old container was given
|
||||
`redis://valkey:6379` and that is exactly what this playbook sets.
|
||||
3. This playbook — **run it after the file copy below**, not before: the
|
||||
entrypoint migrates the schema on first start, so an empty
|
||||
`~/podman-data/yamtrack/data` must not be what the app wakes up to.
|
||||
|
||||
[postgres/playbook.yml](../postgres/playbook.yml) is **not** part of this
|
||||
deployment: there is no `yamtrack` database in the restored cluster, because the
|
||||
app never used Postgres.
|
||||
|
||||
This playbook attaches `yamtrack_net` to Valkey. Until it has, `valkey` does not
|
||||
resolve on that network, so running this playbook is what makes the app
|
||||
reachable — there is no separate step.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
yamtrack/
|
||||
├── playbook.yml
|
||||
├── vars/default.yml
|
||||
└── roles/
|
||||
├── valkey/ # attaches this app's network to the shared cache (+ its handler)
|
||||
├── yamtrack/ # data dir, the 0600 env file, the Quadlet unit, start + verify
|
||||
└── caddy/ # appends this site's block to the shared Caddyfile
|
||||
```
|
||||
|
||||
Quadlet units and state:
|
||||
|
||||
| On the host | What |
|
||||
|---|---|
|
||||
| `~/.config/containers/systemd/yamtrack.container` | the generated unit (Ansible-templated) |
|
||||
| `~/.config/containers/systemd/yamtrack.network` | the app network, written by `caddy/roles/register` |
|
||||
| `~/.config/containers/systemd/caddy.container.d/50-yamtrack.conf` | attaches Caddy to that network |
|
||||
| `~/.config/containers/systemd/valkey.container.d/50-yamtrack.conf` | attaches Valkey to it |
|
||||
| `~/.config/containers/systemd/valkey.container.d/10-network-aliases.conf` | the shared `redis` alias |
|
||||
| `~/podman-data/yamtrack/data/` | the image's `/yamtrack/db`: `db.sqlite3` (+ `-wal`/`-shm`) |
|
||||
| `~/podman-data/yamtrack/yamtrack.env` | `SECRET`, `TZ`, `URLS`, `REDIS_URL`, `PUID`/`PGID`, mode `0600`, read by systemd |
|
||||
|
||||
Two Docker-era pieces are **deleted rather than ported**: `roles/network`, which
|
||||
existed to attach an already-created container to a network
|
||||
(`docker_network`'s `connected:`/`appends:`, which `podman_network` has no
|
||||
equivalent for — [docs/podman-compat-audit.md §3.3](../docs/podman-compat-audit.md)),
|
||||
and `files/nginx.conf.j2`, which was a copy-pasted *Vaultwarden* nginx template
|
||||
that this playbook never used. Nothing has to be connected after the fact now:
|
||||
the app's own unit names its `Network=`, and Caddy and Valkey learn about the
|
||||
network from drop-ins written in the same run.
|
||||
|
||||
## How it registers with Caddy
|
||||
|
||||
Caddy is the base unit and knows nothing about its upstreams. This playbook calls
|
||||
the shared include in
|
||||
[caddy/roles/register](../caddy/roles/register/tasks/main.yaml), which writes the
|
||||
network unit **and** the drop-in that pulls it into Caddy:
|
||||
|
||||
```yaml
|
||||
- name: Register Yamtrack's network with Caddy
|
||||
ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/register/tasks/main.yaml"
|
||||
```
|
||||
|
||||
It runs in `pre_tasks`, **before** the app unit is started, because
|
||||
`yamtrack.container` references `yamtrack.network`: Quadlet fails *unit
|
||||
generation* — so `systemctl --user start` reports "unit not found" — when a
|
||||
referenced `.network` file does not exist yet. The Valkey attach below depends on
|
||||
it for the same reason.
|
||||
|
||||
`network_name` is `yamtrack`, so the *file* is `yamtrack.network` and the *Podman
|
||||
network* is `yamtrack_net` (`NetworkName=` in the network unit pins it, keeping
|
||||
the Docker-era name).
|
||||
|
||||
The site block itself is appended to the shared Caddyfile by
|
||||
[roles/caddy](roles/caddy/tasks/main.yaml) with `blockinfile` — the same bytes
|
||||
the Docker-era Caddyfile on `monoco` already carries for this host:
|
||||
|
||||
```
|
||||
track.tanshu.com {
|
||||
reverse_proxy yamtrack:8000
|
||||
}
|
||||
```
|
||||
|
||||
`8000` is the port the image's own nginx listens on inside the container (it
|
||||
proxies to gunicorn); the image `EXPOSE`s exactly that, and
|
||||
`vars/default.yml`'s `container_port` is where it is written down. The old
|
||||
`roles/caddy` slurped and compared the Caddyfile before editing it, which
|
||||
*fails* on a host where Caddy has not run yet; `blockinfile` is idempotent on
|
||||
its own marker, so that whole dance is gone.
|
||||
|
||||
## How it attaches to Valkey
|
||||
|
||||
Same idea at the other end: Valkey knows nothing about its clients, and this
|
||||
playbook uses the shared include in
|
||||
[valkey/roles/attach](../valkey/roles/attach/tasks/main.yaml) rather than writing
|
||||
those drop-ins itself. That writes two files, both mergeable into
|
||||
`valkey.container` by Quadlet:
|
||||
|
||||
| File | Content |
|
||||
|---|---|
|
||||
| `valkey.container.d/50-yamtrack.conf` | `Network=yamtrack.network` |
|
||||
| `valkey.container.d/10-network-aliases.conf` | `NetworkAlias=redis` |
|
||||
|
||||
`REDIS_URL` uses the host `valkey`, which is `ContainerName=valkey` and is what
|
||||
the Docker-era container was given. The `redis` alias in the shared file is for
|
||||
the *other* cache clients (Nextcloud writes `REDIS_HOST=redis` into its
|
||||
`config.php`), not for this app; the play checks that `valkey` resolves.
|
||||
|
||||
[roles/valkey](roles/valkey/tasks/main.yaml) wraps that include for two reasons:
|
||||
|
||||
- it defines the `Restart Valkey` handler the include notifies; and
|
||||
- it **flushes handlers** immediately afterwards. On a first attach the drop-in
|
||||
is new, so Valkey has to be *recreated* to join `yamtrack_net` — and that
|
||||
restart is a handler, which by default runs at the end of the play, after the
|
||||
`yamtrack` role has already started the app. The app would then come up before
|
||||
`valkey` resolves, fail its own health check, and fail the play on a machine
|
||||
where nothing is wrong. Flushing applies the drop-in first; on later runs
|
||||
nothing is pending and it is a no-op.
|
||||
|
||||
Unlike Postgres, the cost here is **not** downtime for a database: Valkey is a
|
||||
cache, so every attached app simply starts with a cold cache and the first
|
||||
requests after the restart are slower. See
|
||||
[valkey/readme.md](../valkey/readme.md), "The trade-off". Attached means *every*
|
||||
cache client, not just Yamtrack.
|
||||
|
||||
## Who runs as whom
|
||||
|
||||
`UserNS=keep-id:uid=0,gid=0` — container root is mapped onto `podman_user`, the
|
||||
same choice Caddy, Vaultwarden and Tandoor make. That is what keeps
|
||||
`~/podman-data/yamtrack` owned by `podman_user` rather than by a subuid, so
|
||||
plain `tar`, `cp` and `rm -rf` work there without `podman unshare` or `sudo`,
|
||||
and extracting the old data as `podman_user` is already correct.
|
||||
|
||||
**uid 0 is required here, not a convenience.** `--userns=keep-id` does two
|
||||
things: it maps one container uid onto the connecting user, *and it runs the
|
||||
container as that uid*. Yamtrack's entrypoint begins by rewriting `/etc/group`
|
||||
and `/etc/passwd` (`groupmod`/`usermod`, applying `PUID`/`PGID`) — files owned by
|
||||
container-root in the image. With `keep-id:uid=1000` the entrypoint runs as
|
||||
`abc`, cannot lock `/etc/group`, and the unit dies with:
|
||||
|
||||
```
|
||||
groupmod: /etc/group.5: Permission denied
|
||||
groupmod: cannot lock /etc/group; try again later.
|
||||
yamtrack.service: Main process exited, code=exited, status=10/n/a
|
||||
```
|
||||
|
||||
before supervisord is ever reached. `manage.py migrate` has *already* run and
|
||||
succeeded at that point, because the bind mount is writable as `abc` — which
|
||||
makes the failure look like a database problem when it is not. Docker on
|
||||
`monoco` ran the same entrypoint as real root, which is why it never surfaced
|
||||
there.
|
||||
|
||||
`PUID`/`PGID` are therefore pinned to the same `0` in the environment file. The
|
||||
entrypoint's own default is `1000`, and under this mapping container uid `1000`
|
||||
is a subuid on the host, so its `chown -R abc:abc db` would leave the database
|
||||
owned by e.g. `165536` — unreadable by a plain `tar` backup. With both pinned to
|
||||
`0`, `abc` resolves to container root, which `keep-id` maps onto `podman_user`.
|
||||
The entrypoint still chowns `db` on every start, so the copy only has to be
|
||||
*readable* by `podman_user`.
|
||||
|
||||
The one visible side effect is this warning from Celery, once per start, in
|
||||
`podman logs yamtrack`:
|
||||
|
||||
```
|
||||
SecurityWarning: You're running the worker with superuser privileges
|
||||
```
|
||||
|
||||
That is Celery spotting uid 0 *inside the container*, where uid 0 is an
|
||||
unprivileged user on the host. Expected, and harmless in a rootless unit.
|
||||
|
||||
Like Caddy, `podman_user` is derived from the connection
|
||||
(`ansible_facts['user_id']`) and the play carries no `become`. Because the podman
|
||||
tasks are not run here, there is **no guard against connecting as `root`**; the
|
||||
prerequisites, including that check, live in `caddy/playbook.yml`.
|
||||
|
||||
## The database stays SQLite
|
||||
|
||||
Yamtrack supports Postgres (`DB_HOST`, `DB_NAME`, `DB_USER`, `DB_PASSWORD`,
|
||||
`DB_PORT`), and this repo now has a shared Postgres unit. It is still not used
|
||||
here, on purpose:
|
||||
|
||||
- **The old data is SQLite.** 2.3 MB in one `db.sqlite3` file, owned by uid
|
||||
`1000` ([docs/backup-plan.md §2.1](../docs/backup-plan.md)). Moving it to
|
||||
Postgres is a *data conversion* — `dumpdata`/`loaddata` or a
|
||||
`sqlite`→`postgres` copy — with its own failure modes, and it buys nothing
|
||||
this deployment needs.
|
||||
- **There is no `yamtrack` database in the restored cluster.** The twelve
|
||||
databases brought across in
|
||||
[postgres/migrate-from-monoco.md](../postgres/migrate-from-monoco.md) do not
|
||||
include one, which is the same fact seen from the other side.
|
||||
- **A single file is not a worse fit here than a shared server.** The app is a
|
||||
one-user media tracker; the whole database is smaller than a single query
|
||||
result in some of the other services.
|
||||
|
||||
The consequence to be deliberate about: **the entrypoint runs
|
||||
`manage.py migrate` on every start**, so an image bump with `latest` can migrate
|
||||
the schema, and that migration is not reversible by rolling the image back. The
|
||||
copy in [Migrating the file data](#migrating-the-file-data-from-monoco) is the
|
||||
rollback, which is one more reason to take it before the first start and to keep
|
||||
`monoco` intact until this deployment has been lived with.
|
||||
|
||||
Moving to Postgres later is a normal, separate change: point the app at it,
|
||||
convert the data, and add a `postgres` role here exactly like Nextcloud's or
|
||||
Tandoor's (`postgres/roles/attach` plus a handler and a flush).
|
||||
|
||||
## Settings
|
||||
|
||||
| Variable | Value / purpose |
|
||||
|---|---|
|
||||
| `podman_user` | **Derived** from the connection (`ansible_facts['user_id']`), not set by hand |
|
||||
| `podman_data_dir` | `~/podman-data` |
|
||||
| `quadlet_dir` | `~/.config/containers/systemd` |
|
||||
| `host` | `track.tanshu.com` — the site block Caddy serves |
|
||||
| `container_name` / `unit_name` | `yamtrack`, and the Quadlet unit / container name |
|
||||
| `image` | `ghcr.io/fuzzygrim/yamtrack:latest` — fully qualified on purpose |
|
||||
| `network_name` | `yamtrack` → `yamtrack.network` / `yamtrack_net` |
|
||||
| `host_directory` | `~/podman-data/yamtrack` |
|
||||
| `data_dir` | `~/podman-data/yamtrack/data` → the image's `/yamtrack/db` |
|
||||
| `container_port` | `8000`, the port inside the container that Caddy proxies to |
|
||||
| `env_file` | `~/podman-data/yamtrack/yamtrack.env`, mode `0600` |
|
||||
| `app_uid` / `app_gid` | `0` — container root, mapped by `keep-id` onto `podman_user`, and the entrypoint's `PUID`/`PGID` |
|
||||
| `yamtrack_tz` | `Asia/Kolkata`, the `TZ` the old container was given |
|
||||
| `yamtrack_secret` | The Django `SECRET`, carried over verbatim |
|
||||
| `urls` | `https://track.tanshu.com`, the old container's `URLS` (CSRF origins) |
|
||||
| `valkey_container` / `valkey_unit` | The shared cache unit this app attaches to |
|
||||
| `redis_url` | `redis://valkey:6379`, unchanged from the Docker-era container |
|
||||
| `caddy_name` / `caddyfile_path` | The shared Caddy unit and its Caddyfile |
|
||||
|
||||
The three `podman_*` / `quadlet_dir` vars are **duplicated** from
|
||||
[caddy/vars/default.yml](../caddy/vars/default.yml), as in every service
|
||||
playbook. That is a deliberate trade: a play cannot include the Caddy vars file
|
||||
wholesale, because it also sets `host_directory` (to the *Caddy* data directory)
|
||||
and every service defines that for itself. Keep them in sync. Caddy's fourth,
|
||||
`podman_unprivileged_port_start`, belongs to the podman tasks this play does not
|
||||
run — Yamtrack publishes nothing host-side.
|
||||
|
||||
`SECRET`, `TZ` and `REDIS_URL` are written to `yamtrack.env` at mode `0600` and
|
||||
read by systemd via `EnvironmentFile=`, not by Ansible. `SECRET` is the same
|
||||
value `monoco` uses, so existing sessions and password-reset tokens keep working
|
||||
— rotating it logs every user out. It is committed in plaintext as before;
|
||||
flagged as a follow-up, not changed here.
|
||||
|
||||
`ALLOWED_HOSTS` is deliberately absent: Yamtrack's default is `*`, which is what
|
||||
the Docker-era container relied on. Setting `URLS` — as the old playbook did — is
|
||||
what fills in the CSRF trusted origins.
|
||||
|
||||
## Running
|
||||
|
||||
```command
|
||||
ansible-playbook -u tanshu yamtrack/playbook.yml
|
||||
```
|
||||
|
||||
The play targets `emily` directly, so the inventory must contain it — the one in
|
||||
`prepare-server/` does, and its `ansible.cfg` names it.
|
||||
|
||||
This play applies **only** Yamtrack's own units and data. Its `pre_tasks` run
|
||||
Caddy's `register` include and nothing else. The rootless-Podman prerequisites —
|
||||
installing podman, `loginctl enable-linger`, the low-port sysctl and the Quadlet
|
||||
directory — are the shared task file at
|
||||
[caddy/roles/podman/tasks/main.yaml](../caddy/roles/podman/tasks/main.yaml), and
|
||||
are **not** run here. A host that has already run `caddy/playbook.yml` has them.
|
||||
Neither is Valkey deployed here: run `valkey/playbook.yml` first, or the flush in
|
||||
`roles/valkey` fails on a `valkey.service` that does not exist.
|
||||
|
||||
`systemctl --user enable yamtrack` is neither needed nor possible: the unit is
|
||||
*generated*, and Quadlet applies `[Install] WantedBy=default.target` itself.
|
||||
|
||||
## Migrating the file data from monoco
|
||||
|
||||
There is no database to restore from Postgres for this app: the whole of its
|
||||
state is `db.sqlite3`, and the playbook does not move it because the file is on
|
||||
the old host while the playbook runs on the new one. Run this from the machine
|
||||
that holds the `monoco` and `emily` SSH aliases.
|
||||
|
||||
**Copy before deploying.** The entrypoint migrates the schema on first start, so
|
||||
a first run against an empty directory creates a new, empty database and the
|
||||
copy would then be overwriting a live one.
|
||||
|
||||
If a first run has *already* happened — the unit crash-looped, or a deploy was
|
||||
tried before the copy — that directory now holds a fresh database with every
|
||||
migration applied and no rows. Delete it before copying rather than copying on
|
||||
top of it, so no `-wal`/`-shm` from the empty database can mix with the real
|
||||
one:
|
||||
|
||||
```bash
|
||||
# Prove it is the empty one before removing it: 0 users, 0 items, 0 episodes.
|
||||
ssh emily 'podman exec -i yamtrack python3 -' <<'PY'
|
||||
import sqlite3
|
||||
c = sqlite3.connect("/yamtrack/db/db.sqlite3")
|
||||
for t in ("users_user", "app_item", "app_episode"):
|
||||
print(t, c.execute("select count(*) from " + t).fetchone()[0])
|
||||
PY
|
||||
|
||||
ssh emily 'systemctl --user stop yamtrack.service; rm -rf ~/podman-data/yamtrack/data'
|
||||
```
|
||||
|
||||
```bash
|
||||
# 1. Look before you copy: size, contents, and whether it is readable without
|
||||
# sudo. `ls -ln` shows numeric owners — the old data is uid 1000, which is
|
||||
# `ubuntu` on monoco and the container's `abc` inside it.
|
||||
ssh monoco 'du -sh /var/lib/yamtrack/data; ls -ln /var/lib/yamtrack/data'
|
||||
ssh monoco 'find /var/lib/yamtrack/data -type f ! -readable' # empty = no sudo needed
|
||||
|
||||
# 2. Stop the old container so nothing writes while you copy. Leave it stopped:
|
||||
# two live copies of one SQLite database would diverge.
|
||||
#
|
||||
# This is also what makes the copy *consistent*. SQLite runs in WAL mode here,
|
||||
# and a plain tar of a live database can produce a file that will not open —
|
||||
# the main file and the WAL are copied at different instants
|
||||
# (docs/backup-plan.md §3.1). A clean stop checkpoints the WAL into
|
||||
# db.sqlite3 and removes the sidecars, so the copy is a single frozen file.
|
||||
ssh monoco 'docker stop yamtrack'
|
||||
|
||||
# 3. Create the destination.
|
||||
ssh emily 'mkdir -p ~/podman-data/yamtrack/data'
|
||||
|
||||
# 4. Copy. tar streams it across the workstation; rsync cannot copy
|
||||
# remote-to-remote in one command.
|
||||
#
|
||||
# The -wal/-shm sidecars are copied rather than filtered out on purpose: if
|
||||
# anything is still in the WAL, it is part of the database. After a clean stop
|
||||
# they are usually already gone, and copying "nothing" costs nothing.
|
||||
#
|
||||
# `set -o pipefail` matters here: without it the pipeline reports only the
|
||||
# *receiving* tar's status, so a read error or a dropped stream on the source
|
||||
# side exits 0 and leaves a silently short copy.
|
||||
set -o pipefail
|
||||
ssh monoco 'tar -C /var/lib/yamtrack/data -cf - .' \
|
||||
| ssh emily 'tar --no-same-owner -C ~/podman-data/yamtrack/data -xf -'
|
||||
|
||||
# If step 1 listed unreadable files, read them as root instead:
|
||||
# ssh monoco 'sudo tar -C /var/lib/yamtrack/data -cf - .' \
|
||||
# | ssh emily 'tar --no-same-owner -C ~/podman-data/yamtrack/data -xf -'
|
||||
|
||||
# 5. Verify by content. Both sides are frozen by now (the old container is
|
||||
# stopped and nothing writes on emily yet), so identical digests mean a
|
||||
# byte-faithful copy.
|
||||
ssh monoco 'cd /var/lib/yamtrack/data && find . -type f -print0 | sort -z | xargs -0 -r sha256sum | sha256sum'
|
||||
ssh emily 'cd ~/podman-data/yamtrack/data && find . -type f -print0 | sort -z | xargs -0 -r sha256sum | sha256sum'
|
||||
|
||||
# 6. Nothing may end up owned by root or by a subuid.
|
||||
ssh emily 'find ~/podman-data/yamtrack ! -user tanshu' # empty = correct
|
||||
|
||||
# 7. Deploy.
|
||||
cd prepare-server && ansible-playbook -u tanshu ../yamtrack/playbook.yml
|
||||
```
|
||||
|
||||
Ownership needs no `chown` pass: the app runs as uid `1000` in-container, which
|
||||
`keep-id` maps to `tanshu`, so files extracted as `tanshu` are already right, and
|
||||
the entrypoint's own `chown abc:abc db` runs on every start anyway. That is also
|
||||
why `--no-same-owner` is explicit on the receiving side — the source is owned by
|
||||
uid `1000`, and restoring that ownership would be wrong here.
|
||||
|
||||
If the link is slow or drops, do it in two hops instead: resumable, and it shows
|
||||
progress.
|
||||
|
||||
```bash
|
||||
rsync -aP monoco:/var/lib/yamtrack/data/ ~/yamtrack-copy/
|
||||
rsync -aP ~/yamtrack-copy/ emily:~/podman-data/yamtrack/data/
|
||||
```
|
||||
|
||||
If stopping the app is unacceptable — for instance while rehearsing this
|
||||
deployment before the DNS move — ask SQLite for a consistent snapshot instead of
|
||||
copying the files while they are hot:
|
||||
|
||||
```bash
|
||||
ssh monoco 'docker exec -i yamtrack python3 -' <<'PY'
|
||||
import sqlite3
|
||||
sqlite3.connect("/yamtrack/db/db.sqlite3").execute(
|
||||
"VACUUM INTO '/yamtrack/db/pre-migration.sqlite3'"
|
||||
)
|
||||
print("wrote /yamtrack/db/pre-migration.sqlite3")
|
||||
PY
|
||||
|
||||
# then copy it as the database, and delete the snapshot on monoco afterwards
|
||||
ssh monoco 'cat /var/lib/yamtrack/data/pre-migration.sqlite3' \
|
||||
| ssh emily 'cat > ~/podman-data/yamtrack/data/db.sqlite3'
|
||||
```
|
||||
|
||||
`VACUUM INTO` needs SQLite 3.27+; the image's Python 3.12 has far newer. This
|
||||
still has to be followed by stopping the container before DNS moves: it shortens
|
||||
the window, it does not remove the cutover.
|
||||
|
||||
Rolling back means rolling back both halves: `ssh monoco 'docker start yamtrack'`
|
||||
is only meaningful while `monoco`'s data and reverse proxy are intact, and the
|
||||
DNS record has to point back. `monoco` was never modified — its container was
|
||||
only stopped — so the rollback is starting it again.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
systemctl --user status yamtrack
|
||||
journalctl --user -u yamtrack -f
|
||||
podman logs yamtrack
|
||||
|
||||
# `podman ps` shows (healthy) once the image's own probe passes.
|
||||
podman ps --filter name=yamtrack
|
||||
podman healthcheck run yamtrack
|
||||
|
||||
# When it does not pass, the health view names the failing check — cache,
|
||||
# database, Redis or Celery — instead of just "unhealthy":
|
||||
podman exec yamtrack wget -qO- http://127.0.0.1:8000/health/
|
||||
|
||||
# What the playbook checks for you:
|
||||
podman exec yamtrack getent hosts valkey
|
||||
|
||||
# The file really is the migrated database, untouched by a first-start migrate:
|
||||
podman exec yamtrack python3 -c "import sqlite3; print(sqlite3.connect('/yamtrack/db/db.sqlite3').execute('pragma integrity_check').fetchone()[0])"
|
||||
|
||||
# Row counts compared with monoco — equal numbers mean the library came across.
|
||||
# (1517 items and 1312 episodes when this was written. Use `docker exec` on
|
||||
# monoco, `podman exec` on emily.)
|
||||
ssh monoco 'docker exec yamtrack python3 -c "import sqlite3; print(sqlite3.connect(\"/yamtrack/db/db.sqlite3\").execute(\"select count(*) from app_item\").fetchone()[0])"'
|
||||
ssh emily 'podman exec yamtrack python3 -c "import sqlite3; print(sqlite3.connect(\"/yamtrack/db/db.sqlite3\").execute(\"select count(*) from app_item\").fetchone()[0])"'
|
||||
|
||||
# The site, end to end:
|
||||
curl -sSI https://track.tanshu.com
|
||||
```
|
||||
|
||||
Then log in **in a browser** and confirm the library, an episode's history and
|
||||
one metadata lookup feel right — the health check proves the process, not the
|
||||
data. Back it up with a **write**: mark an episode watched, then confirm on the
|
||||
host that `db.sqlite3` changed and is owned by `tanshu`:
|
||||
|
||||
```bash
|
||||
ssh emily 'ls -ln ~/podman-data/yamtrack/data'
|
||||
```
|
||||
|
||||
A write landing as root or as a `165536+` subuid is the `keep-id` check, and a
|
||||
read-only smoke test will not catch it.
|
||||
|
||||
Then reboot the host and confirm the unit returns unattended — without
|
||||
`loginctl enable-linger tanshu` it will not, and the failure is silent.
|
||||
|
||||
## Notes and gotchas
|
||||
|
||||
- **`SECRET` is committed in plaintext**, as it was before. It is the same value
|
||||
`monoco` used, so sessions survive the move; changing it is a separate,
|
||||
user-visible decision (everyone is logged out, pending password-reset links
|
||||
break).
|
||||
- **`latest` floats, and the entrypoint migrates.** Neither this playbook nor the
|
||||
old one pins a digest or a version, so an image bump can run new Django
|
||||
migrations against the SQLite file on first start. Rolling the image back does
|
||||
not roll the schema back; the copy taken in
|
||||
[Migrating the file data](#migrating-the-file-data-from-monoco) is the
|
||||
rollback. This is not hypothetical: `monoco` is running an image pulled in
|
||||
April (`v0.25.2`, 200 applied migrations) while `emily` pulled a newer
|
||||
`latest`, so the copied database gets 12 migrations on its first start under
|
||||
the new unit.
|
||||
- **The entrypoint needs root, so `keep-id`'s uid matters.** See
|
||||
[Who runs as whom](#who-runs-as-whom) — `keep-id:uid=1000` produces a crash
|
||||
loop whose log looks like a database or permission problem and is neither.
|
||||
After that failure the data directory contains a **fresh, empty** database
|
||||
(0 users, 0 items) with every migration applied, which will happily serve an
|
||||
empty library if the copy has not landed. Check
|
||||
`select count(*) from app_item` before pointing DNS anywhere.
|
||||
- **Yamtrack requires Redis.** There is no "cache disabled" mode: the cache
|
||||
backend is `django_redis` and Celery needs the broker. If `valkey` is down, the
|
||||
site does not degrade — it fails.
|
||||
- **`staticfiles/` is not on the host.** Unlike Tandoor, this image bakes its
|
||||
collected static files in, so `~/podman-data/yamtrack/data` is the whole of
|
||||
what has to be backed up. The old container's `/yamtrack` showed
|
||||
`staticfiles/` owned by `abc`; that is image content, not state.
|
||||
- **Client IPs.** Yamtrack's nginx sits behind Caddy, and whether the real client
|
||||
address survives the rootless port forward depends on the forwarder. See
|
||||
[caddy/readme.md](../caddy/readme.md), "Client IPs", before relying on it.
|
||||
- **Registering restarts all of Caddy**, and attaching restarts Valkey for every
|
||||
cache client — a brief restart of every hosted site, and a cold cache
|
||||
everywhere. See the trade-offs in [caddy/readme.md](../caddy/readme.md) and
|
||||
[valkey/readme.md](../valkey/readme.md).
|
||||
- **If the Caddyfile does not exist yet**, this playbook creates it containing
|
||||
only the site block. That is a valid Caddyfile; the Caddy playbook's seeded
|
||||
global-options block is then not added, because its seed only fires on a
|
||||
missing or zero-length file. Deploy Caddy first on a fresh host.
|
||||
- **DNS.** `track.tanshu.com` currently points at `monoco` (`13.202.6.2`) and
|
||||
must be repointed at `emily` (`35.154.14.59`) before Caddy can obtain a
|
||||
certificate for it. Until then the site answers from the old host.
|
||||
- **Do not restart the container on `monoco`** after the cutover. It would keep
|
||||
writing to the old database, and two live copies behind one hostname diverge
|
||||
silently.
|
||||
- **There is no `yamtrack-backup.sh`.** `~/bin` on `monoco` has scripts for
|
||||
Nextcloud, Gitea, Bitwarden, Tandoor and others, but none for this app, and
|
||||
root's crontab has only the Nextcloud entry — so nothing has ever been
|
||||
scheduled ([docs/backup-plan.md §11](../docs/backup-plan.md)). Nothing has to
|
||||
be ported here, but the 2.3 MB file that *is* this service has no backup
|
||||
either: a `systemd --user` timer doing `podman exec yamtrack python3 -c "…"
|
||||
"VACUUM INTO …"` next to the Quadlet units is the shape that fits this host,
|
||||
because root cannot `podman exec` into another user's rootless containers.
|
||||
|
||||
## Not done here
|
||||
|
||||
- Rotating `SECRET` and the other secrets that are committed in plaintext.
|
||||
- Moving the database to the shared Postgres unit (see
|
||||
[The database stays SQLite](#the-database-stays-sqlite)).
|
||||
- Replacing the missing backup with a systemd user timer (see above).
|
||||
Reference in new issue
Block a user