tanshu/playbooks
1
0 Fork 0
Code Issues Pull Requests Releases Wiki Activity

Run Gitea as a rootless Podman Quadlet, with git over SSH on the host's sshd

Browse Source 
Replaces the docker_container deployment on monoco with a Quadlet systemd user
unit on emily, and moves state from /var/lib/gitea to ~/podman-data/gitea. Gitea
is SQLite -- the live app.ini on monoco says DB_TYPE = sqlite3 with
PATH = /data/gitea/gitea.db -- so there is no postgres role and the database
travels inside the data tree.

git-over-SSH is the part that could not simply be moved. Port 22 stays the host's
sshd, so no client changes its remote, but the second hop does: the Docker-era
design sshed into an sshd *inside* the container, and under keep-id that sshd runs
as a subuid which cannot read the authorized_keys Gitea writes as tanshu. It is
replaced by a `podman exec`, reached through a shim sshd runs as a new `git` login
user, a one-command sudoers rule, and an AuthorizedKeysCommand that hands sshd the
key file without the ownership check StrictModes would otherwise apply.

Two things stopped it starting, and both only showed up when it was run:

keep-id runs the container *process* as the uid it maps, not just the file
ownership, so the whole container came up as `git` while the image's start-up is
written to begin as root. /usr/bin/entrypoint rewrites /etc/passwd,
/etc/s6/gitea/setup prepares /data before the run script `su-exec git`s the
server, and the image's own CMD is s6-svscan, which has to create
.s6-svscan/lock inside root-owned /etc/s6. The container died on the last two --
"sed: can't create temp file '/etc/passwdXXXXXX': Permission denied" and
"s6-svscan: fatal: unable to open .s6-svscan/lock: Permission denied" -- and
Restart=always turned that into a crash loop, which `is-active` reports as
`activating` rather than as a failure. User=0 restores root for the start-up and
leaves the mapping alone: the server still runs as git through su-exec, which is
what satisfies mustNotRunAsRoot. (nextcloud.container carries the same pair for
the same reason.)

The `ssh/` directory under /data is only ever a mount point -- the in-image
sshd's host keys are mounted over it from ~/podman-data/gitea-sshd -- so it is
hidden inside the container and nothing ever chowns it. When Podman has to create
it, it lands in the data tree owned by the subuid, and the role's own "nothing
here belongs to another user" check fails on it, correctly. Creating it in the
role first is what keeps the tree wholly tanshu's.

Start, is-active and the health check are now one block whose rescue dumps
status, journal and podman logs, because they fail the same way: `systemctl
start` reports success as soon as systemd holds the unit, so a container that
dies a second later surfaces only as `activating` -- which is exactly how the two
failures above were reported, with the reason in the journal and nowhere the play
looked.

Verified on emily: the play runs to ok=27/changed=7/failed=0, the health check
and the stray-owner assertion both pass, and the ssh second hop still returns the
container's Gitea 28.1.0 through the wrapper. The role's comments and the readme
were corrected where they explained the old failures as EPERM from the root
phases: container root holds the usual capabilities inside its user namespace, so
those phases succeed and leave subuid-owned files behind, which is the real reason
the tree is created up front. The readme's list of "expected" chmod EPERM lines is
gone, because the log is clean. The data directory was emptied of the fresh
app.ini afterwards and the unit left stopped, ready for the copy from monoco.
This commit is contained in:
tanshu committed 2026-10-16 16:44:14 +05:30
1 parent 3d4ccdd6d1
commit fc70acb66b
14 files changed
+1259 -105
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

gitea/playbook.yml
+50 -9
View File
@@ -1,16 +1,57 @@
#################################################
# DO Community Playbooks: Docker
#################################################
---
- hosts: monoco
become: true
# Gitea as a rootless Podman Quadlet unit on emily, behind the shared Caddy
# unit (caddy/playbook.yml), with git-over-SSH still arriving on the host's own
# port 22 so that no client has to change its remote.
#
# Connects as the container user and runs everything as that user: the Quadlet
# units belong to it, so the connection's PAM session is what makes
# `systemctl --user` work without any environment juggling.
#
# gather_facts is required: podman_user and the home directory come from the
# user_id / user_dir facts.
#
# The rootless-Podman prerequisites (installing podman, enabling lingering, the
# low-port sysctl, creating the Quadlet directory) are deliberately NOT applied
# here. They are the shared task file at caddy/roles/podman/tasks/main.yaml, and
# a host that has already run caddy/playbook.yml has them:
#
# - name: Rootless Podman prerequisites
# ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/podman/tasks/main.yaml"
#
# The variables this play needs are duplicated at the top of vars/default.yml, so
# it stands on its own. (Applying the commented include above would also need
# podman_unprivileged_port_start, which lives in caddy/vars/default.yml.)
#
# Unlike bitwarden/ and tandoor/ there is no postgres role here: this Gitea runs
# on SQLite, which monoco's live app.ini confirms (`DB_TYPE = sqlite3`,
# `PATH = /data/gitea/gitea.db`). The unused Docker-era Postgres variant that
# used to sit beside this file was removed for the same reason.
#
# Run with: ansible-playbook -u tanshu gitea/playbook.yml
- name: Deploy Gitea as a rootless Podman Quadlet
hosts: emily
gather_facts: true
vars_files:
- vars/default.yml
pre_tasks:
# This must happen before the app unit is started, not after: the generated
# unit references {{ network_name }}.network, and Quadlet fails *unit
# generation* (so `systemctl --user start` reports "unit not found") when a
# referenced .network file does not exist yet. The same include writes the
# Caddy-side drop-in that attaches Caddy to this network.
#
# network_name is a play var, so the include can see it directly.
- name: Register Gitea's network with Caddy
ansible.builtin.include_tasks: "{{ playbook_dir }}/../caddy/roles/register/tasks/main.yaml"
roles:
- user
- passthrough
- sshkey
- network
# The host half of git-over-SSH: the `git` login user, the passthrough shim,
# the authorized_keys hook, the sshd drop-in and the one-command sudoers
# rule. Root-only steps elevate individually — see the role for why.
- ssh
# The app: the data directory tree the image's own root phase would leave
# subuid-owned, the Quadlet unit, start + verify.
- gitea
# Appends this site's block to the shared Caddyfile and reloads Caddy.
- caddy
gitea/readme.md
+480 -35
View File
@@ -1,45 +1,490 @@
# Gitea on Ubuntu
# Gitea (rootless Podman + Quadlet)
This playbook will install Gitea on an Ubuntu 19.10 machine.
A number of containers will be created with the options specified in the `vars/default.yml` variable file.
Runs Gitea on `emily` as a **rootless Podman** container managed by a **systemd
user unit** (Quadlet), behind the shared [rootless Caddy](../caddy/readme.md),
with git-over-SSH still arriving on the **host's own port 22** so that no client
has to change its remote. Replaces the `docker_container` deployment on `monoco`.
Two things are worth knowing before anything else:
- **This Gitea is SQLite.** monoco's live `app.ini` says `DB_TYPE = sqlite3`,
`PATH = /data/gitea/gitea.db`, so there is no `postgres` role here and nothing
to restore into the shared cluster. The unused Docker-era Postgres variant that
used to sit next to the playbook has been deleted.
- **There is no sshd inside the container any more**, and the reason is not a
preference. [Why](#why-there-is-no-sshd-in-the-container) is the heart of this
readme.
## Ordering
1. [caddy/playbook.yml](../caddy/playbook.yml) — the reverse proxy, which owns
the Caddyfile this site appends to and is the one play that applies the
rootless-Podman prerequisites.
2. **The data copy below — before the first run of this playbook.** Gitea writes
into `~/podman-data/gitea` the moment it starts, and a first start on an empty
directory serves the *install wizard* instead of this repository's Gitea.
3. This playbook — **run it after the file copy**, not before.
`postgres/playbook.yml` is not part of this ordering. There is no `postgres`
role in this playbook and no database to attach.
## Layout
```
gitea/
├── playbook.yml
├── vars/default.yml
└── roles/
├── ssh/ # the host half of git-over-SSH: git user, shim, sudoers, sshd drop-in
├── gitea/ # the data tree the image cannot create, the Quadlet unit, start + verify
└── caddy/ # appends this site's block to the shared Caddyfile
```
| On the host | What |
|---|---|
| `~/.config/containers/systemd/gitea.container` | the generated unit (Ansible-templated) |
| `~/.config/containers/systemd/gitea.network` | the app network, written by `caddy/roles/register` |
| `~/.config/containers/systemd/caddy.container.d/50-gitea.conf` | attaches Caddy to that network |
| `~/podman-data/gitea/` | **everything Gitea owns**: `git/` (repositories, LFS), `gitea/` (`app.ini`, `gitea.db`, sessions, avatars, logs), `git/.ssh/authorized_keys` |
| `~/podman-data/gitea/ssh/` | **not Gitea's**: a bare *mount point*, always empty. The in-image sshd's host keys are mounted over it from `gitea-sshd` below, which hides it from the container — so nothing ever writes here |
| `~/podman-data/gitea-sshd/` | scratch space for the *image's* sshd, which this deployment does not use — see below |
| `/home/git/` | the `git` login user's home; **no `.ssh`** — see [The pieces on the host](#the-pieces-on-the-host) |
| `/usr/local/bin/gitea` | the passthrough shim sshd's forced command runs |
| `/usr/local/bin/gitea-container-exec` | `podman exec` wrapper, run as the podman user |
| `/usr/local/bin/gitea-authorized-keys` | what sshd calls to get the git user's keys |
| `/etc/sudoers.d/gitea-ssh-passthrough` | lets `git` run exactly that one wrapper |
| `/etc/ssh/sshd_config.d/60-gitea.conf` | `Match User git` → the keys hook |
## How git over SSH works now
### What the Docker deployment did — and why it cannot be repeated
```
client ──ssh git@git.tanshu.com:22──► host sshd (monoco)
│ authenticates against /home/git/.ssh/authorized_keys
│ each line is a forced command:
│ command="/usr/local/bin/gitea --config=/data/gitea/conf/app.ini serv key-5",…
▼
/usr/local/bin/gitea ← shim, runs as host `git`
│ ssh -p 2222 git@127.0.0.1 "SSH_ORIGINAL_COMMAND=… /usr/local/bin/gitea … serv key-5"
▼
sshd inside the container (published 127.0.0.1:2222)
│ authenticated with the host git user's key
│ — the bare, un-forced "Gitea Host Key" line
▼
container's gitea: `gitea serv key-5`
└─ key-5 → the user, in Gitea's database → serve the repo
```
That works on monoco only because **Docker does not remap uids**: container-uid
1003 *is* host-uid 1003, container-root *is* host-root, so the app, the sshd and
the `git` login user all share one identity and can read each other's files.
Under rootless Podman with `UserNS=keep-id:uid=1003,gid=1003`, exactly **one**
container uid is shared with the host — 1003 ↔ `tanshu` — and everything else is a
subuid. Verified on `emily` from the running postgres unit, which uses the same
mechanism:
```
container 0 → parent 1 → host subuid 165536
container 70 → parent 0 → host tanshu
```
So the container's own sshd has an impossible job:
| Inside the container | On the host | Consequence |
|---|---|---|
| Gitea (`1003`) | `tanshu` | writes `authorized_keys`, mode `0600` |
| sshd (`0`) | subuid `165536` | **cannot read a `0600` tanshu-owned file** |
and it cannot be fixed by swapping the mapping: sshd must be root in the
container, while Gitea refuses to be root at all — the image aborts with
`mustNotRunAsRoot() [F] Gitea is not supposed to be run as root`. There is no uid
that satisfies both, and `keep-id` can only map one of them onto the host user.
### The replacement
The second hop stops being an ssh and becomes a **`podman exec`**, run as
`tanshu`, the user who owns the container. The public entry point — the host's
sshd on port 22 — is unchanged, and so is every `command=` line Gitea has already
written, because the passthrough shim keeps its path and its arguments:
```
client ──ssh git@git.tanshu.com:22──► host sshd (emily)
│ keys from gitea-authorized-keys (AuthorizedKeysCommand,
│ because StrictModes refuses a file git does not own)
▼
/usr/local/bin/gitea ← shim, runs as host `git`
│ sudo -n -u tanshu /usr/local/bin/gitea-container-exec \
│ "$SSH_ORIGINAL_COMMAND" --config=… serv key-5
▼
podman exec -i --user 1003:1003 … gitea \
/usr/local/bin/gitea --config=/data/gitea/conf/app.ini serv key-5
└─ env SSH_ORIGINAL_COMMAND="git-upload-pack '…'"
key-5 → the user → serve the repo
```
The identity still travels in the command string (`key-5`), so Gitea's per-user
authorisation is exactly what it was. What is gone: the 2222 publish, the
container's sshd as a hop, `/data/ssh` in the data tree, and the "Gitea Host Key"
that only existed to authenticate that hop.
### Why there is no sshd in the container
- sshd must run as the container's root, which under `keep-id` is a subuid.
- Gitea refuses to run as root, so it runs as container-uid 1003 = `tanshu`.
- Gitea maintains `authorized_keys` itself, as `tanshu`, mode `0600`.
- A subuid cannot read it. `StrictModes` would also reject it.
- Rewriting the modes is not a fix: `sshd` ignores world-readable host keys, and
the file is rewritten by Gitea after every key change.
- `podman exec` needs no uid to be shared with the container at all, so it works.
The image still *starts* an sshd of its own (`/etc/s6/openssh`), because that is
baked into the image. It is unpublished and nothing connects to it, but it does
want host keys, and it cannot write them into the data tree for the same subuid
reason. It therefore gets `~/podman-data/gitea-sshd`, a scratch directory mounted
over `/data/ssh` **outside** the data tree — so no subuid-owned file ever lands in
`~/podman-data/gitea`, a plain `tar` of the data tree stays complete, and the
play's "nothing in here belongs to another user" check stays meaningful.
> If you prefer no stray directory at all, a tmpfs works in principle:
> `Mount=type=tmpfs,destination=/data/ssh,tmpfs-mode=1777` instead of the
> `Volume=` line. **Not verified on emily** — try it in a scratch copy of the unit
> and watch `podman logs gitea` for the image's openssh service coming up.
## The pieces on the host
**The `git` login user** exists only so sshd has someone to authenticate as. It
has `/bin/bash` because sshd runs a forced command through the login shell
(`/bin/bash -c '…'`); a nologin shell would break git-over-SSH outright. Its uid
is deliberately *not* pinned: on monoco it was 1003 because it had to equal the
container's git uid, and under `keep-id` that pairing no longer exists.
**There is no `/home/git/.ssh`.** It used to hold the shim's client key (the
"Gitea Host Key", now obsolete — nothing authenticates over ssh any more) and was
bind-mounted into the container as `/data/git/.ssh`. The file that matters now
lives in the data tree, where Gitea writes it. The migration below moves it there.
**`/usr/local/bin/gitea`** is the shim. Its path is a contract: it is baked into
every `command=` line Gitea has already written, so renaming it locks every user
out until Gitea rewrites the file.
**`/usr/local/bin/gitea-container-exec`** is the `podman exec`, with the uid,
`HOME`, `GITEA_CUSTOM`, `SSH_ORIGINAL_COMMAND` and `XDG_RUNTIME_DIR` that the
container's sshd used to provide. `XDG_RUNTIME_DIR` is set explicitly because
`sudo -u` creates no PAM session — the same trick
[caddy/readme.md](../caddy/readme.md) shows for `sudo -u … systemctl --user`.
**`/etc/sudoers.d/gitea-ssh-passthrough`** grants `git` exactly one command as
`tanshu`. It is installed with `visudo -cf` validation, because a syntax error in
`/etc/sudoers.d` does not break one rule, it breaks *every* `sudo` on the host.
The trust boundary is worth stating plainly: whoever can obtain a `git` shell on
this host can run containers as `tanshu`. Nothing gets that shell — every key
Gitea accepts carries a forced command — and the hook below is what keeps it that
way.
**`/etc/ssh/sshd_config.d/60-gitea.conf`** points sshd at
`gitea-authorized-keys` for `Match User git`. It has to be a hook rather than the
plain file because the file belongs to `tanshu`, which is exactly what
`StrictModes` refuses — and `StrictModes` cannot be relaxed for one user:
OpenSSH rejects it inside a `Match` block (`Directive 'StrictModes' is not
allowed within a Match block`, confirmed on emily, OpenSSH 10.2p1).
`AuthorizedKeysCommand` output is used without an ownership check, so that is the
mechanism. The drop-in is validated with a full `sshd -t` before sshd is
reloaded, and removed again if it does not parse.
**`/usr/local/bin/gitea-authorized-keys`** prints Gitea's file, filtered to the
lines that force the shim. The unforced "Gitea Host Key" line is deliberately
*dropped*: nothing needs it now, and printing it would hand an interactive `git`
login — and with it rootless Podman, via the sudoers rule — to anyone holding a
copy of that key, a backup included.
## Who runs as whom
`UserNS=keep-id:uid=1003,gid=1003` plus `Environment=USER_UID/USER_GID=1003`, all
three from `gitea_uid` in [vars/default.yml](vars/default.yml). Gitea therefore
writes as container-1003, which is `tanshu` on the host: everything under
`~/podman-data/gitea` is `tanshu`-owned, and plain `tar`/`rsync` backups keep
working.
**`User=0` is the fourth line, and without it nothing starts.** `keep-id` does not
only map the uid, it also runs the container *process* as it — so the whole
container came up as `git`, while the image's start-up is written to begin as
root: `/usr/bin/entrypoint` rewrites `/etc/passwd`, `/etc/s6/gitea/setup` (sourced
by `/etc/s6/gitea/run` before it `su-exec git`s the server) prepares `/data`, and
the image's own CMD is `s6-svscan /etc/s6`, which has to create
`.s6-svscan/lock` inside root-owned `/etc/s6`. Denied all three, the container
exited immediately and `Restart=always` turned that into a crash loop — which
`is-active` reports as `activating`, not as a failure. `User=0` restores root for
the start-up and changes nothing about the mapping above: the *server* still runs
as `git` through `su-exec`, which is what satisfies Gitea's own
`mustNotRunAsRoot` check. (`nextcloud.container` carries the same pair for the
same reason.)
The consequence to internalise: **container root is a subuid on the host.** It
holds the usual capabilities *inside* its user namespace, so the start-up's
`mkdir`, `chmod` and `chown` all succeed — and what they create is left owned by
that subuid. The image chowns the directories it knows about back to `git`
(`chown -R ${USER}:git /data/gitea` and `/data/git` in `/etc/s6/gitea/setup`), but
not everything, so [roles/gitea](roles/gitea/tasks/main.yaml) creates the tree up
front — `gitea/`, `gitea/conf/`, `gitea/log/`, `git/`, `git/.ssh/`, `ssh/` — owned
by `tanshu`, with the modes those scripts would have set. Two of them are not
housekeeping at all:
- **`git/.ssh/environment`** is written by the play rather than left to the setup
script, because the setup's copy is the one file under `/data` it never chowns
back to `git`.
- **`ssh/`** is the mount point for the in-image sshd's host keys. It is hidden
inside the container by that mount, so nothing ever chowns it: if Podman has to
create it, it lands in the data tree owned by the subuid, and the check at the
end of the role fails on it. That is exactly what happened before this was added.
This is the same trap [postgres/readme.md](../postgres/readme.md) documents for
`PGDATA`, arrived at from the other direction.
### When it does not start
The container's start-up output is in the journal, and the play prints it for you:
every step from `Start Gitea` through the health check is in one block whose
`rescue` dumps `systemctl status`, `journalctl` and `podman logs`. The lines that
have actually shown up, and what each means:
| In the journal | Cause |
|---|---|
| `s6-svscan: fatal: unable to open .s6-svscan/lock: Permission denied` | the container process is not root — `User=0` is missing |
| `sed: can't create temp file '/etc/passwdXXXXXX': Permission denied` | same cause: the entrypoint may not rewrite `/etc/passwd` |
| `DB not initialized` / the install page in a browser | the copy from monoco has not run, so this is an empty Gitea |
Beyond those, `podman logs gitea` on a healthy container is **clean** — no
permission errors at all. An earlier draft of this readme listed four
`chmod … Operation not permitted` lines as expected noise; they do not occur, and
if you see something like them they are worth reading rather than skimming.
## Settings
- `create_containers`: number of containers to create.
- `default_container_name`: default name for new containers.
- `default_container_image`: default image for new containers.
- `default_container_command`: default command to run on new containers.
| Variable | Purpose |
|---|---|
| `podman_user`, `podman_data_dir`, `quadlet_dir` | Rootless-Podman contract, duplicated from [caddy/vars/default.yml](../caddy/vars/default.yml) |
| `container_name` | Container, unit and network prefix — `gitea` |
| `image` | `docker.io/gitea/gitea:latest`, fully qualified on purpose |
| `host_directory` | Where all state lives (`~/podman-data/gitea`) |
| `container_port` | `3000`, the port inside the container Caddy proxies to |
| `gitea_uid` / `gitea_gid` | `1003` — the image's own `git` account. Drives `UserNS`, `USER_UID`/`USER_GID` and the wrapper's `--user`; see [Who runs as whom](#who-runs-as-whom) |
| `gitea_custom` | `/data/gitea`, the image's `GITEA_CUSTOM` |
| `timezone` | `TZ` for log timestamps only; Gitea stores each user's own timezone |
| `health_cmd` | The unit's `HealthCmd` — `/api/healthz`, Gitea's unauthenticated probe |
| `container_sshd_hostkeys` | Scratch dir for the image's unused sshd, mounted over `/data/ssh` |
| `git_user`, `git_home` | The SSH login account (`git`, `/home/git`) |
| `shim_path` | `/usr/local/bin/gitea` — a **contract**, not a knob |
| `podman_exec_wrapper`, `authorized_keys_command` | The two helper scripts |
| `sshd_dropin_path`, `sshd_service`, `sudoers_path` | `/etc/ssh/sshd_config.d/60-gitea.conf`, `ssh`, `/etc/sudoers.d/gitea-ssh-passthrough` |
| `host`, `caddy_name`, `caddyfile_path`, `snippet_block` | The Caddy side, as in every service playbook |
Keep the three `podman_*`/`quadlet_dir` vars in sync with
[caddy/vars/default.yml](../caddy/vars/default.yml) — they are duplicated rather
than imported because a play cannot include Caddy's vars file wholesale (it also
sets `host_directory`, which every service defines for itself).
## Running this Playbook
Quick Steps:
### 1. Obtain the playbook
```shell
git clone https://github.com/do-community/ansible-playbooks.git
cd ansible-playbooks/docker_ubuntu1804
```
### 2. Customize Options
```shell
nano vars/default.yml
```
```yml
#vars/default.yml
---
create_containers: 4
default_container_name: docker
default_container_image: ubuntu
default_container_command: sleep 1d
```
### 3. Run the Playbook
## Running
```command
ansible-playbook -l [target] -i [inventory file] -u [remote user] playbook.yml
ansible-playbook -u tanshu gitea/playbook.yml
```
For more information on how to run this Ansible setup, please check this guide: [How to Use Ansible to Install and Set Up Docker on Ubuntu 18.04](https://www.digitalocean.com/community/tutorials/how-to-use-ansible-to-install-and-set-up-docker-on-ubuntu-18-04).
The play targets `emily` directly, so the inventory must contain it — the one in
`prepare-server/` does, and its `ansible.cfg` names it.
This play applies **only** this app's units, data directories and host-side SSH
plumbing. The rootless-Podman prerequisites — installing podman,
`loginctl enable-linger`, the low-port sysctl and the Quadlet directory — are the
shared task file at
[caddy/roles/podman/tasks/main.yaml](../caddy/roles/podman/tasks/main.yaml), and
are **not** run here. A host that has already run `caddy/playbook.yml` has them.
`systemctl --user enable gitea` is neither needed nor possible: the unit is
*generated*, and Quadlet applies `[Install] WantedBy=default.target` itself.
## Migrating the data from monoco
Everything here is about **files Gitea owns**. There is no database dump to take:
this Gitea is SQLite, and `gitea.db` travels inside the tree. Run the commands
from the machine that holds the `monoco` and `emily` SSH aliases.
Two things in the source tree are deliberately **not** copied:
- **`/var/lib/gitea/ssh`** — host keys for the sshd inside the container. That
sshd is no longer a hop, nothing connects to it, and the new deployment lets it
generate fresh keys in `~/podman-data/gitea-sshd`. The Docker-era audit told you
to copy them so clients would not see host-key warnings; with the container's
sshd out of the path, no client ever sees them.
- **`/home/git/.ssh/id_ed25519` and friends** — the "Gitea Host Key" that
authenticated the old second hop. Nothing authenticates over ssh now. Only
`authorized_keys` (and optionally `environment`) is carried across, into the
container's data tree, which is where Gitea expects to maintain it.
```bash
# 1. Look before you copy: what is there, and how big is it. `ls -ln` shows
# numeric owners, because the old container wrote as uid 1003 (`git`).
ssh monoco 'du -sh /var/lib/gitea/*; ls -ln /var/lib/gitea'
ssh monoco 'sudo ls -ln /home/git/.ssh'
# 2. Stop the old container so nothing writes while you copy. SQLite in WAL mode
# is exactly the thing the backup plan warns about — a live copy can be torn.
# Leave it stopped: emily is about to become the only writer, and two live
# copies behind the same hostname would diverge.
ssh monoco 'sudo docker stop gitea'
# 3. Create the destination tree (the playbook would create it too, but the copy
# needs somewhere to land).
ssh emily 'mkdir -p ~/podman-data/gitea/git/.ssh'
# If the playbook has already run here — it says so when it does — the app has
# left a fresh `gitea/conf/app.ini` (and possibly logs) behind, and the digests
# in step 6 will not match. Clear it first:
# ssh emily 'systemctl --user stop gitea.service; find ~/podman-data/gitea/gitea ~/podman-data/gitea/git -mindepth 1 -delete'
# 4. Copy the data tree, minus the obsolete container-sshd host keys.
# tar streams it across the workstation; rsync cannot copy remote-to-remote in
# one command.
#
# `set -o pipefail` matters: without it the pipeline reports only the
# *receiving* tar's status, so a read error or a dropped stream on the source
# side exits 0 and leaves a silently short copy.
#
# The source needs sudo — the files belong to `git`, not to you.
set -o pipefail
ssh monoco 'sudo tar --exclude="./ssh" -C /var/lib/gitea -cf - .' \
| ssh emily 'tar --no-same-owner -C ~/podman-data/gitea -xf -'
# 5. Copy the authorized_keys Gitea maintains. It lives outside /var/lib/gitea on
# the old host, because it was bind-mounted in from the git user's home.
set -o pipefail
ssh monoco 'sudo tar -C /home/git/.ssh -cf - authorized_keys environment' \
| ssh emily 'tar --no-same-owner -C ~/podman-data/gitea/git/.ssh -xf -'
# 6. Verify by content, over both trees, with the same exclusions on each side so
# the digests are comparable. `.ssh` is excluded here because it is compared
# on its own in the next step, and `ssh/` never left monoco.
ssh monoco 'cd /var/lib/gitea && sudo find ./git ./gitea ! -path "./git/.ssh*" -type f -print0 \
| sort -z | xargs -0 -r sudo sha256sum | sha256sum'
ssh emily 'cd ~/podman-data/gitea && find ./git ./gitea ! -path "./git/.ssh*" -type f -print0 \
| sort -z | xargs -0 -r sha256sum | sha256sum'
# identical digests = a byte-faithful copy
ssh monoco 'sudo sha256sum /home/git/.ssh/authorized_keys | cut -d" " -f1'
ssh emily 'sha256sum ~/podman-data/gitea/git/.ssh/authorized_keys | cut -d" " -f1'
# 7. Nothing may end up owned by root or by a subuid, and sshd's hook must be
# able to read the file it is about to be handed.
ssh emily 'find ~/podman-data/gitea ! -user tanshu' # empty = correct
ssh emily 'ls -ln ~/podman-data/gitea/git/.ssh/authorized_keys'
# 8. Deploy — after the copy, so Gitea finds the real data on first start.
cd prepare-server && ansible-playbook -u tanshu ../gitea/playbook.yml
```
If the link is slow or drops, do it in two hops instead: resumable, and it shows
progress.
```bash
rsync -aP --exclude='ssh' monoco:/var/lib/gitea/ ~/gitea-copy/
rsync -aP ~/gitea-copy/ emily:~/podman-data/gitea/
rsync -aP monoco:'/home/git/.ssh/{authorized_keys,environment}' ~/gitea-ssh-copy/
rsync -aP ~/gitea-ssh-copy/ emily:~/podman-data/gitea/git/.ssh/
```
Rolling back means rolling back all of it: `ssh monoco 'sudo docker start gitea'`
is only meaningful while monoco's data is still intact, and it would then be
writing to a `gitea.db` that emily's copy has already moved past. **Never run
both at once.**
## Verification
```bash
systemctl --user status gitea
journalctl --user -u gitea -f
podman logs gitea # clean — see "When it does not start" above
# What the playbook checks for you:
podman healthcheck run gitea # /api/healthz
find ~/podman-data/gitea ! -user tanshu # empty
# The unit is where it should be, and the image's sshd has its scratch space:
podman inspect gitea --format '{{range .Mounts}}{{.Source}} -> {{.Destination}}{{"\n"}}{{end}}'
# The host's SSH half, piece by piece:
sudo sshd -T -C user=git | grep -i authorizedkeys # the hook, and only for git
sudo -u tanshu /usr/local/bin/gitea-authorized-keys | head -3
sudo -u tanshu /usr/local/bin/gitea-authorized-keys | grep -c '^ssh-' # 0 — the bare host-key line is dropped
sudo visudo -cf /etc/sudoers.d/gitea-ssh-passthrough
# the wrapper reaches the container as the right user:
sudo -u tanshu /usr/local/bin/gitea-container-exec "" /usr/local/bin/gitea --version
# and the git user is allowed to run exactly that wrapper:
sudo -u git sudo -n -u tanshu /usr/local/bin/gitea-container-exec "" /usr/local/bin/gitea --version
# The site, end to end.
curl -sSI https://git.tanshu.com
```
Then, from a client whose key is registered in Gitea:
```bash
ssh -T git@git.tanshu.com # "Hi <user>! You've successfully authenticated…"
git clone git@git.tanshu.com:tanshu/playbooks.git
```
and push to a repository you own — that is the check that per-user authorisation
still works, because the identity travels in the `serv key-N` argument and *not*
in the shim's own credentials. Then exercise the web UI and confirm a repository
list renders, which is the check that `gitea.db` came across.
Finally, reboot the host and confirm the unit returns unattended — without
`loginctl enable-linger tanshu` it will not, and the failure is silent.
## Notes and gotchas
- **`DISABLE_SSH` must stay `false`, and Gitea's built-in SSH server must stay
off** (`START_SSH_SERVER` absent or `false`) in `app.ini`. Gitea only maintains
`authorized_keys` for an *external* sshd — which the host's sshd is here. Turn
the built-in server on and Gitea stops writing the file the host's sshd reads,
and git-over-SSH stops working with no error anywhere obvious. The copied
`app.ini` already has this right.
- **`SSH_PORT = 22` stays as it is.** It is what makes Gitea offer
`git@git.tanshu.com:user/repo.git` rather than a URL with a port in it, and the
host's sshd is what answers on 22. `SSH_LISTEN_PORT = 22` refers to the
container's own sshd port, which this deployment does not use.
- **The `git` user can reach rootless Podman as `tanshu`** through one sudoers
rule. That is the price of a two-user design and it is bounded to one root-owned
wrapper; see [The pieces on the host](#the-pieces-on-the-host).
- **The authorized_keys filter is a format dependency.** If Gitea ever stops
writing `command="/usr/local/bin/gitea …` lines, the hook prints nothing and
git-over-SSH breaks while HTTPS keeps serving. Diagnose with
`sudo -u tanshu /usr/local/bin/gitea-authorized-keys` and
`podman exec gitea cat /data/git/.ssh/authorized_keys`.
- **Pushing over SSH runs no hooks through the host user.** Hooks execute inside
the container, as `git` = `tanshu`, so a repository hook can write anywhere
`tanshu` can. Same as before the move, but worth knowing before enabling
server-side hooks.
- **`latest` floats.** Neither the old playbook nor this one pins a digest, so an
image bump is a normal `podman pull` away and may run new database migrations on
first start.
- **The old backup script's `docker exec -u git gitea gitea dump …` needs
rewriting** for this deployment: `sudo podman exec` from root's crontab cannot
enter another user's rootless containers, so anything scheduled belongs in a
**systemd user timer** next to the Quadlet units. See the Backups section of
[postgres/readme.md](../postgres/readme.md) for the shape of it.
- **DNS.** `git.tanshu.com` must resolve to `emily` before Caddy can obtain a
certificate for it.
- **Do not restart the container on monoco** while emily is serving. It would
write to the old data tree, which is being kept intact purely as a rollback.
## Not done here
- Rewriting the backup script for rootless Podman and moving it to a systemd user
timer (above).
- Rotating the secrets that live in `app.ini` in plaintext — `SECRET_KEY`,
`INTERNAL_TOKEN`, `LFS_JWT_SECRET`, `JWT_SECRET` — and the fleet-wide
`123456`-style passwords elsewhere in this repo.
- Templating `app.ini`. Gitea's own configuration travels with the data, and it
carries those secrets; a fresh instance with no data gets the image's install
wizard instead, which is a reasonable outcome but not a supported path here.
gitea/roles/caddy/handlers/main.yaml
+28 -2
View File
@@ -1,5 +1,31 @@
---
# Handlers for Caddy role
# Caddy is its own playbook (caddy/playbook.yml) and a rootless Quadlet unit
# there. These handlers run as the connection user, so `systemctl --user` and
# `podman exec` work without any XDG_RUNTIME_DIR juggling.
# Notified by this role when the Caddyfile changes. A reload keeps existing
# connections; a restart would briefly drop every hosted site.
#
# Note the config path: /etc/caddy/Caddyfile is the mount inside the container,
# not {{ caddyfile_path }}, which is the host side of it.
- name: Reload Caddy configuration
ansible.builtin.command: "docker exec -w /etc/caddy {{ caddy_container }} caddy reload"
ansible.builtin.command: >-
podman exec {{ caddy_name }} caddy reload --config /etc/caddy/Caddyfile
changed_when: true
listen: "Reload Caddy"
# Notified by caddy/roles/register when this service's network unit, or the
# drop-in that attaches Caddy to it, is written. Quadlet has no live equivalent
# of `docker network connect`, so the container has to be recreated — a brief
# restart of every hosted site, not just this one. See caddy/readme.md,
# "Trade-off worth knowing".
#
# Defined after the reload above so that when both are notified the cheap config
# reload happens first.
- name: Restart Caddy
ansible.builtin.systemd_service:
name: "{{ caddy_name }}.service"
scope: user
state: restarted
daemon_reload: true
listen: Restart Caddy
gitea/roles/caddy/tasks/main.yaml
+16 -16
View File
@@ -1,22 +1,22 @@
---
- name: Read current Caddyfile
ansible.builtin.slurp:
path: "{{ caddyfile_path }}"
register: caddyfile_raw
# Append this site's block to the shared Caddyfile.
#
# The Caddyfile is a bind mount into the Caddy container, so Caddy sees the
# change as soon as the file is written; the handler only asks Caddy to re-read
# it. That is a reload, not a restart, so the other hosted sites are untouched.
#
# blockinfile is idempotent on its own via the marker, so there is no need to
# slurp the file and compare first — which also matters because slurp *fails*
# when the Caddyfile does not exist yet, on a host where the Caddy playbook has
# not run.
- name: Decode Caddyfile content
set_fact:
caddyfile_content: "{{ caddyfile_raw['content'] | b64decode }}"
- name: Check if snippet already exists
set_fact:
snippet_present: "{{ snippet_block in caddyfile_content }}"
- name: Add snippet if missing
- name: "Add the {{ host }} site block to the Caddyfile"
ansible.builtin.blockinfile:
path: "{{ caddyfile_path }}"
marker: "# {mark} Ansible managed Caddy snippet for {{ http_host }}"
marker: "# {mark} Ansible managed Caddy snippet for {{ host }}"
block: "{{ snippet_block }}"
create: yes
when: not snippet_present
notify: "Reload Caddy"
owner: "{{ podman_user }}"
group: "{{ podman_user }}"
mode: '0644'
notify: Reload Caddy
gitea/roles/gitea/handlers/main.yaml
+19
View File
@@ -0,0 +1,19 @@
---
# All handlers run as the connection user ({{ podman_user }}), so systemctl
# --user works without setting XDG_RUNTIME_DIR.
- name: Reload systemd user units
ansible.builtin.command: systemctl --user daemon-reload
changed_when: true
listen: Reload systemd user units
# A changed unit file changes the generated unit, so Gitea is recreated. All of
# its state is on the bind mount or in the SQLite database, so a recreate loses
# nothing — but it is a few seconds of downtime for the one site.
- name: Restart Gitea
ansible.builtin.systemd_service:
name: "{{ unit_name }}"
scope: user
state: restarted
daemon_reload: true
listen: Restart Gitea
gitea/roles/gitea/tasks/main.yaml
+207 -32
View File
@@ -1,36 +1,211 @@
---
# Tasks for Gitea role
- name: Pull Gitea image
docker_image:
name: "{{ container_image }}"
source: pull
state: present
force_source: yes
# Gitea as a rootless Podman Quadlet unit.
#
# Everything here runs as {{ podman_user }} — the connection user. No become, no
# XDG_RUNTIME_DIR: the SSH connection gave us a real PAM session, so systemd's
# user manager is reachable through the standard environment.
- getent:
database: passwd
key: "{{ user }}"
split: ":"
# The data tree is created here rather than left to the image, and that is not
# housekeeping. With keep-id:uid={{ gitea_uid }} the container process runs as
# `git`, so /usr/bin/entrypoint is the *only* part of the image's start-up that
# root can do — and it needs root for /etc/passwd, /etc/s6 and this tree.
#
# Container root is a subuid on the host, but inside its user namespace it holds
# the usual capabilities, so `mkdir`, `chmod` and `chown` all succeed; what it
# creates is simply left owned by that subuid. The image chowns the directories it
# knows about back to `git` — `chown -R ${USER}:git /data/gitea` and `/data/git`
# in /etc/s6/gitea/setup, which is where app.ini comes from — so those work either
# way. The things it does *not* chown are what this task is for, and they are the
# ones that would otherwise be subuid-owned and invisible to the app and to a
# plain tar backup:
#
# gitea/ app.ini, gitea.db, sessions, avatars, indexers
# gitea/conf where the entrypoint would generate app.ini
# gitea/log Gitea's file log
# git/ repositories, LFS, and the git account's HOME
# git/.ssh authorized_keys — written by Gitea, read by the host's sshd
# ssh/ only ever a *mount point* — see below
#
# The sshd *inside* the image is the exception; it cannot use this tree at all.
# See {{ container_sshd_hostkeys }} below.
- name: Ensure the Gitea data directory exists
ansible.builtin.file:
path: "{{ host_directory }}"
state: directory
owner: "{{ podman_user }}"
group: "{{ podman_user }}"
mode: '0755'
- name: Create gitea container
docker_container:
name: "{{ container_name }}"
image: "{{ container_image }}"
- name: Ensure the directories the image expects under /data exist
ansible.builtin.file:
path: "{{ host_directory }}/{{ item.path }}"
state: directory
owner: "{{ podman_user }}"
group: "{{ podman_user }}"
mode: "{{ item.mode }}"
loop:
- { path: gitea, mode: '0755' }
- { path: gitea/conf, mode: '0755' }
- { path: gitea/log, mode: '0755' }
- { path: git, mode: '0755' }
# 0700 because that is what the image's own setup script chmods it to
# (`chmod 700 /data/git/.ssh` in /etc/s6/gitea/setup, which runs as root
# before `su-exec git`). Set here so the mode never depends on that script
# having run.
- { path: git/.ssh, mode: '0700' }
# The in-image sshd's host keys are mounted over /data/ssh from
# {{ container_sshd_hostkeys }}, so this directory is never written to — it
# has to *exist*, because it is the mount point. If it does not, Podman
# creates it during container set-up, inside the container's user namespace,
# and it lands here owned by the subuid rather than by {{ podman_user }}:
#
# drwxr-xr-t 2 165536 165536 4096 ssh
#
# which the check at the end of this role then (correctly) fails on. Creating
# it first is what keeps the data tree wholly {{ podman_user }}'s.
- { path: ssh, mode: '0755' }
# The image's setup creates this file if it is missing, as root — and it is the
# one file under /data the setup never chowns back to `git`, so on a fresh host it
# would be left subuid-owned. Writing it here, as {{ podman_user }} and 0600,
# both avoids that and states the value the image's own environment carries.
- name: Write the container-side .ssh/environment file
ansible.builtin.copy:
dest: "{{ host_directory }}/git/.ssh/environment"
content: |
GITEA_CUSTOM={{ gitea_custom }}
owner: "{{ podman_user }}"
group: "{{ podman_user }}"
mode: '0600'
# The image always starts an sshd of its own inside the container, and that sshd
# wants host keys. This deployment does not use it — git-over-SSH arrives on the
# host's sshd on port 22 — but it has to have somewhere to write, and the process
# that runs it maps to a subuid that cannot touch the data directory. Keeping it
# out here means no subuid-owned file ever lands in {{ host_directory }}, so a
# plain tar/rsync of the data tree stays complete and error-free, and the check
# at the end of this role stays meaningful.
#
# 0777 is deliberate and bounded: /home/{{ podman_user }} is 0750, so no other
# host user can reach this path at all. The mode only matters inside the userns.
- name: Ensure the scratch host key directory exists
ansible.builtin.file:
path: "{{ container_sshd_hostkeys }}"
state: directory
owner: "{{ podman_user }}"
group: "{{ podman_user }}"
mode: '0777'
- name: Install the Gitea Quadlet unit
ansible.builtin.template:
src: gitea.container.j2
dest: "{{ quadlet_dir }}/{{ container_name }}.container"
owner: "{{ podman_user }}"
group: "{{ podman_user }}"
mode: '0644'
notify:
- Reload systemd user units
- Restart Gitea
# Quadlet honours [Install] WantedBy= when it generates the unit, so
# `systemctl --user enable` is neither needed nor possible on a generated unit.
#
# All three steps are in one block because they fail the same way and want the
# same evidence. `systemctl start` reports success as soon as it has handed the
# unit to systemd, so a container that dies a second later — a `User=0` missing,
# an unwritable /data, a subuid that cannot run the image's start-up — surfaces
# only as `activating` from `is-active`, or as a health check that never passes,
# with the reason in the journal and nowhere else. Restart=always makes that a
# crash loop rather than an error.
- name: Start and verify Gitea
block:
- name: Start Gitea
ansible.builtin.systemd_service:
name: "{{ unit_name }}"
scope: user
state: started
restart_policy: "unless-stopped"
cpus: .10
memory: "512M"
memory_swap: "512M"
env:
USER_UID: "{{ ansible_facts['getent_passwd'][user][1] }}"
USER_GID: "{{ ansible_facts['getent_passwd'][user][2] }}"
published_ports:
- "127.0.0.1:3000:3000"
- "127.0.0.1:2222:22"
volumes:
- /var/lib/gitea:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
- "/home/{{ user }}/.ssh/:/data/git/.ssh"
networks:
- name: "{{ docker_network }}"
daemon_reload: true
# Retried rather than checked once: podman reports `activating` for the
# interval between systemd starting the container and its readiness
# notification.
- name: Verify Gitea is running
ansible.builtin.command: "systemctl --user is-active {{ unit_name }}"
register: gitea_active
until: gitea_active.stdout == 'active'
retries: 12
delay: 5
changed_when: false
# `is-active` says the container is running; it does not say the app can
# serve. The image ships no HEALTHCHECK, so the unit defines one (HealthCmd=,
# see the template) and this runs exactly that command once, synchronously.
- name: Wait for Gitea to pass its health check
ansible.builtin.command: "podman healthcheck run {{ container_name }}"
register: gitea_health
until: gitea_health.rc == 0
retries: 30
delay: 5
changed_when: false
rescue:
# systemd reports "control process exited with error code" and nothing else,
# which is useless for the usual failures. The journal is where the image's
# own start-up output is — that is where a s6 or entrypoint permission
# failure shows up.
- name: Collect Gitea diagnostics
ansible.builtin.command: "{{ item }}"
loop:
- "systemctl --user status {{ unit_name }} --no-pager"
- "journalctl --user -u {{ unit_name }} --no-pager -n 40"
- "podman logs --tail 40 {{ container_name }}"
register: gitea_diag
changed_when: false
failed_when: false
- name: Show Gitea diagnostics
ansible.builtin.debug:
msg: "{{ gitea_diag.results | map(attribute='stdout') | list }}"
- name: Fail
ansible.builtin.fail:
msg: "Gitea did not start — see the diagnostics above."
# A wrong keep-id is the one failure here that does not look like one: Gitea
# still serves, but from files it owns as an unmapped subuid on the host — files
# that then disappear from a tar backup and cannot be deleted without
# `podman unshare`. Check the whole tree, not a sample.
#
# Deliberately not `failed_when: false`: if `find` cannot read something in the
# data directory, the tree is not provably clean, and that is the finding.
- name: List anything in the data tree not owned by {{ podman_user }}
ansible.builtin.command: >-
find {{ host_directory }} ! -user {{ podman_user }}
register: gitea_stray_owners
changed_when: false
- name: Fail if the data tree holds files owned by another user
ansible.builtin.assert:
that: gitea_stray_owners.stdout_lines | length == 0
fail_msg: >-
{{ gitea_stray_owners.stdout_lines | join(', ') }} — every file under
{{ host_directory }} must belong to {{ podman_user }}. A subuid in there
means the uid Gitea writes as and the uid mapped onto {{ podman_user }}
have drifted apart: compare gitea_uid in vars/default.yml with UserNS and
USER_UID/USER_GID in {{ quadlet_dir }}/{{ container_name }}.container.
- name: Check whether the data has been restored
ansible.builtin.stat:
path: "{{ host_directory }}/gitea/gitea.db"
register: gitea_db
# Not a failure: a fresh instance legitimately has no database yet. But on a
# migration it means the copy in readme.md has not run, and the instance that is
# now serving is an empty installer rather than this repository's Gitea.
- name: Report a data directory with no database in it
ansible.builtin.debug:
msg: >-
{{ host_directory }}/gitea/gitea.db is missing, so this Gitea has no data:
it will serve its install page and write a fresh app.ini. Expected only on
a brand-new instance — a migrated one is given its data first. See
readme.md, "Migrating the data from monoco".
when: not gitea_db.stat.exists
gitea/roles/gitea/templates/gitea.container.j2
+83
View File
@@ -0,0 +1,83 @@
# Gitea — rootless Podman Quadlet unit.
# Managed by Ansible (gitea/roles/gitea). Do not edit by hand.
#
# Joins {{ network_name }}.network, written by caddy/roles/register — the same
# include that attaches Caddy to it. Caddy reaches Gitea by container name, so
# nothing is published on the host for HTTP.
[Unit]
Description=Gitea
Documentation=https://docs.gitea.com/
After=network-online.target
Wants=network-online.target
[Container]
ContainerName={{ container_name }}
Image={{ image }}
# Gitea refuses to run as root (`mustNotRunAsRoot`), so the app cannot simply be
# container-root the way Caddy and Audiobookshelf are. Instead container uid
# {{ gitea_uid }} — the image's own `git` account — is mapped onto
# {{ podman_user }}, which keeps every file Gitea writes owned by
# {{ podman_user }} on the host: plain tar/rsync backups, no subuids, no
# `podman unshare`. The cost is that the image's *root* phases run as a subuid,
# which is why roles/gitea creates the data tree up front.
#
# USER_UID/USER_GID are not decoration: they make the entrypoint keep the
# account at {{ gitea_uid }} instead of the image default, which is what holds
# the file layout and this mapping together. All of it comes from gitea_uid in
# vars/default.yml — change one and the others have to follow.
UserNS=keep-id:uid={{ gitea_uid }},gid={{ gitea_gid }}
Environment=USER_UID={{ gitea_uid }}
Environment=USER_GID={{ gitea_gid }}
# keep-id also runs the container *process* as that mapped uid, i.e. as `git`,
# while the image's start-up is written to begin as root: `s6-svscan`, which is
# the image's own CMD, creates `.s6-svscan/lock` inside root-owned /etc/s6, and
# /usr/bin/entrypoint rewrites /etc/passwd and /etc/group to apply
# USER_UID/USER_GID. Denied both, the container died with
#
# sed: can't create temp file '/etc/passwdXXXXXX': Permission denied
# s6-svscan: fatal: unable to open .s6-svscan/lock: Permission denied
#
# and Restart=always turned that into a crash loop — which `is-active` reports as
# `activating`, not as a failure. User=0 forces the process back to root, and the
# mapping above is untouched by it: the server itself still runs as `git`, which
# is what satisfies mustNotRunAsRoot. (nextcloud.container carries the same pair
# for the same reason.)
User=0
# Only log timestamps: Gitea keeps each user's own timezone in the database.
Environment=TZ={{ timezone }}
# The image's VOLUME. Repositories, LFS, the SQLite database, app.ini, sessions,
# logs, and the authorized_keys file the host's sshd reads.
Volume={{ host_directory }}:/data
# The image starts an sshd of its own on port 22 inside the container. This
# deployment does not use it — git-over-SSH arrives on the *host's* sshd and is
# forwarded with `podman exec` — but it still wants host keys, and it runs as the
# container's root, which under keep-id is a subuid that cannot write into the
# data directory. It gets a scratch directory of its own instead, outside the
# data tree, so nothing it writes can end up in a backup. It is not published,
# and nothing connects to it.
Volume={{ container_sshd_hostkeys }}:/data/ssh
Network={{ network_name }}.network
# The image ships no HEALTHCHECK, so the unit defines one and the play gates on
# it. /api/healthz is Gitea's own unauthenticated health endpoint.
HealthCmd={{ health_cmd }}
HealthInterval=30s
HealthTimeout=5s
HealthRetries=3
HealthStartPeriod=60s
[Service]
Restart=always
# The first start initialises the SQLite database and runs its migrations before
# the server answers.
TimeoutStartSec=180
[Install]
WantedBy=default.target
gitea/roles/ssh/tasks/main.yaml
+140
View File
@@ -0,0 +1,140 @@
---
# The host half of Gitea's git-over-SSH support.
#
# Port 22 belongs to the host's sshd and stays that way: clients keep using
# git@git.tanshu.com:user/repo.git, and no remote URL changes. Four host-side
# pieces make that work:
#
# 1. a `{{ git_user }}` login user with no interactive access — every key
# Gitea accepts carries a forced command, so sshd runs the shim and
# nothing else;
# 2. {{ shim_path }}, the shim those forced commands invoke;
# 3. {{ podman_exec_wrapper }} plus a one-command sudoers rule — how the shim,
# running as `{{ git_user }}`, reaches a container owned by
# {{ podman_user }};
# 4. the sshd and sudoers drop-ins that tie the above together.
#
# What differs from the Docker-era role is the second hop. It used to be an ssh
# into an sshd running *inside* the container; under keep-id that sshd runs as a
# subuid and cannot read the authorized_keys Gitea writes as {{ podman_user }},
# so the hop is a `podman exec` instead. readme.md, "How git over SSH works
# now", is where the reasoning lives.
#
# Every task here is root-only (users, /usr/local/bin, /etc/ssh, /etc/sudoers),
# so each elevates on its own. The play carries no become, because the Quadlet
# units have to belong to the connection user.
- name: Ensure the git group exists
ansible.builtin.group:
name: "{{ git_user }}"
become: true
# The shell matters even though nobody ever gets one: sshd runs a forced command
# through the login user's shell (`/bin/bash -c '...'`), so a nologin shell would
# break git-over-SSH outright.
#
# The uid is deliberately left to the system. On monoco the host git account was
# uid 1003 because the container's git account was 1003 and the two shared one
# uid space; under keep-id the container's 1003 is mapped onto {{ podman_user }}
# instead, so nothing on this host depends on this number any more — not the
# key file (owned by name), not the sudoers rule (matched by name), not the
# authorized_keys hook (run as {{ podman_user }}).
- name: Add the git login user
ansible.builtin.user:
name: "{{ git_user }}"
comment: Gitea SSH login user
group: "{{ git_user }}"
home: "{{ git_home }}"
shell: /bin/bash
create_home: true
become: true
# There is deliberately no ~/.ssh for this user any more. Its old contents were
# the shim's client key and the authorized_keys Gitea maintains; the key is
# obsolete (nothing authenticates over ssh in this design) and the file lives in
# the container's data tree now, where Gitea writes it. The migration notes in
# readme.md move the file into place.
- name: Install the git-over-SSH passthrough shim
ansible.builtin.template:
src: gitea.j2
dest: "{{ shim_path }}"
owner: root
group: root
mode: '0755'
become: true
- name: Install the container exec wrapper
ansible.builtin.template:
src: gitea-container-exec.j2
dest: "{{ podman_exec_wrapper }}"
owner: root
group: root
mode: '0755'
become: true
- name: Install the authorized_keys hook sshd runs for the git user
ansible.builtin.template:
src: gitea-authorized-keys.j2
dest: "{{ authorized_keys_command }}"
owner: root
group: root
mode: '0755'
become: true
# valid with `visudo -cf %s`: a syntax error in /etc/sudoers.d does not fail one
# rule, it fails *every* sudo invocation on the host, which on a server reached
# by ssh is a nasty way to find out.
- name: Let the git user reach the container as {{ podman_user }}
ansible.builtin.template:
src: sudoers-gitea.j2
dest: "{{ sudoers_path }}"
owner: root
group: root
mode: '0440'
validate: /usr/sbin/visudo -cf %s
become: true
- name: Point sshd at Gitea's authorized_keys
block:
- name: Install the sshd drop-in for the git user
ansible.builtin.template:
src: sshd-gitea.conf.j2
dest: "{{ sshd_dropin_path }}"
owner: root
group: root
mode: '0644'
register: gitea_sshd_dropin
become: true
# A drop-in that does not parse is not a failed play, it is a host nobody can
# log in to after the next reboot: sshd reads the whole directory at start-up.
# Validate the resulting configuration before it is ever loaded, and take the
# file away again if it does not parse.
- name: Validate the resulting sshd configuration
ansible.builtin.command: /usr/sbin/sshd -t
changed_when: false
become: true
when: gitea_sshd_dropin.changed
rescue:
- name: Take the sshd drop-in back out
ansible.builtin.file:
path: "{{ sshd_dropin_path }}"
state: absent
become: true
- name: Fail
ansible.builtin.fail:
msg: >-
{{ sshd_dropin_path }} did not validate, so it has been removed and
sshd is still running — and still restarting — on its previous
configuration. Run `/usr/sbin/sshd -t` on the host to see why.
# `reload`, not `restart`: a reload re-execs sshd without dropping the connection
# this play is running over.
- name: Reload sshd
ansible.builtin.systemd_service:
name: "{{ sshd_service }}"
state: reloaded
become: true
when: gitea_sshd_dropin.changed
gitea/roles/ssh/templates/gitea-authorized-keys.j2
+29
View File
@@ -0,0 +1,29 @@
#!/bin/sh
# sshd's source for the `{{ git_user }}` user's authorized_keys.
# Managed by Ansible (gitea/roles/ssh). Do not edit by hand.
#
# Gitea maintains that file from inside its container, where it runs as
# {{ podman_user }} (container uid {{ gitea_uid }} is mapped onto that user by
# keep-id), so the file belongs to {{ podman_user }} and not to `{{ git_user }}`.
# sshd refuses a file the login user does not own — that is StrictModes — and
# StrictModes cannot be relaxed for one user, because sshd_config(5) does not
# allow the option inside a Match block. So the file is fed to sshd through
# AuthorizedKeysCommand instead: sshd uses whatever this prints, and applies no
# ownership check to it.
#
# Only lines that force the passthrough shim are printed. Gitea's file also
# carries an unforced entry for the "Gitea Host Key" that the Docker-era
# deployment needed so that its shim could ssh into a second sshd inside the
# container. Nothing reads that entry any more, and printing it would hand an
# interactive `{{ git_user }}` login on this host — and therefore rootless
# Podman, through the sudoers drop-in — to anyone holding a copy of that key,
# a backup included.
#
# The pattern is exactly the shape Gitea writes. If it ever changes,
# git-over-SSH stops working while HTTPS keeps serving, and
# podman exec {{ container_name }} cat /data/git/.ssh/authorized_keys
# shows what Gitea writes now.
keys="{{ host_directory }}/git/.ssh/authorized_keys"
[ -r "$keys" ] || exit 0
grep -E '^command="{{ shim_path }} ' "$keys"
exit 0
gitea/roles/ssh/templates/gitea-container-exec.j2
+34
View File
@@ -0,0 +1,34 @@
#!/bin/sh
# Runs one authorized_keys command inside Gitea's rootless container.
# Managed by Ansible (gitea/roles/ssh). Do not edit by hand.
#
# This runs as {{ podman_user }} — the user whose rootless Podman owns the
# container, which is the entire reason the wrapper exists. The `{{ git_user }}`
# login user cannot enter another user's rootless container, and there is no
# sshd inside the container to hop through any more (see readme.md, "How git
# over SSH works now"), so the forced command sshd runs for a git client ends up
# here.
#
# $1 SSH_ORIGINAL_COMMAND — the client's original request, passed through so
# Gitea can serve it.
# $@ the rest of the authorized_keys line, i.e.
# /usr/local/bin/gitea --config=/data/gitea/conf/app.ini serv key-N —
# the *container's* Gitea binary, not the host's shim at the same path.
#
# --user, HOME and GITEA_CUSTOM are what sshd used to supply: it ran the command
# as the container's git account, with HOME from /etc/passwd and GITEA_CUSTOM
# from /data/git/.ssh/environment. GITEA_CUSTOM and USER are image defaults and
# come through `podman exec` on their own; HOME does not, and git looks there for
# /data/git/.gitconfig, so it is set explicitly.
#
# XDG_RUNTIME_DIR is set explicitly because `sudo -u` creates no PAM session, so
# Podman would not otherwise find this user's runtime directory — the same reason
# caddy/readme.md shows the `sudo -u … systemctl --user` form with it.
original_command="$1"
shift
exec env XDG_RUNTIME_DIR="/run/user/$(id -u)" \
podman exec --interactive \
--user {{ gitea_uid }}:{{ gitea_gid }} \
--env "HOME=/data/git" \
--env "SSH_ORIGINAL_COMMAND=$original_command" \
{{ container_name }} "$@"
gitea/roles/ssh/templates/gitea.j2
+25
View File
@@ -0,0 +1,25 @@
#!/bin/sh
# Gitea's git-over-SSH passthrough shim — {{ shim_path }}
# Managed by Ansible (gitea/roles/ssh). Do not edit by hand.
#
# The host's sshd runs this, as the `{{ git_user }}` login user, whenever a
# client presents a key whose authorized_keys line is a forced command. Gitea
# writes those lines itself, from inside its container, and each one looks like
# this:
#
# command="/usr/local/bin/gitea --config=/data/gitea/conf/app.ini serv key-5",...
#
# So $@ here is `--config=/data/gitea/conf/app.ini serv key-5`, and $0 is the
# path of Gitea's own binary *inside the container*, which the wrapper resolves
# in there. The path is a contract, not a knob: it is already baked into every
# line of the authorized_keys file copied from monoco.
#
# `{{ git_user }}` cannot reach rootless containers that belong to
# {{ podman_user }}, so the command is handed to {{ podman_exec_wrapper }}, which
# sudo runs as {{ podman_user }}. SSH_ORIGINAL_COMMAND — the client's own
# git-upload-pack / git-receive-pack request — crosses as an argument, because
# sudo strips the environment.
#
# `sudo -n` on purpose: there is no TTY on this path, so a password prompt would
# hang a git push instead of failing it.
exec sudo -n -u {{ podman_user }} {{ podman_exec_wrapper }} "$SSH_ORIGINAL_COMMAND" "$@"
gitea/roles/ssh/templates/sshd-gitea.conf.j2
+20
View File
@@ -0,0 +1,20 @@
# git-over-SSH for Gitea's `{{ git_user }}` login user.
# Managed by Ansible (gitea/roles/ssh). Do not edit by hand.
#
# The authorized_keys file is maintained by Gitea from inside its container, as
# {{ podman_user }}, so `{{ git_user }}` does not own it — which is precisely what
# StrictModes refuses. StrictModes cannot be relaxed for a single user
# (sshd_config(5) does not permit it inside a Match block), so the keys are
# handed to sshd by {{ authorized_keys_command }} instead, which sshd uses
# without any ownership check.
#
# AuthorizedKeysFile=none turns the file lookup off. Without it sshd would still
# read (and log a StrictModes complaint about) ~/.ssh/authorized_keys on every
# connection before falling through to the command.
#
# This drop-in must stay in one file, and the Match block must stay last in it:
# sshd applies everything after a Match to the matched connections only.
Match User {{ git_user }}
AuthorizedKeysFile none
AuthorizedKeysCommand {{ authorized_keys_command }}
AuthorizedKeysCommandUser {{ podman_user }}
gitea/roles/ssh/templates/sudoers-gitea.j2
+17
View File
@@ -0,0 +1,17 @@
# Let Gitea's `{{ git_user }}` login user run exactly one command as
# {{ podman_user }}: the wrapper that execs into Gitea's rootless container.
# Managed by Ansible (gitea/roles/ssh). Do not edit by hand.
#
# This is what replaces the ssh hop the Docker-era deployment used. A rootless
# container belongs to one user and cannot be entered by another, and no sshd
# runs inside the container any more, so the forced command sshd runs for a git
# client has to become `podman exec` — run as {{ podman_user }}.
#
# No arguments are pinned, on purpose. The command line comes from the
# authorized_keys lines Gitea maintains (… serv key-N), and that file is already
# the thing deciding who may reach which repository; restating its format here
# would only add a second place to update whenever Gitea changes it, and pinning
# it loosely enough to be useful (… serv *) would not restrict anything the
# wrapper does not already fix — container, uid and environment are all baked
# into {{ podman_exec_wrapper }}.
{{ git_user }} ALL=({{ podman_user }}) NOPASSWD: {{ podman_exec_wrapper }}
gitea/vars/default.yml
+111 -11
View File
@@ -1,18 +1,118 @@
---
http_host: "git.tanshu.com"
http_conf: "git.tanshu.com.conf"
http_url: "https://git.tanshu.com"
# ---------------------------------------------------------------------------
# Rootless Podman / Quadlet
# ---------------------------------------------------------------------------
# Duplicated on purpose from caddy/vars/default.yml: a play cannot include that
# file wholesale, because it also sets `host_directory` — to the Caddy data
# directory — and every playbook defines that for itself. Keep these three in
# sync with caddy/vars/default.yml. (Its fourth, podman_unprivileged_port_start,
# belongs to the podman tasks, which this play does not run: Gitea publishes
# nothing host-side.)
#
# podman_user is derived from the connection: this play carries no become at
# play level, so the Quadlet units belong to whoever Ansible connects as.
# Requires gather_facts (set in playbook.yml).
podman_user: "{{ ansible_facts['user_id'] }}"
podman_data_dir: "{{ ansible_facts['user_dir'] }}/podman-data"
quadlet_dir: "{{ ansible_facts['user_dir'] }}/.config/containers/systemd"
# ---------------------------------------------------------------------------
# Gitea
# ---------------------------------------------------------------------------
host: git.tanshu.com
container_name: gitea
container_image: gitea/gitea:latest
unit_name: "{{ container_name }}.service"
docker_network: "{{ container_name }}_net"
caddy_container: caddy
# Fully qualified on purpose: rootless Podman resolves short names against
# unqualified-search-registries, and there is no TTY to answer a prompt when
# the systemd unit starts.
image: docker.io/gitea/gitea:latest
# The Quadlet file is <network_name>.network. caddy/roles/register pins the
# Podman network name to <network_name>_net (so: gitea.network, gitea_net).
network_name: "{{ container_name }}"
host_directory: "{{ podman_data_dir }}/{{ container_name }}"
# Port inside the container that Caddy proxies to — app.ini's HTTP_PORT. Not
# published on the host: Caddy reaches it by container name.
container_port: 3000
# The uid/gid of the `git` account the image creates for itself
# (git:x:1003:1003, verified in the image), and therefore the uid Gitea runs as.
# The number is load-bearing three times over:
#
# * UserNS in the Quadlet unit maps it onto podman_user, which is what keeps
# every file Gitea writes owned by podman_user on the host — plain tar and
# rsync backups, no subuids;
# * USER_UID/USER_GID force the entrypoint to keep it, so the file layout and
# the mapping cannot drift apart;
# * roles/ssh execs into the container as exactly this uid.
#
# Change one and the other two have to change with it.
gitea_uid: 1003
gitea_gid: 1003
# Where the image keeps its data *inside* the container, and therefore the
# GITEA_CUSTOM the image's own environment already sets.
gitea_custom: /data/gitea
# Gitea's own unauthenticated health endpoint. The image ships no HEALTHCHECK,
# so the unit defines one from this and the play gates on it.
health_cmd: "curl -fsS -o /dev/null http://127.0.0.1:{{ container_port }}/api/healthz"
# Only log timestamps are affected: Gitea stores each user's own timezone. The
# Docker-era container bind-mounted the host's /etc/localtime and /etc/timezone
# instead; TZ is the same intent without depending on either file existing.
timezone: Asia/Kolkata
# A scratch directory for the sshd *inside* the image, which this deployment
# does not use but which still has to write host keys somewhere. It is kept
# outside the data tree on purpose: the process that runs it is the container's
# root, which under keep-id is a subuid, and a subuid-owned file inside
# {{ host_directory }} would break both a plain tar backup and the play's
# "nothing in here belongs to another user" check. See readme.md.
container_sshd_hostkeys: "{{ podman_data_dir }}/{{ container_name }}-sshd"
# ---------------------------------------------------------------------------
# The git-over-SSH entry point on the host
# ---------------------------------------------------------------------------
# Clients keep using git@git.tanshu.com:user/repo.git on port 22, served by the
# *host's* sshd, exactly as they did on monoco. No sshd runs inside the
# container: under keep-id the container's root is a subuid that cannot read the
# authorized_keys Gitea writes as podman_user, so the second hop is a `podman
# exec` instead of an ssh. See readme.md, "How git over SSH works now".
#
# `git_home` only has to exist — sshd runs the forced command from it — and its
# uid is deliberately left to the system: nothing in this design pairs the host
# account with the container's uid. See roles/ssh/tasks/main.yaml.
git_user: git
git_home: /home/git
# The command Gitea writes into every authorized_keys line it maintains. This is
# a contract, not a knob: it is already baked into the lines sitting in the file
# copied from monoco, and renaming it locks every user out of git-over-SSH until
# Gitea rewrites them. See roles/ssh/templates/gitea.j2.
shim_path: /usr/local/bin/gitea
# Runs as podman_user, via the sudoers drop-in, and does the podman exec.
podman_exec_wrapper: /usr/local/bin/gitea-container-exec
# What sshd calls to obtain the git user's authorized_keys.
authorized_keys_command: /usr/local/bin/gitea-authorized-keys
sshd_dropin_path: /etc/ssh/sshd_config.d/60-gitea.conf
sshd_service: ssh
sudoers_path: /etc/sudoers.d/gitea-ssh-passthrough
# ---------------------------------------------------------------------------
# The shared Caddy unit and its Caddyfile, both owned by caddy/playbook.yml
# ---------------------------------------------------------------------------
caddy_name: caddy
caddyfile_path: "{{ podman_data_dir }}/{{ caddy_name }}/conf/Caddyfile"
caddyfile_path: /var/lib/caddy/conf/Caddyfile
snippet_block: |
{{ http_host }} {
reverse_proxy {{ container_name }}:3000
{{ host }} {
reverse_proxy {{ container_name }}:{{ container_port }}
}
user: 'git'