playbooks/rustdesk

RustDesk Server on Docker, behind Caddy

Deploys the self-hosted RustDesk server (OSS) on monoco: the ID/rendezvous server hbbs and the relay server hbbr, both supervised by s6 inside the official rustdesk/rustdesk-server-s6 container, reachable only through Caddy on rust.tanshu.com:443. No RustDesk port is published on the host.

How the traffic is carried

RustDesk's native transports on 21115/21116/21117 are a custom TCP/UDP protocol, not HTTP, so an HTTP reverse proxy cannot carry them. What is HTTP is the WebSocket transport that hbbs and hbbr also listen on (21118 and 21119), and clients that enable WebSocket use it for both rendezvous and relay. Caddy terminates TLS on 443 and forwards two paths into the container over the Docker network:

Path	Upstream	Purpose
/ws/id	rustdesk:21118 (hbbs)	ID registration and signalling
/ws/relay	rustdesk:21119 (hbbr)	Relayed sessions

The code that decides this is in the client: hbb_common::websocket::check_ws maps a domain host:21116 to wss://host/ws/id and host:21117 to wss://host/ws/relay, and the path names are fixed there. Caddy sets X-Real-IP, which hbbs/hbbr trust to keep the real client address for rate limiting and logs (upstream warns that these ports must therefore never be exposed directly).

Settings

• host: the public name, rust.tanshu.com. Both the Caddy site and the address hbbs advertises come from it.
• docker_container, image_name, image_version: container name and the rustdesk/rustdesk-server-s6 tag to run.
• data_location: host directory bind-mounted to /data (/var/lib/rustdesk/data), holding the SQLite database and the server key pair.
• rustdesk_relay_server: handed to clients by hbbs as the relay address. It keeps the public name, so clients rewrite it to wss://rust.tanshu.com/ws/relay.
• rustdesk_publish_raw_ports / rustdesk_raw_ports: publish 21115/21116/21117 as well. Off by default, see "Clients that cannot use WebSocket" below.
• rustdesk_encrypted_only: ENCRYPTED_ONLY=1, so hbbr validates the server key too instead of accepting unkeyed clients.
• rustdesk_always_use_relay: ALWAYS_USE_RELAY=Y, forcing relayed sessions. WebSocket clients are relay-only anyway, so this only affects clients on the raw ports.

Prerequisites

• A DNS A record for rust.tanshu.com pointing at monoco; Caddy issues the certificate over ports 80/443.
• The Caddy container already running on monoco (caddy/playbook.yml), with its Caddyfile at /var/lib/caddy/conf/Caddyfile.

Running this Playbook

The inventory in /etc/ansible/hosts already lists monoco, which is what hosts: targets:

ansible-playbook playbook.yml

The playbook creates the rustdesk_net network, attaches Caddy to it, starts the container, appends the site block to the Caddyfile, reloads Caddy, and prints the public key.

Configuring the clients

On each client, open Settings → Network → ID/Relay Server and set:

Field	Value
ID Server	rust.tanshu.com
API Server	https://rust.tanshu.com
Key	the printed value, i.e. /var/lib/rustdesk/data/id_ed25519.pub on the host

then enable Settings → Network → Use WebSocket (allow-websocket=Y).

The API Server field looks odd for an OSS deployment, which has no API, but it is what selects the TLS scheme: the client uses wss:// only when the API server starts with https, and otherwise silently falls back to plaintext ws://rust.tanshu.com/... on port 80. Because the client also skips its own client↔server key exchange whenever WebSocket is enabled — on the assumption that the transport is already encrypted — that fallback would leave the connection unprotected rather than merely failing.

Trade-offs

• Relayed sessions only. With WebSocket enabled the client never hole-punches (force_relay is set, and hbbs skips the same-intranet path for WebSocket peers), so every session flows through monoco. Remote control works, but throughput is bounded by the server, and latency is higher than a direct connection.
• Upstream labels this a Pro feature. The client's own tooltip says "RustDesk server OSS doesn't include this feature", even though the OSS binaries do listen on 21118/21119 and the OSS docs describe proxying them. Treat the WSS path as needing a real test: if clients fail to register, the fallback is the raw ports below.
• The address has to be the domain. check_ws only builds a TLS URL and the /ws paths for hostnames; an IP would become ws://ip:21118 and bypass Caddy entirely.

Clients that cannot use WebSocket

Clients without the WebSocket option (older builds, third-party clients, or anything that must connect directly) need the native ports. Set rustdesk_publish_raw_ports: true and re-run the playbook, then open 21115-21116 (TCP) plus 21116/udp and 21117 (TCP) on the host and in any cloud firewall:

ufw allow 21115:21117/tcp
ufw allow 21116/udp

WebSocket clients keep working at the same time; they ignore the raw ports.

Notes

• Logs. docker logs rustdesk covers both services.
• Server key. If /var/lib/rustdesk/data/id_ed25519 is lost, every client has to be reconfigured with the new public key. Back it up.
• Upgrading. Re-running the playbook pulls the tag in image_version and recreates the container; the key pair and database live in data_location and survive.