Initial VPS operations handbook

2026-08-03 12:26:42 +08:00
commit b73125e5bc
97 changed files with 3641 additions and 0 deletions
@@ -0,0 +1,100 @@
+# synapse.chans.xyz — Matrix homeserver (ESS)
+
+## Role
+
+Matrix homeserver running **Element Server Suite (ESS) Community**: Synapse + Matrix Authentication Service (MAS) + Element Web + Element Admin.
+
+## SSH
+
+```
+ssh -4 windy@synapse.chans.xyz
+```
+
+- User: `windy` (sudo via Ansible)
+- IPv4: `169.58.86.13`
+- OS: Ubuntu 26.04
+
+## Stack
+
+| Layer | Detail |
+|-------|--------|
+| Runtime | K3s v1.36.2+k3s1 (single-node) |
+| Ingress | K3s built-in Traefik (hostPort `:80` / `:443`) |
+| TLS | cert-manager v1.19.3 + `letsencrypt-prod` ClusterIssuer (HTTP-01) |
+| DB | ESS embedded PostgreSQL 17 (PVC 20Gi, local-path) |
+| Cache | ESS embedded Redis (PVC 2Gi) |
+| Chart | `oci://ghcr.io/element-hq/ess-helm/matrix-stack`, version `26.7.2` |
+
+### Matrix service endpoints
+
+| URL | Component |
+|-----|-----------|
+| https://chat.chans.xyz | Element Web |
+| https://synapse.chans.xyz | Synapse Client-Server / Federation API |
+| https://account.chans.xyz | Matrix Authentication Service |
+| https://admin.chans.xyz | Element Admin console |
+| https://chans.xyz/.well-known/matrix/server | Federation discovery → `synapse.chans.xyz:443` |
+| https://chans.xyz/.well-known/matrix/client | Client discovery → `https://synapse.chans.xyz` |
+| `mrtc.chans.xyz` | Reserved for MatrixRTC (not deployed) |
+
+## Public ports
+
+| Port | Protocol | Service |
+|------|----------|---------|
+| 22/tcp | SSH | OpenSSH |
+| 80/tcp | HTTP | Let's Encrypt HTTP-01 / Traefik redirect |
+| 443/tcp | HTTPS | Matrix Client-Server, Federation, Element, MAS, Admin |
+
+All other ports internal only (no K3s API, no database, no Redis exposed).
+
+## Namespace layout
+
+- `ess` — all ESS workloads (Synapse, MAS, Element, Postgres, Redis, HAProxy)
+- `matrix-system` — cluster base resources (ResourceQuota, LimitRange, mrtc-placeholder)
+- `cert-manager` — cert-manager
+
+## Local backup
+
+| Item | Detail |
+|------|--------|
+| Path | `/var/backups/matrix` (root:root, 0700) |
+| Schedule | **Paused** — `matrix-backup.timer` is disabled pending a reviewed backup design |
+| Retention | 7 days |
+| Disk warning | 80% (healthcheck), 90% (backup stops) |
+| Content | Planned: PostgreSQL `synapse` + `mas` logical dumps, media store archive, `/etc/matrix-bootstrap` |
+| Status | **Not operational** — no current Matrix backup or recovery tier |
+
+## Health checks
+
+Daily via `matrix-k3s-healthcheck.timer`, outputs to `/var/lib/vps-health/latest.json`. Checks:
+
+- K3s API / namespace health
+- ESS pod restart counts
+- Backup freshness
+- Root disk usage
+- Certificate expiry
+- External HTTPS endpoints
+
+## Bootstrap secrets
+
+Path: `/etc/matrix-bootstrap/` (root:root, 0700) — individual files per secret key (0600). SMTP password, DB passwords, signing key etc. Included in local backup.
+
+## Identity
+
+- **Server name**: `chans.xyz` (user IDs: `@user:chans.xyz`)
+- **Deployment type**: New instance (no recovery from previous homeserver)
+- **Public registration**: Disabled
+- **Authentication**: MAS local passwords
+
+## Ansible
+
+Inventory group: `matrix_production` (but NOT in `managed` which expects Docker Compose).
+
+Playbooks in `ansible/playbooks/matrix-*.yml`.
+
+## Known limitations
+
+- MatrixRTC / Element Call / LiveKit / Coturn not deployed (`mrtc.chans.xyz` reserved only)
+- SMTP email not yet configured (requires manual bootstrap + Helm upgrade)
+- No off-site Restic backup
+- Single-node K3s (no HA for control plane)