Initial VPS operations handbook

2026-08-03 12:26:42 +08:00
commit b73125e5bc
97 changed files with 3641 additions and 0 deletions
@@ -0,0 +1,86 @@
+# PowerDNS health (hk2)
+
+Read-only checks for the `/opt/pdns` stack on **hk2.chans.xyz** (`ns1.wsvc.info`).
+
+Facts: [hosts/hk2.chans.xyz.md](../hosts/hk2.chans.xyz.md) · Upstream: [docs/pdns-upstream.md](../docs/pdns-upstream.md)
+
+## Quick remote
+
+```bash
+ssh -4 windy@hk2.chans.xyz 'cd /opt/pdns && docker compose ps -a'
+```
+
+Expect `auth`, `db`, `poweradmin` healthy; `backup` Up; `pgweb` Up. Ignore stopped orphan `powerdns-admin` unless cleaning orphans.
+
+## Version / security poll
+
+Expect Authoritative **5.0.6** (or newer). Startup logs should **not** show a Mandatory `Security Update` banner. If they do, check https://doc.powerdns.com/authoritative/security-advisories/ and bump the `powerdns/pdns-auth-50` image pin in `/opt/pdns/compose.yml`.
+
+```bash
+ssh -4 windy@hk2.chans.xyz 'docker logs pdns-auth 2>&1 | grep -i "Security Update" | tail -5'
+ssh -4 windy@hk2.chans.xyz 'docker inspect pdns-auth --format "{{.Config.Image}}"'
+```
+
+## DNS (from laptop / WSL)
+
+Prefer querying auth/secondary IPs directly (local stub / public resolvers may be flaky or cached):
+
+```bash
+NS1=154.36.174.161
+NS2=202.91.35.141
+for z in windy.me wsvc.info chans.xyz; do
+  echo "== $z =="
+  dig @$NS1 +short NS "$z"
+  dig @$NS1 +short SOA "$z"
+  dig @$NS2 +short SOA "$z"
+done
+dig @$NS1 +short A ns1.wsvc.info   # expect 154.36.174.161
+dig @$NS1 +short A ns2.wsvc.info   # expect 202.91.35.141
+```
+
+Served SOA serial can differ from the backend/API serial when `SOA-EDIT=INCEPTION-INCREMENT` is active — compare **ns1 vs ns2 served SOA**, not dig vs `pdnsutil`/API.
+
+## API (on host)
+
+```bash
+ssh -4 windy@hk2.chans.xyz 'cd /opt/pdns && docker compose exec -T auth \
+  python3 -c "import json,os,urllib.request; print(json.load(urllib.request.urlopen(urllib.request.Request(\"http://127.0.0.1:8081/api/v1/servers/localhost\", headers={\"X-API-Key\": os.environ[\"PDNS_API_KEY\"]})))[\"version\")"'
+```
+
+Or use host loopback with the key from `.env` (do not paste the key into chat/logs).
+
+## Web UIs
+
+```bash
+# If local stub DNS is flaky, pin SNI to the auth IP:
+curl -4 -sS -o /dev/null -w "%{http_code}\n" --resolve pdns.wsvc.info:443:154.36.174.161 https://pdns.wsvc.info/   # 302 → login
+curl -4 -sS -o /dev/null -w "%{http_code}\n" --resolve pgweb.wsvc.info:443:154.36.174.161 https://pgweb.wsvc.info/  # 401 without basic auth
+```
+
+## Backup
+
+```bash
+ssh -4 windy@hk2.chans.xyz 'cd /opt/pdns && ./scripts/manual-backup.sh && ls -lt backup/*.sql.gz | head'
+ssh -4 windy@hk2.chans.xyz 'tail -30 /opt/pdns/backup/backup.log'
+```
+
+Scheduler should log `scheduler started … (daily 03:00)`. If you see `can't execute 'bash'` or Alpine `apk` errors, the backup image/entrypoint was reverted — see [hosts/hk2.chans.xyz.md](../hosts/hk2.chans.xyz.md).
+
+## Config sanity (docs-aligned)
+
+```bash
+ssh -4 windy@hk2.chans.xyz 'grep -E "^(primary|also-notify|only-notify|allow-axfr-ips|gpgsql-dnssec|default-soa-edit)" /opt/pdns/auth/pdns.conf'
+```
+
+Expect: `primary=yes`, `also-notify=202.91.35.141`, `only-notify=` empty, `gpgsql-dnssec=yes`, SOA-EDIT `INCEPTION-INCREMENT`.
+
+## Scheduled local check
+
+The sanitized Ansible health profile is `pdns` (`ansible/playbooks/healthchecks.yml`). It runs locally through `vps-healthcheck.timer`, writes a sanitized JSON result to `/var/lib/vps-health/latest.json`, and uses the API key only inside the PowerDNS container. It does not modify PowerDNS, DNS records, or secrets.
+
+
+## After config changes
+
+- `auth/pdns.conf`, `auth/templates.d/secrets.j2`, or auth-related `.env` → `docker compose up -d --force-recreate auth`
+- Poweradmin env / Traefik labels → `docker compose up -d --no-deps --force-recreate poweradmin` (restart Traefik if UI 404s)
+- backup schedule / retention env → `docker compose up -d --force-recreate --no-deps backup`