Files
vps/runbooks/pdns-health.md
T

3.7 KiB

PowerDNS health (hk2)

Read-only checks for the /opt/pdns stack on hk2.chans.xyz (ns1.wsvc.info).

Facts: hosts/hk2.chans.xyz.md · Upstream: docs/pdns-upstream.md

Quick remote

ssh -4 windy@hk2.chans.xyz 'cd /opt/pdns && docker compose ps -a'

Expect auth, db, poweradmin healthy; backup Up; pgweb Up. Ignore stopped orphan powerdns-admin unless cleaning orphans.

Version / security poll

Expect Authoritative 5.0.6 (or newer). Startup logs should not show a Mandatory Security Update banner. If they do, check https://doc.powerdns.com/authoritative/security-advisories/ and bump the powerdns/pdns-auth-50 image pin in /opt/pdns/compose.yml.

ssh -4 windy@hk2.chans.xyz 'docker logs pdns-auth 2>&1 | grep -i "Security Update" | tail -5'
ssh -4 windy@hk2.chans.xyz 'docker inspect pdns-auth --format "{{.Config.Image}}"'

DNS (from laptop / WSL)

Prefer querying auth/secondary IPs directly (local stub / public resolvers may be flaky or cached):

NS1=154.36.174.161
NS2=202.91.35.141
for z in windy.me wsvc.info chans.xyz; do
  echo "== $z =="
  dig @$NS1 +short NS "$z"
  dig @$NS1 +short SOA "$z"
  dig @$NS2 +short SOA "$z"
done
dig @$NS1 +short A ns1.wsvc.info   # expect 154.36.174.161
dig @$NS1 +short A ns2.wsvc.info   # expect 202.91.35.141

Served SOA serial can differ from the backend/API serial when SOA-EDIT=INCEPTION-INCREMENT is active — compare ns1 vs ns2 served SOA, not dig vs pdnsutil/API.

API (on host)

ssh -4 windy@hk2.chans.xyz 'cd /opt/pdns && docker compose exec -T auth \
  python3 -c "import json,os,urllib.request; print(json.load(urllib.request.urlopen(urllib.request.Request(\"http://127.0.0.1:8081/api/v1/servers/localhost\", headers={\"X-API-Key\": os.environ[\"PDNS_API_KEY\"]})))[\"version\")"'

Or use host loopback with the key from .env (do not paste the key into chat/logs).

Web UIs

# If local stub DNS is flaky, pin SNI to the auth IP:
curl -4 -sS -o /dev/null -w "%{http_code}\n" --resolve pdns.wsvc.info:443:154.36.174.161 https://pdns.wsvc.info/   # 302 → login
curl -4 -sS -o /dev/null -w "%{http_code}\n" --resolve pgweb.wsvc.info:443:154.36.174.161 https://pgweb.wsvc.info/  # 401 without basic auth

Backup

ssh -4 windy@hk2.chans.xyz 'cd /opt/pdns && ./scripts/manual-backup.sh && ls -lt backup/*.sql.gz | head'
ssh -4 windy@hk2.chans.xyz 'tail -30 /opt/pdns/backup/backup.log'

Scheduler should log scheduler started … (daily 03:00). If you see can't execute 'bash' or Alpine apk errors, the backup image/entrypoint was reverted — see hosts/hk2.chans.xyz.md.

Config sanity (docs-aligned)

ssh -4 windy@hk2.chans.xyz 'grep -E "^(primary|also-notify|only-notify|allow-axfr-ips|gpgsql-dnssec|default-soa-edit)" /opt/pdns/auth/pdns.conf'

Expect: primary=yes, also-notify=202.91.35.141, only-notify= empty, gpgsql-dnssec=yes, SOA-EDIT INCEPTION-INCREMENT.

Scheduled local check

The sanitized Ansible health profile is pdns (ansible/playbooks/healthchecks.yml). It runs locally through vps-healthcheck.timer, writes a sanitized JSON result to /var/lib/vps-health/latest.json, and uses the API key only inside the PowerDNS container. It does not modify PowerDNS, DNS records, or secrets.

After config changes

  • auth/pdns.conf, auth/templates.d/secrets.j2, or auth-related .envdocker compose up -d --force-recreate auth
  • Poweradmin env / Traefik labels → docker compose up -d --no-deps --force-recreate poweradmin (restart Traefik if UI 404s)
  • backup schedule / retention env → docker compose up -d --force-recreate --no-deps backup