106 lines
4.4 KiB
Markdown
106 lines
4.4 KiB
Markdown
# Runbook: Vaultwarden health (us2)
|
|
|
|
Target: [us2.wsvc.info](../hosts/us2.wsvc.info.md)
|
|
Path: `/opt/vaultwarden`
|
|
URL: https://auth.wsvc.info/
|
|
Upstream: [docs/vaultwarden-upstream.md](../docs/vaultwarden-upstream.md)
|
|
|
|
Prefer the Ansible health report (`ansible/playbooks/health-report.yml` with
|
|
`--limit vaultwarden`) for the routine check. The SSH commands below are
|
|
focused diagnostics for an already identified problem.
|
|
|
|
## Focused SSH diagnostics
|
|
|
|
Use these only after the Ansible health report needs investigation.
|
|
|
|
### 1. Containers
|
|
|
|
```bash
|
|
ssh -4 windy@us2.wsvc.info 'cd /opt/vaultwarden && docker compose ps -a'
|
|
```
|
|
|
|
Expect `vaultwarden` + `vw-db` **Up (healthy)**. `pgweb` should be stopped unless started with `--profile debug`.
|
|
|
|
### 2. Live DB is Postgres
|
|
|
|
```bash
|
|
ssh -4 windy@us2.wsvc.info 'cd /opt/vaultwarden && docker compose exec -T vaultwarden sh -c "printenv DATABASE_URL" | sed -E "s#://[^@]+@#://REDACTED@#"'
|
|
ssh -4 windy@us2.wsvc.info 'cd /opt/vaultwarden && docker compose exec -T pg psql -U vaultwarden -d vaultwarden -c "SELECT COUNT(*) AS users FROM users; SELECT COUNT(*) AS ciphers FROM ciphers;"'
|
|
```
|
|
|
|
### 3. Effective config (config.json wins)
|
|
|
|
```bash
|
|
ssh -4 windy@us2.wsvc.info 'cd /opt/vaultwarden && docker compose exec -T vaultwarden cat /data/config.json' \
|
|
| python3 -c "import sys,json;d=json.load(sys.stdin);print({k:d.get(k) for k in ['domain','smtp_host','smtp_port','smtp_security','smtp_username','smtp_from','ip_header','signups_allowed']});print('smtp_password_set',bool(d.get('smtp_password')))"
|
|
```
|
|
|
|
Expect SMTP **mx2.windy.me** / **587** / **starttls**, `ip_header` **X-Forwarded-For**.
|
|
|
|
Also confirm `.env` and `config.json` SMTP passwords match (lengths/hashes only — never print secrets):
|
|
|
|
```bash
|
|
ssh -4 windy@us2.wsvc.info 'cd /opt/vaultwarden && python3 - << "EOF"
|
|
import json, re, pathlib, hashlib, subprocess
|
|
env = pathlib.Path(".env").read_text()
|
|
def grab(k):
|
|
m = re.search(rf"^{re.escape(k)}=(.*)$", env, re.M)
|
|
return m.group(1).strip().strip("\"'\''") if m else None
|
|
cfg = json.loads(subprocess.check_output(
|
|
["docker","compose","exec","-T","vaultwarden","cat","/data/config.json"]))
|
|
ep, cp = grab("SMTP_PASSWORD"), cfg.get("smtp_password") or ""
|
|
def fp(s): return {"len": len(s), "sha256_8": hashlib.sha256(s.encode()).hexdigest()[:8]}
|
|
print("env_pass", fp(ep or ""), "cfg_pass", fp(cp), "match", ep == cp)
|
|
EOF'
|
|
```
|
|
|
|
If they diverge, **auth uses `config.json`** — sync password there (or via `/admin`) and update `.smtp-credentials`.
|
|
|
|
### 4. External HTTPS
|
|
|
|
```bash
|
|
curl -4 -sS -I --max-time 15 https://auth.wsvc.info/ | head -15
|
|
```
|
|
|
|
If **404** right after recreating VW: use the Ansible Compose reconcile playbook
|
|
with `service_reconcile_restart_traefik=true`, then retry.
|
|
|
|
### 5. SMTP reachability + AUTH
|
|
|
|
TCP (expect `587:0`; `465` often times out from us2):
|
|
|
|
```bash
|
|
ssh -4 windy@us2.wsvc.info 'docker compose -f /opt/vaultwarden/docker-compose.yml exec -T vaultwarden sh -c "timeout 5 bash -c \"cat </dev/null >/dev/tcp/mx2.windy.me/587\"; echo 587:\$?"'
|
|
```
|
|
|
|
AUTH with the **effective** `config.json` password (run on us2; do not print the password):
|
|
|
|
```bash
|
|
ssh -4 windy@us2.wsvc.info 'cd /opt/vaultwarden && python3 - << "EOF"
|
|
import json, smtplib, ssl, subprocess
|
|
cfg = json.loads(subprocess.check_output(
|
|
["docker","compose","exec","-T","vaultwarden","cat","/data/config.json"]))
|
|
host, port = cfg["smtp_host"], int(cfg["smtp_port"])
|
|
user, pw = cfg["smtp_username"], cfg["smtp_password"]
|
|
ctx = ssl.create_default_context()
|
|
with smtplib.SMTP(host, port, timeout=15) as s:
|
|
s.ehlo(); s.starttls(context=ctx); s.ehlo(); s.login(user, pw)
|
|
print("AUTH_OK", host, port, user)
|
|
EOF'
|
|
```
|
|
|
|
Expect `AUTH_OK`. `535` usually means stale password in `config.json` (see step 3).
|
|
|
|
## Scheduled local check
|
|
|
|
The sanitized Ansible health profile is `vaultwarden` (`ansible/playbooks/healthchecks.yml`). It runs locally through `vps-healthcheck.timer`, writes a sanitized JSON result to `/var/lib/vps-health/latest.json`, and uses only server-side credentials for the SMTP AUTH probe. It does not modify Vaultwarden, Traefik, SMTP, or secrets.
|
|
|
|
|
|
## Pass criteria
|
|
|
|
- Healthy compose; `DATABASE_URL` points at `pg`
|
|
- HTTPS 200 + `/admin` 200; PG counts sane
|
|
- Effective SMTP 587/starttls; `.env` ↔ `config.json` password match; **AUTH_OK**
|
|
- Wrong-password login returns 400 business error (not 500)
|
|
- Update **Verified** on [hosts/us2.wsvc.info.md](../hosts/us2.wsvc.info.md)
|