vps/runbooks/mailcow-health.md

# Runbook: mailcow health (mx2)

Target: [mx2.windy.me](../hosts/mx2.windy.me.md)  
Path: `/opt/mail`  
Prefer: the Ansible health report (`ansible/playbooks/health-report.yml`), or
`./scripts/check-mx2.sh` from this repo. The SSH commands below remain useful
for focused, interactive diagnosis after the report identifies a problem.

## Focused SSH diagnostics

Use these only after the Ansible health report needs investigation.

### 1. Containers

```bash
ssh windy@mx2.windy.me 'cd /opt/mail && docker compose ps -a'
```

Expect mailcow (`cow-*`) services **Up**. Note any Exit/Restarting.

### 2. Watchdog

```bash
ssh windy@mx2.windy.me 'cd /opt/mail && docker compose logs --tail=40 watchdog-mailcow'
```

Expect recent lines at **100%** for Nginx, Postfix, Dovecot, MySQL, Redis, SOGo, Rspamd, ACME, etc.

### 3. Queue

```bash
ssh windy@mx2.windy.me 'cd /opt/mail && docker compose exec -T postfix-mailcow postqueue -p'
```

Healthy: empty / `Mail queue is empty`.

### 4. Local listeners

```bash
ssh windy@mx2.windy.me 'ss -tlnp | grep -E ":(25|465|587|143|993|995|80|443|4190)\\s"'
```

## 5. External checks (from laptop / agent host)

```bash
curl -sI --max-time 10 http://mx2.windy.me/ | head -5
curl -skI --max-time 10 https://mx2.windy.me/ | head -15
echo | openssl s_client -connect mx2.windy.me:443 -servername mx2.windy.me 2>/dev/null \
  | openssl x509 -noout -subject -issuer -dates
timeout 8 bash -c 'exec 3<>/dev/tcp/mx2.windy.me/25; echo -e "EHLO test.local\r\nQUIT\r\n" >&3; cat <&3' | head -20
```

Expect: HTTP 301 → HTTPS; HTTPS 200; LE cert for `mx2.windy.me`; SMTP banner `Postcow`.

## 6. DNS summary

Use a public resolver if the local stub is flaky (WSL `127.0.0.53`):

```bash
dig @1.1.1.1 +short mx2.windy.me A
dig @1.1.1.1 +short mx2.windy.me AAAA
dig @1.1.1.1 +short -x 194.163.160.244
dig @1.1.1.1 +short windy.me MX
dig @1.1.1.1 +short windy.me TXT | grep -i spf
```

## Scheduled local check

The sanitized Ansible health profile is `mailcow` (`ansible/playbooks/healthchecks.yml`). It preserves the existing standalone `scripts/check-mx2.sh` as the richer operator check, while the server-local timer emits a sanitized result at `/var/lib/vps-health/latest.json`. Neither check changes Mailcow configuration or service state.

## Pass criteria

- Compose stack up; watchdog ~100%
- Queue empty (or explain backlog)
- TLS valid; mail ports reachable externally
- A/AAAA/PTR/MX consistent with [host facts](../hosts/mx2.windy.me.md)

Update the **Verified** line on the host page after a successful check.