windyboyandCursor
343c5db415
Keep sanitized Compose sources in-repo with a confirmation-gated Ansible playbook, add repo-wide validation, tighten runbook ownership/STOP/review metadata, and archive stale research docs. Co-authored-by: Cursor <cursoragent@cursor.com>
102 lines
2.9 KiB
Markdown
102 lines
2.9 KiB
Markdown
# Runbook: mailcow health (mx2)
|
|
|
|
## Purpose
|
|
|
|
Read-only health check of the mailcow stack on mx2.
|
|
|
|
## Scope
|
|
|
|
- Applicable: [mx2.windy.me](../hosts/mx2.windy.me.md), `/opt/mail`.
|
|
- Read-only: does not change mailcow configuration or service state.
|
|
|
|
## Ownership
|
|
|
|
- Owner: personal ops (Windy)
|
|
- Last reviewed: 2026-08-17
|
|
- Related systems: mx2.windy.me (`/opt/mail`)
|
|
|
|
Target: [mx2.windy.me](../hosts/mx2.windy.me.md)
|
|
Path: `/opt/mail`
|
|
Prefer: the Ansible health report (`ansible/playbooks/health-report.yml --limit mailcow`),
|
|
or the server-local daily timer result at `/var/lib/vps-health/latest.json`. The
|
|
SSH commands below remain useful for focused, interactive diagnosis after the
|
|
report identifies a problem.
|
|
|
|
## Focused SSH diagnostics
|
|
|
|
Use these only after the Ansible health report needs investigation.
|
|
|
|
### 1. Containers
|
|
|
|
```bash
|
|
ssh windy@mx2.windy.me 'cd /opt/mail && docker compose ps -a'
|
|
```
|
|
|
|
Expect mailcow (`cow-*`) services **Up**. Note any Exit/Restarting.
|
|
|
|
### 2. Watchdog
|
|
|
|
```bash
|
|
ssh windy@mx2.windy.me 'cd /opt/mail && docker compose logs --tail=40 watchdog-mailcow'
|
|
```
|
|
|
|
Expect recent lines at **100%** for Nginx, Postfix, Dovecot, MySQL, Redis, SOGo, Rspamd, ACME, etc.
|
|
|
|
### 3. Queue
|
|
|
|
```bash
|
|
ssh windy@mx2.windy.me 'cd /opt/mail && docker compose exec -T postfix-mailcow postqueue -p'
|
|
```
|
|
|
|
Healthy: empty / `Mail queue is empty`.
|
|
|
|
### 4. Local listeners
|
|
|
|
```bash
|
|
ssh windy@mx2.windy.me 'ss -tlnp | grep -E ":(25|465|587|143|993|995|80|443|4190)\\s"'
|
|
```
|
|
|
|
## 5. External checks (from laptop / agent host)
|
|
|
|
```bash
|
|
curl -sI --max-time 10 http://mx2.windy.me/ | head -5
|
|
curl -skI --max-time 10 https://mx2.windy.me/ | head -15
|
|
echo | openssl s_client -connect mx2.windy.me:443 -servername mx2.windy.me 2>/dev/null \
|
|
| openssl x509 -noout -subject -issuer -dates
|
|
timeout 8 bash -c 'exec 3<>/dev/tcp/mx2.windy.me/25; echo -e "EHLO test.local\r\nQUIT\r\n" >&3; cat <&3' | head -20
|
|
```
|
|
|
|
Expect: HTTP 301 → HTTPS; HTTPS 200; LE cert for `mx2.windy.me`; SMTP banner `Postcow`.
|
|
|
|
## 6. DNS summary
|
|
|
|
Use a public resolver if the local stub is flaky (WSL `127.0.0.53`):
|
|
|
|
```bash
|
|
dig @1.1.1.1 +short mx2.windy.me A
|
|
dig @1.1.1.1 +short mx2.windy.me AAAA
|
|
dig @1.1.1.1 +short -x 194.163.160.244
|
|
dig @1.1.1.1 +short windy.me MX
|
|
dig @1.1.1.1 +short windy.me TXT | grep -i spf
|
|
```
|
|
|
|
## Scheduled local check
|
|
|
|
The sanitized Ansible health profile is `mailcow` (`ansible/playbooks/healthchecks.yml`).
|
|
The server-local timer emits a sanitized result at `/var/lib/vps-health/latest.json`.
|
|
It does not change Mailcow configuration or service state.
|
|
|
|
## Safety
|
|
|
|
- Read-only: never mutate configuration or service state during this check.
|
|
- If live state conflicts with an expected value below, `STOP` and report; do not "fix" on the fly.
|
|
|
|
## Pass criteria
|
|
|
|
- Compose stack up; watchdog ~100%
|
|
- Queue empty (or explain backlog)
|
|
- TLS valid; mail ports reachable externally
|
|
- A/AAAA/PTR/MX consistent with [host facts](../hosts/mx2.windy.me.md)
|
|
|
|
Update the **Verified** line on the host page after a successful check.
|