- RUNBOOKS.md: repo-level spec (six-field model, naming, safety, maturity path) - runbooks/_template.md + README.md: standard template and 16-entry routing index - new: issue-to-merge, fix-ci, release, rollback, network-change, network-recovery - light-enhance 10 existing runbooks with Purpose/Scope/Safety headers - AGENTS.md: point step 3 at index/spec, add runbook execution rules - docs/agent-runbook-guide.md: archive of Manus AI guide
1.8 KiB
1.8 KiB
Runbook: roll back a change
Purpose
Restore a known-good state after a change failed verification or caused a regression.
Scope
- Applicable: Compose services, DNS records, firewall rules, and Ansible-managed config that have a rollback point.
- Not applicable: destructive migrations without a backup (route to the owning runbook, e.g.
vaultwarden-sqlite-to-postgres.md§Rollback).
Preconditions
- A rollback point exists: backup, prior config, or known-good commit/playbook state.
- The failing change and its symptoms are recorded.
Safety
- Confirm the rollback target before acting; do not roll back unrelated changes.
- Verify after rollback; a rollback that is not verified is not complete.
Procedure
Step 1 — Confirm the target
Action — record the change to revert (playbook run, config edit, DNS change) and its intended prior state.
Expected — the exact prior state is known and reachable.
Decision — prior state known → Step 2; unknown → STOP and escalate.
Step 2 — Revert
Action — apply the documented reversal for the change type:
- Compose config: re-apply the prior config and reconcile (or
git revertthe change then reconcile). - DNS: restore the previous record value on PowerDNS.
- Firewall: re-apply the prior allowlist via the gated playbook.
- General repo change:
git revert <sha>.
Expected — the prior configuration is in place.
Verification — re-run the relevant health report / endpoint check and confirm green.
Rollback — if the rollback itself fails, STOP; keep the original management session open and escalate.
Final Verification
- Health report green after rollback.
- The incident and rollback are recorded in the Linear
vpsissue.