docs(runbooks): add runbook spec, template, index and 6 first-batch runbooks; light-enhance existing 10
- RUNBOOKS.md: repo-level spec (six-field model, naming, safety, maturity path) - runbooks/_template.md + README.md: standard template and 16-entry routing index - new: issue-to-merge, fix-ci, release, rollback, network-change, network-recovery - light-enhance 10 existing runbooks with Purpose/Scope/Safety headers - AGENTS.md: point step 3 at index/spec, add runbook execution rules - docs/agent-runbook-guide.md: archive of Manus AI guide
This commit is contained in:
@@ -0,0 +1,56 @@
|
||||
# Runbook: roll back a change
|
||||
|
||||
## Purpose
|
||||
|
||||
Restore a known-good state after a change failed verification or caused a
|
||||
regression.
|
||||
|
||||
## Scope
|
||||
|
||||
- Applicable: Compose services, DNS records, firewall rules, and Ansible-managed config that have a rollback point.
|
||||
- Not applicable: destructive migrations without a backup (route to the owning runbook, e.g. `vaultwarden-sqlite-to-postgres.md` §Rollback).
|
||||
|
||||
## Preconditions
|
||||
|
||||
- A rollback point exists: backup, prior config, or known-good commit/playbook state.
|
||||
- The failing change and its symptoms are recorded.
|
||||
|
||||
## Safety
|
||||
|
||||
- Confirm the rollback target before acting; do not roll back unrelated changes.
|
||||
- Verify after rollback; a rollback that is not verified is not complete.
|
||||
|
||||
## Procedure
|
||||
|
||||
### Step 1 — Confirm the target
|
||||
|
||||
**Action** — record the change to revert (playbook run, config edit, DNS change) and its intended prior state.
|
||||
|
||||
**Expected** — the exact prior state is known and reachable.
|
||||
|
||||
**Decision** — prior state known → Step 2; unknown → `STOP` and escalate.
|
||||
|
||||
### Step 2 — Revert
|
||||
|
||||
**Action** — apply the documented reversal for the change type:
|
||||
|
||||
- Compose config: re-apply the prior config and reconcile (or `git revert` the change then reconcile).
|
||||
- DNS: restore the previous record value on PowerDNS.
|
||||
- Firewall: re-apply the prior allowlist via the gated playbook.
|
||||
- General repo change: `git revert <sha>`.
|
||||
|
||||
**Expected** — the prior configuration is in place.
|
||||
|
||||
**Verification** — re-run the relevant health report / endpoint check and confirm green.
|
||||
|
||||
**Rollback** — if the rollback itself fails, `STOP`; keep the original management session open and escalate.
|
||||
|
||||
## Final Verification
|
||||
|
||||
- Health report green after rollback.
|
||||
- The incident and rollback are recorded in the Linear `vps` issue.
|
||||
|
||||
## References
|
||||
|
||||
- [`release.md`](release.md)
|
||||
- [`ansible-operations.md`](ansible-operations.md)
|
||||
Reference in New Issue
Block a user