74 lines
2.4 KiB
Markdown
74 lines
2.4 KiB
Markdown
|
# Runbook: release a reviewed change to production
|
||
|
|
|
||
|
|
## Purpose
|
||
|
|
|
||
|
|
Apply a reviewed configuration change to production hosts in a controlled,
|
||
|
|
verifiable, rollback-safe way.
|
||
|
|
|
||
|
|
## Scope
|
||
|
|
|
||
|
|
- Applicable: reviewed changes to Compose services, DNS, firewall, or Ansible-managed config on production hosts.
|
||
|
|
- Not applicable: destructive migrations (`vaultwarden-sqlite-to-postgres.md`), SSH access-policy changes (`AGENTS.md` §SSH access safety), or network changes needing a change ticket (`network-change.md`).
|
||
|
|
|
||
|
|
## Preconditions
|
||
|
|
|
||
|
|
- The change is reviewed and its intent matches a Linear issue / change record.
|
||
|
|
- Read-only health pass (or `maintenance-preview`) has been run and is green.
|
||
|
|
- A rollback point exists (backup, prior config, or known-good git/playbook state).
|
||
|
|
|
||
|
|
## Safety
|
||
|
|
|
||
|
|
- Never release without a verification and a rollback point.
|
||
|
|
- Never bypass gated confirmation variables.
|
||
|
|
- One host at a time; re-run health after each.
|
||
|
|
|
||
|
|
## Procedure
|
||
|
|
|
||
|
|
### Step 1 — Pre-flight
|
||
|
|
|
||
|
|
**Action** — run the read-only health report for the target host(s).
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cd ansible && ansible-playbook playbooks/health-report.yml --limit <host>
|
||
|
|
```
|
||
|
|
|
||
|
|
**Expected** — green baseline.
|
||
|
|
|
||
|
|
**Decision** — green → Step 2; warning/critical → fix first (`fix-ci.md`), `STOP` the release.
|
||
|
|
|
||
|
|
### Step 2 — Approve
|
||
|
|
|
||
|
|
**Action** — confirm the change is approved in the issue/change record; confirm the window.
|
||
|
|
|
||
|
|
**Decision** — approval recorded → Step 3; missing → `STOP`.
|
||
|
|
|
||
|
|
### Step 3 — Apply
|
||
|
|
|
||
|
|
**Action** — run the gated playbook for the change, with its confirmation variable.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# example: reviewed Compose reconcile
|
||
|
|
cd ansible && ansible-playbook playbooks/compose-reconcile.yml --limit <host> \
|
||
|
|
-e '{"service_reconcile_confirm": true, "service_reconcile_targets": ["<svc>"]}'
|
||
|
|
```
|
||
|
|
|
||
|
|
**Expected** — playbook completes; service reports the new state.
|
||
|
|
|
||
|
|
**Verification** — re-run the health report and confirm green; confirm the specific change took effect (version, config, endpoint).
|
||
|
|
|
||
|
|
**Rollback** — on verification failure, revert the change and re-verify (see `rollback.md`).
|
||
|
|
|
||
|
|
## Final Verification
|
||
|
|
|
||
|
|
- Health report green post-release.
|
||
|
|
- The change is visible and the issue/change record is updated.
|
||
|
|
|
||
|
|
## Failure Handling
|
||
|
|
|
||
|
|
On failure: stop further changes, collect the playbook output and health report, and route to `rollback.md` if the change was applied.
|
||
|
|
|
||
|
|
## References
|
||
|
|
|
||
|
|
- [`ansible-operations.md`](ansible-operations.md)
|
||
|
|
- [`rollback.md`](rollback.md)
|