diff --git a/AGENTS.md b/AGENTS.md index 1bb70ca..e970b06 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -76,6 +76,7 @@ Also readable as `agent.md` (symlink → this file). | Matrix health | [runbooks/matrix-health.md](runbooks/matrix-health.md) | | RustDesk health (hk2) | [runbooks/rustdesk-health.md](runbooks/rustdesk-health.md) | | AdGuard Home health | [runbooks/adguard-home-health.md](runbooks/adguard-home-health.md) | +| Home Assistant maintenance | [runbooks/home-assistant-maintenance.md](runbooks/home-assistant-maintenance.md) + [scripts/ha-maintenance.sh](runbooks/scripts/ha-maintenance.sh) | | Matrix upstream refs | [docs/matrix-upstream.md](docs/matrix-upstream.md) | | Hermes Agent Matrix channel | [docs/hermes-matrix.md](docs/hermes-matrix.md) | | UniFi local-service proxy bypass | [docs/unifi-openclash-localhost.md](docs/unifi-openclash-localhost.md) | diff --git a/hosts/hass.windy.lan.md b/hosts/hass.windy.lan.md index 97fbff4..6bfe126 100644 --- a/hosts/hass.windy.lan.md +++ b/hosts/hass.windy.lan.md @@ -13,10 +13,20 @@ | Web UI | `http://hass.windy.lan:8123` (LAN); WAN port-forward `hass` on gw → `:8123` | Use `hassio` for routine SSH inspection. Key-only login was verified on -2026-08-13 from the WSL client (`BatchMode=yes`). The `ha` supervisor CLI is -present (`/usr/bin/ha`) but returns `unauthorized` without a supervisor API -token; use the web UI or SSH shell for read-only checks unless a token is -configured out of band. +2026-08-13 from the WSL client (`BatchMode=yes`). + +The `ha` supervisor CLI (`/usr/bin/ha`) authenticates with `SUPERVISOR_TOKEN`. +Interactive login works because `~hassio/.zprofile` runs `exec sudo -i`, which +loads a root environment carrying the supervisor API token. Non-interactive +`ssh hassio 'command'` does not source `.zprofile` and fails with +`unauthorized: missing or invalid API token`. Run `ha` non-interactively via: + +```bash +ssh -o BatchMode=yes hassio@hass.windy.lan 'sudo -n -i ha core info' +``` + +Verified 2026-08-13 that `sudo -n -i ha core info` works from the WSL client. +Never copy the supervisor token into this repository. The current SSH ED25519 host-key fingerprint is `SHA256:DMcMOgDzFsFTon1fndXowEP7jlyOK3/AX3PVK8BATvk` (verified 2026-08-13). @@ -99,5 +109,6 @@ getent hosts hass.windy.lan ## Related docs +- [runbooks/home-assistant-maintenance.md](../runbooks/home-assistant-maintenance.md) — `ha` CLI maintenance runbook + [script](../runbooks/scripts/ha-maintenance.sh) - [docs/lan-overview.md](../docs/lan-overview.md) — LAN map and gw port-forward - [hosts/dns.windy.lan.md](dns.windy.lan.md) — `hass.windy.lan` / `hass.local` rewrites diff --git a/runbooks/home-assistant-maintenance.md b/runbooks/home-assistant-maintenance.md new file mode 100644 index 0000000..0a111a2 --- /dev/null +++ b/runbooks/home-assistant-maintenance.md @@ -0,0 +1,210 @@ +# Runbook: Home Assistant maintenance (hass.windy.lan) + +Target: [hass.windy.lan](../hosts/hass.windy.lan.md) (HAOS, `machine: green`) +Upstream: HAOS 18.1 / Core 2026.8.x / Supervisor 2026.07.5 (verified 2026-08-13) + +This runbook covers routine Home Assistant maintenance through the **`ha` +supervisor CLI**. All commands are wrapped by a single script +[`scripts/ha-maintenance.sh`](scripts/ha-maintenance.sh); the sections below +document the exact commands it runs, for manual/agent use. + +## Access pattern + +`ha` authenticates to the Supervisor with `SUPERVISOR_TOKEN`. Interactive SSH +login works because `~hassio/.zprofile` runs `exec sudo -i`; the root login +environment carries the token. Non-interactive use must be: + +```bash +ssh -o BatchMode=yes hassio@hass.windy.lan 'sudo -n -i ha ' +``` + +Running `ha` as `hassio` directly (or `sudo -n` without `-i`) returns +`unauthorized: missing or invalid API token`. + +**MOTD:** every `sudo -n -i` login prints the SSH & Web Terminal MOTD banner. +The script runs its whole procedure in one remote login (`sudo -n -i bash -s`) +so the banner appears once, then strips it with `awk` up to the +`System is ready! Use browser or app to configure.` line. + +## Script usage + +```bash +cd runbooks/scripts + +./ha-maintenance.sh # read-only health snapshot +./ha-maintenance.sh --check-config # validate core configuration +./ha-maintenance.sh --logs core 200 # tail core logs +./ha-maintenance.sh --logs supervisor # tail supervisor logs (default 100) +./ha-maintenance.sh --logs host 50 # tail host journald logs +./ha-maintenance.sh --logs apps: # tail an add-on log + +# Mutating — refuse to run without --yes: +./ha-maintenance.sh --update --yes # refresh + update core(--backup)/supervisor/os +./ha-maintenance.sh --restart-core --yes # restart Core +./ha-maintenance.sh --restart-core --safe-mode --yes # restart Core in safe mode +./ha-maintenance.sh --rebuild-core --yes # rebuild Core image (after options change) +./ha-maintenance.sh --rollback-os --yes # boot previous OS slot (A/B rollback) +./ha-maintenance.sh --reboot --yes # reboot the HAOS host +./ha-maintenance.sh --backup [NAME] --yes # full backup (optionally named) +./ha-maintenance.sh --restore --yes # restore a backup (DESTRUCTIVE) +./ha-maintenance.sh --app restart core_mosquitto --yes # add-on lifecycle +``` + +- `--app` action is one of `start|stop|restart|update`; needs an add-on slug. +- `HA_HOST` / `HA_SSH_USER` override the defaults (`hass.windy.lan` / `hassio`). +- `--restore` overwrites the current installation — run only from a planned + recovery, with the backup verified. + +## Command reference (verified 2026-08-13) + +All verified against the live host. MOTD prepends each command's output; strip +with the `awk` pattern above or read the last block. + +### Routine / read-only + +| Purpose | Command | +|---|---| +| General overview | `ha info` | +| Core version/status | `ha core info` | +| Core config validation | `ha core check` | +| Core stats | `ha core stats` | +| Supervisor status | `ha supervisor info` (incl. add-on list) | +| Supervisor stats | `ha supervisor stats` | +| OS status | `ha os info` (boot slots A/B) | +| Host status | `ha host info` (disk free/total, kernel) | +| Network | `ha network info` (`supervisor_internet`) | +| Hardware | `ha hardware info` | +| Pending updates | `ha available-updates` | +| Reload stores/versions | `ha refresh-updates` | +| Job manager | `ha jobs info` | +| Resolution center | `ha resolution info` | +| Core logs | `ha core logs -n 100` (`-f` follow, `-b` boot id) | +| Supervisor logs | `ha supervisor logs -n 100` | +| Host journald logs | `ha host logs -n 100` | +| Add-on logs | `ha apps logs -n 100` | +| Add-on list | `ha supervisor info` → `addons:` (started/stopped/error) | +| Security integrity | `ha security integrity` | + +### Mutating (require --yes) + +| Purpose | Command | +|---|---| +| Update core (with partial backup) | `ha core update --backup` | +| Update supervisor | `ha supervisor update` | +| Update OS | `ha os update` | +| Update add-on | `ha apps update ` | +| Restart core | `ha core restart` / `ha core restart --safe-mode` | +| Rebuild core | `ha core rebuild` | +| OS rollback | `ha os boot-slot other` | +| Reboot host | `ha host reboot` | +| Full backup | `ha backups new [--name NAME]` | +| Restore backup | `ha backups restore ` | +| Add-on start/stop/restart | `ha apps start\|stop\|restart ` | + +## Procedure + +### 1. Health snapshot (read-only) + +```bash +./ha-maintenance.sh +``` + +Review: supervisor `healthy: true`/`supported: true`; core/OS `update_available`; +add-on states (any `state: error`?); `resolution info` issues; disk free. + +### 2. Validate config after any `configuration.yaml` change + +```bash +./ha-maintenance.sh --check-config +``` + +Expect `Command completed successfully.` before a Core restart. + +### 3. Inspect logs + +```bash +./ha-maintenance.sh --logs core 200 +./ha-maintenance.sh --logs supervisor +./ha-maintenance.sh --logs apps:core_mosquitto +``` + +### 4. Apply updates (mutating) + +```bash +./ha-maintenance.sh --update --yes +``` + +Runs `refresh-updates` → `core update --backup` (partial backup first) → +`supervisor update` → `os update`, then re-prints pending updates. Prefer the +web UI (**Settings → System → Updates**) for a human-supervised pass. + +### 5. Recovery operations (mutating, only when needed) + +```bash +./ha-maintenance.sh --restart-core --safe-mode --yes # start Core without custom integrations +./ha-maintenance.sh --rollback-os --yes # OS update broke boot? go back one slot +./ha-maintenance.sh --restore --yes # full restore; overwrites current install +``` + +OS update policy: HAOS uses two boot slots (A/B); `ha os info` shows which slot +booted. After a bad OS update, `ha os boot-slot other` boots the previous slot. + +### 6. Backup before major changes + +```bash +./ha-maintenance.sh --backup pre-migration --yes # named backup +``` + +## Command families intentionally NOT scripted + +These exist in `ha` but are either rare, dangerous, or better done in the web +UI; documented here so nothing is a surprise. Use `ha --help` on the +host for exact syntax. + +| Family | Notes | +|---|---| +| `ha audio` | Audio device management; peripheral. | +| `ha authentication` | `auth list/reset/cache`; user password ops — do in web UI. `auth list` is local-terminal only. | +| `ha cli` | Internal CLI backend info/update; self-maintained. | +| `ha dns` | Internal DNS server; only relevant if Supervisor DNS add-on in use. | +| `ha docker` | Host Docker backend info/options/registries; HAOS-managed. | +| `ha mounts` | Network storage (NFS/CIFS) mounts — configure in **Settings → System → Storage**. | +| `ha multicast` / `ha observer` | Internal services; self-maintained. | +| `ha network scan/update/vlan` | WiFi AP scan & interface config — prefer web UI networking. | +| `ha host disks/options/shutdown/reload` | Disk ops / host options; `shutdown` is equivalent to `--reboot` but off. | +| `ha os datadisk list/move/wipe` | Data-disk migration; `wipe` is **local-terminal only** and erases all data. | +| `ha os import` | Import config from USB stick. | +| `ha os boards` / `os config` | Board / OS settings. | +| `ha core options` / `supervisor options` | Core/OS config options (e.g. `--duplicate-log-file`); changes need `ha core rebuild` + restart. | +| `ha backups freeze/thaw/remove/options` | Freeze/thaw for external backup tools; removal is destructive. | +| `ha jobs options/reset` | Job-manager tuning. | +| `ha resolution check/healthcheck/issue/suggestion` | Resolution center management; `healthcheck` runs fixups. | +| `ha store add/delete/repair` | Repository management — add repos in web UI app store. | +| `ha security info/options` | Security backend options. | + +## Docs vs actual CLI discrepancies + +The [official HAOS common-tasks docs](https://www.home-assistant.io/common-tasks/os/) +reference `ha backups list` and `ha host update`. **Neither exists in the +installed CLI** (2026-08-13): backups are inspected with `ha backups info +` (a slug is required) and `host` has no `update` subcommand. Trust the +server CLI (`ha --help`) over the docs. + +## Known issues on hass.windy.lan (observed 2026-08-13) + +From the health snapshot — follow-ups are optional, no action taken: + +- **2 add-ons in `state: error`**: `core_openthread_border_router`, + `a0d7b954_ssh` (duplicate Advanced SSH & Web Terminal install). +- **`ha resolution info` issues**: `systemd_unit_failed` + (`systemd-vconsole-setup.service`), `no_current_backup`, 2× + `corrupt_repository` (store `d5369777`, `a0d7b954`). +- `host info` reports `disk_life_time: 10` (disk lifetime warning threshold). + +## Pass criteria + +- Health snapshot completes; supervisor `healthy`/`supported: true` +- Mutating modes refuse to run without `--yes` (incl. `--restore`, `--app`) +- `--check-config` returns success +- Update / rollback / restore / reboot confirmed only after explicit `--yes` +- Update the **Verified** line on [hosts/hass.windy.lan.md](../hosts/hass.windy.lan.md) diff --git a/runbooks/scripts/ha-maintenance.sh b/runbooks/scripts/ha-maintenance.sh new file mode 100755 index 0000000..912cb7b --- /dev/null +++ b/runbooks/scripts/ha-maintenance.sh @@ -0,0 +1,180 @@ +#!/usr/bin/env bash +# +# ha-maintenance.sh — Home Assistant maintenance via the `ha` supervisor CLI. +# +# Target : hass.windy.lan (HAOS / Core), SSH user `hassio`. +# Access : non-interactive `sudo -n -i ha `; the root login environment +# carries the supervisor API token (interactive login loads +# ~hassio/.zprofile which runs `exec sudo -i`). The whole remote +# procedure runs in ONE `sudo -n -i bash -s` login so the SSH & Web +# Terminal MOTD is printed only once, then stripped locally. +# See hosts/hass.windy.lan.md and runbooks/home-assistant-maintenance.md. +# +# Usage: +# ./ha-maintenance.sh read-only health snapshot +# ./ha-maintenance.sh --check-config validate core configuration +# ./ha-maintenance.sh --logs core|supervisor|host|apps: [N] +# ./ha-maintenance.sh --update --yes refresh + update core(--backup)/supervisor/os +# ./ha-maintenance.sh --restart-core [--safe-mode] --yes +# ./ha-maintenance.sh --rebuild-core --yes rebuild core image (after options change) +# ./ha-maintenance.sh --rollback-os --yes boot the previous OS slot (A/B rollback) +# ./ha-maintenance.sh --backup [NAME] --yes full backup (optionally named) +# ./ha-maintenance.sh --restore --yes restore a backup (DESTRUCTIVE) +# ./ha-maintenance.sh --app start|stop|restart|update --yes +# +# Environment: HA_HOST (default hass.windy.lan), HA_SSH_USER (default hassio). +# Mutating actions always require --yes; this script never prompts. + +set -uo pipefail + +HOST="${HA_HOST:-hass.windy.lan}" +SSH_USER="${HA_SSH_USER:-hassio}" + +MODE="health" +TARGET="core" +LINES=100 +SAFE_MODE=0 +CONFIRM=0 +ARG1="" +ARG2="" + +while [ $# -gt 0 ]; do + case "$1" in + --check-config) MODE="check-config" ;; + --logs) + MODE="logs" + if [ $# -gt 1 ]; then TARGET="$2"; shift; fi + if [ $# -gt 1 ] && [[ "$2" =~ ^[0-9]+$ ]]; then LINES="$2"; shift; fi + ;; + --update) MODE="update" ;; + --restart-core) MODE="restart-core" ;; + --safe-mode) SAFE_MODE=1 ;; + --rebuild-core) MODE="rebuild-core" ;; + --rollback-os) MODE="rollback-os" ;; + --reboot) MODE="reboot" ;; + --backup) + MODE="backup" + if [ $# -gt 1 ] && [[ "$2" != --* ]]; then ARG1="$2"; shift; fi + ;; + --restore) + MODE="restore" + if [ $# -gt 1 ] && [[ "$2" != --* ]]; then ARG1="$2"; shift + else echo "ha-maintenance: --restore needs a backup slug" >&2; exit 2; fi + ;; + --app) + MODE="app" + if [ $# -gt 1 ] && [[ "$2" != --* ]]; then ARG1="$2"; shift; fi + if [ $# -gt 1 ] && [[ "$2" != --* ]]; then ARG2="$2"; shift; fi + ;; + --yes) CONFIRM=1 ;; + *) + echo "ha-maintenance: unknown option '$1'" >&2 + echo "usage: $0 [--check-config|--logs TARGET [N]|--update|--restart-core [--safe-mode]|--rebuild-core|--rollback-os|--reboot|--backup [NAME]|--restore SLUG|--app ACTION SLUG] [--yes]" >&2 + exit 2 + ;; + esac + shift +done + +if [ "$MODE" = "app" ]; then + case "$ARG1" in + start|stop|restart|update) ;; + *) echo "ha-maintenance: --app action must be start|stop|restart|update (got '$ARG1')" >&2; exit 2 ;; + esac + if [ -z "$ARG2" ]; then + echo "ha-maintenance: --app needs an app slug, e.g. --app restart core_mosquitto" >&2 + exit 2 + fi +fi + +case "$MODE" in + update|restart-core|rebuild-core|rollback-os|reboot|backup|restore|app) + if [ "$CONFIRM" -ne 1 ]; then + echo "ha-maintenance: refusing '$MODE' without --yes (mutating action)" >&2 + exit 1 + fi + ;; +esac + +REMOTE_SCRIPT=$(cat <<'EOF' +set -u +ha() { command ha --no-progress "$@"; } + +case "$1" in + health) + printf '\n=== CORE ===\n' + ha core info + printf '\n=== SUPERVISOR ===\n' + ha supervisor info | grep -E '^(healthy|supported|version|version_latest|update_available|auto_update|timezone|logging):' + printf '\n=== ADDON STATES ===\n' + ha supervisor info | grep -E '^ (name|state):' + printf '\n=== OS ===\n' + ha os info + printf '\n=== HOST ===\n' + ha host info + printf '\n=== NETWORK ===\n' + ha network info | grep -E '^(address|ip_address|supervisor_internet|vlan|wifi):' + printf '\n=== AVAILABLE UPDATES ===\n' + ha available-updates + printf '\n=== JOBS ===\n' + ha jobs info + printf '\n=== RESOLUTION ===\n' + ha resolution info + printf '\n=== CORE STATS ===\n' + ha core stats + ;; + check-config) + ha core check + ;; + logs) + case "$2" in + core) ha core logs -n "$3" ;; + supervisor) ha supervisor logs -n "$3" ;; + host) ha host logs -n "$3" ;; + apps:*) ha apps logs -n "$3" "${2#apps:}" ;; + *) echo "unknown log target: $2" >&2; exit 2 ;; + esac + ;; + update) + ha refresh-updates + printf '\n=== UPDATE CORE (with backup) ===\n'; ha core update --backup + printf '\n=== UPDATE SUPERVISOR ===\n'; ha supervisor update + printf '\n=== UPDATE OS ===\n'; ha os update + printf '\n=== PENDING AFTER ===\n'; ha available-updates + ;; + restart-core) + if [ "$4" = "1" ]; then ha core restart --safe-mode; else ha core restart; fi + ;; + rebuild-core) + ha core rebuild + ;; + rollback-os) + ha os boot-slot other + ;; + reboot) + ha host reboot + ;; + backup) + if [ -n "${2:-}" ]; then ha backups new --name "$2"; else ha backups new; fi + ;; + restore) + echo "WARNING: restoring backup $2" + ha backups restore "$2" + ;; + app) + ha apps "$2" "$3" + ;; +esac +EOF +) + +if [ "$MODE" = "logs" ]; then + ARG1="$TARGET" + ARG2="$LINES" +fi + +# One remote login shell (MOTD printed once), then drop the MOTD locally. +printf '%s\n' "$REMOTE_SCRIPT" \ + | ssh -o BatchMode=yes -o ConnectTimeout=10 "${SSH_USER}@${HOST}" \ + 'sudo -n -i bash -s' "$MODE" "$ARG1" "$ARG2" "$SAFE_MODE" 2>/dev/null \ + | awk 'BEGIN{on=0} /^System is ready/{on=1; next} on'