vps/runbooks/matrix-e2ee-update.md

# matrix_e2ee update (hass.windy.lan)

## Purpose

Update the custom **`matrix_e2ee`** integration on `hass.windy.lan` while
preserving a verified rollback point and confirming that Home Assistant loads it.

## Scope

- Applicable: deploying a reviewed `matrix_e2ee` source revision to
  `hass.windy.lan`.
- Not applicable: Home Assistant Core upgrades, integration configuration
  changes, or recovery without a usable live-tree backup.

## Ownership

- Owner: personal ops (Windy)
- Last reviewed: 2026-08-23
- Related systems: hass.windy.lan (HAOS, `machine: green`)

## Approval gates

| Action | Risk | Explicit approval |
|---|---|---|
| Replace the live integration tree and restart Home Assistant Core | Medium | Yes — user confirmation required |

## Safety

- Do not replace the live tree or restart Core without explicit user confirmation.
- If any precondition or verification fails, `STOP` and record evidence before continuing.

## Preconditions

- The source repo at `/home/windy/project/ha-matrix-e2ee` is on the **target
  state**: either a release tag (`git tag -l 'v*'`) or a commit whose
  `manifest.json` `version` is the target. Note v0.3.0 was deployed from an
  **untagged** `main` HEAD (`216cc99`), so the tag check alone is not enough —
  confirm the working-tree `custom_components/matrix_e2ee/manifest.json`.
- The working tree matches HEAD: `git status --short` clean (only ignorables)
  and `git diff HEAD -- custom_components/` empty. Record
  `git rev-parse HEAD` for the docs/Linear record — HEAD can move during a
  session, so re-check right before rsync (verified 2026-08-18: HEAD moved
  from a `w1n-180` branch merge to `main` mid-deploy).
- The remote host is reachable and `sudo -n -i ha core info` succeeds.
- The workstation HTTP proxy does not interfere — LAN hosts must be reachable
  without proxying (unset `http_proxy` / `HTTP_PROXY` if needed).
- If the agent sandbox hits `Bad owner or permissions on /etc/ssh/ssh_config.d/20-systemd-ssh-proxy.conf`, add `-F /dev/null` to the `ssh` / `rsync` commands below.
- Domain is **`matrix_e2ee`** (double-e). Older notes may say `matrix_e2e`;
  paths, events, and services all use `matrix_e2ee`.

## Procedure

### 1. Backup the live tree

```bash
ssh -o BatchMode=yes hassio@hass.windy.lan \
  'sudo -n -i mkdir -p /homeassistant/.matrix-e2ee-backups &&
   sudo -n -i cp -a /homeassistant/custom_components/matrix_e2ee \
     /homeassistant/.matrix-e2ee-backups/matrix_e2ee.bak-$(date +%Y%m%d)-v<OLD_VERSION>'
```

The backup lives in `/homeassistant/.matrix-e2ee-backups/` — a directory
separated from `custom_components/` to avoid HA scanning it as a custom
component domain.

### 2. Rsync the new source

```bash
rsync -a --delete -e 'ssh -o BatchMode=yes' \
  /home/windy/project/ha-matrix-e2ee/custom_components/matrix_e2ee/ \
  hassio@hass.windy.lan:/homeassistant/custom_components/matrix_e2ee/
```

The `--delete` cannot remove Core-owned `__pycache__` — that is handled
in the next step. Source `.py` files and `manifest.json` are transferred
correctly even with the `__pycache__` errors, but **rsync exits with code 23
(`some files/attrs were not transferred`)** — that is expected, not a failure.
Confirm the transfer by checking the manifest on the host before restarting.

### 3. Wipe `__pycache__` (as root) and restart Core

Quote the nested `__pycache__` glob — remote login shell is zsh and will
fail with `no matches found` if left unquoted.

```bash
ssh -o BatchMode=yes hassio@hass.windy.lan \
  "sudo -n -i rm -rf /homeassistant/custom_components/matrix_e2ee/__pycache__ \
     '/homeassistant/custom_components/matrix_e2ee/*/__pycache__' &&
   sudo -n -i ha core restart"
```

Stale `cpython-314` bytecode in Core-owned `__pycache__` keeps the old
coordinator in memory until restart. Wipe before restart.

Wait for `Command completed successfully.` (typically 1–2 min).

### 4. Verify the deployment

#### 4a. Confirm manifest version

```bash
ssh -o BatchMode=yes hassio@hass.windy.lan \
  'sudo -n -i cat /homeassistant/custom_components/matrix_e2ee/manifest.json'
```

Expect `"version": "<NEW_VERSION>"`.

#### 4b. Check Core logs for matrix_e2ee

```bash
ssh -o BatchMode=yes hassio@hass.windy.lan \
  'sudo -n -i ha core logs -n 2500' | grep -E 'matrix_e2ee|Setting up matrix' | head -20
```

Expect:
- `Setup of domain matrix_e2ee took ...` (older wording `Setting up matrix_e2ee` may appear)
- `matrix_e2ee restored existing device; user=@hass:chans.xyz device=rO1R915ncu`
- No `ERROR` level messages from `custom_components.matrix_e2ee`
- Blocking-call WARNINGs from `_patch_nio_sas_timeout` / nio store I/O are expected

#### 4c. Verify the entry is loaded (optional, via Supervisor API)

No trailing slash on the entries URL (trailing `/` returns 404 on Core 2026.8.1).

```bash
ssh -o BatchMode=yes hassio@hass.windy.lan \
  "sudo -n -i python3 - <<'PY'
import os, json, urllib.request
req = urllib.request.Request(
    'http://supervisor/core/api/config/config_entries/entry',
    headers={'Authorization': 'Bearer ' + os.environ['SUPERVISOR_TOKEN']},
)
entries = json.loads(urllib.request.urlopen(req, timeout=30).read())
for e in entries:
    if e['domain'] == 'matrix_e2ee':
        print(f\"{e['domain']}: state={e['state']} source={e['source']}\")
PY"
```

Expect `state: loaded`.

### 5. Record the deployment

- Update the `matrix_e2ee` live-tree section in
  [hosts/hass.windy.lan.md](../hosts/hass.windy.lan.md): new version, source
  commit (`git rev-parse HEAD`), backup name, and any new feature notes.
- Record the operation in the Linear `vps` project (scope, action,
  verification, follow-up); see [docs/agents/issue-tracker.md](../docs/agents/issue-tracker.md).

## Rollback

If Core fails to start after the update:

```bash
# Restore the backup
ssh -o BatchMode=yes hassio@hass.windy.lan \
  'sudo -n -i rm -rf /homeassistant/custom_components/matrix_e2ee &&
   sudo -n -i cp -a /homeassistant/.matrix-e2ee-backups/matrix_e2ee.bak-<DATE>-v<OLD_VERSION> \
     /homeassistant/custom_components/matrix_e2ee &&
   sudo -n -i rm -rf /homeassistant/custom_components/matrix_e2ee/__pycache__ &&
   sudo -n -i ha core restart'
```

If a full HA backup exists (pre-update), restore via `ha backups restore <slug>`.

## References

- [hosts/hass.windy.lan.md](../hosts/hass.windy.lan.md) — current live version and config
- [docs/home-assistant-matrix.md](../docs/home-assistant-matrix.md) — integration architecture and verification model
- [home-assistant-maintenance.md](home-assistant-maintenance.md) — general HA maintenance procedures
- [ha-matrix-e2ee source](https://github.com/windyboy/ha-matrix-e2ee) — GitHub repo