Author SHA1 Message Date
windyboy ff1a92110c docs(us2): Soft Serve → Gitea 迁移事实与参考镜像 (Plane VPS-94)
- hosts/us2: Gitea 1.27.3-rootless 部署实况 (repo.windy.me SSH:2222/Web), 16 仓迁移核对, 备份/回滚; soft-serve 停用保留作回滚
- compose/gitea: 参考镜像 (rootless compose + 备份 sidecar + 一次性迁移脚本留档)
- AGENTS/inventory/compose README: 服务表与索引同步
2026-09-18 17:24:35 +08:00
windyboy cc3fb99c14 docs(us2): 根盘清理 71%→23% 事实记录 (Plane VPS-93) 2026-09-18 11:46:17 +08:00
windyboy 6879d79cc6 docs(hass,pgdb): scribe 4.4.0 升级核对 + stats_io_interval 300 + sensor_minute 体积诊断
- hass: Scribe 3.8.0 → 4.4.0(HACS jonathan-gtd/scribe,2026-09-13 随 Core
  2026.9.1 / HAOS 18.2 升级)。记录 4.0 两个 breaking change 在本机均无需动作
  (3.x 结构 DB + PK 在、TimescaleDB 2.29.2 已装)、配置优先级
  YAML > options > entry data > 默认值、以及「YAML 改动必须重启 Core」。
- hass: 新增 stats_io_interval: 300(备份 scribe.yaml.bak-20260913-191558)。
  变更前 24h scribe 自写 11 019/87 461 行(12.6%);实测发布间隔 300s。
  retention_states/retention_events 可用但刻意不设;flush_interval 仍被 entry
  data 钉在 5s(采用新默认 30s 需显式写 YAML)。
- pgdb: sensor_minute 2.26 GB 是压缩窗口内的正常暂存(compress_after 按 chunk
  结束时间判断,09-17 才合格),稳态 4–5 GB 平台期,暂不处理;复查点 09-17
  之后。顺带修正 hass/scribe 库体积事实。

校验:scripts/validate-repo.sh PASS (0 warnings)。
2026-09-13 19:34:04 +08:00
windyboy c445c5f512 Merge remote-tracking branch 'origin/main' into main
AGENTS.md 冲突(两侧都改了 Linear→Plane 记录源):取本地更完整的表述
(self-hosted plane.chans.xyz + mcp__plane__* + 2026-09-03 停用日期 +
issue-tracker.md 已过时),并吸收远端的 `plane-workflow` skill 指引。
2026-09-13 19:13:06 +08:00
windyboy 3c83246f24 docs(pgdb,gfw): pgdb VM 重启根因 (W1N-263) + gfw Quad9 上游移除记录 2026-09-13 19:12:32 +08:00
windyboy c0c975584a docs(plane): 自托管 Plane 落地事实入仓库 + plane-health runbook + hardening 草稿
记录源 Linear→Plane (2026-09-03 起, Plane MCP) + plane.chans.xyz 服务行/upstream 段;
inventory + hosts/synapse.chans.xyz.md 补 Plane 部署事实 (Helm plane-ce-1.8.0 / app v1.4.1,
ns plane, IngressRoute/自有证书 issuer/PVC 5+5Gi local-path/无备份层);
新增 runbooks/plane-health.md (只读健康检查) 与 docs/plane-hardening/ 草稿
(values.hardened.yaml、secrets.yaml.example 占位、backup/ CronJob), 均为未应用设计稿;
.gitignore 增加 .tmp-* agent 临时文件。
2026-09-13 19:12:32 +08:00
windyboy 0034cec925 Merge remote-tracking branch 'origin/main' into HEAD 2026-09-13 19:11:13 +08:00
windyboy c9dcde1274 docs(hass): Quick 面板时间范围扩容 — 功率/环境/人体感应加 3d-30d 档,用电量加 3mo (VPS-92)
卡片 JS 只接受 <n>m|<n>h|<n>d 与命名档(today/week/month/3mo/6mo/year/custom);
记录 scribe sensor_minute 数据下界 2026-08-29 对 >15d 档的影响。
2026-09-13 19:09:54 +08:00
windyboy 037c4ccaa5 docs(hass): 马桶换气电源 Matter 插座接入 + 功率积分补电量 (VPS-92)
Matter Smart Plug (SIXWGH model 3596) 的 cluster 0x0091 声明 IMPE+CUME+PERE 但
CumulativeEnergyImported 恒为 null -> HA 能量实体永久 unknown。功率计量正常
(实测 24.7 W),故用 Integration (Riemann sum) 辅助元素补电量实体
sensor.wei_sheng_jian_ma_tong_huan_qi_dian_yuan_energy:
能源仪表盘 grid 源 [8] 与 Quick「用电量(按插座)」图换到该实体,Quick 另加
span-2「开关」区块。记录 config-flow 经 supervisor 代理走 REST 的 Agent 方法、
备份路径与回滚方式;energy/validate 已全绿。
2026-09-13 18:39:22 +08:00
windyboy 17bb171578 docs(hass): CSG 面板重构入维护 runbook — avg price 传感器 + 季节提醒 automation + WS 改法 (VPS-90) 2026-09-04 21:55:40 +08:00
windyboy cecf7e6331 docs(hass): CSG 电力监控面板重构 — avg price 传感器 + 去重/改名 + 季节切换提醒 (VPS-90) 2026-09-04 21:44:43 +08:00
windyboy e957bc2bb1 docs: add host-disk-cleanup runbook + hk2 disk facts; record source -> Plane vps (VPS-81) 2026-09-02 18:02:22 +08:00
windyboy de52cb8b57 docs(hass): 地图卡 CARTO 水印修复 — custom:map-card v1.16.0 + keyed tiles (W1N-261) 2026-08-30 13:39:42 +08:00
windyboy b15e19bce9 docs(pgdb): compose 开机竞态故障修复 + 自愈 unit (W1N-260)
- 根因:开机时 docker 恢复容器绑定 192.168.55.15:5432/8081 失败(EADDRNOTAVAIL,IP 尚未可绑)→ timescaledb/pgweb 启动失败且不重试,停摆 3h15m;pg-backup 开机备份失败 → unhealthy
- 处置:docker compose up -d --force-recreate(三容器回 database_default、端口发布、备份恢复、pgweb 恢复);用户重启 HA Core 后写入管道恢复
- 防复发:新增开机自愈 systemd oneshot pgdb-compose.service(enabled),源码 compose/pgdb/pgdb-compose.service
2026-08-30 13:19:05 +08:00
windyboy bee54a6858 docs(hass): CSG 长期归档 csg_history + recorder 365d 补录 (W1N-243) 2026-08-30 13:19:05 +08:00
windyboy d6747028b4 docs(soft-serve): 镜像固定 v0.12.2 + 备份 sidecar + 非 root 运行 (W1N-244..248)
- 镜像 pinned charmcli/soft-serve:v0.12.2(GHCR 为 dev/nightly 源,无 v0.12.x tag)
- soft-serve-backup sidecar:每日 02:00 sqlite .backup + repos-config 打包,03:00 prune 保留 14 份
- 非 root 运行(user 1000:1000),data chown;ssh.public_url 修复
- compose 源码参考:compose/soft-serve/(服务器文件为准)
2026-08-30 13:19:05 +08:00
windyboy f174aa1219 docs(hass): CSG 复核遗留修复 — 模板 days[-1] 补排序 + 本月日均/预测进度上屏 (W1N-242) 2026-08-29 21:08:13 +08:00
windyboy 5b5f6042e6 docs(hass): CSG 模板/面板修复记录 — availability 硬化 (W1N-239)、off-by-one + 新传感器 (W1N-241)、gauge 阶梯对齐 + 年度统计 (W1N-240)
- hosts/hass.windy.lan.md: csg_sensors 三次变更记录 + 季节性 gauge 切换已知事项 (11-01/5-01)
- runbooks/home-assistant-maintenance.md: float(0) fake-zero follow-up 标记已解决
2026-08-29 21:02:33 +08:00
windyboy 50136b2ffd docs(hass): W1N-238 — scribe config split to scribe.yaml + templates/ merge include
- Scribe 3.8.0 block moved verbatim from configuration.yaml to
  /homeassistant/scribe.yaml (scribe: !include scribe.yaml); YAML stays
  authoritative, import semantics unchanged.
- template: switched to !include_dir_merge_list templates; new
  quick_sensors.yaml scaffold (top-level list, quick_ prefix, unique_id
  required; pure sums stay min_max per W1N-233).
- Verified post-restart 2026-08-29 20:19 CST: core check ok, scribe
  connection on, states_written 18581→19426, template entities = 12,
  no scribe/template log errors. Backup configuration.yaml.bak-20260829-201724-w1n238.
2026-08-29 20:25:53 +08:00
windyboy 6707cebc88 chore: gitignore .agent-work/ agent scratch dir
Untracked agent scratch made validate-repo.sh link scan fail; same
category as the already-ignored .agents/ and .claude/ dirs.
2026-08-29 20:25:53 +08:00
windyboy 908ff5412a docs(hass): Quick dashboard round-2 state — badges, total-power helper, span-2 power pair, fill/colors (W1N-231)
Sync Quick dashboard section with live config: 2x2 mushroom light grid,
kong_diao AC entity fix, heading badges (env temps / AC / PC / total power),
min_max sum helper sensor.dang_qian_zong_gong_lu (fail-closed), selective
tozeroy fill on base-load chart, power charts paired as span-2 sections with
card titles removed, motion per-entity colors.
2026-08-29 19:54:35 +08:00
windyboy e58283210a chore(vaultwarden,healthcheck): upgrade 1.37.2 (Bitwarden 2026.8+); fix vps-health checks
- vaultwarden/server:1.37.1 -> 1.37.2 (required for Bitwarden clients 2026.8.0+)
- compose probe: flag only active services (config --services) so debug-profile
  pgweb 'Exited' no longer false-positives
- runner: build aggregate args line-by-line (robust vs Jinja trim_blocks)
- SMTP AUTH probe moved host-side (vaultwarden image has no python3); never
  prints the SMTP password
- us2 facts: probe refresh 2026-08-29, image/version, vps-health install
2026-08-29 19:54:32 +08:00
windyboy 8c73d1f894 docs(hass,pgdb): timescale-plotly-card chart stack — reader+card install, sensor_minute pipeline, Quick dashboard
- reader timescale_database_reader v1.1.0 (bb8776a) + card timescale-plotly-card
  2.2.0 (217961d), manual installs; config entry, Lovelace resource id recorded
- pgdb scribe: sensor_minute_aggregate cagg + sensor_minute hypertable + jobs
  1005/1006/1007; states_raw 3-month retention/compression statements deliberately
  skipped (permanent archive per host doc)
- sensor_minute_refresh local patch ELSE 0 → ELSE NULL (unavailable-minute zeros
  poison diff-mode energy charts) + one-time cleanup (505 head rows, 26 impossible
  zeros); re-apply after re-running upstream 02 SQL
- Quick dashboard: 5 chart cards via WS lovelace/config/save; documented section
  column_span (absent → span 1) vs card grid_options sizing rules
2026-08-29 15:54:40 +08:00
windyboy bc0a86245d docs: Scribe retention v4.x status (user declined RCs, 2026-08-29); HA Core 2026.8.3 verified 2026-08-29 14:56:05 +08:00
windyboy 13032fd0bb Merge origin/main (production Makefile) into pgdb runbooks delivery 2026-08-29 14:52:10 +08:00
windyboy d2063e7496 Merge pgdb ops runbooks — health/restore/update (W1N-228) 2026-08-29 14:51:31 +08:00
windyboy ab808088b8 Add production Makefile for routine VPS ops
Wrap validate-repo.sh and routine Ansible playbooks with safe-by-default
targets: read-only health/audit flows, CONFIRM=1 gates for mutating work,
and LIMIT/TARGETS guards. Document entry point in AGENTS.md.
2026-08-26 11:27:02 +08:00
39 changed files with 1715 additions and 45 deletions
+3
View File
@@ -30,3 +30,6 @@ skills-lock.json
.vscode/
.idea/
*~
# Agent working scratch (not repo content).
.agent-work/
.tmp-*
+21 -5
View File
@@ -30,9 +30,13 @@ This repo is the **agent ops handbook + fact source** for maintaining personal V
> read-only unless a change is approved).
7. Record each material VPS operation, incident, configuration change, or
verification outcome in the corresponding **Linear `vps` project**. Include
scope, action, verification, and remaining follow-up; never put passwords,
tokens, private keys, recovery keys, or private room IDs in Linear.
verification outcome in the corresponding **Plane `vps` project**
(self-hosted `plane.chans.xyz`, Plane MCP `mcp__plane__*`, following the
`plane-workflow` skill). **Linear is retired as a record source (2026-09-03)
— do not create Linear issues;** existing W1N-* entries are read-only
history. Include scope, action, verification, and remaining follow-up; never
put passwords, tokens, private keys, recovery keys, or private room IDs in
Plane or Linear.
### Runbook execution rules
@@ -59,7 +63,7 @@ not maintain a second copy of the machine table here.
| SMTP `mx2.windy.me:587` (STARTTLS) or `:465` | mx2 | client submission; full email + mailbox password — [runbook](runbooks/mailcow-smtp-client.md) |
| IMAP `mx2.windy.me:993` | mx2 | same mailbox credentials |
| https://auth.wsvc.info | us2 (`/opt/vaultwarden`) | Vaultwarden (Postgres, **operational**) — client Server URL |
| `repo.windy.me:2222` | us2 (`/opt/soft-serve`) | Soft Serve (stub details) |
| `repo.windy.me` (git SSH `:2222` / web HTTPS) | us2 (`/opt/gitea`) | Gitea — 1.27.3-rootless pinned, backup sidecar; details in [hosts/us2.wsvc.info.md](hosts/us2.wsvc.info.md) |
| DNS `ns1.wsvc.info:53` | hk2 (`/opt/pdns`, Auth **5.0.6**) | PowerDNS auth — zones `windy.me`, `wsvc.info`, `chans.xyz` |
| https://pdns.wsvc.info | hk2 (`poweradmin`) | Poweradmin UI |
| https://pgweb.wsvc.info | hk2 (`pgweb`) | PowerDNS Postgres browser |
@@ -67,6 +71,7 @@ not maintain a second copy of the machine table here.
| https://synapse.chans.xyz | synapse | Synapse Client-Server + Federation API |
| https://account.chans.xyz | synapse | Matrix Authentication Service (local passwords) |
| https://admin.chans.xyz | synapse | Element Admin console (MAS admin auth) |
| https://plane.chans.xyz | synapse (`plane`, Helm `plane-ce` 1.8.0 / v1.4.1) | Plane project management (self-hosted, K3s) |
### Upstream docs
@@ -76,6 +81,8 @@ not maintain a second copy of the machine table here.
**Matrix (ESS on synapse):** Matrix homeserver running on `synapse.chans.xyz` via the official ESS (Element Server Suite) Helm chart with Synapse + MAS + Element Web + Admin. DNS zone `chans.xyz` managed by hk2 PowerDNS. Before changing config, read [docs/matrix-upstream.md](docs/matrix-upstream.md) and [hosts/synapse.chans.xyz.md](hosts/synapse.chans.xyz.md). K3s cluster on this node has hostPort 80/443 for Traefik (no ServiceLB). Health: [matrix-health](runbooks/matrix-health.md).
**Plane (on synapse):** Self-hosted Plane project management at `plane.chans.xyz`, Helm release `plane-app` (chart `plane-ce-1.8.0`, app `v1.4.1`) in ns `plane` on the same K3s node as Matrix. Config from `/home/windy/plane-k3s/values.yaml`; workload/cert/ingress details in [hosts/synapse.chans.xyz.md](hosts/synapse.chans.xyz.md). Its Postgres/MinIO PVCs are **not** backed up.
**RustDesk:** Self-hosted RustDesk server on `hk2.chans.xyz` (`/opt/rustdesk`, containers `hbbs`/`hbbr`, image pinned `1.1.14`). The `hbbs -r` relay hostname must resolve to the host's public IP `154.36.174.161` — use `hk2.chans.xyz` (never `hk2.wsvc.info`, which has no DNS record). Health: [rustdesk-health](runbooks/rustdesk-health.md).
## Runbooks & scripts
@@ -91,8 +98,10 @@ not maintain a second copy of the machine table here.
| PowerDNS health (hk2) | [runbooks/pdns-health.md](runbooks/pdns-health.md) |
| PowerDNS upstream refs | [docs/pdns-upstream.md](docs/pdns-upstream.md) |
| Matrix health | [runbooks/matrix-health.md](runbooks/matrix-health.md) |
| Plane health | [runbooks/plane-health.md](runbooks/plane-health.md) |
| RustDesk health (hk2) | [runbooks/rustdesk-health.md](runbooks/rustdesk-health.md) |
| AdGuard Home health | [runbooks/adguard-home-health.md](runbooks/adguard-home-health.md) |
| Host disk cleanup | [runbooks/host-disk-cleanup.md](runbooks/host-disk-cleanup.md) |
| Home Assistant maintenance | [runbooks/home-assistant-maintenance.md](runbooks/home-assistant-maintenance.md) + [scripts/ha-maintenance.sh](runbooks/scripts/ha-maintenance.sh) |
| matrix_e2ee update (hass.windy.lan) | [runbooks/matrix-e2ee-update.md](runbooks/matrix-e2ee-update.md) |
| Matrix upstream refs | [docs/matrix-upstream.md](docs/matrix-upstream.md) |
@@ -100,6 +109,7 @@ not maintain a second copy of the machine table here.
| UniFi local-service proxy bypass | [docs/unifi-openclash-localhost.md](docs/unifi-openclash-localhost.md) |
| UniFi SSO login setting (Ansible) | `cd ansible && ansible-playbook playbooks/unifi-sso.yml --limit unifi` |
| Routine Ansible operations | [runbooks/ansible-operations.md](runbooks/ansible-operations.md) |
| Routine make commands | `make help` (wraps `ansible-operations.md` read-only + gated flows) |
| Issue → mergeable change | [runbooks/issue-to-merge.md](runbooks/issue-to-merge.md) |
| Fix failing health/playbook run | [runbooks/fix-ci.md](runbooks/fix-ci.md) |
| Release a reviewed change | [runbooks/release.md](runbooks/release.md) |
@@ -115,7 +125,12 @@ Routine mailcow health: `cd ansible && ansible-playbook playbooks/health-report.
### Issue tracker
Issues are tracked in Linear and created/updated via the Linear MCP (`vps` project). See `docs/agents/issue-tracker.md`.
Issues are tracked in **Plane** — self-hosted at `plane.chans.xyz`, project
`vps` — and created/updated via the Plane MCP (`mcp__plane__*`), following the
`plane-workflow` skill. **Linear is retired as a record source (2026-09-03); do
not create Linear issues.** Existing W1N-* entries are read-only history.
`docs/agents/issue-tracker.md` documents the retired Linear workflow and is
stale; treat this section as authoritative.
### Triage labels
@@ -166,4 +181,5 @@ hosts/ # per-host facts
runbooks/ # step-by-step ops (README.md = index, _template.md = template)
docs/ # upstream refs / design notes / research records (active + archive/)
scripts/validate-repo.sh # repo-wide validation (run before merging)
Makefile # routine validate / health / gated ansible wrappers
```
+183
View File
@@ -0,0 +1,183 @@
# VPS ops hub — routine validate / health / gated Ansible wrappers.
# See runbooks/ansible-operations.md for playbook semantics.
SHELL := /usr/bin/env bash
.SHELLFLAGS := -eu -o pipefail -c
.DEFAULT_GOAL := help
REPO_ROOT := $(CURDIR)
ANSIBLE_DIR := $(REPO_ROOT)/ansible
export ANSIBLE_LOCAL_TEMP := $(REPO_ROOT)/.ansible/tmp
export ANSIBLE_HOME := $(REPO_ROOT)/.ansible
LIMIT ?=
EXTRA ?=
VERBOSE ?= 0
CONFIRM ?= 0
TARGETS ?=
TRAEFIK ?= 0
LIMIT_FLAG := $(if $(LIMIT),--limit $(LIMIT),)
VERBOSE_FLAG := $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vvv,))
.PHONY: help validate check deps galaxy syntax ansible-prep \
ping inventory audit health health-mailcow health-matrix \
maint-preview baseline compose-check \
install-healthchecks install-matrix-healthchecks compose-deploy reconcile
help:
@printf '%s\n' \
'VPS ops hub — make targets (run from repo root)' \
'' \
'Variables: LIMIT=<group|host> CONFIRM=1 TARGETS=<svc[,svc]> TRAEFIK=1 VERBOSE=0|1|2 EXTRA=...' \
'' \
'Local / repo:' \
' validate, check scripts/validate-repo.sh (pre-merge gate)' \
' deps, galaxy ansible-galaxy collection install' \
' syntax ansible-playbook --syntax-check all playbooks' \
'' \
'Read-only remote (ansible):' \
' ping ansible managed -m ping' \
' inventory ansible-inventory --graph' \
' audit playbooks/audit.yml' \
' health [LIMIT=…] playbooks/health-report.yml' \
' health-mailcow health --limit mailcow' \
' health-matrix health --limit matrix' \
' maint-preview playbooks/maintenance-preview.yml' \
' baseline playbooks/baseline.yml' \
' compose-check compose-deploy --check --diff (requires LIMIT=)' \
'' \
'Mutating (require CONFIRM=1; host-scoped targets require LIMIT=):' \
' install-healthchecks playbooks/healthchecks.yml' \
' install-matrix-healthchecks playbooks/matrix-healthchecks.yml' \
' compose-deploy playbooks/compose-deploy.yml' \
' reconcile playbooks/compose-reconcile.yml (requires TARGETS=)' \
'' \
'Examples:' \
' make validate' \
' make health LIMIT=mailcow' \
' make compose-check LIMIT=vaultwarden' \
' make compose-deploy LIMIT=vaultwarden CONFIRM=1' \
' make reconcile LIMIT=powerdns TARGETS=auth CONFIRM=1' \
' make reconcile LIMIT=vaultwarden TARGETS=vaultwarden TRAEFIK=1 CONFIRM=1' \
'' \
'Advanced (not wrapped — use ansible-playbook directly):' \
' us4-firewalld, unifi-sso, k3s-server, matrix-stack, wireguard-harden,' \
' restic, rustdesk, email-alerts, mailcow update runbook'
validate check:
@bash "$(REPO_ROOT)/scripts/validate-repo.sh"
deps galaxy: ansible-prep
@command -v ansible-galaxy >/dev/null 2>&1 || { echo "ansible-galaxy not found; install Ansible first." >&2; exit 1; }
@cd "$(ANSIBLE_DIR)" && ansible-galaxy collection install -r requirements.yml
syntax: ansible-prep
@if ! command -v ansible-playbook >/dev/null 2>&1; then \
echo "ansible-playbook not found; syntax check skipped." >&2; \
exit 0; \
fi
@fail=0; \
for p in "$(ANSIBLE_DIR)"/playbooks/*.yml; do \
if ! (cd "$(ANSIBLE_DIR)" && ansible-playbook --syntax-check "playbooks/$$(basename "$$p")" >/dev/null 2>&1); then \
echo "syntax-check failed: $$p" >&2; \
fail=1; \
fi; \
done; \
exit $$fail
ansible-prep:
@mkdir -p "$(ANSIBLE_HOME)/tmp" "$(ANSIBLE_HOME)/ssh-control"
define require_ansible
@command -v ansible-playbook >/dev/null 2>&1 || { echo "ansible-playbook not found; install Ansible first." >&2; exit 1; }
endef
define require_limit
@if [ -z "$(LIMIT)" ]; then \
echo "LIMIT is required (e.g. LIMIT=mailcow, LIMIT=vaultwarden, LIMIT=powerdns)." >&2; \
exit 1; \
fi
endef
define require_confirm
@if [ "$(CONFIRM)" != "1" ]; then \
echo "Mutating operation blocked. Re-run with CONFIRM=1" >&2; \
exit 1; \
fi
endef
define require_targets
@if [ -z "$(TARGETS)" ]; then \
echo "TARGETS is required (comma-separated service names, e.g. TARGETS=auth or TARGETS=vaultwarden)." >&2; \
exit 1; \
fi
endef
ping: ansible-prep
$(require_ansible)
@cd "$(ANSIBLE_DIR)" && ansible managed -m ping $(LIMIT_FLAG) $(VERBOSE_FLAG) $(EXTRA)
inventory: ansible-prep
$(require_ansible)
@cd "$(ANSIBLE_DIR)" && ansible-inventory --graph $(EXTRA)
audit: ansible-prep
$(require_ansible)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/audit.yml $(LIMIT_FLAG) $(VERBOSE_FLAG) $(EXTRA)
health: ansible-prep
$(require_ansible)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/health-report.yml $(LIMIT_FLAG) $(VERBOSE_FLAG) $(EXTRA)
health-mailcow: ansible-prep
$(require_ansible)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/health-report.yml --limit mailcow $(VERBOSE_FLAG) $(EXTRA)
health-matrix: ansible-prep
$(require_ansible)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/health-report.yml --limit matrix $(VERBOSE_FLAG) $(EXTRA)
maint-preview: ansible-prep
$(require_ansible)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/maintenance-preview.yml $(LIMIT_FLAG) $(VERBOSE_FLAG) $(EXTRA)
baseline: ansible-prep
$(require_ansible)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/baseline.yml $(LIMIT_FLAG) $(VERBOSE_FLAG) $(EXTRA)
compose-check: ansible-prep
$(require_ansible)
$(require_limit)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/compose-deploy.yml --check --diff --limit $(LIMIT) $(VERBOSE_FLAG) $(EXTRA)
install-healthchecks: ansible-prep
$(require_ansible)
$(require_confirm)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/healthchecks.yml $(LIMIT_FLAG) $(VERBOSE_FLAG) $(EXTRA)
install-matrix-healthchecks: ansible-prep
$(require_ansible)
$(require_confirm)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/matrix-healthchecks.yml $(LIMIT_FLAG) $(VERBOSE_FLAG) $(EXTRA)
compose-deploy: ansible-prep
$(require_ansible)
$(require_limit)
$(require_confirm)
@cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/compose-deploy.yml --limit $(LIMIT) \
-e '{"compose_deploy_confirm": true}' $(VERBOSE_FLAG) $(EXTRA)
reconcile: ansible-prep
$(require_ansible)
$(require_limit)
$(require_targets)
$(require_confirm)
@json=$$(python3 -c 'import json,sys; t=[x.strip() for x in sys.argv[1].split(",") if x.strip()]; \
(not t) and sys.exit("TARGETS must contain at least one non-empty service name"); \
d={"service_reconcile_confirm": True, "service_reconcile_targets": t}; \
(sys.argv[2]=="1") and d.update({"service_reconcile_restart_traefik": True}); \
print(json.dumps(d))' "$(TARGETS)" "$(TRAEFIK)"); \
cd "$(ANSIBLE_DIR)" && ansible-playbook playbooks/compose-reconcile.yml --limit $(LIMIT) \
-e "$$json" $(VERBOSE_FLAG) $(EXTRA)
@@ -31,9 +31,27 @@ compose_ps() {
}
check_compose() {
local output
output="$(compose_ps)" || { record critical 'compose_ps_failed'; return; }
if grep -qiE 'Exited|Restarting|[[:space:]]Dead[[:space:]]' <<<"$output"; then
local output services bad
# Only flag containers of *active* services (config --services excludes
# debug/profile-gated services such as vaultwarden's pgweb, which is
# intentionally stopped unless started with --profile debug).
services="$(docker compose --project-directory '{{ compose_project_dir }}' config --services 2>/dev/null)" || { record critical 'compose_ps_failed'; return; }
output="$(docker compose --project-directory '{{ compose_project_dir }}' ps --all --format json 2>&1)" || { record critical 'compose_ps_failed'; return; }
bad="$(printf '%s\n' "$output" | python3 -c '
import json, sys
services = set(sys.argv[1].split())
for line in sys.stdin:
line = line.strip()
if not line:
continue
try:
c = json.loads(line)
except Exception:
continue
if c.get("Service") in services and c.get("State") in ("exited", "restarting", "dead"):
print(c.get("Service"))
' "$services")"
if [[ -n "$bad" ]]; then
record critical 'compose_unhealthy_container'
else
record ok 'compose_ok'
@@ -14,5 +14,11 @@ rm -f '{{ healthcheck_state_dir }}/latest-{{ healthcheck_profile_scripts[profile
this_rc="${PIPESTATUS[0]}"
[ "$this_rc" -gt "$rc" ] && rc="$this_rc"
{% endfor %}
aggregate_result{% for profile in healthcheck_profiles %} {{ healthcheck_profile_scripts[profile] | replace('.sh', '') }}{% endfor %}
# Collect profile check names line-by-line (robust against Jinja trim_blocks
# whitespace control, which would otherwise merge this into one line).
aggregate_args=""
{% for profile in healthcheck_profiles %}
aggregate_args="$aggregate_args {{ healthcheck_profile_scripts[profile] | replace('.sh', '') }}"
{% endfor %}
aggregate_result $aggregate_args
exit "$rc"
@@ -15,11 +15,12 @@ grep -Fq 'vw-db' <<<"$health" || record critical 'postgres_missing'
check_https 'https://auth.wsvc.info/' '^200$'
check_tls_days auth.wsvc.info 443
# Read effective config only inside the service and report booleans/fingerprints,
# never its SMTP password or other secret fields.
smtp_result="$(docker compose --project-directory '{{ compose_project_dir }}' exec -T vaultwarden python3 - <<'PY' 2>&1
# Read effective config from the mounted vw-data dir on the host and run the
# SMTP AUTH probe from the host (the vaultwarden image has no python3; the
# host does). Never print the SMTP password.
smtp_result="$(python3 - <<'PY' 2>&1
import json, pathlib, smtplib, ssl
cfg=json.loads(pathlib.Path('/data/config.json').read_text())
cfg=json.loads(pathlib.Path('{{ compose_project_dir }}/vw-data/config.json').read_text())
host=cfg.get('smtp_host'); port=int(cfg.get('smtp_port') or 0)
user=cfg.get('smtp_username')
smtp_secret=cfg.get('smtp_password')
+2
View File
@@ -11,6 +11,8 @@ files: every secret is a `${VAR}` reference resolved from the **server-local
| `vaultwarden` | us2 (`/opt/vaultwarden`) | `compose/vaultwarden/compose.yml` | static file + `compose-deploy.yml` |
| `pdns` | hk2 (`/opt/pdns`) | `compose/pdns/compose.yml` | static file + `compose-deploy.yml` |
| `pgdb` | pgdb (`/opt/database`, 无 ansible) | `compose/pgdb/compose.yml` | static file(手动部署:scp → `docker compose config -q``up -d`;服务器文件名 `docker-compose.yml` |
| `soft-serve` | us2 (`/opt/soft-serve`, 已退役停用) | `compose/soft-serve/compose.yml` (+ `Dockerfile.backup`, `scripts/`) | static file(参考镜像; 2026-09-18 被 gitea 替换 VPS-94, 数据保留作回滚) |
| `gitea` | us2 (`/opt/gitea`) | `compose/gitea/compose.yml` (+ `Dockerfile.backup`, `scripts/`) | static file(参考镜像, 未接入 compose-deploy; 服务器文件为准; 2026-09-18 替换 soft-serve, VPS-94 |
| `adguardhome` | dns.windy.lan (`/opt/adguardhome`) | — (待从 LAN 提取) | static file (pending) |
| `unifi` | ubnt (`/home/windy/unifi-9`) | — (待从 LAN 提取) | static file (pending) |
| `wireguard` | us4 (`/opt/wireguard`) | `ansible/templates/wireguard-compose.yml.j2` | role-rendered (inventory vars) |
+4
View File
@@ -0,0 +1,4 @@
# compose/gitea — 秘密一律走服务器本地 .env, 不入库
# 迁移期一次性: Gitea 管理员生成的 token (mirror-migrate.sh 读取, 用后撤销)
GITEA_MIGRATE_USER=
GITEA_MIGRATE_TOKEN=
+3
View File
@@ -0,0 +1,3 @@
FROM alpine:3.20
RUN apk add --no-cache sqlite rsync tzdata
WORKDIR /scripts
+59
View File
@@ -0,0 +1,59 @@
# Gitea on us2 — reference compose (Plane VPS-94, 迁移完成 2026-09-18)
# 参考镜像, 服务器 /opt/gitea 文件为准 (同 soft-serve 约定, 未接入 compose-deploy)
# rootless 镜像: uid 1000 原生非 root; 数据 /var/lib/gitea (宿主 ./data), 配置 /etc/gitea (宿主 ./config)
# SSH: 容器内监听 2322 (非特权, SSH_LISTEN_PORT), 对外 repo.windy.me:2222 经 Traefik TCP entrypoint `ssh`
services:
gitea:
image: gitea/gitea@sha256:1c17ecaead42eb3b5391553d8708103a4beb0e86edf5b9ebc1eb269c318845f2 # 1.27.3-rootless
container_name: gitea
restart: unless-stopped
user: "1000:1000"
environment:
TZ: Asia/Shanghai
volumes:
- ./data:/var/lib/gitea
- ./config:/etc/gitea
- ./secrets:/secrets:ro # 复用的 soft-serve host key (SSH_SERVER_HOST_KEYS)
networks:
- traefik
labels:
- traefik.enable=true
# Web UI: repo.windy.me (2026-09-18 操作者决定复用现有域名, 免 DNS 变更)
- traefik.http.routers.gitea-web.rule=Host(`repo.windy.me`)
- traefik.http.routers.gitea-web.entrypoints=websecure
- traefik.http.routers.gitea-web.tls.certresolver=letsencrypt
- traefik.http.services.gitea-web.loadbalancer.server.port=3000
# SSH: 接管 :2222 (entrypoint 已存在, router 动态生效, 无需重启 Traefik)
- traefik.tcp.routers.gitea-ssh.entrypoints=ssh
- traefik.tcp.routers.gitea-ssh.rule=HostSNI(`*`)
- traefik.tcp.routers.gitea-ssh.tls=false
- traefik.tcp.services.gitea-ssh.loadbalancer.server.port=2322
gitea-backup:
build:
context: .
dockerfile: Dockerfile.backup
container_name: gitea-backup
restart: unless-stopped
volumes:
- ./data:/data:ro
- ./config:/config:ro
- ./backups:/backup
- ./scripts:/scripts
environment:
TZ: Asia/Shanghai
BACKUP_UID: 1000
BACKUP_GID: 1000
entrypoint: >
/bin/sh -ec "
umask 077 &&
touch /backup/backup.log &&
crontab /scripts/crontab.txt &&
echo '[INFO] gitea backup cron installed' &&
crond -f -l 8
"
networks:
traefik:
external: true
name: vw-net
+20
View File
@@ -0,0 +1,20 @@
#!/bin/sh
set -eu
umask 077
D() { date "+%Y-%m-%d %H:%M:%S"; }
TS=$(date +%Y%m%d_%H%M%S)
OUT="/backup/gitea_${TS}"
mkdir -p "$OUT"
echo "[$(D)] Starting gitea backup -> $OUT"
# rootless 布局: app.ini=/etc/gitea(宿主 ./config), db+repos=/var/lib/gitea/data(宿主 ./data/data)
# app.ini 含 SECRET_KEY/INTERNAL_TOKEN — 恢复 2FA/session/mirror 凭据必需
tar czf "$OUT/app.ini.tar.gz" -C /config app.ini
sqlite3 /data/data/gitea.db ".backup '$OUT/gitea.db'"
rsync -a /data/data/git/repositories/ "$OUT/repos/"
tar czf "$OUT/repos.tar.gz" -C "$OUT" repos
rm -rf "$OUT/repos"
chmod 600 "$OUT"/*
if [ -n "${BACKUP_UID:-}" ] && [ -n "${BACKUP_GID:-}" ]; then
chown -R "$BACKUP_UID:$BACKUP_GID" "$OUT" /backup/backup.log
fi
echo "[$(D)] Backup OK: $(du -sh "$OUT" | cut -f1)"
+4
View File
@@ -0,0 +1,4 @@
# Run gitea backup daily at 02:00
0 2 * * * /bin/sh /scripts/backup.sh >> /backup/backup.log 2>&1
# Prune backups older than 14 days daily at 03:00
0 3 * * * /bin/sh /scripts/prune.sh >> /backup/backup.log 2>&1
+41
View File
@@ -0,0 +1,41 @@
#!/bin/sh
# 一次性迁移辅助 (Plane VPS-94 Phase 2): 在 gitea 容器内执行。
# 已于 2026-09-18 执行完成 (16 仓), 留档备查; 复用时按 VPS-94 流程重生成一次性 token。
# 用法:
# GITEA_MIGRATE_USER=<user> GITEA_MIGRATE_TOKEN=<token> \
# docker exec -e GITEA_MIGRATE_USER -e GITEA_MIGRATE_TOKEN gitea \
# /scripts/mirror-migrate.sh [public_repo ...]
# 每仓: API 建仓 (默认 private, 参数中列出的为 public) -> push --mirror。
# default_branch 按源仓 symbolic-ref HEAD 设置, 避免非 main 源仓在 Gitea 显示为空。
# 结束后按 VPS-94 Phase 3 逐仓核对 git ls-remote ref 全集。
set -eu
MUSER="${GITEA_MIGRATE_USER:?need GITEA_MIGRATE_USER}"
TOKEN="${GITEA_MIGRATE_TOKEN:?need GITEA_MIGRATE_TOKEN}"
SRC="/migration-src"
API="http://localhost:3000/api/v1"
PUBLIC_REPOS=" $* "
migrate_one() {
dir="$1"
git -C "$dir" rev-parse --git-dir >/dev/null 2>&1 || { echo "[SKIP] $dir (not a git repo)"; return 0; }
name=$(basename "$dir"); name=${name%.git}
def_branch=$(git -C "$dir" symbolic-ref --short HEAD)
case "$PUBLIC_REPOS" in *" $name "*) private=false ;; *) private=true ;; esac
echo "[MIGRATE] $name (default=$def_branch private=$private)"
code=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$API/user/repos" \
-H "Authorization: token $TOKEN" -H "Content-Type: application/json" \
-d "{\"name\":\"$name\",\"private\":$private,\"default_branch\":\"$def_branch\",\"auto_init\":false}")
case "$code" in
201) : ;;
409) echo " [WARN] $name 已存在, 直接补推" ;;
*) echo " [FAIL] create HTTP $code"; return 1 ;;
esac
git -C "$dir" push --mirror "http://$MUSER:$TOKEN@localhost:3000/$MUSER/$name.git"
echo " [OK] $name pushed"
}
for dir in "$SRC"/*.git "$SRC"/cdia; do
[ -d "$dir" ] || continue
migrate_one "$dir"
done
echo "[DONE] 全部处理完毕; 迁移后记得撤销一次性 token"
+5
View File
@@ -0,0 +1,5 @@
#!/bin/sh
set -eu
D() { date "+%Y-%m-%d %H:%M:%S"; }
ls -dt /backup/gitea_* 2>/dev/null | tail -n +15 | xargs -r rm -rf
echo "[$(D)] Pruned. Kept $(ls -d /backup/gitea_* 2>/dev/null | wc -l) backups (max 14)"
+24
View File
@@ -0,0 +1,24 @@
[Unit]
Description=Reconcile pgdb compose stack (timescaledb + pgweb + pg-backup) at boot
Documentation=file:///opt/database/docker-compose.yml
After=network-online.target docker.service
Wants=network-online.target
Requires=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/opt/database
# Idempotent boot-time reconcile. docker's own restore can fail to bind the
# published ports (192.168.55.15:5432/8081) when the VM IP is not yet usable
# right after boot (EADDRNOTAVAIL, observed 2026-08-30): timescaledb/pgweb
# then stay stopped until a manual `docker compose up`. This unit retries
# `docker compose up -d` (a no-op when the stack is healthy) until the port
# listens, and force-recreates as a last resort to recover a network-detached
# container. Data lives on bind mounts (/srv/pgdata, /opt/database/backups),
# so recreation is safe.
ExecStart=/bin/bash -c 'for i in $(seq 1 12); do docker compose up -d --remove-orphans; sleep 2; if ss -tln | grep -q "192.168.55.15:5432"; then exit 0; fi; sleep 3; done; echo "pgdb-compose: retries exhausted, force-recreating"; docker compose up -d --force-recreate; sleep 10; ss -tln | grep -q "192.168.55.15:5432"'
TimeoutStartSec=180
[Install]
WantedBy=multi-user.target
+2
View File
@@ -0,0 +1,2 @@
# Soft Serve initial admin public key (used only on first boot)
SOFT_SERVE_INITIAL_ADMIN_KEYS=ssh-ed25519 AAAA... # replace with admin public key
+3
View File
@@ -0,0 +1,3 @@
FROM alpine:3.20
RUN apk add --no-cache sqlite tzdata
WORKDIR /scripts
+59
View File
@@ -0,0 +1,59 @@
services:
soft-serve:
image: charmcli/soft-serve:v0.12.2
container_name: soft-serve
restart: unless-stopped
# non-root (uid 1000 = windy; 与 backup sidecar BACKUP_UID 一致)
user: "1000:1000"
environment:
SOFT_SERVE_DATA_PATH: /var/lib/soft-serve
SOFT_SERVE_INITIAL_ADMIN: windy
SOFT_SERVE_INITIAL_ADMIN_KEYS: ${SOFT_SERVE_INITIAL_ADMIN_KEYS}
volumes:
- ./data:/var/lib/soft-serve
- soft-serve-app:/soft-serve
networks:
- traefik
labels:
- traefik.enable=true
# SSH over TCP via Traefik (entryPoint ssh -> container port 23231)
- traefik.tcp.routers.softserve-ssh.entrypoints=ssh
- traefik.tcp.routers.softserve-ssh.rule=HostSNI(`*`)
- traefik.tcp.routers.softserve-ssh.tls=false
- traefik.tcp.services.softserve-ssh.loadbalancer.server.port=23231
soft-serve-backup:
build:
context: .
dockerfile: Dockerfile.backup
container_name: soft-serve-backup
restart: unless-stopped
volumes:
- ./data:/data:ro
- ./backups:/backup
- ./scripts:/scripts
environment:
TZ: Asia/Shanghai
BACKUP_UID: 1000
BACKUP_GID: 1000
entrypoint: >
/bin/sh -ec "
umask 077 &&
touch /backup/backup.log &&
crontab /scripts/crontab.txt &&
echo '[INFO] soft-serve backup cron installed' &&
crond -f -l 8
"
volumes:
soft-serve-app:
networks:
traefik:
external: true
name: vw-net
+15
View File
@@ -0,0 +1,15 @@
#!/bin/sh
set -eu
umask 077
D() { date "+%Y-%m-%d %H:%M:%S"; }
TS=$(date +%Y%m%d_%H%M%S)
OUT="/backup/soft-serve_${TS}"
mkdir -p "$OUT"
echo "[$(D)] Starting soft-serve backup -> $OUT"
tar czf "$OUT/repos-config.tar.gz" -C /data repos hooks config.yaml ssh
sqlite3 /data/soft-serve.db ".backup '$OUT/soft-serve.db'"
chmod 600 "$OUT/repos-config.tar.gz" "$OUT/soft-serve.db"
if [ -n "${BACKUP_UID:-}" ] && [ -n "${BACKUP_GID:-}" ]; then
chown -R "$BACKUP_UID:$BACKUP_GID" "$OUT" /backup/backup.log
fi
echo "[$(D)] Backup OK: $(du -sh "$OUT" | cut -f1)"
+4
View File
@@ -0,0 +1,4 @@
# Run soft-serve backup daily at 02:00
0 2 * * * /bin/sh /scripts/backup.sh >> /backup/backup.log 2>&1
# Prune backups older than 14 days daily at 03:00
0 3 * * * /bin/sh /scripts/prune.sh >> /backup/backup.log 2>&1
+5
View File
@@ -0,0 +1,5 @@
#!/bin/sh
set -eu
D() { date "+%Y-%m-%d %H:%M:%S"; }
ls -dt /backup/soft-serve_* 2>/dev/null | tail -n +15 | xargs -r rm -rf
echo "[$(D)] Pruned. Kept $(ls -d /backup/soft-serve_* 2>/dev/null | wc -l) backups (max 14)"
+1 -1
View File
@@ -1,6 +1,6 @@
services:
vaultwarden:
image: vaultwarden/server:1.37.1
image: vaultwarden/server:1.37.2
container_name: vaultwarden
restart: unless-stopped
env_file: ".env"
+45
View File
@@ -0,0 +1,45 @@
# Plane CE 加固草稿(docs/plane-hardening/
> **状态:草稿,未应用、未提交。** 对应追踪:Plane vps 项目条目(2026-09-03**记录源**Linear W1N-277 已取消,Linear 自 2026-09-03 起不再作为记录源)。
> 线上实例:`plane.chans.xyz`synapse K3sns `plane`release `plane-app` = chart `plane-ce-1.8.0` / app `v1.4.1`)。
> 依据:2026-09-03 只读核查(13 条审查意见中 11 条属实、#3 基本属实、#9 指标归属错误)+ 上游 chart 模板逐条核对。
## 文件
| 文件 | 内容 |
|------|------|
| `values.hardened.yaml` | 可选硬化 valuesexternal secrets 引用、requireExplicitSecrets、minio pin、上传限额对齐);含 HTTP→HTTPS `extraObjects` 示例 |
| `secrets.yaml.example` | 6 组外部 Secret 结构占位(只含 key 名,真实值仅存宿主机) |
| `backup/plane-backup.yaml` | **PostgreSQL 备份 CronJob**pg_dump `-Fc`hostPath `/var/backups/plane`MinIO 已按实际用量剔除) |
| `backup/README.md` | 备份方案说明(排程/容量/保留/还原/阻塞) |
## 应用顺序(每步先 diff 后执行,全部需用户逐项确认)
### 现在就值得做:DB 备份(P0,见 backup/
`plane-backup.yaml` 部署 + 手动触发验证一次即可;88 MB 库每日快照几乎零成本。
### 可选(顺手做一次,不是必须)
- **Phase A 密钥外部化**(零行为变化、无停机,约 15 分钟):按 `secrets.yaml.example`
在宿主机建 6 个 Secret(值先复制当前集群),用 `values.hardened.yaml`
`helm diff upgrade``helm upgrade`;验证后删除 chart 生成的旧 Secret。
价值:默认密钥不再落在 chart 公开常量上,作为保险。
- **MCP API Key 轮换**:若审查对话出过你的环境,Plane 后台重生成 + 更新
`/home/windy/plane-k3s/mcp/mcp.env`0600+ 重启 Cursor MCP。
- **/god-mode IP 白名单**:若在意管理后台被公网爆破。chart 1.8.0 的 IngressRoute
不支持给单条路由追加 middleware → 需 post-renderer 或 upgrade 后 `kubectl patch`
(升级会覆盖,需固化);源 IP 清单待提供。
### 明确暂缓/跳过(个人单节点,等出现症状再处理)
- SECRET_KEY 等轮换(Phase B):等真要配 SMTP/OAuth 前再做(避免旧密文不可解)。
- NetworkPolicy、有状态组件 resources limitschart 无 values 开关,需 post-render/patch)、
HTTP→HTTPS(草稿已给 `extraObjects` 示例)、metrics-server/Sentry。
## 关键限制(chart 1.8.0 模板已核对)
- `external_secrets.*_existingSecret` 设置后,对应 Secret **必须**包含模板所需全部 key
(缺失不自动补),见 `secrets.yaml.example` 注释。
- `app_keys_existingSecret` 的 envFrom 在所有 workload 上**最后注入**(后置生效),
保证 app/live 共享密钥一致——不要在其后再放同名 key 的 Secret。
- `DATABASE_URL`/`AMQP_URL`/`REDIS_URL` 是 chart 生成的派生 URL,内嵌明文密码;
外部化后轮换 DB/队列密码时必须同步更新 `plane-app-env`
- minio 的 `MINIO_ROOT_*``AWS_*` 同源于一个 Secret;升级时 bucket Job 会重跑
(需 admin 权限凭据)——换 svcacct 前先确认权限覆盖该 Job。
+48
View File
@@ -0,0 +1,48 @@
# Plane CE 备份方案(DB-only)— DRAFT (2026-09-03), 未应用
> 关联:`plane-backup.yaml`CronJob);追踪:Plane vps 项目条目(记录源,2026-09-03 起不用 Linear)。
> 现状(实测):pg 全库 **88 MB**310 issues / 1 user);MinIO uploads **264 KB**(几乎空)。
## 范围决策(2026-09-03,实际角度)
- **做:PostgreSQL 逻辑备份** —— 覆盖现实故障(误删、升级失败、磁盘坏、重装),成本≈0。
- **不做:MinIO/附件备份** —— 桶仅 264 KB,个人实例附件可接受丢失;不为它付日常维护。
日后附件明显变多再按原完整版思路加 `mc mirror`(历史版本见本目录 git 历史/Plane 条目评论)。
- 异机同步暂不启用(见下"局限/阻塞")。
## 方案
集群内 CronJobns `plane`,每天 **01:30 UTC = 03:30 本地**,控制器按 UTC 跑):
1. 单容器 `postgres:15.7-alpine``pg_dump -Fc`(自定义压缩格式)打 `plane`
`/var/backups/plane/pg/plane-<UTC时间戳>.dump`hostPath `DirectoryOrCreate`
2. 保留 7 天(`find -mtime +7 -delete`),成功/失败历史各留 3/2
3. 凭据:现 chart Secret `plane-app-pgdb-secrets`Phase A 外部化后改 `plane-pgdb-credentials`
## 容量
- 库 88 MB → `-Fc` 快照约 10–40 MB/天 × 7 天 ≈ **<300 MB**,对 83 G 可用盘可忽略。
## 还原(未演练;应用前先做一次隔离测试)
```bash
# 目标 PG15 实例(临时起一个 postgres:15.7-alpine 容器或另一台机):
# 先建空库: createdb plane (user=plane)
pg_restore -h <target> -U plane -d plane --clean --if-exists /var/backups/plane/pg/plane-<TS>.dump
# 还原后确认 310 issues 量级一致;附件为空属预期(未备份 MinIO)
```
## 验收(应用前逐项过)
- [ ] CronJob 建立后手动触发一次:`kubectl -n plane create job --from=cronjob/plane-backup plane-backup-manual-1`Job `Completed`
- [ ] `/var/backups/plane/pg/plane-*.dump` 可被 `pg_restore -l` 列出
- [ ] 备份 Job 只依赖 pgdb 服务,不依赖 Plane 应用 Pod(应用故障期间也能出备份)
- [ ] 保留清理 dry-run`find ... -print`)正确;`df -h /` 前后对比记录
## 局限 / 阻塞
- **本地方案不是离机备份**:单节点磁盘/整机故障即丢。如日后要离机,纳入
[Restic 异机 repository 决策与存取隔离](https://plane.chans.xyz/space/projects/56874283-7e1d-43a8-afa4-631cf1c4ad5b/issues/7825d564-ae15-446b-bced-be26b648346b/)
(与 Matrix 备份同一决策);恢复演练纪律见
[服务级 restore runbook 与隔离复元演练](https://plane.chans.xyz/space/projects/56874283-7e1d-43a8-afa4-631cf1c4ad5b/issues/a9bea3ba-c958-4a74-b2f1-6bbb653f21d3/)。
- 提醒:同一节点 **Matrix 数据价值远高于 Plane 且同样无备份** —— 若投入备份精力,顺序上 Matrix 优先。
@@ -0,0 +1,63 @@
# Plane CE PostgreSQL backup CronJob — DRAFT (2026-09-03), NOT applied.
# ns: plane (synapse K3s single node). Output: hostPath /var/backups/plane (root disk, auto-created).
#
# Scope decision (2026-09-03, practical): DB-only. MinIO dropped — uploads bucket
# measured at 264 KB / 444 KB total; attachments are acceptable loss for this
# personal 1-user instance (310 issues / 88 MB DB). Revisit only if usage grows.
#
# Credentials: read from the CURRENT chart-generated Secret (works today). After the
# optional external-secrets migration (docs/plane-hardening/README.md Phase A) switch
# the secretKeyRef name to plane-pgdb-credentials.
#
# Apply:
# ssh windy@synapse.chans.xyz 'sudo k3s kubectl apply -n plane -f -' < plane-backup.yaml
# Manual run + verify:
# sudo k3s kubectl -n plane create job --from=cronjob/plane-backup plane-backup-manual-1
# sudo k3s kubectl -n plane get cronjob,job,pods | grep plane-backup
# sudo ls -lh /var/backups/plane/pg
# Restore steps + tuning: see backup/README.md
apiVersion: batch/v1
kind: CronJob
metadata:
name: plane-backup
namespace: plane
spec:
# 01:30 UTC daily = 03:30 local (CEST). CronJob controller runs in UTC.
schedule: "30 1 * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 2
jobTemplate:
spec:
backoffLimit: 2
template:
spec:
restartPolicy: OnFailure
volumes:
- name: backup
hostPath:
path: /var/backups/plane
type: DirectoryOrCreate
containers:
- name: pg-dump
image: postgres:15.7-alpine
env:
- name: PGPASSWORD
valueFrom:
secretKeyRef:
name: plane-app-pgdb-secrets # -> plane-pgdb-credentials after Phase A
key: POSTGRES_PASSWORD
command: ["/bin/sh", "-c"]
args:
- |
set -euo pipefail
TS=$(date -u +%Y%m%dT%H%M%SZ)
mkdir -p /backup/pg
pg_dump -h plane-app-pgdb.plane.svc.cluster.local -U plane -d plane \
-Fc -f "/backup/pg/plane-${TS}.dump"
find /backup/pg -type f -name 'plane-*.dump' -mtime +7 -delete
echo "pg_dump done: /backup/pg/plane-${TS}.dump ($(du -h /backup/pg/plane-${TS}.dump | cut -f1))"
volumeMounts:
- name: backup
mountPath: /backup
+91
View File
@@ -0,0 +1,91 @@
# External Secret structure for Plane CE hardening — EXAMPLE ONLY.
# No real values here; this file is safe to commit. Real values live only on the
# host (/home/windy/plane-k3s, 0600/0700) and in the cluster.
#
# Phase A — create each Secret with the CURRENT cluster values first (zero change):
# # current source Secrets (chart-generated):
# kubectl -n plane get secret plane-app-app-secrets -o jsonpath='{.data.SECRET_KEY}' | base64 -d
# kubectl -n plane get secret plane-app-live-secrets -o jsonpath='{.data.REDIS_URL}' | base64 -d
# kubectl -n plane get secret plane-app-pgdb-secrets -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d
# kubectl -n plane get secret plane-app-rabbitmq-secrets -o jsonpath='{.data.RABBITMQ_DEFAULT_PASS}' | base64 -d
# kubectl -n plane get secret plane-app-doc-store-secrets -o jsonpath='{.data}' | base64 -d
#
# e.g. kubectl -n plane create secret generic plane-app-keys \
# --from-literal=SECRET_KEY="$(<copy from above>)" \
# --from-literal=LIVE_SERVER_SECRET_KEY="$(<copy from above>)"
#
# All keys below are REQUIRED by chart templates/plane-ce-1.8.0 (verified 2026-09-03):
# missing keys are NOT auto-filled once an existingSecret is referenced.
---
apiVersion: v1
kind: Secret
metadata:
name: plane-app-keys # external_secrets.app_keys_existingSecret
namespace: plane
type: Opaque
stringData:
SECRET_KEY: "" # current: copy from plane-app-app-secrets; rotate only in Phase B
LIVE_SERVER_SECRET_KEY: "" # current: same value as above / plane-app-live-secrets
---
apiVersion: v1
kind: Secret
metadata:
name: plane-app-env # external_secrets.app_env_existingSecret
namespace: plane
type: Opaque
stringData:
REDIS_URL: "" # redis://plane-app-redis.plane.svc.cluster.local:6379/
DATABASE_URL: "" # postgresql://plane:plane@plane-app-pgdb.plane.svc.cluster.local/plane
AMQP_URL: "" # amqp://plane:plane@plane-app-rabbitmq.plane.svc.cluster.local/
---
apiVersion: v1
kind: Secret
metadata:
name: plane-live-env # external_secrets.live_env_existingSecret
namespace: plane
type: Opaque
stringData:
REDIS_URL: "" # redis://plane-app-redis.plane.svc.cluster.local:6379/
---
apiVersion: v1
kind: Secret
metadata:
name: plane-pgdb-credentials # external_secrets.pgdb_existingSecret
namespace: plane
type: Opaque
stringData:
POSTGRES_PASSWORD: "" # Phase A: keep current ('plane'); Phase B: ALTER USER first, then sync
POSTGRES_DB: "plane"
POSTGRES_USER: "plane"
---
apiVersion: v1
kind: Secret
metadata:
name: plane-rabbitmq-credentials # external_secrets.rabbitmq_existingSecret
namespace: plane
type: Opaque
stringData:
RABBITMQ_DEFAULT_USER: "plane"
RABBITMQ_DEFAULT_PASS: "" # Phase A: keep current; Phase B: rabbitmqctl change_password first
---
apiVersion: v1
kind: Secret
metadata:
name: plane-minio-credentials # external_secrets.doc_store_existingSecret
namespace: plane
type: Opaque
stringData:
FILE_SIZE_LIMIT: "20971520" # must match env.doc_upload_size_limit
AWS_S3_BUCKET_NAME: "uploads"
USE_MINIO: "1"
MINIO_ROOT_USER: "admin"
MINIO_ROOT_PASSWORD: "" # root creds take effect on first init only
AWS_ACCESS_KEY_ID: "admin"
AWS_SECRET_ACCESS_KEY: "" # == MINIO_ROOT_PASSWORD while minio.local_setup
AWS_S3_ENDPOINT_URL: "http://plane-app-minio:9000"
+109
View File
@@ -0,0 +1,109 @@
# Plane CE hardened values — DRAFT (2026-09-03), NOT applied.
# Target file on host: /home/windy/plane-k3s/values.yaml (synapse.chans.xyz)
# Reference release: plane-app, chart plane-ce-1.8.0 (values.yaml L1-362 + templates verified 2026-09-03).
# No secrets in this file. Secret *values* live only in k8s Secrets (see secrets.yaml.example).
#
# Two phases:
# Phase A: externalize secrets (reference names below) with CURRENT values copied -> zero change.
# Phase B: rotate credentials one by one (see README.md). SECRET_KEY rotation is cheap only while
# SMTP/OAuth are unconfigured (no encrypted config rows yet).
planeVersion: v1.4.1
ingress:
enabled: true
appHost: plane.chans.xyz
ingressClass: traefik
traefik:
# 20 MiB (chart default). Keep aligned with env.doc_upload_size_limit below.
maxRequestBodyBytes: 20971520
ssl:
createIssuer: true
issuer: http # HTTP-01; ssl_token_existingSecret not needed
email: admin@chans.xyz
generateCerts: true
postgres:
storageClass: local-path
volumeSize: 5Gi
# NOTE: chart 1.8.0 exposes NO resources knob for the bundled datastores
# (stateful templates render no resources block). Add limits via
# --post-renderer/kustomize or `kubectl -n plane patch sts ...` re-applied on
# every upgrade (P2 task; see README.md).
redis:
storageClass: local-path
# image: valkey/valkey:7.2.11-alpine # already pinned by chart default; uncomment to make explicit
minio:
# P2: pin. Digest of the currently running :latest (2026-09-03, pod plane-app-minio-wl-0).
image: minio/minio@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
# image_mc: minio/mc@sha256:... # optional: pin one-shot bucket-init client the same way
storageClass: local-path
volumeSize: 5Gi
rabbitmq:
storageClass: local-path
env:
# Fail the render instead of ever falling back to the chart's PUBLIC constants
# (values.yaml L340-341 in chart 1.8.0). Requires external_secrets below.
requireExplicitSecrets: true
# SECRET_KEY / LIVE_SERVER_SECRET_KEY are deliberately OMITTED here.
# They live in k8s Secret `plane-app-keys` (referenced below). With
# requireExplicitSecrets=true and app_keys_existingSecret set, the chart renders
# neither key itself and app+live workloads both envFrom `plane-app-keys` LAST
# (later envFrom wins), which keeps the shared signing key consistent.
pgdb_name: plane
docstore_bucket: uploads
# Align app-side upload cap with the Traefik body limit (was 5242880/5MiB).
# Keep both at 20MiB, or lower both together.
doc_upload_size_limit: "20971520"
external_secrets:
# Shared signing keys (used by app + live). REQUIRED keys: SECRET_KEY, LIVE_SERVER_SECRET_KEY.
app_keys_existingSecret: plane-app-keys
# REQUIRED keys: REDIS_URL, DATABASE_URL, AMQP_URL (chart-derived URLs; update on DB/queue rotation).
app_env_existingSecret: plane-app-env
# REQUIRED keys: REDIS_URL.
live_env_existingSecret: plane-live-env
# REQUIRED keys: POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_USER.
pgdb_existingSecret: plane-pgdb-credentials
# REQUIRED keys: RABBITMQ_DEFAULT_USER, RABBITMQ_DEFAULT_PASS.
rabbitmq_existingSecret: plane-rabbitmq-credentials
# REQUIRED keys: FILE_SIZE_LIMIT, AWS_S3_BUCKET_NAME, USE_MINIO, MINIO_ROOT_USER,
# MINIO_ROOT_PASSWORD, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_S3_ENDPOINT_URL.
doc_store_existingSecret: plane-minio-credentials
# ssl_token_existingSecret: '' # DNS-01 only (cloudflare/digitalocean); unused with HTTP-01
# Optional, P2: HTTP -> HTTPS 301. The chart's own IngressRoute binds only
# 'websecure' (http:// currently 404s). extraObjects is rendered verbatim (toYaml).
# Uncomment and `helm upgrade` once reviewed:
# extraObjects:
# - apiVersion: traefik.io/v1alpha1
# kind: Middleware
# metadata:
# name: plane-https-redirect
# namespace: plane
# spec:
# redirectScheme:
# scheme: https
# permanent: true
# - apiVersion: traefik.io/v1alpha1
# kind: IngressRoute
# metadata:
# name: plane-http-to-https
# namespace: plane
# spec:
# entryPoints: [web]
# routes:
# - match: Host(`plane.chans.xyz`)
# kind: Rule
# middlewares:
# - name: plane-https-redirect
# services:
# - name: plane-app-web
# port: 3000
+8
View File
@@ -60,6 +60,14 @@ OpenClash runs `/etc/openclash/clash` (clash_meta core) with configuration
> AAAA now `2404:6800…` (new upstream answering, was `2607:f8b0…` via hk2),
> taobao/intercept/clash-fake-ip all unchanged. Backup:
> `config.yaml.bak-multi-doh-20260813-105421`.
> 2026-09-01: removed `dns.quad9.net` from `foreign_upstream` — recurring
> `WARN foreign_upstream … unexpected EOF` bursts (481 log entries) against
> Quad9 DoH; endpoint answers on probe but gets intermittently
> connection-reset from this network (same failure class as the excluded
> `dns.quad101.net`). Remaining upstreams `adg.chans.xyz` (hk2) +
> `dns.cloudflare.com` both verified live; google.com A + youtube.com AAAA
> resolve through mosdns :6052 after restart. Backup:
> `config.yaml.bak-quad9-remove-20260901-201801`.
- nft: OpenClash injects TPROXY/redirect + DNS-hijack rules into
`table inet fw4`; a residual `table inet passwall` exists with 0 packets (unused)
+346 -9
View File
@@ -228,6 +228,102 @@ cost/ladder sensors can stay `unknown` because CSG
dashboard uses template ladder/cost entities instead. Do not change
`templates/csg_sensors.yaml` or the 电力监控 dashboard for an install.
**`templates/csg_sensors.yaml` hardened 2026-08-29 (W1N-239):** added
`availability` templates to all 12 `csg_*` sensors (numeric sensors can't
render `unknown`/`unavailable` in `state`; availability suppresses
rendering instead — native CSG down ⇒ derived sensors show `unavailable`,
no more fake zeros / "一档" / `0%`). `csg_yesterday_kwh` now falls back to
`last_month_by_day`'s last entry when `this_month_by_day` is empty (month
start); ladder constants (`t1/t2/p1/p2/p3`) deduped into per-block
`variables:` (Block B + Block D); `csg_mom_change` parses `date`
defensively. Backup:
`/homeassistant/.csg-backups/csg_sensors.yaml.bak-20260829-w1n239`.
**Verified:** `ha core check` OK; Core restart required (trigger-based
template blocks don't settle on `template.reload` — W1N-114 precedent);
post-restart all 12 entities numeric & consistent (302.47 kWh→180.28 元,
324.03 kWh→194.06 元, mom_change -3.6%, yesterday 7.66 kWh/2026-08-28),
no template errors in Core logs.
**`csg_sensors.yaml` off-by-one fixed 2026-08-29 (W1N-241):** CSG data
lags 1 day (`sum(this_month_by_day)` == `this_month_total_usage`, data
stops at yesterday), but templates used `now().day` as "days elapsed" →
`csg_predicted_usage` underestimated ~1 daily avg (~3%) and
`csg_mom_change` compared this-month 28 days vs last-month 29 days
(-3.6% vs true -0.3%). Both now derive the day number from
`this_month_by_day[-1].date` (fallback `now().day` when empty). Added
`sensor.csg_this_month_daily_avg` (month-to-date avg, 302.47/28=10.8) and
`sensor.csg_prediction_progress` (usage/predicted %, 90.3) in Block C
(trigger adds `csg_predicted_usage`). Backup:
`/homeassistant/.csg-backups/csg_sensors.yaml.bak-20260829-w1n241`.
**Verified (8/29):** predicted 324.03→334.81, mom_change -3.6→-0.3,
daily_avg 10.8, progress 90.3, predicted_cost 194.06→200.94 (334.81 kWh
ladder), ladder cost 180.28 unchanged, `ha core check` OK after restart,
no template errors; 14 csg_* entities total.
**电力监控面板(`lovelace.dashboard_unknown` / view `power-monitor`
updated 2026-08-29 (W1N-240 + W1N-242):** 「本月累计」gauge 对齐夏季阶梯:
`max:650`、segments `0/260/600`(绿/橙/红 = 一/二/三档;冬季 11-01 需切
`max:450``0/200/400`**seasonal switch point**,见下文)。「📊 统计
数据」卡新增本年/去年 4 行(原生传感器,口径标注「电费(账单)」、本年
「(至今)」)+ 本月日均/预测进度 2 行(`csg_this_month_daily_avg` /
`csg_prediction_progress`W1N-242);面板共引用 **20** 个实体。改前备份:
`/homeassistant/.lovelace-backups/dashboard-unknown-power-monitor-20260829-204845.json`
W1N-240)、`-20260829-210708.json`W1N-242
(改法:WS `lovelace/config/save`,参数 `url_path: dashboard-unknown` +
`config`;勿直改 `.storage/`)。验证:WS 读回 18→20 实体 diff ✓、gauge
配置一致 ✓、URL `http://hass.windy.lan:8123/dashboard-unknown/power-monitor`
**`csg_sensors.yaml` W1N-242:** `csg_predicted_usage` /
`csg_mom_change` / `csg_this_month_daily_avg` 三处取 `days[-1]` 前补
`sort(attribute='date')`(与 `csg_yesterday_kwh` 一致,防上游乱序取错
数据日)。备份 `csg_sensors.yaml.bak-20260829-w1n242`。验证:Core
restart 后回归值不变(334.81 / -0.3 / 10.8 / 90.3 / 200.94 / 180.28)。
**CSG 面板重构 2026-09-04VPS-90,先核对计价后展示层改动):** 核对
`power-monitor` 计价与 8 月账单一致(198.65 vs 账单 198.64,差 ≤0.01 元,
因模板用公众圆整价 0.589/0.639/0.889、账单用 6 位精确价),不改阶梯常量。
改动:① `csg_sensors.yaml` Block B 新增
`sensor.csg_this_month_avg_price`(本月阶梯电费÷本月用电,`元/kWh`
availability 照 W1N-239 惯例;**csg_* 实体 14→15**);② 面板改名「环比上月」
→「环比上月同期」;glance「本月/上月」grid 去重为单卡「上月」(本月用电/电费
行归 💰核心数据卡);⚡阶梯电价卡加「本月实际均价」行(当前档位/当前电价/
本月实际均价/档位剩余;面板唯一实体引用 20→21);③ `automations.yaml`
2 条提醒:`automation.csg_mian_ban_qie_dong_ji_dang_ti_xing`10-25 09:00
`automation.csg_mian_ban_qie_xia_ji_dang_ti_xing`4-25 09:00)经
`matrix_e2ee.send_message` 提醒切 gauge。④ 金额单位混排(原生 CNY vs 模板
元)**维持**`config/entity_registry/update` 拒绝自定义文本单位
`extra keys not allowed … Got '元'`),用户确认接受。备份:
`.lovelace-backups/dashboard-unknown-power-monitor-20260904-204757-pre-refactor.json`
`.csg-backups/csg_sensors.yaml.bak-20260904-204757-pre-refactor`(及
`-205301-pre-avgprice`)、`.automations-backups/automations.yaml.bak-*`
**WS 改法(2026.8,本机实测)**core/主机 python 无 ws 库、core 容器内经
supervisor 代理 WS 被拒(loop prevention),用
`docker run --rm --network host -e SUPERVISOR_TOKEN`supervisor 镜像
`aarch64-hassio-supervisor:2026.08.0`)连 `ws://172.30.32.2/core/websocket`
命令名 `lovelace/config`(读)+ `lovelace/config/save`(写),
`lovelace/config/get` 已不存在(unknown_command)。验证:新实体
0.589 元/kWh、15 个 csg_* 数值齐全、回归值不变(14.09/198.65/331.22/
304.99/181.89)、automations on、`ha core check` OK、日志无 template 错误。
**CSG 长期归档(W1N-243, 2026-08-29:** scribe 库新增 `csg_history`
表(逐日 usage/cost/ladder/balance + 逐月累计;2026-07-01 起回填,永久),
由 TimescaleDB 每日任务 **1008** `csg_daily_snapshot()`22:30
Asia/Shanghai**TS job 非 pg_cron**upsert 维护。日费用在原生
`latest_day_cost` 缺失时回退 = 昨日用电 × 当前档费率(模板
`csg_current_ladder_tariff` 0.639);月费用回退模板
`csg_this_month_ladder_cost`。**语义**day 行 usage/cost 为该日值,
ladder/balance 为 22:30 快照值。详见 [hosts/pgdb.md](../hosts/pgdb.md)。
> **Seasonal gauge switch (W1N-240 已知事项):** 每年 **11-01** 把
> `power-monitor` 视图「本月累计」gauge 切到冬季 `max:450` /
> `0/200/400`**5-01** 切回夏季 `max:650` / `0/260/600`(与模板
> `now().month` 季节逻辑对齐;模板常量在 Block B/D `variables`)。
> **提醒 automation2026-09-04 起,VPS-90:**
> `automation.csg_mian_ban_qie_dong_ji_dang_ti_xing`10-25)与
> `automation.csg_mian_ban_qie_xia_ji_dang_ti_xing`4-2509:00 经
> `matrix_e2ee.send_message` 发操作步骤提醒;gauge 的 max/segments 无法
> 模板化,仍需人工改卡配置。
Home PPPoE IPv4 to CSG is still blackholed (`curl -4` to `218.19.148.218:443`
times out). `end0` IPv6 is enabled (`ipv6.method: auto`); from HA,
`curl -6 https://95598.csg.cn` returns HTTP 200 via `240e:f9:8060::1:16`.
@@ -319,12 +415,17 @@ deployed 2026-08-18 from `216cc99` (backup
reworked in v0.3.1/v0.3.2 to wait for a peer-initiated inbound SAS from
Element with emoji comparison), see
[docs/home-assistant-matrix.md § Device verification](../docs/home-assistant-matrix.md).
### Scribe long-term history (verified 2026-08-29)
### Scribe long-term history (3.8.0 setup 2026-08-29; 4.4.0 verified 2026-09-13)
- **Scribe 3.8.0** (`/homeassistant/custom_components/scribe/`), configured from
`configuration.yaml` (`scribe:` block; config entry
- **Scribe 4.4.0** (`/homeassistant/custom_components/scribe/`, HACS repo
`jonathan-gtd/scribe`, = latest stable 2026-09-12; upgraded 2026-09-13 together
with Core 2026.9.1 / HAOS 18.2), configured from
`/homeassistant/scribe.yaml` — W1N-238 moved the block out of
`configuration.yaml` on 2026-08-29 (main config now carries
`scribe: !include scribe.yaml`; content moved verbatim; backup
`configuration.yaml.bak-20260829-201724-w1n238`). Config entry
`01KC2VFJWEQ3XDHY6TQKHPDVRB`, `source: import` — UI "Configure → Advanced"
edits are overridden by the YAML on restart; treat YAML as authoritative).
edits are overridden by the YAML on restart; treat YAML as authoritative.
- TimescaleDB at `192.168.55.15:5432/scribe` (DB user `hass`; host in inventory,
see [hosts/pgdb.md](../hosts/pgdb.md)). Database re-initialized 2026-08-29 14:06 CST
(user-handled; earlier `relation "entities" does not exist` errors resolved).
@@ -342,11 +443,166 @@ deployed 2026-08-18 from `216cc99` (backup
entities (`scribe_states_written`, `scribe_events_written`, rates, sizes).
- Verified post-restart 14:23 CST: writer started, `scribe_events_written=1`
(homeassistant_start), states ~110/min, buffer 3, no scribe log errors.
- **Scribe 3.8.0 has no retention option** (upstream feature, newer versions
only). Retention requires upgrading Scribe first; the DB is a fresh 3.x-schema
so an upgrade is safe. Do not expect retention YAML keys to validate on 3.8.0.
- Recorder stays external-Postgres with `purge_keep_days: 30` for daily UI
history; Scribe is the permanent archive.
- **4.x upgrade核对 2026-09-13(只读 + 一处配置变更)**live `manifest.json` =
4.4.0。两个 4.0 breaking change 在本机都不需要动作——数据库是 3.x 结构
`states_raw` PK `(metadata_id, time)` 在,4.2.0 的启动态去重因此可用),
TimescaleDB 2.29.2 已装。4.1.0 修了 `db_url` 优先级,YAML 里的
`!secret scribe_url` 现在是权威。`scribe.yaml` 现有键在 4.4.0 全部仍然合法
(未知键被忽略,`extra=vol.ALLOW_EXTRA`)。**配置优先级 YAML > entry
`options` > entry `data` > 默认值**,而 `_resolve_settings` 读的是
`hass.data[DOMAIN]["yaml_config"]`(只有 `async_setup` 会写),所以
**YAML 改动必须重启 Corereload config entry 不重读 YAML。**
- **`stats_io_interval: 300`2026-09-13 添加**,备份
`/homeassistant/scribe.yaml.bak-20260913-191558`)。4.4.0 不再让 HA 每 30s
轮询 I/O 统计传感器,改由集成自己每 60s 发布,间隔成为配置项。scribe 自己的
传感器此前是本机自写历史的主要来源(变更前 24h:11 019 / 87 461 行状态 =
12.6%),60s → 300s 把这部分降约 5 倍(每个 I/O 传感器约 1440 → 288 行/天)。
验证:`ha core check` OK;重启 88s`ScribeWriter started successfully`
无 scribe error/warningscribe Repairs 问题 0 条;传感器发布间隔实测正好
300s11:19:26 → 11:24:26 UTC)。
- **Retention 现在可用但刻意不设**:`retention_states` / `retention_events`
4.0.0)按间隔丢 chunk,留空 = 永久保留,符合本机定位(Scribe 是永久归档,
recorder 保留 365 天)。注意 retention 是**绕过** entry `data` 副本读取的
`from_entry_data=False`),所以删掉 YAML 行即撤销策略。`db_schema`
`enable_rollups``scribe.purge` 同样未用:图表走 `sensor_minute` +
`timescale_database_reader`(见 [hosts/pgdb.md](pgdb.md)),不吃 scribe 自己的
视图,配置里也没有任何 `scribe.query` 调用。`flush_interval` 仍是 entry
`data` 钉住的 5s——上游下一个版本把默认改成 30s,但 entry 值优先,要采用只能
在 YAML 显式写 `flush_interval: 30`
- Recorder stays external-Postgres with `purge_keep_days: 365` (W1N-243,
2026-08-29, raised from 30 — ~300 MB/yr, 1% of the 30G pgdb disk) for
native UI per-change history; Scribe is the permanent archive. Long-term
statistics stay permanent (not purged by `purge_keep_days`). Note:
extending retention does **not** recover pre-2026-08-29 raw history
(already purged); only `csg_history` day/month values cover that period.
### Config layout: scribe.yaml + templates/ merge (W1N-238, verified 2026-08-29)
- `configuration.yaml` line 29: `scribe: !include scribe.yaml`; line 9:
`template: !include_dir_merge_list templates`. No `packages/`.
- `scribe.yaml` (config root): the Scribe block, content identical to the
former inline one; import semantics unchanged.
- `templates/`: `csg_sensors.yaml` (12 template sensors, top-level **list**)
+ `quick_sensors.yaml` (scaffold for Quick-derived `quick_*` sensors, empty
list with convention header). **`!include_dir_merge_list` merges per-file
lists; non-list files are silently skipped** — every file in `templates/`
must be a top-level list (`- sensor:` blocks). Directory include only picks
up `*.yaml`, so the `.bak` / `.pre-*` backups in the dir are ignored. After
adding sensors, verify template-platform entity count = 12 + N (entity
registry `platform: template`).
- Convention (per review + W1N-233): pure sums/averages stay min_max helpers
(e.g. `sensor.dang_qian_zong_gong_lu`); only template-logic derivations
(ladder pricing, cross-entity conditions) go into `quick_sensors.yaml`.
- Post-change verification 20:19 CST: `ha core check` ok, 92 s restart
(2026.8.3), `binary_sensor.scribe_database_connection` on,
`scribe_states_written` 18581→19426 growing, template entities still 12,
csg sensors numeric, no scribe/template log errors.
### Timescale Plotly card + database reader (verified 2026-08-29)
Chart stack over the Scribe TimescaleDB archive. Upstream pair (no HACS;
manual copies): reader `remmob/timescale_database_reader` **v1.1.0** (main
`bb8776a`) + card `remmob/timescale-plotly-card` **2.2.0** (main `217961d`).
- **Reader integration**: `/homeassistant/custom_components/timescale_database_reader/`.
Config entry `01M165P77QT1FQEAVPNZHDT82W` ("Scribe", `source: user`): connects
`hass@192.168.55.15:5432/scribe` (credentials = `secrets.yaml` `scribe_url`),
`table: sensor_minute`. Exposes no entities/services — it serves WS command
`timescale/query` (window ≤ 365 d, ≤ 50 000 rows, `downsample` bucket seconds).
Benign startup warning `Error executing test query: column "time" does not
exist`: the self-test SQL assumes the LTSS column name; the scribe table uses
`minute` — real queries work (verified: 70 rows for a live power sensor).
- **Card**: `/homeassistant/www/community/timescale-plotly-card/timescale-plotly-card.js`
(root-owned, same convention as HACS dirs). Lovelace resource (storage)
id `2e360d17b5aa4ce59c2fd13c43b51215`
`/hacsfiles/timescale-plotly-card/timescale-plotly-card.js`, type `module`.
Card config matches the entry by `database: scribe` (name from the reader
entry). Updates: replace the file, resource URL unchanged — browsers need a
hard refresh or a bumped `?v=` query on the resource URL.
- **pgdb side** (`sensor_minute_aggregate` cagg + `sensor_minute` hypertable +
every-minute refresh job): see [hosts/pgdb.md](pgdb.md) § Databases.
- **Agent-side HA WebSocket without a long-lived token** (verified 2026-08-29):
connect `ws://supervisor/core/websocket` with header
`Authorization: Bearer $SUPERVISOR_TOKEN`, then send
`{"type":"auth","access_token":"$SUPERVISOR_TOKEN"}` — the Supervisor proxy
swaps it for a core token (works as the internal Supervisor admin user). Note
`lovelace/resources/create` in HA 2026.8 takes `res_type` (NOT
`resource_type`).
- Scribe stores numeric sensor values in `states_raw.value` with `state` NULL,
so `sensor_minute.state` shows `'0'` for numeric sensors; the card plots
`avg_state` (from `value`) — expected, not a bug.
- **Quick 仪表盘(`dashboard-quick`)图表套件**2026-08-29 创建,经 WS
`lovelace/config/save` 写入;W1N-230 修复 + W1N-231 round-2 改进):
5 张 timescale 卡——大功率电器/常驻负载功率(按量级拆图,避免尖峰压扁
<70 W 基线)、按插座用电量(`energy_mode` + cumulative/diff,数据质量前提
见 pgdb 的 refresh 过程补丁)、室内外温湿度(温度左轴/湿度右轴,4 位置同色
配对)、人体感应活动状态(3 个 `motion_state`banded `state_map`
none/small/medium/large → 0-11per-entity `line_color` 红/蓝/绿)。
空调实体引用为 `kong_diao_*``kong_tiao` 是笔误,W1N-230 修复;`grep -c
kong_tiao` 应为 0)。灯区:2×2 嵌套 grid(`grid_options: {columns: "full"}`
内层 `columns: 2`+ 4 卡统一 `mushroom-light-card`(显式 name、
`use_light_color: false`、内联亮度/色温控制),heading icon
`mdi:lightbulb-group`。heading badges:环境 4 温度(迷你/mini数显/数显/广州)、
大功率电器 空调/电脑当前功率、常驻负载 总功率
`sensor.dang_qian_zong_gong_lu`min_max **sum** helper`round_digits: 0`
任一源掉线 fail-closed → unknown)。常驻负载图卡级 `fill: 'tozeroy'` +
冰箱/主网络 per-entity `fill_color`(线色 20% 透明)+ 其余 5 条 `fill: false`
per-entity fill 逐系列退出,卡 JS `seriesConfig.fill !== false`)。
布局:视图 `type: sections` + `max_columns: 4`;灯/用电/环境/人体感应
`column_span: 4`,功率两图拆两个 `column_span: 2` 分区**并排**(等高 280px
桌面并排、手机回落堆叠;去卡内 title 省半宽图垂直空间)。
**分区/卡片是两套尺寸键,不可混用**:分区宽 = `column_span`
`hui-sections-view.ts` 缺省按 1 列渲染,绝不省略);卡片宽 =
`grid_options: {columns: <n|"full">}``hui-card.ts` 只读 `config.grid_options`
写在卡片上的 `column_span` 被静默忽略;缺省 12 列,分区内格 = 12 × 分区
span,故 span-4 分区里缺省卡片只有 1/4 宽)。
修改前备份:`/homeassistant/.lovelace-backups/dashboard-quick-*.json`
W1N-230 修复: `20260829-190256`round-2 改进: `20260829-194040`)。
- **Quick 时间范围扩容 (2026-09-13, VPS-92)**: 用户反馈「48 小时不够」。
各 timescale 卡可选档上调——大功率电器/常驻负载 `…,24h``+3d,7d`
环境 `6h,12h,24h,48h``+7d,14d,30d`;人体感应 `…,24h``+3d,7d`
用电量(按插座) `energy_time_ranges` `today,week,month,custom``+3mo`
**默认档未改**6h / 6h / today / 24h / 12h)。卡片 JS 只接受
`<n>m|<n>h|<n>d``parseDurationToMs` 正则 `/^(\d+)(m|h|d)$/`
仅 m/h/d,无 w)与命名档 `today|week|month|3mo|6mo|year|years|custom`
`energy_mode` 卡必须用后者。**数据下界注意**:scribe `sensor_minute`
目前最早只到 **2026-08-29**,所以 >15d 的档(14d 边缘、30d 明显)前半段
会是空白,等归档继续累积才好看。备份
`.lovelace-backups/dashboard-quick-20260913-190912-pre-timerange.json`
### 地图仪表盘:CARTO keyed tiles via `custom:map-card` (verified 2026-08-30, W1N-261)
- **背景:** CARTO 自 2026-08-26 起对无 key 栅格瓦片打 "API KEY REQUIRED"
水印,内置地图卡/zone 编辑器全部受影响。Core 2026.8.3 的 `MapCardConfig`
**没有任何瓦片配置项**frontend 20260729.7 源码核对:
`setup-leaflet-map.ts` 硬编码 CARTO voyager URL)。上游修复是 2026.9.0b1
起改用 OSMF 矢量瓦片(frontend PR #53816),stable 预计 2026-09-02 前后。
- **变更:** 「地图」仪表盘(url_path `map`storage)唯一 map 卡替换为
`custom:map-card`[nathan-gs/ha-map-card](https://github.com/nathan-gs/ha-map-card)
**v1.16.0**,手动安装非 HACS):`tile_layer_url` =
`https://{s}.basemaps.cartocdn.com/rastertiles/voyager/{z}/{x}/{y}.png?key=<CARTO_KEY>`
(配 `tile_layer_options: {subdomains: abcd, maxZoom: 20}` + OSM/CARTO
attribution)。实体不变:2 person + 4 zonezone 用 `display: icon` +
`circle: auto`circle 读实体 `radius` 属性画半径圈)。
- **CARTO key 是 secret**: 只存在于服务端 lovelace 存储(dashboard `map`
的卡片配置)和用户本人处;勿写入本仓库或 Linear。
- **文件/资源:** `/homeassistant/www/community/ha-map-card/map-card.js`
root:root 644678554 Bsha256
`f30dfb606e858d2216d5198d8cf758ce956d127006ebd7d66d4329153a247ec2`);
Lovelace resourcestorageid `9d2b50b52c60420d89ebd041f722cf60`
`/hacsfiles/ha-map-card/map-card.js`type moduleWS
`lovelace/resources/create`2026.8 参数名 `res_type`)。升级 = 手动替换
该文件(不在 HACS 管理下,浏览器需强刷)。
- **备份:** `/homeassistant/.lovelace-backups/dashboard-map-map-20260830-133714.json`
(还原 = 把备份里的 `views[0].cards[0]` 写回后再 WS `lovelace/config/save`
url_path `map`)。
- **验证 8/30:** 同瓦片无 key=水印 / 带 key=干净(256×256 PNG 视觉对比);
resource HTTP 200 text/javascriptWS 读回卡片配置(type/entities/key/
attribution/options)全部符合;HA 主机 `curl -4` 带 key 瓦片 200。
- **Follow-up:** Core 升 2026.9.0 stable 后内置地图/zone 编辑器自动切
OSMF 矢量瓦片;届时可保留 custom 卡(继续 keyed CARTO)或用备份还原
内置卡。zone 编辑器等其余内置地图的水印在 2026.9 前无解。
## Known issues
@@ -407,6 +663,87 @@ advertised correctly over v4+v6. Both ESP32-C2 bulbs now announce `_matter._tcp`
:5540 sessions** (device-side failure mode C; no errors logged — see
[docs/matter-pairing-troubleshoot.md §8](../docs/matter-pairing-troubleshoot.md)).
## 马桶换气电源(Matter 插座,半计量)+ 电量估算 (2026-09-13)
**设备**Matter `Smart Plug`SIXWGH`model_id 3596`hw 1.0 / sw 1.3.0),node 18
(0x12)`device_id 5ef1850953466d6e7a9c6b901fbebe1c`config entry
`01JF51VQ48PGJGXX3RNAG6MVAA`,区域**卫生间** (`wei_sheng_jian`)label `power`
2026-09-13 17:58 CST 配对。实体:
`switch.wei_sheng_jian_ma_tong_huan_qi_dian_yuan`(插座)、
`sensor.…_dian_yuan`(电源 W)、`sensor.…_dian_ya`(电压 V)、
`sensor.…_you_gong_dian_liu`(有功电流 A)、`sensor.…_dian_li`(电力 kWh
**永久 unknown**)。
**根因(实测 Matter 属性,node 18**:电量簇 0x0091 `FeatureMap = 13`
(IMPE|CUME|PERE,即**声明**支持导入/累计/周期电量),但
`CumulativeEnergyImported (0x0001)` 恒为 `null``PeriodicEnergyImported
(0x0003)` 带载也恒为 `{Energy: 0}``CumulativeEnergyExported (0x0002)`
不存在(EXPE 未声明,自洽)。HA 只用 `CumulativeEnergyImported` 建能量实体
`components/matter/sensor.py:1083``allow_none_value=True`)→ 该实体
**永远不会出数**。**功率计量本身正常**:0x0090 `FeatureMap = 2` (ALTC)
Voltage / ActiveCurrent / ActivePower 都随负载变化(实测 220.3 V / 118 mA /
24.7 WHA `电源` 0.0→24.9 W 有历史)。厂商 `update` 实体报无新固件。
**处理(方案 A:功率积分补电量)**
- 新建 **Integration (Riemann sum) 辅助元素**config entry
`01M2D53T188FW8WEC547ENHSVH`domain `integration`state `loaded`),
source `sensor.wei_sheng_jian_ma_tong_huan_qi_dian_yuan_dian_yuan`
`method: trapezoidal``unit_prefix: k``unit_time: h``round: 3`
`max_sub_interval: 60s`
- 实体 `sensor.wei_sheng_jian_ma_tong_huan_qi_dian_yuan_energy`(创建时 HA
自动生成 `…_dian_yuan_ma_tong_huan_qi_dian_yuan_dian_liang`,随后立即
`config/entity_registry/update` 改名为 `<插座>_energy` 以对齐约定;
该实体新建、无引用,改名安全),friendly name「马桶换气电源 电力」,
unit kWh、`device_class: energy`、**`state_class: total`**——能源仪表盘
允许 `TOTAL``TOTAL_INCREASING``components/energy/validate.py:279`)。
- **能源仪表盘** (`/energy`)grid 源 `[8]``…_dian_li` 改为 `…_energy`
其余 8 条插座源未动。注意这 9 条「插座」全部以 `type: grid` 注册,被当作
全屋用电代理;`switch` 卡片所在的 Grid 卡片此前第 9 行是空的,即本次修复点。
- **Quick 仪表盘**:「用电量(按插座)」图第 11 项由 `…_dian_li` 改为
`…_energy`;新增 `column_span: 2` 的「开关」区块(heading + tile
`switch.…` + `toggle` feature + 功率徽标)→ 视图 6→7 分区。
**口径警告**`…_energy` 是**估算值**Riemann 积分,只在 HA 运行期间累计、
非账单级),与另外 8 个原生计量插座的累计电量口径不同;功率传感器更新
间隔约 510 s(实测 24.9/24.8/25.0 W 抖动),加 `max_sub_interval: 60s`
保证静默时也继续累计。
**Agent 侧建辅助元素的方法(2026-09-13 实测)**HA 的 config flow 走
**REST**WS 只有 `config_entries/flow/progress|subscribe`,没有 start)。
经 supervisor 代理即可,无需 HA 长连接/长寿命 token:
```bash
# SUPERVISOR_TOKEN 由 sudo -n -i 提供
curl -s -X POST -H "Authorization: Bearer $SUPERVISOR_TOKEN" \
-H "Content-Type: application/json" -d '{"handler":"integration"}' \
http://supervisor/core/api/config/config_entries/flow # → {flow_id, step_id:"user", data_schema}
curl -s -X POST -H "Authorization: Bearer $SUPERVISOR_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name":"…","source":"sensor.x","method":"trapezoidal","round":3,
"unit_prefix":"k","unit_time":"h","max_sub_interval":{"minutes":1}}' \
http://supervisor/core/api/config/config_entries/flow/<flow_id> # → create_entry
```
`auth/long_lived_access_token` 在 supervisor 代理身份下**失败**
`unknown_error`),故无法用长寿命 token 开浏览器会话;`DurationSelector`
的值是 `{"minutes":1}` 形式(`cv.time_period`)。
**备份/回滚**`.lovelace-backups/dashboard-quick-20260913-181251-pre-ma-tong-plug.json`
(改动前原件)、`…-20260913-183210-pre-repoint.json`(改名/换源前);
`.ha-backups/energy-20260913-183135-pre-ma-tong-repoint.json`(能源 prefs)。
回滚 = 把能源 prefs 的源 [8] 指回 `…_dian_li` + 还原 Quick 面板 JSON
如需彻底放弃估算电量 = 删除 config entry `01M2D53T188FW8WEC547ENHSVH`
**验证 (2026-09-13 18:3x)**`…_energy` 0.002→0.003 kWh 且随 24.6 W 负载
增长(换气扇关掉后回落 0.0 W,累计值保留);`recorder/list_statistic_ids`
已含该实体;Quick 面板 WS 读回 7 分区、用电量图 11 项指向新实体、旧
`_dian_li` 引用 0 处;能源 prefs 读回 9 源、第 9 条为新实体。
**`energy/validate` 已全绿**9 源 0 issue):创建后 ~5 min 内曾报
`statistics_not_defined`(recorder 的统计任务周期是 5 min,`statistics_meta`
行由该任务建立),18:39 复核时已自动消失——建辅助元素后**不要**把这条
瞬时告警当作失败。
## Related docs
- [runbooks/home-assistant-maintenance.md](../runbooks/home-assistant-maintenance.md) — `ha` CLI maintenance runbook + [script](../runbooks/scripts/ha-maintenance.sh); custom-component zip install is §7
+18
View File
@@ -185,6 +185,23 @@ dig @202.91.35.141 SOA wsvc.info +short
On-server docs: `/opt/pdns/README.md`, `CHANGELOG.md`.
## Disk / logging (VPS-81, 2026-09-02)
Root disk cleanup performed (runbook: [host-disk-cleanup](../runbooks/host-disk-cleanup.md)):
- Root `/` (20G vda1): 76% used → **38% used** (15G → 7.1G; free 4.7G → 12G).
- **AGH log flood root cause fixed**: `/opt/adguard/conf/AdGuardHome.yaml`
`log.verbose: true → false` (backup `AdGuardHome.yaml.bak-20260902-vps81`).
Verbose debug was streaming to stderr → container `json.log` (~120MB/day);
`log.file: ""` makes AGH's own rotation keys inert. Restart only (no recreate).
- Journald capped: `/etc/systemd/journald.conf.d/00-vps81.conf`
`SystemMaxUse=200M`; journal vacuumed to ~96M.
- Docker: engine **29.7.2**; 14 unused images removed (kept `pdns-auth-50:5.0.5`
rollback pin); 12 orphan anonymous volumes + build cache pruned. In-use
volumes intact (`pdns_dbdata`, `b594d738…` PG data, `e855d078…` backup).
- Follow-up: re-check AGH `json.log` growth **2026-09-09** (one-week checkpoint);
global docker log rotation only if still needed.
## Verified
Last checked: **2026-08-01 21:40 CST** — operational; docs audit recorded.
@@ -194,3 +211,4 @@ Last checked: **2026-08-01 21:40 CST** — operational; docs audit recorded.
- `only-notify=` + `also-notify=202.91.35.141`; MASTER `domains.master` cleared
- https://pdns.wsvc.info → **302**; https://pgweb.wsvc.info → **401**
- Hardening backlog: API/DB credential rotation + TSIG rotate (see upstream doc)
- 2026-09-02 (VPS-81): post-cleanup verified — 10 containers Up (adguardhome healthy), DNS SOA/NS + web endpoints OK; see Disk/logging section above.
+12 -4
View File
@@ -8,8 +8,8 @@
| IPv4 | `192.168.55.15` (LAN55) |
| DNS | (none) |
| SSH | `ssh -4 windy@192.168.55.15`key auth 已验证可用 2026-08-29agent 沙箱用 `ssh -F /dev/null -o BatchMode=yes`password auth 亦可) |
| Host | PVE 管理的 QEMU VMi440FX),Debian 13 (trixie),内核 6.12.105 |
| Resources | 2 GB RAM / 30 GB disk26 G 空闲) |
| Host | PVE 管理的 QEMU VMi440FX**VMID 100**),Debian 13 (trixie),内核 6.12.105;宿主机 **pve2 `192.168.55.25`**Proxmox 9.2.2SSH `root@192.168.55.25``onboot: 1`QEMU guest agent 已装;2026-08-31 补记) |
| Resources | 3 GB RAM08-30 13:58 由 2G 上调、删除 balloon/ksm/shares 后重启生效)/ 30 GB disk26 G 空闲) |
| Docker | 29.7.2;容器 `timescaledb` = `timescale/timescaledb:latest-pg18`PG **18.6** + TimescaleDB **2.29.2**Apache-2.0 版) |
| Ports | `192.168.55.15:5432`PGIPv4 only);`192.168.55.15:8081`pgweb GUIbasic auth |
@@ -17,8 +17,8 @@
| DB | Owner | Size | 用途 |
|---|---|---|---|
| `hass` | hass | ~14 MB | HA recorderstates/events/statistics),客户端 HAOS `192.168.55.11` |
| `scribe` | postgres | ~11 MB | HA scribe 集成(entities/areas/devices 注册表同步 + `states_raw` hypertable |
| `hass` | hass | ~406 MB2026-09-13 | HA recorderstates/events/statistics),客户端 HAOS `192.168.55.11` |
| `scribe` | postgres | ~2.6 GB2026-09-13 | HA scribe 集成(entities/areas/devices 注册表同步 + `states_raw`/`events` hypertable + `csg_history` 长期归档表);体积由 `sensor_minute` 图表管道主导(2.26 GB),见 Known issues |
| `postgres` | postgres | ~9 MB | 默认库 |
## Ops notes
@@ -35,17 +35,25 @@
- **备份**:由 `pg-backup` 容器接管(2026-08-29),宿主机 cron 与 `/opt/database/pg-backup.sh` 已退役。恢复用 `pg_restore`custom format)——2026-08-29 已实测还原 hass 库 dumpstates 10014 行)成功。
- **认证**:外部连接 scram-sha-256(密码必填,改密码有效);容器内 loopback 为 trust(官方镜像默认)。
- **回滚**:旧启动命令保留在 `/opt/database/run`(容器无状态,数据在 /srv/pgdata);旧匿名卷 `9375195843b950f4e04c34872409ca095e1136520dd019a8e86e2794be06c236`(根盘 ~82M)保留作兜底,确认稳定后可 `docker volume rm`
- **开机自愈**2026-08-30):新增 systemd oneshot `pgdb-compose.service`enabled,源码在仓库 `compose/pgdb/pgdb-compose.service`):`After=network-online.target docker.service`,开机后幂等执行 `docker compose up -d`,重试直到 `192.168.55.15:5432` 监听,重试耗尽 `--force-recreate` 兜底(数据在 bind mount,无损)。原因:2026-08-30 开机竞态——docker 恢复容器时 VM IP 尚未可绑(EADDRNOTAVAIL),timescaledb/pgweb 启动失败且 docker 不重试。手动重跑:`sudo systemctl restart pgdb-compose.service`
- 本机无防火墙(ufw/nft/iptables 均未装)——待办:如要彻底隔离可加 ufw 白名单 192.168.55.11。
- `/opt/database/backups/` 根下残留 `*-2026-08-29_1359.dump`(compose 化之前旧备份机制产物)与 `backup.log`——健康检查只看 `daily/`,残留可清理。
- **Runbooks**[pgdb-health](../runbooks/pgdb-health.md)(只读健康检查)、[pgdb-restore](../runbooks/pgdb-restore.md)pg_restore 还原)、[pgdb-update](../runbooks/pgdb-update.md)(镜像/compose 升级)。
- **CSG 长期归档(2026-08-29, W1N-243**`csg_history` 表(`period date / kind('day'|'month') / usage_kwh / cost / ladder / balance / updated_at`PK(period,kind)`GRANT SELECT TO hass`)保存南方电网有价值数据:day = 逐日(昨日用电/费用/阶梯/余额,2026-07-01 起),month = 当月累计(用电/费用,2025-01 起)。由 TimescaleDB 每日任务 **1008** `csg_daily_snapshot()`22:30 Asia/Shanghai**TS job 非 pg_cron**,本库未装 pg_cronupsert 维护:取「最新有值行」防瞬态 unknown 竞态;日费用缺原生 `latest_day_cost` 时回退 = 昨日用电 × 当前档费率(模板 `csg_current_ladder_tariff` 0.639);月费用回退模板 `csg_this_month_ladder_cost`。验证:day 08-28 = 7.66 / 4.89474 / 二档 / 0month 08 = 302.47 / 180.28。回填来源:集成 attributes `history_data`59 天)+ `by_month`(19 月)——08-29 前唯一残存历史。回滚:`DROP TABLE csg_history` + `SELECT delete_job(1008)`
## Known issues
- 2026-09-13**`sensor_minute` 体积构成与压缩窗口(只读诊断,暂不处理)**。`scribe` 库 2.6 GB = `sensor_minute` **2.26 GB**850 万行 / 16 天,约 5659 万行/天 = 331 实体 × 1440 分钟 LOCF+ `states_raw` 290 MB + `events` 1.5 MB`hass` 库另 406 MB。2.26 GB 中 1.50 GB 是 chunk `[09-03,09-10]`、0.76 GB 是 `[09-10,09-17]`,**都还没到压缩窗口**——TimescaleDB 的 `compress_after`**chunk 结束时间**判断(09-10 结束 + 7 天 = **09-17** 才合格),所以「7 天 chunk + 7 天 compress_after」的设计下限就是盘上常驻近 14 天原始数据;已压缩的 `[08-27,09-03]` 从 1.04 GB → **1.5 MB**LOCF 重复度极高,~700:1)。任务 1005 健康(30 成功 / 0 失败,最近 09-13 04:18 跑过但无合格 chunk);1002/1003/1006/1007 亦全 Success。稳态估算 ≈ 2 个未压缩 chunk(3–4.5 GB+ 已压缩归档(约 1.5 MB/周 ≈ 80 MB/年)≈ **45 GB 平台期**pgdata 卷 32 G 当前用 3.2 G,可用 27 G,**无需处理**。复查点 **2026-09-17 之后**`_hyper_4_6_chunk` 应转为 `compressed=true` 且库体积回落;若仍为 false 才需动手(手动 `compress_chunk()` 或调小 `compress_after`)。可选调优:chunk 间隔 7 天 → 1 天 + `compress_after` → 2 天,把常驻未压缩量压到 <1 GB(`set_chunk_time_interval` 只对新 chunk 生效,旧 chunk 不重切)。诊断命令:`select chunk_name, is_compressed from timescaledb_information.chunks where hypertable_name='sensor_minute';` + `pg_database_size('scribe')`。**注意:这是 pgdb 侧对象,HA/scribe 的 `retention_states` 管不到它;HA 侧唯一杠杆是少记/少画(等于砍图)。**
- 2026-08-29HA 侧 HACS 集成 `custom_components.scribe`YAML `scribe: db_url:`,连 `scribe` 库)建表被拒(`permission denied for schema public`hass 无 CREATE 权限),之后持续报 `relation "entities" does not exist`。**已解决**:① `GRANT CREATE ON SCHEMA public TO hass;`scribe 库)② 重启 HA Core 触发重跑建表。重启后自动创建 `entities`1591 行)/`users`/`areas`/`devices`/`integrations`/`states_raw` 表并启用 TimescaleDB 时间序列能力。报错已停止(最后一条 06:06 UTC),`states_raw` 持续写入。2026-08-29 复查:scribe 现有**两个** hypertable——`states_raw`segmentby `metadata_id`、orderby `time`)与 `events`segmentby `event_type`、orderby `time`),均 1 维 `time`;压缩已配置(`timescaledb_information.compression_settings` 可见对应行;2.29.x 该视图无 `compression_enabled` 列)。
- 2026-08-29**timescale reader 图表对象**(配套 hass 的 `timescale_database_reader` 集成 + `timescale-plotly-card`,上游 SQL `remmob/timescale_database_reader` `SQL/scribe/01+02` @ `bb8776a`,以 postgres 执行):`sensor_minute_aggregate` 连续聚合(1 分钟桶,last(state)/last(value),实时聚合开启)+ `sensor_minute_aggregate_entity` 视图(join `entities`+ `sensor_minute` hypertable`minute`/`entity_id`/`state`/`value`,LOCF 前向填充)。任务:1005 `sensor_minute` 压缩(7 天)、1006 `sensor_minute` 保留(10 年)、1007 `every_minute_refresh` 每分钟增量刷新(含 5 分钟回溯窗口修正)。授权:`GRANT SELECT ON sensor_minute_aggregate, sensor_minute_aggregate_entity, sensor_minute, entities TO hass`。种子 19529 行(331 实体,自首个数据点起)。**刻意跳过**了上游脚本对 `states_raw` 的 3 个月保留 + 压缩策略语句——与"`states_raw` 永久归档"定位冲突,如需磁盘回收属用户决策(scribe 自己的压缩任务 1000/1001 未动)。
- 2026-08-29**`sensor_minute_refresh` 本地补丁(类比 tianqi 补丁,重跑上游 02 SQL 后需重打)**:值 CASE 的 `ELSE 0``ELSE NULL`。原因:scribe 对 unavailable 分钟 value 为 NULL,上游刷新过程兜底写 0;对差分模式的用电图,0→计数器回升会把插座的**生命周期累计值**(最高 1588 kWh)算进掉线那一小时。同日一次性清理既有脏 0:头部占位行 DELETE 505 行(各实体首次非零分钟之前的 value=0);`sensor.%_energy` 与温湿度实体的 value=0 → NULL(10+16 行,物理上不可能的真 0,图表渲染为断点)。功率实体的中途 0 是真实待机读数,保留。
- `hass` 库的 recorder 表仍为普通表(无 hypertable);`scribe` 集成负责时间序列历史(`states_raw` + `events` hypertable)。
## Verification history
- 2026-08-31**13:58 重启根因确认,非停电**W1N-263):pve2`192.168.55.25`)任务日志显示 08-30 **13:58:00 `root@pam` 在 PVE Web UI 修改 VM 100 配置**`-delete allow-ksm,balloon,shares -memory 3072`),**13:58:06 点 Reboot**`qmreboot` → 客机 13:58:08 干净 ACPI 关机 → 13:58:13 自动重启)。宿主机全程在线(08-30 09:00 开机至今连续运行 1d12h+),`.66.26` PVE 及各 VM 均无重启——排除停电。HA recorder 在窗口(13:58:4647)报 2 次 `Connection refused`,DB 恢复后自动重连,**无数据丢失**(`hass.states`/`scribe.states_raw` 13:5514:02 逐分钟无缺口,recorder 内存队列吸收回写)。13:58:47 三容器已起,13:58:56 自愈单元 `pgdb-compose.service` 执行成功——本次自愈按设计工作。同日下午 12:54–12:55 另有一次**客机内自重启**(无 PVE 任务,工作站 SSH 会话相邻)。08-29 22:19→08-30 09:00 宿主机停机 10h41m 为**干净关机**(systemd 有序关闭,非停电)。
- 2026-08-30**开机竞态故障 + 修复**W1N-260):09:01 开机后 docker 恢复容器时绑定 `192.168.55.15:5432/8081` 失败(EADDRNOTAVAIL)→ timescaledb/pgweb 停摆至 12:16pg-backup 开机备份失败(解析不到 timescaledb)→ unhealthy。12:22 `docker compose up -d --force-recreate` 修复(三容器回 `database_default`、端口发布、今日备份、pgweb 恢复);用户重启 HA Core 后写入管道恢复。12:43 新增开机自愈 unit `pgdb-compose.service`enabled,已实测幂等 reconcile)。pgdb-health 8 项全绿。
- 2026-08-29:首次检查(只读)+ 修复 scribe 权限 + 安装夜间备份。见 Linear vps 项目登记。
- 2026-08-29**compose 改造完成**W1N-227,用户已验收):裸 `docker run``/opt/database/docker-compose.yml` 三服务(timescaledb + pgweb + pg-backup);superuser 换强密码;端口收紧 IPv4;备份容器化(TZ=Asia/Shanghaicron 02:00 本地);`pg_restore` 还原实测通过;pgweb UI 用户确认可查 hass/scribe 数据。源码在仓库 `compose/pgdb/`
- 2026-08-29**运维 runbook 落地**W1N-228,已验收):新增 `runbooks/pgdb-health.md`(只读,8 项诊断全绿)、`pgdb-restore.md`(流程式,temp-DB 安全还原 + 审批门)、`pgdb-update.md`(门控命令式,回滚=/opt/database/run + 旧卷);README 索引与 validate-repo.sh 分类同步更新;runbook 命令已对活主机逐条实测(含 `pg_restore -l` 校验当日 dump)。同日修正:SSH key auth 可用(facts 原记"密钥未安装"已过时);scribe 新增 `events` hypertable。
- 2026-08-29**CSG 长期归档 + recorder 365d**W1N-243):建 `csg_history` 表 + attributes 回填(逐日 59 + 逐月 19)+ 每日任务 1008(函数 v2:最新有值行读取、日费用阶梯回退);hass `purge_keep_days` 30→365(备份 `configuration.yaml.bak-20260829-purge365`)。见 Linear vps W1N-243。
+45 -1
View File
@@ -24,6 +24,7 @@ ssh -4 windy@synapse.chans.xyz
| DB | ESS embedded PostgreSQL 17 (PVC 20Gi, local-path) |
| Cache | ESS embedded Redis (PVC 2Gi) |
| Chart | `oci://ghcr.io/element-hq/ess-helm/matrix-stack`, version `26.7.2` |
| Plane | Helm `plane-ce-1.8.0` (app `v1.4.1`), namespace `plane` — self-hosted Plane project management |
### Matrix service endpoints
@@ -51,8 +52,50 @@ All other ports internal only (no K3s API, no database, no Redis exposed).
- `ess` — all ESS workloads (Synapse, MAS, Element, Postgres, Redis, HAProxy)
- `matrix-system` — cluster base resources (ResourceQuota, LimitRange, mrtc-placeholder)
- `plane` — Plane project management (Helm release `plane-app`)
- `cert-manager` — cert-manager
## Plane (project management)
Self-hosted [Plane](https://github.com/makeplane/plane) on the same K3s node, deployed via the official `plane-ce` Helm chart.
| Item | Detail |
|------|--------|
| Release | `plane-app` (ns `plane`), chart `plane-ce-1.8.0`, app `v1.4.1`, revision 1 |
| URL | https://plane.chans.xyz |
| Install date | 2026-09-01 |
| Values source | `/home/windy/plane-k3s/values.yaml` (plain file, not a git repo) |
| Images | `artifacts.plane.so/makeplane/*` (`plane-frontend`, `plane-backend`, `plane-admin`, `plane-live`), pullPolicy `Always` |
| Ingress | Traefik `IngressRoute` `plane-app-ingress``/`→web, `/api` `/auth`→api, `/spaces`→space, `/god-mode`→admin, `/live`→live, `/uploads`→minio; `maxRequestBodyBytes` 20Mi |
| TLS | Own namespace `Issuer` `plane-app-cert-issuer` (HTTP-01, LE prod, `admin@chans.xyz`); cert `plane-app-ssl-cert` (CN `plane.chans.xyz`) |
| DB | Bundled Postgres `15.7-alpine` (PVC 5Gi, local-path) |
| Cache/queue | Bundled Redis (PVC 100Mi), RabbitMQ `3.13.6-management-alpine` (PVC 100Mi) |
| Storage | Bundled MinIO (`minio/minio:latest`, root user `admin`, PVC 5Gi) — S3 for uploads/docs |
| Resources | Every workload: cpu 50m/500m, mem 50Mi/1000Mi, replicas 1 |
| SMTP | Not configured (no `smtp` values) — Plane invites/password resets won't email yet |
Workloads (all 1/1 Running): 7 Deployments (`plane-app-{admin,api,beat-worker,live,space,web,worker}-wl`) + 4 StatefulSets (`plane-app-{minio,pgdb,rabbitmq,redis}-wl`); init Jobs `api-migrate-1` / `minio-bucket-1` Completed. All PVCs Bound on `local-path` (root disk).
### Plane configuration notes
- **`planeVersion: v1.4.1`** pinned in values.yaml; chart tracks Plane's own tags.
- **Secrets**: Helm-generated Opaque secrets (`plane-app-app-secrets`, `-doc-store-secrets`, `-pgdb-secrets`, `-rabbitmq-secrets`, `-live-secrets`); `requireExplicitSecrets: false`. Values live in `$SECRET_KEY`, `DATABASE_URL`, `AMQP_URL`, `REDIS_URL` etc.
- **Sentry / CORS**: `sentry_dsn` and `cors_allowed_origins` empty (defaults fine for single-host).
- **MinIO is `latest` tag** — pin a version for reproducibility.
- **Backup**: NOT covered by `/var/backups/matrix` (which is paused anyway) — Plane Postgres/MinIO PVCs have no backup tier yet.
### Plane verification
```bash
# Release + workloads
sudo helm list -A
sudo k3s kubectl -n plane get deploy,sts,pods -o wide
# Cert + ingress
sudo k3s kubectl -n plane get certificate,ingressroute
# Endpoint
curl -4 -s -o /dev/null -w '%{http_code}\n' https://plane.chans.xyz/
```
## Local backup
| Item | Detail |
@@ -62,7 +105,7 @@ All other ports internal only (no K3s API, no database, no Redis exposed).
| Retention | 7 days |
| Disk warning | 80% (healthcheck), 90% (backup stops) |
| Content | Planned: PostgreSQL `synapse` + `mas` logical dumps, media store archive, `/etc/matrix-bootstrap` |
| Status | **Not operational** — no current Matrix backup or recovery tier |
| Status | **Not operational** — no current Matrix backup or recovery tier. **Plane data (its own Postgres + MinIO PVCs in ns `plane`) is also not covered by any backup.** |
## Health checks
@@ -101,5 +144,6 @@ diagnosis and imperative recovery work.
- MatrixRTC / Element Call / LiveKit / Coturn not deployed (`mrtc.chans.xyz` reserved only)
- SMTP email not yet configured (requires manual secret bootstrap followed by a
reviewed Ansible stack deployment)
- Plane `minio` image uses `latest` tag (pin a version)
- No off-site Restic backup
- Single-node K3s (no HA for control plane)
+26 -12
View File
@@ -5,12 +5,12 @@
| Role | Multi-service VPS (Vaultwarden, Traefik, Soft Serve, …) |
| SSH | `ssh -4 windy@us2.wsvc.info` (prefer IPv4 from WSL) |
| IPv4 | `193.9.44.165` |
| Also DNS | `auth.wsvc.info` → this host; `repo.windy.me` → this host (Soft Serve) |
| Also DNS | `auth.wsvc.info` → this host; `repo.windy.me` → this host (Gitea) |
| Public HTTPS | Traefik on `:80` / `:443` (`/opt/traefik`) |
## Vaultwarden (Bitwarden-compatible)
**Status: operational** (Postgres live, HTTPS 200, healthy containers, SMTP AUTH OK — last probe 2026-08-01 18:55 CST).
**Status: operational** (Postgres live, HTTPS 200, healthy containers, SMTP AUTH OK — last probe 2026-08-29).
Upstream docs: [docs/vaultwarden-upstream.md](../docs/vaultwarden-upstream.md)
@@ -21,9 +21,9 @@ Upstream docs: [docs/vaultwarden-upstream.md](../docs/vaultwarden-upstream.md)
| Env file | `/opt/vaultwarden/.env` |
| Admin overrides | `/opt/vaultwarden/vw-data/config.json` (**wins over env**) |
| Public URL / `DOMAIN` | `https://auth.wsvc.info` |
| Image | `vaultwarden/server:1.37.1` (pinned) |
| Image | `vaultwarden/server:1.37.2` (pinned) |
| Live DB | **Postgres 16** (`vw-db` / service `pg`) via compose `DATABASE_URL` |
| Data (probe) | users=1, ciphers=1327 |
| Data (probe) | users=1, ciphers=1360 |
| Cold SQLite | `backups/sqlite-cold/db.sqlite3.pre-pg-20260801` (not used live) |
| Pre-migrate backup | `backups/pre-pg-migrate-20260801_161204/` |
| Data dir | `./vw-data``/data` (attachments, rsa keys, `config.json`) |
@@ -50,7 +50,7 @@ Upstream docs: [docs/vaultwarden-upstream.md](../docs/vaultwarden-upstream.md)
| Container | Status |
|-----------|--------|
| `vaultwarden` | Up (healthy), `vaultwarden/server:1.37.1` |
| `vaultwarden` | Up (healthy), `vaultwarden/server:1.37.2` |
| `vw-db` | Up (healthy) — **live** Postgres |
| `vaultwarden-backup` | Up (`pg_dump`) |
| `vaultwarden-pgweb` | Exited (profile `debug`) |
@@ -80,19 +80,33 @@ ansible-playbook playbooks/compose-reconcile.yml --limit vaultwarden \
| Container | Status | Image / notes |
|-----------|--------|---------------|
| `soft-serve` | Up | `ghcr.io/charmbracelet/soft-serve:latest` (`repo.windy.me:2222`) |
| `gitea` | Up | `gitea/gitea@sha256:1c17ecaead42e…` (1.27.3-rootless) — SSH `repo.windy.me:2222`, web `https://repo.windy.me` |
| `gitea-backup` | Up | alpine + sqlite3/rsync sidecar (daily backup 02:00 / prune 03:00, crond) |
### Gitea (replaced Soft Serve 2026-09-18; [Plane VPS-94](https://plane.chans.xyz))
- `/opt/gitea/compose.yml` (+ `Dockerfile.backup`, `scripts/`, `config/app.ini`, `data/`, `secrets/`, `backups/`); 镜像: `compose/gitea/`(参考, 服务器文件为准)
- **rootless 镜像** uid 1000:1000; SQLite `/opt/gitea/data/data/gitea.db`; repos `/opt/gitea/data/data/git/repositories/`; app.ini `/opt/gitea/config/app.ini`(600, 含 SECRET_KEY)
- SSH: 内置 server 容器内 `:2322`(`SSH_LISTEN_PORT` 非特权), Traefik TCP entrypoint `ssh`(`:2222``gitea:2322`, `HostSNI(*)`, `tls=false`) on `vw-net`; clone URL `ssh://git@repo.windy.me:2222/windy/<repo>.git`(owner 段 `windy`)
- **host key 复用 soft-serve**(`SSH_SERVER_HOST_KEYS=/secrets/soft_serve_host_ed25519`, ed25519, 指纹 `SHA256:PdxZRe74…`): 客户端 known_hosts 零变更; 仅公钥认证(密码认证未启用)
- Web: `https://repo.windy.me`(Traefik websecure + letsencrypt); `DISABLE_REGISTRATION=true`, Actions 关闭; 管理员 `windy`(凭据仅存服务器 `/opt/gitea/.admin-credentials`, 勿入库/入 Plane)
- 仓库: 16 个(顶层 11 + `cdia/` 4 + `windyboy/go-caatsm`), 2026-09-18 自 soft-serve `push --mirror` 迁移, 逐仓 `ls-remote` ref 全集 + HEAD symref 两端一致; 可见性仅 `dotfiles-personal` private, 其余 public(与 soft-serve 现状一致)
- 备份: sidecar 每日 02:00 → `backups/gitea_<TS>/{app.ini.tar.gz, gitea.db, repos.tar.gz}`(app.ini 含恢复必需 SECRET_KEY), 03:00 prune 保留 14 份; 已验证手动备份产物 109.9M
- 回滚: `/opt/soft-serve` 未删(compose stop + sidecar 停, 数据与旧备份冻结保留), 回滚 = Traefik `:2222` 指回 `soft-serve:23231` + 客户端 remote 回改旧无 owner 段路径; 观察 24 周后清理(历史: W1N-244~248)
| `traefik` | Up | `traefik:v3.6.2` (`/opt/traefik`, public `:80`/`:443`) |
| `nghttpx-proxy` + `squid-backend` | Up | HTTP forward-proxy stack (`/opt/nghttpx`), network `nghttpx_internal-net`; details TBD |
Directories for `authelia`, `conduit`, `dendrite`, `mastodon`, `rustdesk`, `zitadel`, etc. exist under `/opt` but have no running containers; treat them as dormant, not documented services.
**Disk cleanup 2026-09-18** ([Plane vps VPS-93](https://plane.chans.xyz)): root 71% → **23%** (~33G freed) keeping soft-serve / vaultwarden / traefik (nghttpx kept running per operator choice). Removed: unused Docker images + orphan volumes (incl. `zitadel_data` 801M), dormant `/opt` dirs (dendrite + its disabled `dendrite.service` unit, mastodon, dailysync, keycloak, media-repo, authelia, conduit, npm, manager, fusion, zitadel, rustdesk), rootless podman storage (6.4G stale goauthentik), home dev caches, apt cache, journal 3.8G→162M (+`SystemMaxUse=200M` drop-in, active next boot), truncated container logs (nghttpx 550M / traefik / squid). Follow-up: nghttpx-proxy logs grow ~25M/day (INFO per-connection); root-cause log-level/rotation fix still open (needs container restart approval).
Remaining running services on this host: `gitea`, `vaultwarden` stack, `traefik`, `nghttpx-proxy` + `squid-backend` (undocumented forward proxy, `/opt/nghttpx`). `/opt/soft-serve` kept stopped as rollback (24 weeks, data intact). `/home/windy/authelia` (76M) left in place — outside approved cleanup scope.
## Verified
Last checked: **2026-08-01 18:55 CST** — operational.
Last checked: **2026-09-18** — operational; disk cleanup done (see note above, Plane vps VPS-93). Prior full probe: 2026-08-29.
- `vaultwarden` + `vw-db` healthy; `DATABASE_URL``pg:5432/vaultwarden`
- `https://auth.wsvc.info/` **200**, `/admin` **200**, `/api/config` OK (`disableUserRegistration: true`)
- Identity wrong-password → **400** business error (DB readable, not 500)
- SMTP: container → `mx2:587` OK; STARTTLS cert CN=`mx2.windy.me`; **AUTH OK** with effective `config.json` password (synced with `.env` / `.smtp-credentials`)
- LE cert CN=`auth.wsvc.info`
- PG counts: users=1, ciphers=1327
- SMTP: container → `mx2:587` OK; **AUTH OK** with effective `config.json` password (synced with `.env` / `.smtp-credentials`, fingerprint match)
- PG counts: users=1, ciphers=1360
- Image `vaultwarden/server:1.37.2` (**upgraded 2026-08-29** from 1.37.1; required for Bitwarden clients 2026.8.0+); post-upgrade 404 fixed by Traefik restart, then 200
- vps-health local check **installed 2026-08-29** (`vps-healthcheck.timer` daily 06:15 + `/usr/local/lib/vps-health/run`); `health-report.yml --limit vaultwarden` now passes (**ok**, was failing due to missing check infra + script bugs fixed: trim_blocks render, pgweb debug-profile false positive, SMTP probe moved host-side since image lacks python3)
+2 -1
View File
@@ -17,7 +17,7 @@ the software deployed there, see [the LAN overview](../docs/lan-overview.md).
| mx2.windy.me | mailcow (primary MX prio 20) | `ssh -4 windy@mx2.windy.me` | 194.163.160.244 | ✓ (mx2) | active | [hosts/mx2.windy.me.md](../hosts/mx2.windy.me.md) |
| us2.wsvc.info | Vaultwarden/Postgres (+ Traefik, Soft Serve, …) | `ssh -4 windy@us2.wsvc.info` | 193.9.44.165 | ✓ (us2) | active | [hosts/us2.wsvc.info.md](../hosts/us2.wsvc.info.md) |
| mx.windy.me | mail (secondary MX prio 30) | TBD | see AAAA/A | — (stub) | stub | — |
| repo.windy.me | Soft Serve git (on us2) | `ssh -p 2222 windy@repo.windy.me` | 193.9.44.165 | — (service on us2) | stub | see us2 |
| repo.windy.me | Gitea git (on us2) | `ssh -p 2222 windy@repo.windy.me` | 193.9.44.165 | — (service on us2) | stub | see us2 |
| auth.wsvc.info | Vaultwarden public hostname | — (HTTPS) | → us2 | — (alias) | active | see us2 |
| us1.wsvc.info | PowerDNS secondary (ns2 host) | TBD | 202.91.35.141 | — (stub) | stub | Auth 5.0.5; see hk2 |
| us4.wsvc.info | WireGuard VPN | `ssh -4 windy@us4.wsvc.info` | 185.201.226.122 | ✓ (us4) | active | [hosts/us4.wsvc.info.md](../hosts/us4.wsvc.info.md) |
@@ -48,4 +48,5 @@ the software deployed there, see [the LAN overview](../docs/lan-overview.md).
| https://synapse.chans.xyz | Synapse API | Client-Server + Federation API |
| https://account.chans.xyz | MAS | Matrix Authentication Service (local passwords) |
| https://admin.chans.xyz | Element Admin | Admin console (MAS admin auth) |
| https://plane.chans.xyz | Plane | Project management (Helm `plane-ce` v1.4.1, ns `plane`) |
| `mrtc.chans.xyz` | MatrixRTC | **Reserved** not deployed |
+2
View File
@@ -16,10 +16,12 @@ runbooks start from [`_template.md`](_template.md).
| PowerDNS health check | [pdns-health.md](pdns-health.md) | read-only |
| RustDesk health check | [rustdesk-health.md](rustdesk-health.md) | read-only |
| Matrix health check | [matrix-health.md](matrix-health.md) | read-only |
| Plane health check | [plane-health.md](plane-health.md) | read-only |
| pgdb health check | [pgdb-health.md](pgdb-health.md) | read-only |
| pgdb DB restore (pg_restore) | [pgdb-restore.md](pgdb-restore.md) | change (procedure) |
| pgdb image/compose update | [pgdb-update.md](pgdb-update.md) | change (gated) |
| AdGuard Home health check | [adguard-home-health.md](adguard-home-health.md) | read-only |
| Host disk cleanup (logs/apt/docker) | [host-disk-cleanup.md](host-disk-cleanup.md) | change (gated) |
| Matter packet capture | [matter-packet-capture.md](matter-packet-capture.md) | read-only |
| Home Assistant maintenance | [home-assistant-maintenance.md](home-assistant-maintenance.md) | change (gated) |
| matrix_e2ee integration update | [matrix-e2ee-update.md](matrix-e2ee-update.md) | change (gated) |
+39 -3
View File
@@ -1,7 +1,7 @@
# Runbook: Home Assistant maintenance (hass.windy.lan)
Target: [hass.windy.lan](../hosts/hass.windy.lan.md) (physical x88 Pro box, HAOS `machine: green`)
Upstream: HAOS 18.1 / Core 2026.8.1 / Supervisor 2026.07.5 (verified 2026-08-14)
Upstream: HAOS 18.2 / Supervisor 2026.09.0 / Core 2026.9.1 (verified 2026-09-13)
This runbook covers routine Home Assistant maintenance through the **`ha`
supervisor CLI**. All commands are wrapped by a single script
@@ -284,8 +284,10 @@ print(urllib.request.urlopen(req, timeout=60).status)
7. **Do not edit the dashboard or `templates/csg_sensors.yaml` for an
install.** Entity IDs did not change across v1.3.0/v1.3.1/v1.3.2.
Template `| float(0)` turns native `unavailable` into fake zeros; that
is a follow-up, not part of the zip install.
(The old `| float(0)` fake-zero follow-up was resolved 2026-08-29 by
W1N-239: template sensors now carry `availability` templates and show
`unavailable` instead of fake zeros when native CSG sensors are down.
Template edits go through that issue, not the install path.)
#### Verify (CSG, after v1.3.2 / W1N-118)
@@ -305,6 +307,40 @@ There is no long-lived `HA_TOKEN` in the agent environment. Read entity
states via Supervisor (`SUPERVISOR_TOKEN` after `sudo -n -i`) at
`http://supervisor/core/api/states/<entity_id>`.
#### CSG display refactor 2026-09-04 (VPS-90)
Template/dashboard changes made **after** pricing cross-check (8月账单
198.64 元 vs 模板 198.65 元,≤0.01 元;阶梯常量 0.589/0.639/0.889、
260/600 夏档未动):
- `templates/csg_sensors.yaml` Block B 新增
`sensor.csg_this_month_avg_price`(本月阶梯电费÷本月用电,`元/kWh`);
**csg_* template sensors = 15**
- Panel `power-monitor``lovelace.dashboard_unknown`):环比行改名
「环比上月同期」;glance「本月/上月」去重为单卡「上月」(本月行归
💰核心数据卡);⚡阶梯电价卡加「本月实际均价」行。实体引用 20→21。
- `automations.yaml` +2 提醒:`automation.csg_mian_ban_qie_dong_ji_dang_ti_xing`
10-25/ `automation.csg_mian_ban_qie_xia_ji_dang_ti_xing`4-25
09:00 Matrix 提醒人工切「本月累计」gauge 季节档(max/segments 不可模板化)。
- 金额单位混排(原生 CNY vs 模板 元)**保留**`config/entity_registry/update`
拒绝自定义文本单位(`extra keys not allowed … Got '元'`),已定案接受。
**WS 改面板(2026.8,本机实测,后续沿用)**: core/主机 python 无 ws 库、
core 容器内经 supervisor 代理 WS 被拒(loop prevention)。用
```bash
ssh -o BatchMode=yes hassio@hass.windy.lan \
'sudo -n -i sh -c "docker run --rm -i --network host -e SUPERVISOR_TOKEN \
--entrypoint python3 r.hassbus.com/home-assistant/aarch64-hassio-supervisor:2026.08.0 \
- < /tmp/x.py"'
```
`ws://172.30.32.2/core/websocket`aiohttpheader `Authorization: Bearer
$SUPERVISOR_TOKEN`,随后 auth 帧同 token)。命令名 **`lovelace/config`**(读)
+ **`lovelace/config/save`**(写,url_path + 全量 config);`lovelace/config/get`
已不存在(unknown_command)。备份与细节见
[hosts/hass.windy.lan.md](../hosts/hass.windy.lan.md) § CSG 面板重构 2026-09-04。
## Command families intentionally NOT scripted
These exist in `ha` but are either rare, dangerous, or better done in the web
+276
View File
@@ -0,0 +1,276 @@
# Runbook: Host disk cleanup (unbounded container logs / apt cache / docker artifacts)
## Purpose
Reclaim space on a root filesystem that is filling up (≥70% used) on a Docker
Compose host, by fixing unbounded container log growth at the source, clearing
apt/journal caches, and removing unused Docker images/volumes. Success: root
usage drops to a safe band (≤55% used, or per acceptance in the tracking issue)
and log growth stays bounded afterwards.
## Scope
- 适用环境: production single-root-fs hosts running Docker Compose stacks
(first application: `hk2.chans.xyz`; reusable for `mx2.windy.me` / `us2.wsvc.info`
which run the same unbounded-`json.log` pattern).
- 适用对象: root filesystem usage; container stdout/stderr log files
(`/var/lib/docker/containers/*/*-json.log`); `/var/cache/apt`; systemd journal;
unused Docker images / anonymous volumes / build cache.
- 不适用情形: hosts without systemd-journald or without Docker; LAN/HAOS hosts
(use their own runbooks); cases needing disk *growth* (provider resize) rather
than cleanup; anything touching service data volumes or `/opt/*` configs
(STOP and use the service-specific runbook instead).
## Ownership
- Owner: windy (operator) + agent executing per approval
- Last reviewed: 2026-09-02
- Related systems: hk2.chans.xyz (PowerDNS auth / AdGuard Home / Traefik / RustDesk compose stacks)
## Preconditions
- SSH access to the target host with **passwordless sudo** (`sudo -n true` must succeed).
- A recorded `df -h` baseline and `docker system df` baseline.
- **Explicit user approval** for every service touch listed in Approval gates
(recorded in the tracking issue, e.g. Plane `vps` VPS-81).
- No open incident on the target host.
- Container log growth root cause identified in Diagnose before mutating.
## Inputs
| Input | Source | Required | Validation |
|---|---:|---|
| Target host | inventory/hosts.md | yes | SSH login + `uname -r` |
| df/docker baseline | live read-only probe | yes | recorded before first mutation |
| Approved service touches | user confirmation in tracking issue | yes | issue comment states approval |
| Image keep-list (rollback pins) | operator decision in issue | yes | review `docker image ls` before rmi |
| Backup of any config edited | local copy with timestamp | yes | exists before edit |
## Safety
### Non-negotiable rules
- Prefer read-only diagnosis before mutation (never mutate on an unmeasured disk).
- Never use `rm` on a live container log — use `truncate -s 0` (keeps the fd valid).
- Never run `docker image prune -a` when a keep-list is intended — no keep-list
exists; delete explicitly with `docker rmi`.
- Never run `docker volume prune -a` — plain `docker volume prune` (no `-a`)
removes only unused anonymous volumes; named/in-use volumes stay.
- After every mutation, verify the expected state (`df -h`, container status).
- Destructive actions require explicit approval (Approval gates).
### Stop conditions
- Live state conflicts with this runbook's preconditions or expectations (e.g.
root usage differs wildly from baseline, or a container is unhealthy).
- Missing approval, missing backup, or missing rollback ability.
- A verification step fails with no documented next step.
- Any step would touch a volume/container/mount that is not on the approved list.
### Approval gates
| Action | Risk | Explicit approval | Approval record |
|---|---:|---|---|
| `docker restart <chatty container>` | low (sec-level blip of that service only) | yes | tracking issue (VPS-81 T1) |
| `systemctl restart systemd-journald` | low (sec-level, no state loss) | yes | tracking issue (VPS-81 T2) |
| `apt-get clean` | low (re-downloadable) | no | — |
| `journalctl --vacuum-*` / journald drop-in | low | no (restart above is gated) | — |
| `docker rmi` of unused images | medium (rollback pin removed unless kept) | yes (keep-list) | tracking issue (VPS-81 T3) |
| `docker volume prune` | medium (data in anonymous volumes lost) | yes | tracking issue (VPS-81 T4) |
| `docker builder prune` | low | no | — |
## Procedure
### Step 1 — Diagnose
**Action**
Read-only: `df -h`, `df -i`, `sudo du -x -h --max-depth=1 /`, `docker system df`,
and locate oversized container logs:
`sudo ls -la /var/lib/docker/containers/*/*-json.log`. Map a big log to its
container (`docker inspect -f '{{.Name}} {{.LogPath}}' <id>`), then inspect what
it logs (`sudo tail -c 400000 <logpath>`; count `[debug]` lines) and find the
config flag driving it (e.g. AGH `log.verbose` in its YAML; note `log.file: ""`
means the app's own rotation keys are inert and output goes to the container log).
**Expected**
A full accounting of root usage and identification of: (a) any unbounded
container log and its root-cause flag; (b) reclaimable apt cache; (c) journal
size and journald limits; (d) unused images (0 dangling expected) and unused
anonymous volumes.
**Decision**
- If root is ≥70% used or any container log is unbounded → Step 2.
- If root is healthy and logs are bounded → STOP (no change needed; record evidence).
- If state conflicts with expectations (e.g. missing sudo, unexpected mount) → STOP.
### Step 2 — Fix noisy container logging at the source, then truncate
**Action**
1. Back up the app config: `sudo cp <config> <config>.bak-YYYYMMDD-<issue>`.
2. Disable the debug/verbose flag (e.g. `log.verbose: true → false` in the AGH YAML).
3. Apply config with a container restart: `docker restart <container>` (config-level
change; **no recreate** needed and daemon.json rotation would not apply anyway).
4. Truncate the accumulated logs: `sudo truncate -s 0 <json.log>` for the chatty
container(s) (and any other oversized ones, e.g. traefik).
5. Record `df -h` before/after.
**Expected**
`docker logs <container>` no longer shows the `[debug]` flood; the `*-json.log`
stops growing; several GB reclaimed.
**Verification**
- `sudo tail -c 200000 <json.log>` after ≥1 minute → no new debug lines.
- `df -h` improvement recorded.
- Container still `Up (healthy)`.
**Rollback**
- Trigger: log volume unchanged, service degraded, or debug output is actually needed.
- Action: restore the config backup and `docker restart <container>`.
- Verify: original verbose behaviour back; container healthy.
### Step 3 — Clear apt cache and cap journald
**Action**
1. `sudo apt-get clean` (clears only `/var/cache/apt/archives`; `/var/lib/apt/lists`
is not cleared by it and regenerates on `apt update` — optional/low value, skip).
2. `sudo journalctl --vacuum-size=100M`.
3. Write drop-in `/etc/systemd/journald.conf.d/00-disk-<issue>.conf`:
`[Journal]` + `SystemMaxUse=200M`.
4. `sudo systemctl restart systemd-journald` (approved service touch).
5. Record `df -h` before/after.
**Expected**
Archives cleared (~1.4G on hk2), journal ≤100M, future journal capped at 200M.
**Verification**
- `du -sh /var/cache/apt/archives` → ~0.
- `journalctl --disk-usage` → ≤100M.
- `systemctl show systemd-journald -p ...` or restart log confirms new limit;
`journalctl -b` still readable.
**Rollback**
- Trigger: journald fails to start or logs lost unexpectedly.
- Action: remove the drop-in, `sudo systemctl restart systemd-journald`.
- Verify: journald active, prior journal entries still listed.
### Step 4 — Remove unused Docker images (explicit keep-list)
**Action**
1. Enumerate unused images: `docker image ls` cross-checked against the images of
running containers (`docker ps --format '{{.Image}}'`). Re-enumerate at
execution time — the list drifts.
2. Present the exact removal list to the operator; keep the agreed rollback pin(s)
(e.g. `powerdns/pdns-auth-50:5.0.5`) and delete the rest explicitly:
`docker rmi <repo:tag> ...` (per image).
3. Record `df -h` before/after.
**Expected**
Only in-use images + kept pins remain; ~12.5G reclaimed (reclaim is an upper
bound — layers shared with kept images are not freed; measure with `df`, do not
promise the estimate).
**Verification**
- `docker image ls` shows only the expected set.
- `docker system df` images reclaimable ≈ 0 for the removed set.
- All containers still `Up`.
**Rollback**
- Trigger: an image that was actually needed was removed.
- Action: re-pull it from the registry (`docker pull <repo:tag>`); if a kept pin
must change, update the compose pin and `up -d`.
- Verify: image present; affected service healthy.
### Step 5 — Remove unused anonymous volumes and build cache
**Action**
1. Enumerate volumes: `docker volume ls`, and confirm which are referenced by
containers (`docker inspect` Mounts). Expected targets: anonymous volumes with
no container reference.
2. `docker volume prune` (**no `-a`**) — engine ≥ v23 removes only unused
anonymous volumes; in-use volumes (e.g. PG data) are protected by container
references in every version.
3. `docker builder prune -f`.
4. Record `df -h` before/after.
**Expected**
Unused anonymous volumes (~1.2G on hk2) and build cache gone; in-use volumes intact.
**Verification**
- `docker volume ls` shows only in-use volumes.
- Services that own volumes (e.g. postgres) report healthy and data present.
- `df -h` improvement recorded.
**Rollback**
- Trigger: data loss suspected in a removed volume.
- Action: restore from backup if the volume ever contained data; verify against
the pre-prune enumeration (targets must be anonymous + unreferenced before prune).
- Note: this is why target enumeration is recorded before pruning.
## Troubleshooting
### Troubleshooting A — Log still grows after disabling verbose
- Evidence: `sudo tail -c 200000 <json.log>` still shows new lines; app config re-checked.
- Allowed actions: check for a second verbose source (container entrypoint flags,
other apps in the same log); check `docker inspect <c> --format '{{.HostConfig.LogConfig}}'`.
- Next step: back to Step 2 or STOP if a container-level log-opts change (recreate)
would be needed — that is a separate approval.
### Troubleshooting B — `docker rmi` fails (image in use)
- Evidence: `image is being used by stopped container ...`.
- Allowed actions: identify the stopped container (`docker ps -a`); confirm it is
not needed; remove it only with explicit approval.
- Next step: re-run rmi for the remaining images; never force-delete blindly.
### Troubleshooting C — `docker volume prune` would remove more than expected
- Evidence: prune dry-run/listing includes a named or referenced volume.
- Allowed actions: abort; do not add `-a`; re-check references.
- Next step: STOP and report to the operator with the enumeration.
## Final Verification
The flow is successful only when all of the following hold:
- `df -h` root usage is in the agreed band (VPS-81: 76% → ≤55% used; measure, do not assume).
- `docker system df` shows reclaimable ≈ 0 for images/volumes targeted.
- All containers `Up` (health checks pass); public services verified
(`dig @<host-ip> SOA <zone>` for DNS hosts; service URLs reachable).
- Tracking issue updated with before/after `df`, actions, and the one-week
observation checkpoint for log growth.
## Failure Handling
If the flow cannot complete:
1. Stop further mutation.
2. Collect command output, timestamps, and the exact step that failed.
3. Record completed steps, actual results, unmet expectations, and whether a
rollback ran.
4. Hand over per the tracking issue with evidence; do not guess further.
## References
- Plane `vps` issue VPS-81 "hk2: 释放根盘空间" (+ subtasks VPS-82…88) — plan, review findings, approvals.
- [hosts/hk2.chans.xyz.md](../hosts/hk2.chans.xyz.md) — host facts.
- [RUNBOOKS.md](../RUNBOOKS.md) — runbook spec; [runbooks/README.md](README.md) — index.
+93
View File
@@ -0,0 +1,93 @@
# Runbook: Plane Health Check
## Purpose
Read-only health check of the self-hosted Plane project-management instance
(`plane.chans.xyz`) running on the synapse K3s cluster.
## Scope
- Applicable: [synapse.chans.xyz](../hosts/synapse.chans.xyz.md), namespace `plane`.
- Read-only: does not change pods, ingress, certificates, secrets, or configuration.
- Not applicable: Plane upgrade, values changes, or data recovery — those need a
reviewed change (see `ansible-operations.md` / `release.md`).
## Ownership
- Owner: personal ops (Windy)
- Last reviewed: 2026-09-02
- Related systems: synapse.chans.xyz (Helm `plane-app`, chart `plane-ce-1.8.0`, app `v1.4.1`)
## Pass criteria
All of the following must hold; any conflict means `STOP` and record evidence.
- `sudo helm list -A` shows `plane-app` in ns `plane`, STATUS `deployed`.
- All 7 Deployments + 4 StatefulSets in ns `plane` are `1/1 Running` with 0 recent restarts.
- Init Jobs `api-migrate-*` / `minio-bucket-*` are `Completed`.
- Certificate `plane-app-ssl-cert` is `READY=True` (CN `plane.chans.xyz`).
- `https://plane.chans.xyz/` returns HTTP 200 with a valid Let's Encrypt cert.
- Root disk usage below the 80% warning threshold.
## Procedure
### 1. Release and workloads
```bash
ssh -4 windy@synapse.chans.xyz 'sudo helm list -A'
ssh -4 windy@synapse.chans.xyz 'sudo k3s kubectl -n plane get deploy,sts,pods -o wide'
```
Expected: `plane-app` deployed; all workloads `1/1 Running`, `RESTARTS` low;
no `CrashLoopBackOff` / `Evicted`. Otherwise `STOP` and record evidence.
### 2. TLS certificate and ingress
```bash
ssh -4 windy@synapse.chans.xyz 'sudo k3s kubectl -n plane get certificate,issuer,ingressroute'
```
Expected: `plane-app-ssl-cert` `READY=True`; `plane-app-ingress` present with
routes for `/`, `/api`, `/spaces`, `/god-mode`, `/live`, `/uploads`.
If `READY=False` with a pending HTTP-01 challenge, the usual cause is the node
DNS chain (coreDNS → systemd-resolved → uplink) failing to resolve
`plane.chans.xyz` — check `resolvectl query plane.chans.xyz` vs
`dig +short plane.chans.xyz @8.8.8.8`. If the record exists publicly but the
node fails, `sudo resolvectl flush-caches` and wait for the cert-manager retry;
do not mutate the issuer.
### 3. Endpoint verification
```bash
curl -4 -s -o /dev/null -w '%{http_code}\n' https://plane.chans.xyz/
echo | openssl s_client -connect plane.chans.xyz:443 -servername plane.chans.xyz 2>/dev/null | openssl x509 -noout -subject -issuer -dates
```
Expected: HTTP 200, cert `CN=plane.chans.xyz` issued by Let's Encrypt with a
future `notAfter`. `https://plane.chans.xyz/api/` may 404 — the API serves
under `/api/...` paths only, so a bare 404 there is not a failure.
### 4. Storage and disk
```bash
ssh -4 windy@synapse.chans.xyz 'sudo k3s kubectl -n plane get pvc'
ssh -4 windy@synapse.chans.xyz 'df -hP /'
```
Expected: all 4 PVCs `Bound` (minio 5Gi, pgdb 5Gi, rabbitmq 100Mi, redis 100Mi
on `local-path`); root disk < 80%.
## Safety
- Read-only: never mutate pods, ingress, certificates, or configuration during
this check.
- Plane data (ns `plane` Postgres + MinIO PVCs) has **no backup tier**; treat
the instance as at-risk until a backup design exists.
- If live state conflicts with the expected values above, `STOP` and record
evidence; do not "fix in passing".
## References
- [hosts/synapse.chans.xyz.md](../hosts/synapse.chans.xyz.md) — plane stack facts
- [matrix-health.md](matrix-health.md) — sibling service on the same cluster
+1 -1
View File
@@ -138,7 +138,7 @@ fi
# ---------------------------------------------------------------------------
say "== 4. Runbook spec compliance (RUNBOOKS.md) =="
PROCEDURE_RUNBOOKS="fix-ci.md issue-to-merge.md network-change.md network-recovery.md pgdb-restore.md release.md rollback.md"
PROCEDURE_RUNBOOKS="fix-ci.md host-disk-cleanup.md issue-to-merge.md network-change.md network-recovery.md pgdb-restore.md release.md rollback.md"
GATED_REF_RUNBOOKS="mailcow-update.md ansible-operations.md home-assistant-maintenance.md matrix-e2ee-update.md pgdb-update.md vaultwarden-sqlite-to-postgres.md"
RUNBOOK_INDEX="runbooks/README.md"
for f in runbooks/*.md; do