11 Commits
Author SHA1 Message Date
windyboy cc3fb99c14 docs(us2): 根盘清理 71%→23% 事实记录 (Plane VPS-93) 2026-09-18 11:46:17 +08:00
windyboy 6879d79cc6 docs(hass,pgdb): scribe 4.4.0 升级核对 + stats_io_interval 300 + sensor_minute 体积诊断
- hass: Scribe 3.8.0 → 4.4.0(HACS jonathan-gtd/scribe,2026-09-13 随 Core
  2026.9.1 / HAOS 18.2 升级)。记录 4.0 两个 breaking change 在本机均无需动作
  (3.x 结构 DB + PK 在、TimescaleDB 2.29.2 已装)、配置优先级
  YAML > options > entry data > 默认值、以及「YAML 改动必须重启 Core」。
- hass: 新增 stats_io_interval: 300(备份 scribe.yaml.bak-20260913-191558)。
  变更前 24h scribe 自写 11 019/87 461 行(12.6%);实测发布间隔 300s。
  retention_states/retention_events 可用但刻意不设;flush_interval 仍被 entry
  data 钉在 5s(采用新默认 30s 需显式写 YAML)。
- pgdb: sensor_minute 2.26 GB 是压缩窗口内的正常暂存(compress_after 按 chunk
  结束时间判断,09-17 才合格),稳态 4–5 GB 平台期,暂不处理;复查点 09-17
  之后。顺带修正 hass/scribe 库体积事实。

校验:scripts/validate-repo.sh PASS (0 warnings)。
2026-09-13 19:34:04 +08:00
windyboy c445c5f512 Merge remote-tracking branch 'origin/main' into main
AGENTS.md 冲突(两侧都改了 Linear→Plane 记录源):取本地更完整的表述
(self-hosted plane.chans.xyz + mcp__plane__* + 2026-09-03 停用日期 +
issue-tracker.md 已过时),并吸收远端的 `plane-workflow` skill 指引。
2026-09-13 19:13:06 +08:00
windyboy 3c83246f24 docs(pgdb,gfw): pgdb VM 重启根因 (W1N-263) + gfw Quad9 上游移除记录 2026-09-13 19:12:32 +08:00
windyboy c0c975584a docs(plane): 自托管 Plane 落地事实入仓库 + plane-health runbook + hardening 草稿
记录源 Linear→Plane (2026-09-03 起, Plane MCP) + plane.chans.xyz 服务行/upstream 段;
inventory + hosts/synapse.chans.xyz.md 补 Plane 部署事实 (Helm plane-ce-1.8.0 / app v1.4.1,
ns plane, IngressRoute/自有证书 issuer/PVC 5+5Gi local-path/无备份层);
新增 runbooks/plane-health.md (只读健康检查) 与 docs/plane-hardening/ 草稿
(values.hardened.yaml、secrets.yaml.example 占位、backup/ CronJob), 均为未应用设计稿;
.gitignore 增加 .tmp-* agent 临时文件。
2026-09-13 19:12:32 +08:00
windyboy 0034cec925 Merge remote-tracking branch 'origin/main' into HEAD 2026-09-13 19:11:13 +08:00
windyboy c9dcde1274 docs(hass): Quick 面板时间范围扩容 — 功率/环境/人体感应加 3d-30d 档,用电量加 3mo (VPS-92)
卡片 JS 只接受 <n>m|<n>h|<n>d 与命名档(today/week/month/3mo/6mo/year/custom);
记录 scribe sensor_minute 数据下界 2026-08-29 对 >15d 档的影响。
2026-09-13 19:09:54 +08:00
windyboy 037c4ccaa5 docs(hass): 马桶换气电源 Matter 插座接入 + 功率积分补电量 (VPS-92)
Matter Smart Plug (SIXWGH model 3596) 的 cluster 0x0091 声明 IMPE+CUME+PERE 但
CumulativeEnergyImported 恒为 null -> HA 能量实体永久 unknown。功率计量正常
(实测 24.7 W),故用 Integration (Riemann sum) 辅助元素补电量实体
sensor.wei_sheng_jian_ma_tong_huan_qi_dian_yuan_energy:
能源仪表盘 grid 源 [8] 与 Quick「用电量(按插座)」图换到该实体,Quick 另加
span-2「开关」区块。记录 config-flow 经 supervisor 代理走 REST 的 Agent 方法、
备份路径与回滚方式;energy/validate 已全绿。
2026-09-13 18:39:22 +08:00
windyboy 17bb171578 docs(hass): CSG 面板重构入维护 runbook — avg price 传感器 + 季节提醒 automation + WS 改法 (VPS-90) 2026-09-04 21:55:40 +08:00
windyboy cecf7e6331 docs(hass): CSG 电力监控面板重构 — avg price 传感器 + 去重/改名 + 季节切换提醒 (VPS-90) 2026-09-04 21:44:43 +08:00
windyboy e957bc2bb1 docs: add host-disk-cleanup runbook + hk2 disk facts; record source -> Plane vps (VPS-81) 2026-09-02 18:02:22 +08:00
19 changed files with 1018 additions and 21 deletions
+1
View File
@@ -32,3 +32,4 @@ skills-lock.json
*~
# Agent working scratch (not repo content).
.agent-work/
.tmp-*
+18 -4
View File
@@ -30,9 +30,13 @@ This repo is the **agent ops handbook + fact source** for maintaining personal V
> read-only unless a change is approved).
7. Record each material VPS operation, incident, configuration change, or
verification outcome in the corresponding **Linear `vps` project**. Include
scope, action, verification, and remaining follow-up; never put passwords,
tokens, private keys, recovery keys, or private room IDs in Linear.
verification outcome in the corresponding **Plane `vps` project**
(self-hosted `plane.chans.xyz`, Plane MCP `mcp__plane__*`, following the
`plane-workflow` skill). **Linear is retired as a record source (2026-09-03)
— do not create Linear issues;** existing W1N-* entries are read-only
history. Include scope, action, verification, and remaining follow-up; never
put passwords, tokens, private keys, recovery keys, or private room IDs in
Plane or Linear.
### Runbook execution rules
@@ -67,6 +71,7 @@ not maintain a second copy of the machine table here.
| https://synapse.chans.xyz | synapse | Synapse Client-Server + Federation API |
| https://account.chans.xyz | synapse | Matrix Authentication Service (local passwords) |
| https://admin.chans.xyz | synapse | Element Admin console (MAS admin auth) |
| https://plane.chans.xyz | synapse (`plane`, Helm `plane-ce` 1.8.0 / v1.4.1) | Plane project management (self-hosted, K3s) |
### Upstream docs
@@ -76,6 +81,8 @@ not maintain a second copy of the machine table here.
**Matrix (ESS on synapse):** Matrix homeserver running on `synapse.chans.xyz` via the official ESS (Element Server Suite) Helm chart with Synapse + MAS + Element Web + Admin. DNS zone `chans.xyz` managed by hk2 PowerDNS. Before changing config, read [docs/matrix-upstream.md](docs/matrix-upstream.md) and [hosts/synapse.chans.xyz.md](hosts/synapse.chans.xyz.md). K3s cluster on this node has hostPort 80/443 for Traefik (no ServiceLB). Health: [matrix-health](runbooks/matrix-health.md).
**Plane (on synapse):** Self-hosted Plane project management at `plane.chans.xyz`, Helm release `plane-app` (chart `plane-ce-1.8.0`, app `v1.4.1`) in ns `plane` on the same K3s node as Matrix. Config from `/home/windy/plane-k3s/values.yaml`; workload/cert/ingress details in [hosts/synapse.chans.xyz.md](hosts/synapse.chans.xyz.md). Its Postgres/MinIO PVCs are **not** backed up.
**RustDesk:** Self-hosted RustDesk server on `hk2.chans.xyz` (`/opt/rustdesk`, containers `hbbs`/`hbbr`, image pinned `1.1.14`). The `hbbs -r` relay hostname must resolve to the host's public IP `154.36.174.161` — use `hk2.chans.xyz` (never `hk2.wsvc.info`, which has no DNS record). Health: [rustdesk-health](runbooks/rustdesk-health.md).
## Runbooks & scripts
@@ -91,8 +98,10 @@ not maintain a second copy of the machine table here.
| PowerDNS health (hk2) | [runbooks/pdns-health.md](runbooks/pdns-health.md) |
| PowerDNS upstream refs | [docs/pdns-upstream.md](docs/pdns-upstream.md) |
| Matrix health | [runbooks/matrix-health.md](runbooks/matrix-health.md) |
| Plane health | [runbooks/plane-health.md](runbooks/plane-health.md) |
| RustDesk health (hk2) | [runbooks/rustdesk-health.md](runbooks/rustdesk-health.md) |
| AdGuard Home health | [runbooks/adguard-home-health.md](runbooks/adguard-home-health.md) |
| Host disk cleanup | [runbooks/host-disk-cleanup.md](runbooks/host-disk-cleanup.md) |
| Home Assistant maintenance | [runbooks/home-assistant-maintenance.md](runbooks/home-assistant-maintenance.md) + [scripts/ha-maintenance.sh](runbooks/scripts/ha-maintenance.sh) |
| matrix_e2ee update (hass.windy.lan) | [runbooks/matrix-e2ee-update.md](runbooks/matrix-e2ee-update.md) |
| Matrix upstream refs | [docs/matrix-upstream.md](docs/matrix-upstream.md) |
@@ -116,7 +125,12 @@ Routine mailcow health: `cd ansible && ansible-playbook playbooks/health-report.
### Issue tracker
Issues are tracked in Linear and created/updated via the Linear MCP (`vps` project). See `docs/agents/issue-tracker.md`.
Issues are tracked in **Plane** — self-hosted at `plane.chans.xyz`, project
`vps` — and created/updated via the Plane MCP (`mcp__plane__*`), following the
`plane-workflow` skill. **Linear is retired as a record source (2026-09-03); do
not create Linear issues.** Existing W1N-* entries are read-only history.
`docs/agents/issue-tracker.md` documents the retired Linear workflow and is
stale; treat this section as authoritative.
### Triage labels
+45
View File
@@ -0,0 +1,45 @@
# Plane CE 加固草稿(docs/plane-hardening/
> **状态:草稿,未应用、未提交。** 对应追踪:Plane vps 项目条目(2026-09-03**记录源**Linear W1N-277 已取消,Linear 自 2026-09-03 起不再作为记录源)。
> 线上实例:`plane.chans.xyz`synapse K3sns `plane`release `plane-app` = chart `plane-ce-1.8.0` / app `v1.4.1`)。
> 依据:2026-09-03 只读核查(13 条审查意见中 11 条属实、#3 基本属实、#9 指标归属错误)+ 上游 chart 模板逐条核对。
## 文件
| 文件 | 内容 |
|------|------|
| `values.hardened.yaml` | 可选硬化 valuesexternal secrets 引用、requireExplicitSecrets、minio pin、上传限额对齐);含 HTTP→HTTPS `extraObjects` 示例 |
| `secrets.yaml.example` | 6 组外部 Secret 结构占位(只含 key 名,真实值仅存宿主机) |
| `backup/plane-backup.yaml` | **PostgreSQL 备份 CronJob**pg_dump `-Fc`hostPath `/var/backups/plane`MinIO 已按实际用量剔除) |
| `backup/README.md` | 备份方案说明(排程/容量/保留/还原/阻塞) |
## 应用顺序(每步先 diff 后执行,全部需用户逐项确认)
### 现在就值得做:DB 备份(P0,见 backup/
`plane-backup.yaml` 部署 + 手动触发验证一次即可;88 MB 库每日快照几乎零成本。
### 可选(顺手做一次,不是必须)
- **Phase A 密钥外部化**(零行为变化、无停机,约 15 分钟):按 `secrets.yaml.example`
在宿主机建 6 个 Secret(值先复制当前集群),用 `values.hardened.yaml`
`helm diff upgrade``helm upgrade`;验证后删除 chart 生成的旧 Secret。
价值:默认密钥不再落在 chart 公开常量上,作为保险。
- **MCP API Key 轮换**:若审查对话出过你的环境,Plane 后台重生成 + 更新
`/home/windy/plane-k3s/mcp/mcp.env`0600+ 重启 Cursor MCP。
- **/god-mode IP 白名单**:若在意管理后台被公网爆破。chart 1.8.0 的 IngressRoute
不支持给单条路由追加 middleware → 需 post-renderer 或 upgrade 后 `kubectl patch`
(升级会覆盖,需固化);源 IP 清单待提供。
### 明确暂缓/跳过(个人单节点,等出现症状再处理)
- SECRET_KEY 等轮换(Phase B):等真要配 SMTP/OAuth 前再做(避免旧密文不可解)。
- NetworkPolicy、有状态组件 resources limitschart 无 values 开关,需 post-render/patch)、
HTTP→HTTPS(草稿已给 `extraObjects` 示例)、metrics-server/Sentry。
## 关键限制(chart 1.8.0 模板已核对)
- `external_secrets.*_existingSecret` 设置后,对应 Secret **必须**包含模板所需全部 key
(缺失不自动补),见 `secrets.yaml.example` 注释。
- `app_keys_existingSecret` 的 envFrom 在所有 workload 上**最后注入**(后置生效),
保证 app/live 共享密钥一致——不要在其后再放同名 key 的 Secret。
- `DATABASE_URL`/`AMQP_URL`/`REDIS_URL` 是 chart 生成的派生 URL,内嵌明文密码;
外部化后轮换 DB/队列密码时必须同步更新 `plane-app-env`
- minio 的 `MINIO_ROOT_*``AWS_*` 同源于一个 Secret;升级时 bucket Job 会重跑
(需 admin 权限凭据)——换 svcacct 前先确认权限覆盖该 Job。
+48
View File
@@ -0,0 +1,48 @@
# Plane CE 备份方案(DB-only)— DRAFT (2026-09-03), 未应用
> 关联:`plane-backup.yaml`CronJob);追踪:Plane vps 项目条目(记录源,2026-09-03 起不用 Linear)。
> 现状(实测):pg 全库 **88 MB**310 issues / 1 user);MinIO uploads **264 KB**(几乎空)。
## 范围决策(2026-09-03,实际角度)
- **做:PostgreSQL 逻辑备份** —— 覆盖现实故障(误删、升级失败、磁盘坏、重装),成本≈0。
- **不做:MinIO/附件备份** —— 桶仅 264 KB,个人实例附件可接受丢失;不为它付日常维护。
日后附件明显变多再按原完整版思路加 `mc mirror`(历史版本见本目录 git 历史/Plane 条目评论)。
- 异机同步暂不启用(见下"局限/阻塞")。
## 方案
集群内 CronJobns `plane`,每天 **01:30 UTC = 03:30 本地**,控制器按 UTC 跑):
1. 单容器 `postgres:15.7-alpine``pg_dump -Fc`(自定义压缩格式)打 `plane`
`/var/backups/plane/pg/plane-<UTC时间戳>.dump`hostPath `DirectoryOrCreate`
2. 保留 7 天(`find -mtime +7 -delete`),成功/失败历史各留 3/2
3. 凭据:现 chart Secret `plane-app-pgdb-secrets`Phase A 外部化后改 `plane-pgdb-credentials`
## 容量
- 库 88 MB → `-Fc` 快照约 10–40 MB/天 × 7 天 ≈ **<300 MB**,对 83 G 可用盘可忽略。
## 还原(未演练;应用前先做一次隔离测试)
```bash
# 目标 PG15 实例(临时起一个 postgres:15.7-alpine 容器或另一台机):
# 先建空库: createdb plane (user=plane)
pg_restore -h <target> -U plane -d plane --clean --if-exists /var/backups/plane/pg/plane-<TS>.dump
# 还原后确认 310 issues 量级一致;附件为空属预期(未备份 MinIO)
```
## 验收(应用前逐项过)
- [ ] CronJob 建立后手动触发一次:`kubectl -n plane create job --from=cronjob/plane-backup plane-backup-manual-1`Job `Completed`
- [ ] `/var/backups/plane/pg/plane-*.dump` 可被 `pg_restore -l` 列出
- [ ] 备份 Job 只依赖 pgdb 服务,不依赖 Plane 应用 Pod(应用故障期间也能出备份)
- [ ] 保留清理 dry-run`find ... -print`)正确;`df -h /` 前后对比记录
## 局限 / 阻塞
- **本地方案不是离机备份**:单节点磁盘/整机故障即丢。如日后要离机,纳入
[Restic 异机 repository 决策与存取隔离](https://plane.chans.xyz/space/projects/56874283-7e1d-43a8-afa4-631cf1c4ad5b/issues/7825d564-ae15-446b-bced-be26b648346b/)
(与 Matrix 备份同一决策);恢复演练纪律见
[服务级 restore runbook 与隔离复元演练](https://plane.chans.xyz/space/projects/56874283-7e1d-43a8-afa4-631cf1c4ad5b/issues/a9bea3ba-c958-4a74-b2f1-6bbb653f21d3/)。
- 提醒:同一节点 **Matrix 数据价值远高于 Plane 且同样无备份** —— 若投入备份精力,顺序上 Matrix 优先。
@@ -0,0 +1,63 @@
# Plane CE PostgreSQL backup CronJob — DRAFT (2026-09-03), NOT applied.
# ns: plane (synapse K3s single node). Output: hostPath /var/backups/plane (root disk, auto-created).
#
# Scope decision (2026-09-03, practical): DB-only. MinIO dropped — uploads bucket
# measured at 264 KB / 444 KB total; attachments are acceptable loss for this
# personal 1-user instance (310 issues / 88 MB DB). Revisit only if usage grows.
#
# Credentials: read from the CURRENT chart-generated Secret (works today). After the
# optional external-secrets migration (docs/plane-hardening/README.md Phase A) switch
# the secretKeyRef name to plane-pgdb-credentials.
#
# Apply:
# ssh windy@synapse.chans.xyz 'sudo k3s kubectl apply -n plane -f -' < plane-backup.yaml
# Manual run + verify:
# sudo k3s kubectl -n plane create job --from=cronjob/plane-backup plane-backup-manual-1
# sudo k3s kubectl -n plane get cronjob,job,pods | grep plane-backup
# sudo ls -lh /var/backups/plane/pg
# Restore steps + tuning: see backup/README.md
apiVersion: batch/v1
kind: CronJob
metadata:
name: plane-backup
namespace: plane
spec:
# 01:30 UTC daily = 03:30 local (CEST). CronJob controller runs in UTC.
schedule: "30 1 * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 2
jobTemplate:
spec:
backoffLimit: 2
template:
spec:
restartPolicy: OnFailure
volumes:
- name: backup
hostPath:
path: /var/backups/plane
type: DirectoryOrCreate
containers:
- name: pg-dump
image: postgres:15.7-alpine
env:
- name: PGPASSWORD
valueFrom:
secretKeyRef:
name: plane-app-pgdb-secrets # -> plane-pgdb-credentials after Phase A
key: POSTGRES_PASSWORD
command: ["/bin/sh", "-c"]
args:
- |
set -euo pipefail
TS=$(date -u +%Y%m%dT%H%M%SZ)
mkdir -p /backup/pg
pg_dump -h plane-app-pgdb.plane.svc.cluster.local -U plane -d plane \
-Fc -f "/backup/pg/plane-${TS}.dump"
find /backup/pg -type f -name 'plane-*.dump' -mtime +7 -delete
echo "pg_dump done: /backup/pg/plane-${TS}.dump ($(du -h /backup/pg/plane-${TS}.dump | cut -f1))"
volumeMounts:
- name: backup
mountPath: /backup
+91
View File
@@ -0,0 +1,91 @@
# External Secret structure for Plane CE hardening — EXAMPLE ONLY.
# No real values here; this file is safe to commit. Real values live only on the
# host (/home/windy/plane-k3s, 0600/0700) and in the cluster.
#
# Phase A — create each Secret with the CURRENT cluster values first (zero change):
# # current source Secrets (chart-generated):
# kubectl -n plane get secret plane-app-app-secrets -o jsonpath='{.data.SECRET_KEY}' | base64 -d
# kubectl -n plane get secret plane-app-live-secrets -o jsonpath='{.data.REDIS_URL}' | base64 -d
# kubectl -n plane get secret plane-app-pgdb-secrets -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d
# kubectl -n plane get secret plane-app-rabbitmq-secrets -o jsonpath='{.data.RABBITMQ_DEFAULT_PASS}' | base64 -d
# kubectl -n plane get secret plane-app-doc-store-secrets -o jsonpath='{.data}' | base64 -d
#
# e.g. kubectl -n plane create secret generic plane-app-keys \
# --from-literal=SECRET_KEY="$(<copy from above>)" \
# --from-literal=LIVE_SERVER_SECRET_KEY="$(<copy from above>)"
#
# All keys below are REQUIRED by chart templates/plane-ce-1.8.0 (verified 2026-09-03):
# missing keys are NOT auto-filled once an existingSecret is referenced.
---
apiVersion: v1
kind: Secret
metadata:
name: plane-app-keys # external_secrets.app_keys_existingSecret
namespace: plane
type: Opaque
stringData:
SECRET_KEY: "" # current: copy from plane-app-app-secrets; rotate only in Phase B
LIVE_SERVER_SECRET_KEY: "" # current: same value as above / plane-app-live-secrets
---
apiVersion: v1
kind: Secret
metadata:
name: plane-app-env # external_secrets.app_env_existingSecret
namespace: plane
type: Opaque
stringData:
REDIS_URL: "" # redis://plane-app-redis.plane.svc.cluster.local:6379/
DATABASE_URL: "" # postgresql://plane:plane@plane-app-pgdb.plane.svc.cluster.local/plane
AMQP_URL: "" # amqp://plane:plane@plane-app-rabbitmq.plane.svc.cluster.local/
---
apiVersion: v1
kind: Secret
metadata:
name: plane-live-env # external_secrets.live_env_existingSecret
namespace: plane
type: Opaque
stringData:
REDIS_URL: "" # redis://plane-app-redis.plane.svc.cluster.local:6379/
---
apiVersion: v1
kind: Secret
metadata:
name: plane-pgdb-credentials # external_secrets.pgdb_existingSecret
namespace: plane
type: Opaque
stringData:
POSTGRES_PASSWORD: "" # Phase A: keep current ('plane'); Phase B: ALTER USER first, then sync
POSTGRES_DB: "plane"
POSTGRES_USER: "plane"
---
apiVersion: v1
kind: Secret
metadata:
name: plane-rabbitmq-credentials # external_secrets.rabbitmq_existingSecret
namespace: plane
type: Opaque
stringData:
RABBITMQ_DEFAULT_USER: "plane"
RABBITMQ_DEFAULT_PASS: "" # Phase A: keep current; Phase B: rabbitmqctl change_password first
---
apiVersion: v1
kind: Secret
metadata:
name: plane-minio-credentials # external_secrets.doc_store_existingSecret
namespace: plane
type: Opaque
stringData:
FILE_SIZE_LIMIT: "20971520" # must match env.doc_upload_size_limit
AWS_S3_BUCKET_NAME: "uploads"
USE_MINIO: "1"
MINIO_ROOT_USER: "admin"
MINIO_ROOT_PASSWORD: "" # root creds take effect on first init only
AWS_ACCESS_KEY_ID: "admin"
AWS_SECRET_ACCESS_KEY: "" # == MINIO_ROOT_PASSWORD while minio.local_setup
AWS_S3_ENDPOINT_URL: "http://plane-app-minio:9000"
+109
View File
@@ -0,0 +1,109 @@
# Plane CE hardened values — DRAFT (2026-09-03), NOT applied.
# Target file on host: /home/windy/plane-k3s/values.yaml (synapse.chans.xyz)
# Reference release: plane-app, chart plane-ce-1.8.0 (values.yaml L1-362 + templates verified 2026-09-03).
# No secrets in this file. Secret *values* live only in k8s Secrets (see secrets.yaml.example).
#
# Two phases:
# Phase A: externalize secrets (reference names below) with CURRENT values copied -> zero change.
# Phase B: rotate credentials one by one (see README.md). SECRET_KEY rotation is cheap only while
# SMTP/OAuth are unconfigured (no encrypted config rows yet).
planeVersion: v1.4.1
ingress:
enabled: true
appHost: plane.chans.xyz
ingressClass: traefik
traefik:
# 20 MiB (chart default). Keep aligned with env.doc_upload_size_limit below.
maxRequestBodyBytes: 20971520
ssl:
createIssuer: true
issuer: http # HTTP-01; ssl_token_existingSecret not needed
email: admin@chans.xyz
generateCerts: true
postgres:
storageClass: local-path
volumeSize: 5Gi
# NOTE: chart 1.8.0 exposes NO resources knob for the bundled datastores
# (stateful templates render no resources block). Add limits via
# --post-renderer/kustomize or `kubectl -n plane patch sts ...` re-applied on
# every upgrade (P2 task; see README.md).
redis:
storageClass: local-path
# image: valkey/valkey:7.2.11-alpine # already pinned by chart default; uncomment to make explicit
minio:
# P2: pin. Digest of the currently running :latest (2026-09-03, pod plane-app-minio-wl-0).
image: minio/minio@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
# image_mc: minio/mc@sha256:... # optional: pin one-shot bucket-init client the same way
storageClass: local-path
volumeSize: 5Gi
rabbitmq:
storageClass: local-path
env:
# Fail the render instead of ever falling back to the chart's PUBLIC constants
# (values.yaml L340-341 in chart 1.8.0). Requires external_secrets below.
requireExplicitSecrets: true
# SECRET_KEY / LIVE_SERVER_SECRET_KEY are deliberately OMITTED here.
# They live in k8s Secret `plane-app-keys` (referenced below). With
# requireExplicitSecrets=true and app_keys_existingSecret set, the chart renders
# neither key itself and app+live workloads both envFrom `plane-app-keys` LAST
# (later envFrom wins), which keeps the shared signing key consistent.
pgdb_name: plane
docstore_bucket: uploads
# Align app-side upload cap with the Traefik body limit (was 5242880/5MiB).
# Keep both at 20MiB, or lower both together.
doc_upload_size_limit: "20971520"
external_secrets:
# Shared signing keys (used by app + live). REQUIRED keys: SECRET_KEY, LIVE_SERVER_SECRET_KEY.
app_keys_existingSecret: plane-app-keys
# REQUIRED keys: REDIS_URL, DATABASE_URL, AMQP_URL (chart-derived URLs; update on DB/queue rotation).
app_env_existingSecret: plane-app-env
# REQUIRED keys: REDIS_URL.
live_env_existingSecret: plane-live-env
# REQUIRED keys: POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_USER.
pgdb_existingSecret: plane-pgdb-credentials
# REQUIRED keys: RABBITMQ_DEFAULT_USER, RABBITMQ_DEFAULT_PASS.
rabbitmq_existingSecret: plane-rabbitmq-credentials
# REQUIRED keys: FILE_SIZE_LIMIT, AWS_S3_BUCKET_NAME, USE_MINIO, MINIO_ROOT_USER,
# MINIO_ROOT_PASSWORD, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_S3_ENDPOINT_URL.
doc_store_existingSecret: plane-minio-credentials
# ssl_token_existingSecret: '' # DNS-01 only (cloudflare/digitalocean); unused with HTTP-01
# Optional, P2: HTTP -> HTTPS 301. The chart's own IngressRoute binds only
# 'websecure' (http:// currently 404s). extraObjects is rendered verbatim (toYaml).
# Uncomment and `helm upgrade` once reviewed:
# extraObjects:
# - apiVersion: traefik.io/v1alpha1
# kind: Middleware
# metadata:
# name: plane-https-redirect
# namespace: plane
# spec:
# redirectScheme:
# scheme: https
# permanent: true
# - apiVersion: traefik.io/v1alpha1
# kind: IngressRoute
# metadata:
# name: plane-http-to-https
# namespace: plane
# spec:
# entryPoints: [web]
# routes:
# - match: Host(`plane.chans.xyz`)
# kind: Rule
# middlewares:
# - name: plane-https-redirect
# services:
# - name: plane-app-web
# port: 3000
+8
View File
@@ -60,6 +60,14 @@ OpenClash runs `/etc/openclash/clash` (clash_meta core) with configuration
> AAAA now `2404:6800…` (new upstream answering, was `2607:f8b0…` via hk2),
> taobao/intercept/clash-fake-ip all unchanged. Backup:
> `config.yaml.bak-multi-doh-20260813-105421`.
> 2026-09-01: removed `dns.quad9.net` from `foreign_upstream` — recurring
> `WARN foreign_upstream … unexpected EOF` bursts (481 log entries) against
> Quad9 DoH; endpoint answers on probe but gets intermittently
> connection-reset from this network (same failure class as the excluded
> `dns.quad101.net`). Remaining upstreams `adg.chans.xyz` (hk2) +
> `dns.cloudflare.com` both verified live; google.com A + youtube.com AAAA
> resolve through mosdns :6052 after restart. Backup:
> `config.yaml.bak-quad9-remove-20260901-201801`.
- nft: OpenClash injects TPROXY/redirect + DNS-hijack rules into
`table inet fw4`; a residual `table inet passwall` exists with 0 packets (unused)
+154 -8
View File
@@ -279,6 +279,32 @@ updated 2026-08-29 (W1N-240 + W1N-242):** 「本月累计」gauge 对齐夏季
数据日)。备份 `csg_sensors.yaml.bak-20260829-w1n242`。验证:Core
restart 后回归值不变(334.81 / -0.3 / 10.8 / 90.3 / 200.94 / 180.28)。
**CSG 面板重构 2026-09-04VPS-90,先核对计价后展示层改动):** 核对
`power-monitor` 计价与 8 月账单一致(198.65 vs 账单 198.64,差 ≤0.01 元,
因模板用公众圆整价 0.589/0.639/0.889、账单用 6 位精确价),不改阶梯常量。
改动:① `csg_sensors.yaml` Block B 新增
`sensor.csg_this_month_avg_price`(本月阶梯电费÷本月用电,`元/kWh`
availability 照 W1N-239 惯例;**csg_* 实体 14→15**);② 面板改名「环比上月」
→「环比上月同期」;glance「本月/上月」grid 去重为单卡「上月」(本月用电/电费
行归 💰核心数据卡);⚡阶梯电价卡加「本月实际均价」行(当前档位/当前电价/
本月实际均价/档位剩余;面板唯一实体引用 20→21);③ `automations.yaml`
2 条提醒:`automation.csg_mian_ban_qie_dong_ji_dang_ti_xing`10-25 09:00
`automation.csg_mian_ban_qie_xia_ji_dang_ti_xing`4-25 09:00)经
`matrix_e2ee.send_message` 提醒切 gauge。④ 金额单位混排(原生 CNY vs 模板
元)**维持**`config/entity_registry/update` 拒绝自定义文本单位
`extra keys not allowed … Got '元'`),用户确认接受。备份:
`.lovelace-backups/dashboard-unknown-power-monitor-20260904-204757-pre-refactor.json`
`.csg-backups/csg_sensors.yaml.bak-20260904-204757-pre-refactor`(及
`-205301-pre-avgprice`)、`.automations-backups/automations.yaml.bak-*`
**WS 改法(2026.8,本机实测)**core/主机 python 无 ws 库、core 容器内经
supervisor 代理 WS 被拒(loop prevention),用
`docker run --rm --network host -e SUPERVISOR_TOKEN`supervisor 镜像
`aarch64-hassio-supervisor:2026.08.0`)连 `ws://172.30.32.2/core/websocket`
命令名 `lovelace/config`(读)+ `lovelace/config/save`(写),
`lovelace/config/get` 已不存在(unknown_command)。验证:新实体
0.589 元/kWh、15 个 csg_* 数值齐全、回归值不变(14.09/198.65/331.22/
304.99/181.89)、automations on、`ha core check` OK、日志无 template 错误。
**CSG 长期归档(W1N-243, 2026-08-29:** scribe 库新增 `csg_history`
表(逐日 usage/cost/ladder/balance + 逐月累计;2026-07-01 起回填,永久),
由 TimescaleDB 每日任务 **1008** `csg_daily_snapshot()`22:30
@@ -292,6 +318,11 @@ ladder/balance 为 22:30 快照值。详见 [hosts/pgdb.md](../hosts/pgdb.md)。
> `power-monitor` 视图「本月累计」gauge 切到冬季 `max:450` /
> `0/200/400`**5-01** 切回夏季 `max:650` / `0/260/600`(与模板
> `now().month` 季节逻辑对齐;模板常量在 Block B/D `variables`)。
> **提醒 automation2026-09-04 起,VPS-90:**
> `automation.csg_mian_ban_qie_dong_ji_dang_ti_xing`10-25)与
> `automation.csg_mian_ban_qie_xia_ji_dang_ti_xing`4-2509:00 经
> `matrix_e2ee.send_message` 发操作步骤提醒;gauge 的 max/segments 无法
> 模板化,仍需人工改卡配置。
Home PPPoE IPv4 to CSG is still blackholed (`curl -4` to `218.19.148.218:443`
times out). `end0` IPv6 is enabled (`ipv6.method: auto`); from HA,
@@ -384,9 +415,11 @@ deployed 2026-08-18 from `216cc99` (backup
reworked in v0.3.1/v0.3.2 to wait for a peer-initiated inbound SAS from
Element with emoji comparison), see
[docs/home-assistant-matrix.md § Device verification](../docs/home-assistant-matrix.md).
### Scribe long-term history (verified 2026-08-29)
### Scribe long-term history (3.8.0 setup 2026-08-29; 4.4.0 verified 2026-09-13)
- **Scribe 3.8.0** (`/homeassistant/custom_components/scribe/`), configured from
- **Scribe 4.4.0** (`/homeassistant/custom_components/scribe/`, HACS repo
`jonathan-gtd/scribe`, = latest stable 2026-09-12; upgraded 2026-09-13 together
with Core 2026.9.1 / HAOS 18.2), configured from
`/homeassistant/scribe.yaml` — W1N-238 moved the block out of
`configuration.yaml` on 2026-08-29 (main config now carries
`scribe: !include scribe.yaml`; content moved verbatim; backup
@@ -410,12 +443,32 @@ deployed 2026-08-18 from `216cc99` (backup
entities (`scribe_states_written`, `scribe_events_written`, rates, sizes).
- Verified post-restart 14:23 CST: writer started, `scribe_events_written=1`
(homeassistant_start), states ~110/min, buffer 3, no scribe log errors.
- **Scribe 3.8.0 has no retention option.** Retention ships only in the v4.x line,
which as of 2026-08-29 has no stable release (v4.0.0rc1/v4.1.0rc1 are
prereleases; user declined RCs — data keeps growing until an upgrade). v4.x is
a major rewrite (writer.py largely rewritten, migration.py removed, TimescaleDB
extension required): re-read release notes before upgrading. Do not expect
retention YAML keys to validate on 3.8.0.
- **4.x upgrade核对 2026-09-13(只读 + 一处配置变更)**live `manifest.json` =
4.4.0。两个 4.0 breaking change 在本机都不需要动作——数据库是 3.x 结构
`states_raw` PK `(metadata_id, time)` 在,4.2.0 的启动态去重因此可用),
TimescaleDB 2.29.2 已装。4.1.0 修了 `db_url` 优先级,YAML 里的
`!secret scribe_url` 现在是权威。`scribe.yaml` 现有键在 4.4.0 全部仍然合法
(未知键被忽略,`extra=vol.ALLOW_EXTRA`)。**配置优先级 YAML > entry
`options` > entry `data` > 默认值**,而 `_resolve_settings` 读的是
`hass.data[DOMAIN]["yaml_config"]`(只有 `async_setup` 会写),所以
**YAML 改动必须重启 Corereload config entry 不重读 YAML。**
- **`stats_io_interval: 300`2026-09-13 添加**,备份
`/homeassistant/scribe.yaml.bak-20260913-191558`)。4.4.0 不再让 HA 每 30s
轮询 I/O 统计传感器,改由集成自己每 60s 发布,间隔成为配置项。scribe 自己的
传感器此前是本机自写历史的主要来源(变更前 24h:11 019 / 87 461 行状态 =
12.6%),60s → 300s 把这部分降约 5 倍(每个 I/O 传感器约 1440 → 288 行/天)。
验证:`ha core check` OK;重启 88s`ScribeWriter started successfully`
无 scribe error/warningscribe Repairs 问题 0 条;传感器发布间隔实测正好
300s11:19:26 → 11:24:26 UTC)。
- **Retention 现在可用但刻意不设**:`retention_states` / `retention_events`
4.0.0)按间隔丢 chunk,留空 = 永久保留,符合本机定位(Scribe 是永久归档,
recorder 保留 365 天)。注意 retention 是**绕过** entry `data` 副本读取的
`from_entry_data=False`),所以删掉 YAML 行即撤销策略。`db_schema`
`enable_rollups``scribe.purge` 同样未用:图表走 `sensor_minute` +
`timescale_database_reader`(见 [hosts/pgdb.md](pgdb.md)),不吃 scribe 自己的
视图,配置里也没有任何 `scribe.query` 调用。`flush_interval` 仍是 entry
`data` 钉住的 5s——上游下一个版本把默认改成 30s,但 entry 值优先,要采用只能
在 YAML 显式写 `flush_interval: 30`
- Recorder stays external-Postgres with `purge_keep_days: 365` (W1N-243,
2026-08-29, raised from 30 — ~300 MB/yr, 1% of the 30G pgdb disk) for
native UI per-change history; Scribe is the permanent archive. Long-term
@@ -506,6 +559,18 @@ manual copies): reader `remmob/timescale_database_reader` **v1.1.0** (main
修改前备份:`/homeassistant/.lovelace-backups/dashboard-quick-*.json`
W1N-230 修复: `20260829-190256`round-2 改进: `20260829-194040`)。
- **Quick 时间范围扩容 (2026-09-13, VPS-92)**: 用户反馈「48 小时不够」。
各 timescale 卡可选档上调——大功率电器/常驻负载 `…,24h``+3d,7d`
环境 `6h,12h,24h,48h``+7d,14d,30d`;人体感应 `…,24h``+3d,7d`
用电量(按插座) `energy_time_ranges` `today,week,month,custom``+3mo`
**默认档未改**6h / 6h / today / 24h / 12h)。卡片 JS 只接受
`<n>m|<n>h|<n>d``parseDurationToMs` 正则 `/^(\d+)(m|h|d)$/`
仅 m/h/d,无 w)与命名档 `today|week|month|3mo|6mo|year|years|custom`
`energy_mode` 卡必须用后者。**数据下界注意**:scribe `sensor_minute`
目前最早只到 **2026-08-29**,所以 >15d 的档(14d 边缘、30d 明显)前半段
会是空白,等归档继续累积才好看。备份
`.lovelace-backups/dashboard-quick-20260913-190912-pre-timerange.json`
### 地图仪表盘:CARTO keyed tiles via `custom:map-card` (verified 2026-08-30, W1N-261)
- **背景:** CARTO 自 2026-08-26 起对无 key 栅格瓦片打 "API KEY REQUIRED"
@@ -598,6 +663,87 @@ advertised correctly over v4+v6. Both ESP32-C2 bulbs now announce `_matter._tcp`
:5540 sessions** (device-side failure mode C; no errors logged — see
[docs/matter-pairing-troubleshoot.md §8](../docs/matter-pairing-troubleshoot.md)).
## 马桶换气电源(Matter 插座,半计量)+ 电量估算 (2026-09-13)
**设备**Matter `Smart Plug`SIXWGH`model_id 3596`hw 1.0 / sw 1.3.0),node 18
(0x12)`device_id 5ef1850953466d6e7a9c6b901fbebe1c`config entry
`01JF51VQ48PGJGXX3RNAG6MVAA`,区域**卫生间** (`wei_sheng_jian`)label `power`
2026-09-13 17:58 CST 配对。实体:
`switch.wei_sheng_jian_ma_tong_huan_qi_dian_yuan`(插座)、
`sensor.…_dian_yuan`(电源 W)、`sensor.…_dian_ya`(电压 V)、
`sensor.…_you_gong_dian_liu`(有功电流 A)、`sensor.…_dian_li`(电力 kWh
**永久 unknown**)。
**根因(实测 Matter 属性,node 18**:电量簇 0x0091 `FeatureMap = 13`
(IMPE|CUME|PERE,即**声明**支持导入/累计/周期电量),但
`CumulativeEnergyImported (0x0001)` 恒为 `null``PeriodicEnergyImported
(0x0003)` 带载也恒为 `{Energy: 0}``CumulativeEnergyExported (0x0002)`
不存在(EXPE 未声明,自洽)。HA 只用 `CumulativeEnergyImported` 建能量实体
`components/matter/sensor.py:1083``allow_none_value=True`)→ 该实体
**永远不会出数**。**功率计量本身正常**:0x0090 `FeatureMap = 2` (ALTC)
Voltage / ActiveCurrent / ActivePower 都随负载变化(实测 220.3 V / 118 mA /
24.7 WHA `电源` 0.0→24.9 W 有历史)。厂商 `update` 实体报无新固件。
**处理(方案 A:功率积分补电量)**
- 新建 **Integration (Riemann sum) 辅助元素**config entry
`01M2D53T188FW8WEC547ENHSVH`domain `integration`state `loaded`),
source `sensor.wei_sheng_jian_ma_tong_huan_qi_dian_yuan_dian_yuan`
`method: trapezoidal``unit_prefix: k``unit_time: h``round: 3`
`max_sub_interval: 60s`
- 实体 `sensor.wei_sheng_jian_ma_tong_huan_qi_dian_yuan_energy`(创建时 HA
自动生成 `…_dian_yuan_ma_tong_huan_qi_dian_yuan_dian_liang`,随后立即
`config/entity_registry/update` 改名为 `<插座>_energy` 以对齐约定;
该实体新建、无引用,改名安全),friendly name「马桶换气电源 电力」,
unit kWh、`device_class: energy`、**`state_class: total`**——能源仪表盘
允许 `TOTAL``TOTAL_INCREASING``components/energy/validate.py:279`)。
- **能源仪表盘** (`/energy`)grid 源 `[8]``…_dian_li` 改为 `…_energy`
其余 8 条插座源未动。注意这 9 条「插座」全部以 `type: grid` 注册,被当作
全屋用电代理;`switch` 卡片所在的 Grid 卡片此前第 9 行是空的,即本次修复点。
- **Quick 仪表盘**:「用电量(按插座)」图第 11 项由 `…_dian_li` 改为
`…_energy`;新增 `column_span: 2` 的「开关」区块(heading + tile
`switch.…` + `toggle` feature + 功率徽标)→ 视图 6→7 分区。
**口径警告**`…_energy` 是**估算值**Riemann 积分,只在 HA 运行期间累计、
非账单级),与另外 8 个原生计量插座的累计电量口径不同;功率传感器更新
间隔约 510 s(实测 24.9/24.8/25.0 W 抖动),加 `max_sub_interval: 60s`
保证静默时也继续累计。
**Agent 侧建辅助元素的方法(2026-09-13 实测)**HA 的 config flow 走
**REST**WS 只有 `config_entries/flow/progress|subscribe`,没有 start)。
经 supervisor 代理即可,无需 HA 长连接/长寿命 token:
```bash
# SUPERVISOR_TOKEN 由 sudo -n -i 提供
curl -s -X POST -H "Authorization: Bearer $SUPERVISOR_TOKEN" \
-H "Content-Type: application/json" -d '{"handler":"integration"}' \
http://supervisor/core/api/config/config_entries/flow # → {flow_id, step_id:"user", data_schema}
curl -s -X POST -H "Authorization: Bearer $SUPERVISOR_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name":"…","source":"sensor.x","method":"trapezoidal","round":3,
"unit_prefix":"k","unit_time":"h","max_sub_interval":{"minutes":1}}' \
http://supervisor/core/api/config/config_entries/flow/<flow_id> # → create_entry
```
`auth/long_lived_access_token` 在 supervisor 代理身份下**失败**
`unknown_error`),故无法用长寿命 token 开浏览器会话;`DurationSelector`
的值是 `{"minutes":1}` 形式(`cv.time_period`)。
**备份/回滚**`.lovelace-backups/dashboard-quick-20260913-181251-pre-ma-tong-plug.json`
(改动前原件)、`…-20260913-183210-pre-repoint.json`(改名/换源前);
`.ha-backups/energy-20260913-183135-pre-ma-tong-repoint.json`(能源 prefs)。
回滚 = 把能源 prefs 的源 [8] 指回 `…_dian_li` + 还原 Quick 面板 JSON
如需彻底放弃估算电量 = 删除 config entry `01M2D53T188FW8WEC547ENHSVH`
**验证 (2026-09-13 18:3x)**`…_energy` 0.002→0.003 kWh 且随 24.6 W 负载
增长(换气扇关掉后回落 0.0 W,累计值保留);`recorder/list_statistic_ids`
已含该实体;Quick 面板 WS 读回 7 分区、用电量图 11 项指向新实体、旧
`_dian_li` 引用 0 处;能源 prefs 读回 9 源、第 9 条为新实体。
**`energy/validate` 已全绿**9 源 0 issue):创建后 ~5 min 内曾报
`statistics_not_defined`(recorder 的统计任务周期是 5 min,`statistics_meta`
行由该任务建立),18:39 复核时已自动消失——建辅助元素后**不要**把这条
瞬时告警当作失败。
## Related docs
- [runbooks/home-assistant-maintenance.md](../runbooks/home-assistant-maintenance.md) — `ha` CLI maintenance runbook + [script](../runbooks/scripts/ha-maintenance.sh); custom-component zip install is §7
+18
View File
@@ -185,6 +185,23 @@ dig @202.91.35.141 SOA wsvc.info +short
On-server docs: `/opt/pdns/README.md`, `CHANGELOG.md`.
## Disk / logging (VPS-81, 2026-09-02)
Root disk cleanup performed (runbook: [host-disk-cleanup](../runbooks/host-disk-cleanup.md)):
- Root `/` (20G vda1): 76% used → **38% used** (15G → 7.1G; free 4.7G → 12G).
- **AGH log flood root cause fixed**: `/opt/adguard/conf/AdGuardHome.yaml`
`log.verbose: true → false` (backup `AdGuardHome.yaml.bak-20260902-vps81`).
Verbose debug was streaming to stderr → container `json.log` (~120MB/day);
`log.file: ""` makes AGH's own rotation keys inert. Restart only (no recreate).
- Journald capped: `/etc/systemd/journald.conf.d/00-vps81.conf`
`SystemMaxUse=200M`; journal vacuumed to ~96M.
- Docker: engine **29.7.2**; 14 unused images removed (kept `pdns-auth-50:5.0.5`
rollback pin); 12 orphan anonymous volumes + build cache pruned. In-use
volumes intact (`pdns_dbdata`, `b594d738…` PG data, `e855d078…` backup).
- Follow-up: re-check AGH `json.log` growth **2026-09-09** (one-week checkpoint);
global docker log rotation only if still needed.
## Verified
Last checked: **2026-08-01 21:40 CST** — operational; docs audit recorded.
@@ -194,3 +211,4 @@ Last checked: **2026-08-01 21:40 CST** — operational; docs audit recorded.
- `only-notify=` + `also-notify=202.91.35.141`; MASTER `domains.master` cleared
- https://pdns.wsvc.info → **302**; https://pgweb.wsvc.info → **401**
- Hardening backlog: API/DB credential rotation + TSIG rotate (see upstream doc)
- 2026-09-02 (VPS-81): post-cleanup verified — 10 containers Up (adguardhome healthy), DNS SOA/NS + web endpoints OK; see Disk/logging section above.
+6 -4
View File
@@ -8,8 +8,8 @@
| IPv4 | `192.168.55.15` (LAN55) |
| DNS | (none) |
| SSH | `ssh -4 windy@192.168.55.15`key auth 已验证可用 2026-08-29agent 沙箱用 `ssh -F /dev/null -o BatchMode=yes`password auth 亦可) |
| Host | PVE 管理的 QEMU VMi440FX),Debian 13 (trixie),内核 6.12.105 |
| Resources | 2 GB RAM / 30 GB disk26 G 空闲) |
| Host | PVE 管理的 QEMU VMi440FX**VMID 100**),Debian 13 (trixie),内核 6.12.105;宿主机 **pve2 `192.168.55.25`**Proxmox 9.2.2SSH `root@192.168.55.25``onboot: 1`QEMU guest agent 已装;2026-08-31 补记) |
| Resources | 3 GB RAM08-30 13:58 由 2G 上调、删除 balloon/ksm/shares 后重启生效)/ 30 GB disk26 G 空闲) |
| Docker | 29.7.2;容器 `timescaledb` = `timescale/timescaledb:latest-pg18`PG **18.6** + TimescaleDB **2.29.2**Apache-2.0 版) |
| Ports | `192.168.55.15:5432`PGIPv4 only);`192.168.55.15:8081`pgweb GUIbasic auth |
@@ -17,8 +17,8 @@
| DB | Owner | Size | 用途 |
|---|---|---|---|
| `hass` | hass | ~14 MB | HA recorderstates/events/statistics),客户端 HAOS `192.168.55.11` |
| `scribe` | postgres | ~73 MB | HA scribe 集成(entities/areas/devices 注册表同步 + `states_raw` hypertable + `csg_history` 长期归档表) |
| `hass` | hass | ~406 MB2026-09-13 | HA recorderstates/events/statistics),客户端 HAOS `192.168.55.11` |
| `scribe` | postgres | ~2.6 GB2026-09-13 | HA scribe 集成(entities/areas/devices 注册表同步 + `states_raw`/`events` hypertable + `csg_history` 长期归档表);体积由 `sensor_minute` 图表管道主导(2.26 GB),见 Known issues |
| `postgres` | postgres | ~9 MB | 默认库 |
## Ops notes
@@ -43,6 +43,7 @@
## Known issues
- 2026-09-13**`sensor_minute` 体积构成与压缩窗口(只读诊断,暂不处理)**。`scribe` 库 2.6 GB = `sensor_minute` **2.26 GB**850 万行 / 16 天,约 5659 万行/天 = 331 实体 × 1440 分钟 LOCF+ `states_raw` 290 MB + `events` 1.5 MB`hass` 库另 406 MB。2.26 GB 中 1.50 GB 是 chunk `[09-03,09-10]`、0.76 GB 是 `[09-10,09-17]`,**都还没到压缩窗口**——TimescaleDB 的 `compress_after`**chunk 结束时间**判断(09-10 结束 + 7 天 = **09-17** 才合格),所以「7 天 chunk + 7 天 compress_after」的设计下限就是盘上常驻近 14 天原始数据;已压缩的 `[08-27,09-03]` 从 1.04 GB → **1.5 MB**LOCF 重复度极高,~700:1)。任务 1005 健康(30 成功 / 0 失败,最近 09-13 04:18 跑过但无合格 chunk);1002/1003/1006/1007 亦全 Success。稳态估算 ≈ 2 个未压缩 chunk(3–4.5 GB+ 已压缩归档(约 1.5 MB/周 ≈ 80 MB/年)≈ **45 GB 平台期**pgdata 卷 32 G 当前用 3.2 G,可用 27 G,**无需处理**。复查点 **2026-09-17 之后**`_hyper_4_6_chunk` 应转为 `compressed=true` 且库体积回落;若仍为 false 才需动手(手动 `compress_chunk()` 或调小 `compress_after`)。可选调优:chunk 间隔 7 天 → 1 天 + `compress_after` → 2 天,把常驻未压缩量压到 <1 GB(`set_chunk_time_interval` 只对新 chunk 生效,旧 chunk 不重切)。诊断命令:`select chunk_name, is_compressed from timescaledb_information.chunks where hypertable_name='sensor_minute';` + `pg_database_size('scribe')`。**注意:这是 pgdb 侧对象,HA/scribe 的 `retention_states` 管不到它;HA 侧唯一杠杆是少记/少画(等于砍图)。**
- 2026-08-29HA 侧 HACS 集成 `custom_components.scribe`YAML `scribe: db_url:`,连 `scribe` 库)建表被拒(`permission denied for schema public`hass 无 CREATE 权限),之后持续报 `relation "entities" does not exist`。**已解决**:① `GRANT CREATE ON SCHEMA public TO hass;`scribe 库)② 重启 HA Core 触发重跑建表。重启后自动创建 `entities`1591 行)/`users`/`areas`/`devices`/`integrations`/`states_raw` 表并启用 TimescaleDB 时间序列能力。报错已停止(最后一条 06:06 UTC),`states_raw` 持续写入。2026-08-29 复查:scribe 现有**两个** hypertable——`states_raw`segmentby `metadata_id`、orderby `time`)与 `events`segmentby `event_type`、orderby `time`),均 1 维 `time`;压缩已配置(`timescaledb_information.compression_settings` 可见对应行;2.29.x 该视图无 `compression_enabled` 列)。
- 2026-08-29**timescale reader 图表对象**(配套 hass 的 `timescale_database_reader` 集成 + `timescale-plotly-card`,上游 SQL `remmob/timescale_database_reader` `SQL/scribe/01+02` @ `bb8776a`,以 postgres 执行):`sensor_minute_aggregate` 连续聚合(1 分钟桶,last(state)/last(value),实时聚合开启)+ `sensor_minute_aggregate_entity` 视图(join `entities`+ `sensor_minute` hypertable`minute`/`entity_id`/`state`/`value`,LOCF 前向填充)。任务:1005 `sensor_minute` 压缩(7 天)、1006 `sensor_minute` 保留(10 年)、1007 `every_minute_refresh` 每分钟增量刷新(含 5 分钟回溯窗口修正)。授权:`GRANT SELECT ON sensor_minute_aggregate, sensor_minute_aggregate_entity, sensor_minute, entities TO hass`。种子 19529 行(331 实体,自首个数据点起)。**刻意跳过**了上游脚本对 `states_raw` 的 3 个月保留 + 压缩策略语句——与"`states_raw` 永久归档"定位冲突,如需磁盘回收属用户决策(scribe 自己的压缩任务 1000/1001 未动)。
- 2026-08-29**`sensor_minute_refresh` 本地补丁(类比 tianqi 补丁,重跑上游 02 SQL 后需重打)**:值 CASE 的 `ELSE 0``ELSE NULL`。原因:scribe 对 unavailable 分钟 value 为 NULL,上游刷新过程兜底写 0;对差分模式的用电图,0→计数器回升会把插座的**生命周期累计值**(最高 1588 kWh)算进掉线那一小时。同日一次性清理既有脏 0:头部占位行 DELETE 505 行(各实体首次非零分钟之前的 value=0);`sensor.%_energy` 与温湿度实体的 value=0 → NULL(10+16 行,物理上不可能的真 0,图表渲染为断点)。功率实体的中途 0 是真实待机读数,保留。
@@ -50,6 +51,7 @@
## Verification history
- 2026-08-31**13:58 重启根因确认,非停电**W1N-263):pve2`192.168.55.25`)任务日志显示 08-30 **13:58:00 `root@pam` 在 PVE Web UI 修改 VM 100 配置**`-delete allow-ksm,balloon,shares -memory 3072`),**13:58:06 点 Reboot**`qmreboot` → 客机 13:58:08 干净 ACPI 关机 → 13:58:13 自动重启)。宿主机全程在线(08-30 09:00 开机至今连续运行 1d12h+),`.66.26` PVE 及各 VM 均无重启——排除停电。HA recorder 在窗口(13:58:4647)报 2 次 `Connection refused`,DB 恢复后自动重连,**无数据丢失**(`hass.states`/`scribe.states_raw` 13:5514:02 逐分钟无缺口,recorder 内存队列吸收回写)。13:58:47 三容器已起,13:58:56 自愈单元 `pgdb-compose.service` 执行成功——本次自愈按设计工作。同日下午 12:54–12:55 另有一次**客机内自重启**(无 PVE 任务,工作站 SSH 会话相邻)。08-29 22:19→08-30 09:00 宿主机停机 10h41m 为**干净关机**(systemd 有序关闭,非停电)。
- 2026-08-30**开机竞态故障 + 修复**W1N-260):09:01 开机后 docker 恢复容器时绑定 `192.168.55.15:5432/8081` 失败(EADDRNOTAVAIL)→ timescaledb/pgweb 停摆至 12:16pg-backup 开机备份失败(解析不到 timescaledb)→ unhealthy。12:22 `docker compose up -d --force-recreate` 修复(三容器回 `database_default`、端口发布、今日备份、pgweb 恢复);用户重启 HA Core 后写入管道恢复。12:43 新增开机自愈 unit `pgdb-compose.service`enabled,已实测幂等 reconcile)。pgdb-health 8 项全绿。
- 2026-08-29:首次检查(只读)+ 修复 scribe 权限 + 安装夜间备份。见 Linear vps 项目登记。
- 2026-08-29**compose 改造完成**W1N-227,用户已验收):裸 `docker run``/opt/database/docker-compose.yml` 三服务(timescaledb + pgweb + pg-backup);superuser 换强密码;端口收紧 IPv4;备份容器化(TZ=Asia/Shanghaicron 02:00 本地);`pg_restore` 还原实测通过;pgweb UI 用户确认可查 hass/scribe 数据。源码在仓库 `compose/pgdb/`
+45 -1
View File
@@ -24,6 +24,7 @@ ssh -4 windy@synapse.chans.xyz
| DB | ESS embedded PostgreSQL 17 (PVC 20Gi, local-path) |
| Cache | ESS embedded Redis (PVC 2Gi) |
| Chart | `oci://ghcr.io/element-hq/ess-helm/matrix-stack`, version `26.7.2` |
| Plane | Helm `plane-ce-1.8.0` (app `v1.4.1`), namespace `plane` — self-hosted Plane project management |
### Matrix service endpoints
@@ -51,8 +52,50 @@ All other ports internal only (no K3s API, no database, no Redis exposed).
- `ess` — all ESS workloads (Synapse, MAS, Element, Postgres, Redis, HAProxy)
- `matrix-system` — cluster base resources (ResourceQuota, LimitRange, mrtc-placeholder)
- `plane` — Plane project management (Helm release `plane-app`)
- `cert-manager` — cert-manager
## Plane (project management)
Self-hosted [Plane](https://github.com/makeplane/plane) on the same K3s node, deployed via the official `plane-ce` Helm chart.
| Item | Detail |
|------|--------|
| Release | `plane-app` (ns `plane`), chart `plane-ce-1.8.0`, app `v1.4.1`, revision 1 |
| URL | https://plane.chans.xyz |
| Install date | 2026-09-01 |
| Values source | `/home/windy/plane-k3s/values.yaml` (plain file, not a git repo) |
| Images | `artifacts.plane.so/makeplane/*` (`plane-frontend`, `plane-backend`, `plane-admin`, `plane-live`), pullPolicy `Always` |
| Ingress | Traefik `IngressRoute` `plane-app-ingress``/`→web, `/api` `/auth`→api, `/spaces`→space, `/god-mode`→admin, `/live`→live, `/uploads`→minio; `maxRequestBodyBytes` 20Mi |
| TLS | Own namespace `Issuer` `plane-app-cert-issuer` (HTTP-01, LE prod, `admin@chans.xyz`); cert `plane-app-ssl-cert` (CN `plane.chans.xyz`) |
| DB | Bundled Postgres `15.7-alpine` (PVC 5Gi, local-path) |
| Cache/queue | Bundled Redis (PVC 100Mi), RabbitMQ `3.13.6-management-alpine` (PVC 100Mi) |
| Storage | Bundled MinIO (`minio/minio:latest`, root user `admin`, PVC 5Gi) — S3 for uploads/docs |
| Resources | Every workload: cpu 50m/500m, mem 50Mi/1000Mi, replicas 1 |
| SMTP | Not configured (no `smtp` values) — Plane invites/password resets won't email yet |
Workloads (all 1/1 Running): 7 Deployments (`plane-app-{admin,api,beat-worker,live,space,web,worker}-wl`) + 4 StatefulSets (`plane-app-{minio,pgdb,rabbitmq,redis}-wl`); init Jobs `api-migrate-1` / `minio-bucket-1` Completed. All PVCs Bound on `local-path` (root disk).
### Plane configuration notes
- **`planeVersion: v1.4.1`** pinned in values.yaml; chart tracks Plane's own tags.
- **Secrets**: Helm-generated Opaque secrets (`plane-app-app-secrets`, `-doc-store-secrets`, `-pgdb-secrets`, `-rabbitmq-secrets`, `-live-secrets`); `requireExplicitSecrets: false`. Values live in `$SECRET_KEY`, `DATABASE_URL`, `AMQP_URL`, `REDIS_URL` etc.
- **Sentry / CORS**: `sentry_dsn` and `cors_allowed_origins` empty (defaults fine for single-host).
- **MinIO is `latest` tag** — pin a version for reproducibility.
- **Backup**: NOT covered by `/var/backups/matrix` (which is paused anyway) — Plane Postgres/MinIO PVCs have no backup tier yet.
### Plane verification
```bash
# Release + workloads
sudo helm list -A
sudo k3s kubectl -n plane get deploy,sts,pods -o wide
# Cert + ingress
sudo k3s kubectl -n plane get certificate,ingressroute
# Endpoint
curl -4 -s -o /dev/null -w '%{http_code}\n' https://plane.chans.xyz/
```
## Local backup
| Item | Detail |
@@ -62,7 +105,7 @@ All other ports internal only (no K3s API, no database, no Redis exposed).
| Retention | 7 days |
| Disk warning | 80% (healthcheck), 90% (backup stops) |
| Content | Planned: PostgreSQL `synapse` + `mas` logical dumps, media store archive, `/etc/matrix-bootstrap` |
| Status | **Not operational** — no current Matrix backup or recovery tier |
| Status | **Not operational** — no current Matrix backup or recovery tier. **Plane data (its own Postgres + MinIO PVCs in ns `plane`) is also not covered by any backup.** |
## Health checks
@@ -101,5 +144,6 @@ diagnosis and imperative recovery work.
- MatrixRTC / Element Call / LiveKit / Coturn not deployed (`mrtc.chans.xyz` reserved only)
- SMTP email not yet configured (requires manual secret bootstrap followed by a
reviewed Ansible stack deployment)
- Plane `minio` image uses `latest` tag (pin a version)
- No off-site Restic backup
- Single-node K3s (no HA for control plane)
+4 -2
View File
@@ -93,11 +93,13 @@ Soft Serve details (verified/updated 2026-08-30; 核查 [W1N-244](https://linear
| `traefik` | Up | `traefik:v3.6.2` (`/opt/traefik`, public `:80`/`:443`) |
| `nghttpx-proxy` + `squid-backend` | Up | HTTP forward-proxy stack (`/opt/nghttpx`), network `nghttpx_internal-net`; details TBD |
Directories for `authelia`, `conduit`, `dendrite`, `mastodon`, `rustdesk`, `zitadel`, etc. exist under `/opt` but have no running containers; treat them as dormant, not documented services.
**Disk cleanup 2026-09-18** ([Plane vps VPS-93](https://plane.chans.xyz)): root 71% → **23%** (~33G freed) keeping soft-serve / vaultwarden / traefik (nghttpx kept running per operator choice). Removed: unused Docker images + orphan volumes (incl. `zitadel_data` 801M), dormant `/opt` dirs (dendrite + its disabled `dendrite.service` unit, mastodon, dailysync, keycloak, media-repo, authelia, conduit, npm, manager, fusion, zitadel, rustdesk), rootless podman storage (6.4G stale goauthentik), home dev caches, apt cache, journal 3.8G→162M (+`SystemMaxUse=200M` drop-in, active next boot), truncated container logs (nghttpx 550M / traefik / squid). Follow-up: nghttpx-proxy logs grow ~25M/day (INFO per-connection); root-cause log-level/rotation fix still open (needs container restart approval).
Remaining running services on this host: `soft-serve`, `vaultwarden` stack, `traefik`, `nghttpx-proxy` + `squid-backend` (undocumented forward proxy, `/opt/nghttpx`). `/home/windy/authelia` (76M) left in place — outside approved cleanup scope.
## Verified
Last checked: **2026-08-29** — operational.
Last checked: **2026-09-18** — operational; disk cleanup done (see note above, Plane vps VPS-93). Prior full probe: 2026-08-29.
- `vaultwarden` + `vw-db` healthy; `DATABASE_URL``pg:5432/vaultwarden`
- `https://auth.wsvc.info/` **200**, `/admin` **200**, `/api/config` OK (`disableUserRegistration: true`)
+1
View File
@@ -48,4 +48,5 @@ the software deployed there, see [the LAN overview](../docs/lan-overview.md).
| https://synapse.chans.xyz | Synapse API | Client-Server + Federation API |
| https://account.chans.xyz | MAS | Matrix Authentication Service (local passwords) |
| https://admin.chans.xyz | Element Admin | Admin console (MAS admin auth) |
| https://plane.chans.xyz | Plane | Project management (Helm `plane-ce` v1.4.1, ns `plane`) |
| `mrtc.chans.xyz` | MatrixRTC | **Reserved** not deployed |
+2
View File
@@ -16,10 +16,12 @@ runbooks start from [`_template.md`](_template.md).
| PowerDNS health check | [pdns-health.md](pdns-health.md) | read-only |
| RustDesk health check | [rustdesk-health.md](rustdesk-health.md) | read-only |
| Matrix health check | [matrix-health.md](matrix-health.md) | read-only |
| Plane health check | [plane-health.md](plane-health.md) | read-only |
| pgdb health check | [pgdb-health.md](pgdb-health.md) | read-only |
| pgdb DB restore (pg_restore) | [pgdb-restore.md](pgdb-restore.md) | change (procedure) |
| pgdb image/compose update | [pgdb-update.md](pgdb-update.md) | change (gated) |
| AdGuard Home health check | [adguard-home-health.md](adguard-home-health.md) | read-only |
| Host disk cleanup (logs/apt/docker) | [host-disk-cleanup.md](host-disk-cleanup.md) | change (gated) |
| Matter packet capture | [matter-packet-capture.md](matter-packet-capture.md) | read-only |
| Home Assistant maintenance | [home-assistant-maintenance.md](home-assistant-maintenance.md) | change (gated) |
| matrix_e2ee integration update | [matrix-e2ee-update.md](matrix-e2ee-update.md) | change (gated) |
+35 -1
View File
@@ -1,7 +1,7 @@
# Runbook: Home Assistant maintenance (hass.windy.lan)
Target: [hass.windy.lan](../hosts/hass.windy.lan.md) (physical x88 Pro box, HAOS `machine: green`)
Upstream: HAOS 18.1 / Supervisor 2026.07.5 (verified 2026-08-14); Core 2026.8.3 (verified 2026-08-29)
Upstream: HAOS 18.2 / Supervisor 2026.09.0 / Core 2026.9.1 (verified 2026-09-13)
This runbook covers routine Home Assistant maintenance through the **`ha`
supervisor CLI**. All commands are wrapped by a single script
@@ -307,6 +307,40 @@ There is no long-lived `HA_TOKEN` in the agent environment. Read entity
states via Supervisor (`SUPERVISOR_TOKEN` after `sudo -n -i`) at
`http://supervisor/core/api/states/<entity_id>`.
#### CSG display refactor 2026-09-04 (VPS-90)
Template/dashboard changes made **after** pricing cross-check (8月账单
198.64 元 vs 模板 198.65 元,≤0.01 元;阶梯常量 0.589/0.639/0.889、
260/600 夏档未动):
- `templates/csg_sensors.yaml` Block B 新增
`sensor.csg_this_month_avg_price`(本月阶梯电费÷本月用电,`元/kWh`);
**csg_* template sensors = 15**
- Panel `power-monitor``lovelace.dashboard_unknown`):环比行改名
「环比上月同期」;glance「本月/上月」去重为单卡「上月」(本月行归
💰核心数据卡);⚡阶梯电价卡加「本月实际均价」行。实体引用 20→21。
- `automations.yaml` +2 提醒:`automation.csg_mian_ban_qie_dong_ji_dang_ti_xing`
10-25/ `automation.csg_mian_ban_qie_xia_ji_dang_ti_xing`4-25
09:00 Matrix 提醒人工切「本月累计」gauge 季节档(max/segments 不可模板化)。
- 金额单位混排(原生 CNY vs 模板 元)**保留**`config/entity_registry/update`
拒绝自定义文本单位(`extra keys not allowed … Got '元'`),已定案接受。
**WS 改面板(2026.8,本机实测,后续沿用)**: core/主机 python 无 ws 库、
core 容器内经 supervisor 代理 WS 被拒(loop prevention)。用
```bash
ssh -o BatchMode=yes hassio@hass.windy.lan \
'sudo -n -i sh -c "docker run --rm -i --network host -e SUPERVISOR_TOKEN \
--entrypoint python3 r.hassbus.com/home-assistant/aarch64-hassio-supervisor:2026.08.0 \
- < /tmp/x.py"'
```
`ws://172.30.32.2/core/websocket`aiohttpheader `Authorization: Bearer
$SUPERVISOR_TOKEN`,随后 auth 帧同 token)。命令名 **`lovelace/config`**(读)
+ **`lovelace/config/save`**(写,url_path + 全量 config);`lovelace/config/get`
已不存在(unknown_command)。备份与细节见
[hosts/hass.windy.lan.md](../hosts/hass.windy.lan.md) § CSG 面板重构 2026-09-04。
## Command families intentionally NOT scripted
These exist in `ha` but are either rare, dangerous, or better done in the web
+276
View File
@@ -0,0 +1,276 @@
# Runbook: Host disk cleanup (unbounded container logs / apt cache / docker artifacts)
## Purpose
Reclaim space on a root filesystem that is filling up (≥70% used) on a Docker
Compose host, by fixing unbounded container log growth at the source, clearing
apt/journal caches, and removing unused Docker images/volumes. Success: root
usage drops to a safe band (≤55% used, or per acceptance in the tracking issue)
and log growth stays bounded afterwards.
## Scope
- 适用环境: production single-root-fs hosts running Docker Compose stacks
(first application: `hk2.chans.xyz`; reusable for `mx2.windy.me` / `us2.wsvc.info`
which run the same unbounded-`json.log` pattern).
- 适用对象: root filesystem usage; container stdout/stderr log files
(`/var/lib/docker/containers/*/*-json.log`); `/var/cache/apt`; systemd journal;
unused Docker images / anonymous volumes / build cache.
- 不适用情形: hosts without systemd-journald or without Docker; LAN/HAOS hosts
(use their own runbooks); cases needing disk *growth* (provider resize) rather
than cleanup; anything touching service data volumes or `/opt/*` configs
(STOP and use the service-specific runbook instead).
## Ownership
- Owner: windy (operator) + agent executing per approval
- Last reviewed: 2026-09-02
- Related systems: hk2.chans.xyz (PowerDNS auth / AdGuard Home / Traefik / RustDesk compose stacks)
## Preconditions
- SSH access to the target host with **passwordless sudo** (`sudo -n true` must succeed).
- A recorded `df -h` baseline and `docker system df` baseline.
- **Explicit user approval** for every service touch listed in Approval gates
(recorded in the tracking issue, e.g. Plane `vps` VPS-81).
- No open incident on the target host.
- Container log growth root cause identified in Diagnose before mutating.
## Inputs
| Input | Source | Required | Validation |
|---|---:|---|
| Target host | inventory/hosts.md | yes | SSH login + `uname -r` |
| df/docker baseline | live read-only probe | yes | recorded before first mutation |
| Approved service touches | user confirmation in tracking issue | yes | issue comment states approval |
| Image keep-list (rollback pins) | operator decision in issue | yes | review `docker image ls` before rmi |
| Backup of any config edited | local copy with timestamp | yes | exists before edit |
## Safety
### Non-negotiable rules
- Prefer read-only diagnosis before mutation (never mutate on an unmeasured disk).
- Never use `rm` on a live container log — use `truncate -s 0` (keeps the fd valid).
- Never run `docker image prune -a` when a keep-list is intended — no keep-list
exists; delete explicitly with `docker rmi`.
- Never run `docker volume prune -a` — plain `docker volume prune` (no `-a`)
removes only unused anonymous volumes; named/in-use volumes stay.
- After every mutation, verify the expected state (`df -h`, container status).
- Destructive actions require explicit approval (Approval gates).
### Stop conditions
- Live state conflicts with this runbook's preconditions or expectations (e.g.
root usage differs wildly from baseline, or a container is unhealthy).
- Missing approval, missing backup, or missing rollback ability.
- A verification step fails with no documented next step.
- Any step would touch a volume/container/mount that is not on the approved list.
### Approval gates
| Action | Risk | Explicit approval | Approval record |
|---|---:|---|---|
| `docker restart <chatty container>` | low (sec-level blip of that service only) | yes | tracking issue (VPS-81 T1) |
| `systemctl restart systemd-journald` | low (sec-level, no state loss) | yes | tracking issue (VPS-81 T2) |
| `apt-get clean` | low (re-downloadable) | no | — |
| `journalctl --vacuum-*` / journald drop-in | low | no (restart above is gated) | — |
| `docker rmi` of unused images | medium (rollback pin removed unless kept) | yes (keep-list) | tracking issue (VPS-81 T3) |
| `docker volume prune` | medium (data in anonymous volumes lost) | yes | tracking issue (VPS-81 T4) |
| `docker builder prune` | low | no | — |
## Procedure
### Step 1 — Diagnose
**Action**
Read-only: `df -h`, `df -i`, `sudo du -x -h --max-depth=1 /`, `docker system df`,
and locate oversized container logs:
`sudo ls -la /var/lib/docker/containers/*/*-json.log`. Map a big log to its
container (`docker inspect -f '{{.Name}} {{.LogPath}}' <id>`), then inspect what
it logs (`sudo tail -c 400000 <logpath>`; count `[debug]` lines) and find the
config flag driving it (e.g. AGH `log.verbose` in its YAML; note `log.file: ""`
means the app's own rotation keys are inert and output goes to the container log).
**Expected**
A full accounting of root usage and identification of: (a) any unbounded
container log and its root-cause flag; (b) reclaimable apt cache; (c) journal
size and journald limits; (d) unused images (0 dangling expected) and unused
anonymous volumes.
**Decision**
- If root is ≥70% used or any container log is unbounded → Step 2.
- If root is healthy and logs are bounded → STOP (no change needed; record evidence).
- If state conflicts with expectations (e.g. missing sudo, unexpected mount) → STOP.
### Step 2 — Fix noisy container logging at the source, then truncate
**Action**
1. Back up the app config: `sudo cp <config> <config>.bak-YYYYMMDD-<issue>`.
2. Disable the debug/verbose flag (e.g. `log.verbose: true → false` in the AGH YAML).
3. Apply config with a container restart: `docker restart <container>` (config-level
change; **no recreate** needed and daemon.json rotation would not apply anyway).
4. Truncate the accumulated logs: `sudo truncate -s 0 <json.log>` for the chatty
container(s) (and any other oversized ones, e.g. traefik).
5. Record `df -h` before/after.
**Expected**
`docker logs <container>` no longer shows the `[debug]` flood; the `*-json.log`
stops growing; several GB reclaimed.
**Verification**
- `sudo tail -c 200000 <json.log>` after ≥1 minute → no new debug lines.
- `df -h` improvement recorded.
- Container still `Up (healthy)`.
**Rollback**
- Trigger: log volume unchanged, service degraded, or debug output is actually needed.
- Action: restore the config backup and `docker restart <container>`.
- Verify: original verbose behaviour back; container healthy.
### Step 3 — Clear apt cache and cap journald
**Action**
1. `sudo apt-get clean` (clears only `/var/cache/apt/archives`; `/var/lib/apt/lists`
is not cleared by it and regenerates on `apt update` — optional/low value, skip).
2. `sudo journalctl --vacuum-size=100M`.
3. Write drop-in `/etc/systemd/journald.conf.d/00-disk-<issue>.conf`:
`[Journal]` + `SystemMaxUse=200M`.
4. `sudo systemctl restart systemd-journald` (approved service touch).
5. Record `df -h` before/after.
**Expected**
Archives cleared (~1.4G on hk2), journal ≤100M, future journal capped at 200M.
**Verification**
- `du -sh /var/cache/apt/archives` → ~0.
- `journalctl --disk-usage` → ≤100M.
- `systemctl show systemd-journald -p ...` or restart log confirms new limit;
`journalctl -b` still readable.
**Rollback**
- Trigger: journald fails to start or logs lost unexpectedly.
- Action: remove the drop-in, `sudo systemctl restart systemd-journald`.
- Verify: journald active, prior journal entries still listed.
### Step 4 — Remove unused Docker images (explicit keep-list)
**Action**
1. Enumerate unused images: `docker image ls` cross-checked against the images of
running containers (`docker ps --format '{{.Image}}'`). Re-enumerate at
execution time — the list drifts.
2. Present the exact removal list to the operator; keep the agreed rollback pin(s)
(e.g. `powerdns/pdns-auth-50:5.0.5`) and delete the rest explicitly:
`docker rmi <repo:tag> ...` (per image).
3. Record `df -h` before/after.
**Expected**
Only in-use images + kept pins remain; ~12.5G reclaimed (reclaim is an upper
bound — layers shared with kept images are not freed; measure with `df`, do not
promise the estimate).
**Verification**
- `docker image ls` shows only the expected set.
- `docker system df` images reclaimable ≈ 0 for the removed set.
- All containers still `Up`.
**Rollback**
- Trigger: an image that was actually needed was removed.
- Action: re-pull it from the registry (`docker pull <repo:tag>`); if a kept pin
must change, update the compose pin and `up -d`.
- Verify: image present; affected service healthy.
### Step 5 — Remove unused anonymous volumes and build cache
**Action**
1. Enumerate volumes: `docker volume ls`, and confirm which are referenced by
containers (`docker inspect` Mounts). Expected targets: anonymous volumes with
no container reference.
2. `docker volume prune` (**no `-a`**) — engine ≥ v23 removes only unused
anonymous volumes; in-use volumes (e.g. PG data) are protected by container
references in every version.
3. `docker builder prune -f`.
4. Record `df -h` before/after.
**Expected**
Unused anonymous volumes (~1.2G on hk2) and build cache gone; in-use volumes intact.
**Verification**
- `docker volume ls` shows only in-use volumes.
- Services that own volumes (e.g. postgres) report healthy and data present.
- `df -h` improvement recorded.
**Rollback**
- Trigger: data loss suspected in a removed volume.
- Action: restore from backup if the volume ever contained data; verify against
the pre-prune enumeration (targets must be anonymous + unreferenced before prune).
- Note: this is why target enumeration is recorded before pruning.
## Troubleshooting
### Troubleshooting A — Log still grows after disabling verbose
- Evidence: `sudo tail -c 200000 <json.log>` still shows new lines; app config re-checked.
- Allowed actions: check for a second verbose source (container entrypoint flags,
other apps in the same log); check `docker inspect <c> --format '{{.HostConfig.LogConfig}}'`.
- Next step: back to Step 2 or STOP if a container-level log-opts change (recreate)
would be needed — that is a separate approval.
### Troubleshooting B — `docker rmi` fails (image in use)
- Evidence: `image is being used by stopped container ...`.
- Allowed actions: identify the stopped container (`docker ps -a`); confirm it is
not needed; remove it only with explicit approval.
- Next step: re-run rmi for the remaining images; never force-delete blindly.
### Troubleshooting C — `docker volume prune` would remove more than expected
- Evidence: prune dry-run/listing includes a named or referenced volume.
- Allowed actions: abort; do not add `-a`; re-check references.
- Next step: STOP and report to the operator with the enumeration.
## Final Verification
The flow is successful only when all of the following hold:
- `df -h` root usage is in the agreed band (VPS-81: 76% → ≤55% used; measure, do not assume).
- `docker system df` shows reclaimable ≈ 0 for images/volumes targeted.
- All containers `Up` (health checks pass); public services verified
(`dig @<host-ip> SOA <zone>` for DNS hosts; service URLs reachable).
- Tracking issue updated with before/after `df`, actions, and the one-week
observation checkpoint for log growth.
## Failure Handling
If the flow cannot complete:
1. Stop further mutation.
2. Collect command output, timestamps, and the exact step that failed.
3. Record completed steps, actual results, unmet expectations, and whether a
rollback ran.
4. Hand over per the tracking issue with evidence; do not guess further.
## References
- Plane `vps` issue VPS-81 "hk2: 释放根盘空间" (+ subtasks VPS-82…88) — plan, review findings, approvals.
- [hosts/hk2.chans.xyz.md](../hosts/hk2.chans.xyz.md) — host facts.
- [RUNBOOKS.md](../RUNBOOKS.md) — runbook spec; [runbooks/README.md](README.md) — index.
+93
View File
@@ -0,0 +1,93 @@
# Runbook: Plane Health Check
## Purpose
Read-only health check of the self-hosted Plane project-management instance
(`plane.chans.xyz`) running on the synapse K3s cluster.
## Scope
- Applicable: [synapse.chans.xyz](../hosts/synapse.chans.xyz.md), namespace `plane`.
- Read-only: does not change pods, ingress, certificates, secrets, or configuration.
- Not applicable: Plane upgrade, values changes, or data recovery — those need a
reviewed change (see `ansible-operations.md` / `release.md`).
## Ownership
- Owner: personal ops (Windy)
- Last reviewed: 2026-09-02
- Related systems: synapse.chans.xyz (Helm `plane-app`, chart `plane-ce-1.8.0`, app `v1.4.1`)
## Pass criteria
All of the following must hold; any conflict means `STOP` and record evidence.
- `sudo helm list -A` shows `plane-app` in ns `plane`, STATUS `deployed`.
- All 7 Deployments + 4 StatefulSets in ns `plane` are `1/1 Running` with 0 recent restarts.
- Init Jobs `api-migrate-*` / `minio-bucket-*` are `Completed`.
- Certificate `plane-app-ssl-cert` is `READY=True` (CN `plane.chans.xyz`).
- `https://plane.chans.xyz/` returns HTTP 200 with a valid Let's Encrypt cert.
- Root disk usage below the 80% warning threshold.
## Procedure
### 1. Release and workloads
```bash
ssh -4 windy@synapse.chans.xyz 'sudo helm list -A'
ssh -4 windy@synapse.chans.xyz 'sudo k3s kubectl -n plane get deploy,sts,pods -o wide'
```
Expected: `plane-app` deployed; all workloads `1/1 Running`, `RESTARTS` low;
no `CrashLoopBackOff` / `Evicted`. Otherwise `STOP` and record evidence.
### 2. TLS certificate and ingress
```bash
ssh -4 windy@synapse.chans.xyz 'sudo k3s kubectl -n plane get certificate,issuer,ingressroute'
```
Expected: `plane-app-ssl-cert` `READY=True`; `plane-app-ingress` present with
routes for `/`, `/api`, `/spaces`, `/god-mode`, `/live`, `/uploads`.
If `READY=False` with a pending HTTP-01 challenge, the usual cause is the node
DNS chain (coreDNS → systemd-resolved → uplink) failing to resolve
`plane.chans.xyz` — check `resolvectl query plane.chans.xyz` vs
`dig +short plane.chans.xyz @8.8.8.8`. If the record exists publicly but the
node fails, `sudo resolvectl flush-caches` and wait for the cert-manager retry;
do not mutate the issuer.
### 3. Endpoint verification
```bash
curl -4 -s -o /dev/null -w '%{http_code}\n' https://plane.chans.xyz/
echo | openssl s_client -connect plane.chans.xyz:443 -servername plane.chans.xyz 2>/dev/null | openssl x509 -noout -subject -issuer -dates
```
Expected: HTTP 200, cert `CN=plane.chans.xyz` issued by Let's Encrypt with a
future `notAfter`. `https://plane.chans.xyz/api/` may 404 — the API serves
under `/api/...` paths only, so a bare 404 there is not a failure.
### 4. Storage and disk
```bash
ssh -4 windy@synapse.chans.xyz 'sudo k3s kubectl -n plane get pvc'
ssh -4 windy@synapse.chans.xyz 'df -hP /'
```
Expected: all 4 PVCs `Bound` (minio 5Gi, pgdb 5Gi, rabbitmq 100Mi, redis 100Mi
on `local-path`); root disk < 80%.
## Safety
- Read-only: never mutate pods, ingress, certificates, or configuration during
this check.
- Plane data (ns `plane` Postgres + MinIO PVCs) has **no backup tier**; treat
the instance as at-risk until a backup design exists.
- If live state conflicts with the expected values above, `STOP` and record
evidence; do not "fix in passing".
## References
- [hosts/synapse.chans.xyz.md](../hosts/synapse.chans.xyz.md) — plane stack facts
- [matrix-health.md](matrix-health.md) — sibling service on the same cluster
+1 -1
View File
@@ -138,7 +138,7 @@ fi
# ---------------------------------------------------------------------------
say "== 4. Runbook spec compliance (RUNBOOKS.md) =="
PROCEDURE_RUNBOOKS="fix-ci.md issue-to-merge.md network-change.md network-recovery.md pgdb-restore.md release.md rollback.md"
PROCEDURE_RUNBOOKS="fix-ci.md host-disk-cleanup.md issue-to-merge.md network-change.md network-recovery.md pgdb-restore.md release.md rollback.md"
GATED_REF_RUNBOOKS="mailcow-update.md ansible-operations.md home-assistant-maintenance.md matrix-e2ee-update.md pgdb-update.md vaultwarden-sqlite-to-postgres.md"
RUNBOOK_INDEX="runbooks/README.md"
for f in runbooks/*.md; do