docs(hass,pgdb): scribe 4.4.0 升级核对 + stats_io_interval 300 + sensor_minute 体积诊断

- hass: Scribe 3.8.0 → 4.4.0(HACS jonathan-gtd/scribe,2026-09-13 随 Core
  2026.9.1 / HAOS 18.2 升级)。记录 4.0 两个 breaking change 在本机均无需动作
  (3.x 结构 DB + PK 在、TimescaleDB 2.29.2 已装)、配置优先级
  YAML > options > entry data > 默认值、以及「YAML 改动必须重启 Core」。
- hass: 新增 stats_io_interval: 300(备份 scribe.yaml.bak-20260913-191558)。
  变更前 24h scribe 自写 11 019/87 461 行(12.6%);实测发布间隔 300s。
  retention_states/retention_events 可用但刻意不设;flush_interval 仍被 entry
  data 钉在 5s(采用新默认 30s 需显式写 YAML)。
- pgdb: sensor_minute 2.26 GB 是压缩窗口内的正常暂存(compress_after 按 chunk
  结束时间判断,09-17 才合格),稳态 4–5 GB 平台期,暂不处理;复查点 09-17
  之后。顺带修正 hass/scribe 库体积事实。

校验:scripts/validate-repo.sh PASS (0 warnings)。
This commit is contained in:
windyboy
2026-09-13 19:34:04 +08:00
parent c445c5f512
commit 6879d79cc6
3 changed files with 34 additions and 11 deletions
+30 -8
View File
@@ -415,9 +415,11 @@ deployed 2026-08-18 from `216cc99` (backup
reworked in v0.3.1/v0.3.2 to wait for a peer-initiated inbound SAS from reworked in v0.3.1/v0.3.2 to wait for a peer-initiated inbound SAS from
Element with emoji comparison), see Element with emoji comparison), see
[docs/home-assistant-matrix.md § Device verification](../docs/home-assistant-matrix.md). [docs/home-assistant-matrix.md § Device verification](../docs/home-assistant-matrix.md).
### Scribe long-term history (verified 2026-08-29) ### Scribe long-term history (3.8.0 setup 2026-08-29; 4.4.0 verified 2026-09-13)
- **Scribe 3.8.0** (`/homeassistant/custom_components/scribe/`), configured from - **Scribe 4.4.0** (`/homeassistant/custom_components/scribe/`, HACS repo
`jonathan-gtd/scribe`, = latest stable 2026-09-12; upgraded 2026-09-13 together
with Core 2026.9.1 / HAOS 18.2), configured from
`/homeassistant/scribe.yaml` — W1N-238 moved the block out of `/homeassistant/scribe.yaml` — W1N-238 moved the block out of
`configuration.yaml` on 2026-08-29 (main config now carries `configuration.yaml` on 2026-08-29 (main config now carries
`scribe: !include scribe.yaml`; content moved verbatim; backup `scribe: !include scribe.yaml`; content moved verbatim; backup
@@ -441,12 +443,32 @@ deployed 2026-08-18 from `216cc99` (backup
entities (`scribe_states_written`, `scribe_events_written`, rates, sizes). entities (`scribe_states_written`, `scribe_events_written`, rates, sizes).
- Verified post-restart 14:23 CST: writer started, `scribe_events_written=1` - Verified post-restart 14:23 CST: writer started, `scribe_events_written=1`
(homeassistant_start), states ~110/min, buffer 3, no scribe log errors. (homeassistant_start), states ~110/min, buffer 3, no scribe log errors.
- **Scribe 3.8.0 has no retention option.** Retention ships only in the v4.x line, - **4.x upgrade核对 2026-09-13(只读 + 一处配置变更)**live `manifest.json` =
which as of 2026-08-29 has no stable release (v4.0.0rc1/v4.1.0rc1 are 4.4.0。两个 4.0 breaking change 在本机都不需要动作——数据库是 3.x 结构
prereleases; user declined RCs — data keeps growing until an upgrade). v4.x is `states_raw` PK `(metadata_id, time)` 在,4.2.0 的启动态去重因此可用),
a major rewrite (writer.py largely rewritten, migration.py removed, TimescaleDB TimescaleDB 2.29.2 已装。4.1.0 修了 `db_url` 优先级,YAML 里的
extension required): re-read release notes before upgrading. Do not expect `!secret scribe_url` 现在是权威。`scribe.yaml` 现有键在 4.4.0 全部仍然合法
retention YAML keys to validate on 3.8.0. (未知键被忽略,`extra=vol.ALLOW_EXTRA`)。**配置优先级 YAML > entry
`options` > entry `data` > 默认值**,而 `_resolve_settings` 读的是
`hass.data[DOMAIN]["yaml_config"]`(只有 `async_setup` 会写),所以
**YAML 改动必须重启 Corereload config entry 不重读 YAML。**
- **`stats_io_interval: 300`2026-09-13 添加**,备份
`/homeassistant/scribe.yaml.bak-20260913-191558`)。4.4.0 不再让 HA 每 30s
轮询 I/O 统计传感器,改由集成自己每 60s 发布,间隔成为配置项。scribe 自己的
传感器此前是本机自写历史的主要来源(变更前 24h:11 019 / 87 461 行状态 =
12.6%),60s → 300s 把这部分降约 5 倍(每个 I/O 传感器约 1440 → 288 行/天)。
验证:`ha core check` OK;重启 88s`ScribeWriter started successfully`
无 scribe error/warningscribe Repairs 问题 0 条;传感器发布间隔实测正好
300s11:19:26 → 11:24:26 UTC)。
- **Retention 现在可用但刻意不设**:`retention_states` / `retention_events`
4.0.0)按间隔丢 chunk,留空 = 永久保留,符合本机定位(Scribe 是永久归档,
recorder 保留 365 天)。注意 retention 是**绕过** entry `data` 副本读取的
`from_entry_data=False`),所以删掉 YAML 行即撤销策略。`db_schema`
`enable_rollups``scribe.purge` 同样未用:图表走 `sensor_minute` +
`timescale_database_reader`(见 [hosts/pgdb.md](pgdb.md)),不吃 scribe 自己的
视图,配置里也没有任何 `scribe.query` 调用。`flush_interval` 仍是 entry
`data` 钉住的 5s——上游下一个版本把默认改成 30s,但 entry 值优先,要采用只能
在 YAML 显式写 `flush_interval: 30`
- Recorder stays external-Postgres with `purge_keep_days: 365` (W1N-243, - Recorder stays external-Postgres with `purge_keep_days: 365` (W1N-243,
2026-08-29, raised from 30 — ~300 MB/yr, 1% of the 30G pgdb disk) for 2026-08-29, raised from 30 — ~300 MB/yr, 1% of the 30G pgdb disk) for
native UI per-change history; Scribe is the permanent archive. Long-term native UI per-change history; Scribe is the permanent archive. Long-term
+3 -2
View File
@@ -17,8 +17,8 @@
| DB | Owner | Size | 用途 | | DB | Owner | Size | 用途 |
|---|---|---|---| |---|---|---|---|
| `hass` | hass | ~14 MB | HA recorderstates/events/statistics),客户端 HAOS `192.168.55.11` | | `hass` | hass | ~406 MB2026-09-13 | HA recorderstates/events/statistics),客户端 HAOS `192.168.55.11` |
| `scribe` | postgres | ~73 MB | HA scribe 集成(entities/areas/devices 注册表同步 + `states_raw` hypertable + `csg_history` 长期归档表) | | `scribe` | postgres | ~2.6 GB2026-09-13 | HA scribe 集成(entities/areas/devices 注册表同步 + `states_raw`/`events` hypertable + `csg_history` 长期归档表);体积由 `sensor_minute` 图表管道主导(2.26 GB),见 Known issues |
| `postgres` | postgres | ~9 MB | 默认库 | | `postgres` | postgres | ~9 MB | 默认库 |
## Ops notes ## Ops notes
@@ -43,6 +43,7 @@
## Known issues ## Known issues
- 2026-09-13**`sensor_minute` 体积构成与压缩窗口(只读诊断,暂不处理)**。`scribe` 库 2.6 GB = `sensor_minute` **2.26 GB**850 万行 / 16 天,约 5659 万行/天 = 331 实体 × 1440 分钟 LOCF+ `states_raw` 290 MB + `events` 1.5 MB`hass` 库另 406 MB。2.26 GB 中 1.50 GB 是 chunk `[09-03,09-10]`、0.76 GB 是 `[09-10,09-17]`,**都还没到压缩窗口**——TimescaleDB 的 `compress_after`**chunk 结束时间**判断(09-10 结束 + 7 天 = **09-17** 才合格),所以「7 天 chunk + 7 天 compress_after」的设计下限就是盘上常驻近 14 天原始数据;已压缩的 `[08-27,09-03]` 从 1.04 GB → **1.5 MB**LOCF 重复度极高,~700:1)。任务 1005 健康(30 成功 / 0 失败,最近 09-13 04:18 跑过但无合格 chunk);1002/1003/1006/1007 亦全 Success。稳态估算 ≈ 2 个未压缩 chunk(3–4.5 GB+ 已压缩归档(约 1.5 MB/周 ≈ 80 MB/年)≈ **45 GB 平台期**pgdata 卷 32 G 当前用 3.2 G,可用 27 G,**无需处理**。复查点 **2026-09-17 之后**`_hyper_4_6_chunk` 应转为 `compressed=true` 且库体积回落;若仍为 false 才需动手(手动 `compress_chunk()` 或调小 `compress_after`)。可选调优:chunk 间隔 7 天 → 1 天 + `compress_after` → 2 天,把常驻未压缩量压到 <1 GB(`set_chunk_time_interval` 只对新 chunk 生效,旧 chunk 不重切)。诊断命令:`select chunk_name, is_compressed from timescaledb_information.chunks where hypertable_name='sensor_minute';` + `pg_database_size('scribe')`。**注意:这是 pgdb 侧对象,HA/scribe 的 `retention_states` 管不到它;HA 侧唯一杠杆是少记/少画(等于砍图)。**
- 2026-08-29HA 侧 HACS 集成 `custom_components.scribe`YAML `scribe: db_url:`,连 `scribe` 库)建表被拒(`permission denied for schema public`hass 无 CREATE 权限),之后持续报 `relation "entities" does not exist`。**已解决**:① `GRANT CREATE ON SCHEMA public TO hass;`scribe 库)② 重启 HA Core 触发重跑建表。重启后自动创建 `entities`1591 行)/`users`/`areas`/`devices`/`integrations`/`states_raw` 表并启用 TimescaleDB 时间序列能力。报错已停止(最后一条 06:06 UTC),`states_raw` 持续写入。2026-08-29 复查:scribe 现有**两个** hypertable——`states_raw`segmentby `metadata_id`、orderby `time`)与 `events`segmentby `event_type`、orderby `time`),均 1 维 `time`;压缩已配置(`timescaledb_information.compression_settings` 可见对应行;2.29.x 该视图无 `compression_enabled` 列)。 - 2026-08-29HA 侧 HACS 集成 `custom_components.scribe`YAML `scribe: db_url:`,连 `scribe` 库)建表被拒(`permission denied for schema public`hass 无 CREATE 权限),之后持续报 `relation "entities" does not exist`。**已解决**:① `GRANT CREATE ON SCHEMA public TO hass;`scribe 库)② 重启 HA Core 触发重跑建表。重启后自动创建 `entities`1591 行)/`users`/`areas`/`devices`/`integrations`/`states_raw` 表并启用 TimescaleDB 时间序列能力。报错已停止(最后一条 06:06 UTC),`states_raw` 持续写入。2026-08-29 复查:scribe 现有**两个** hypertable——`states_raw`segmentby `metadata_id`、orderby `time`)与 `events`segmentby `event_type`、orderby `time`),均 1 维 `time`;压缩已配置(`timescaledb_information.compression_settings` 可见对应行;2.29.x 该视图无 `compression_enabled` 列)。
- 2026-08-29**timescale reader 图表对象**(配套 hass 的 `timescale_database_reader` 集成 + `timescale-plotly-card`,上游 SQL `remmob/timescale_database_reader` `SQL/scribe/01+02` @ `bb8776a`,以 postgres 执行):`sensor_minute_aggregate` 连续聚合(1 分钟桶,last(state)/last(value),实时聚合开启)+ `sensor_minute_aggregate_entity` 视图(join `entities`+ `sensor_minute` hypertable`minute`/`entity_id`/`state`/`value`,LOCF 前向填充)。任务:1005 `sensor_minute` 压缩(7 天)、1006 `sensor_minute` 保留(10 年)、1007 `every_minute_refresh` 每分钟增量刷新(含 5 分钟回溯窗口修正)。授权:`GRANT SELECT ON sensor_minute_aggregate, sensor_minute_aggregate_entity, sensor_minute, entities TO hass`。种子 19529 行(331 实体,自首个数据点起)。**刻意跳过**了上游脚本对 `states_raw` 的 3 个月保留 + 压缩策略语句——与"`states_raw` 永久归档"定位冲突,如需磁盘回收属用户决策(scribe 自己的压缩任务 1000/1001 未动)。 - 2026-08-29**timescale reader 图表对象**(配套 hass 的 `timescale_database_reader` 集成 + `timescale-plotly-card`,上游 SQL `remmob/timescale_database_reader` `SQL/scribe/01+02` @ `bb8776a`,以 postgres 执行):`sensor_minute_aggregate` 连续聚合(1 分钟桶,last(state)/last(value),实时聚合开启)+ `sensor_minute_aggregate_entity` 视图(join `entities`+ `sensor_minute` hypertable`minute`/`entity_id`/`state`/`value`,LOCF 前向填充)。任务:1005 `sensor_minute` 压缩(7 天)、1006 `sensor_minute` 保留(10 年)、1007 `every_minute_refresh` 每分钟增量刷新(含 5 分钟回溯窗口修正)。授权:`GRANT SELECT ON sensor_minute_aggregate, sensor_minute_aggregate_entity, sensor_minute, entities TO hass`。种子 19529 行(331 实体,自首个数据点起)。**刻意跳过**了上游脚本对 `states_raw` 的 3 个月保留 + 压缩策略语句——与"`states_raw` 永久归档"定位冲突,如需磁盘回收属用户决策(scribe 自己的压缩任务 1000/1001 未动)。
- 2026-08-29**`sensor_minute_refresh` 本地补丁(类比 tianqi 补丁,重跑上游 02 SQL 后需重打)**:值 CASE 的 `ELSE 0``ELSE NULL`。原因:scribe 对 unavailable 分钟 value 为 NULL,上游刷新过程兜底写 0;对差分模式的用电图,0→计数器回升会把插座的**生命周期累计值**(最高 1588 kWh)算进掉线那一小时。同日一次性清理既有脏 0:头部占位行 DELETE 505 行(各实体首次非零分钟之前的 value=0);`sensor.%_energy` 与温湿度实体的 value=0 → NULL(10+16 行,物理上不可能的真 0,图表渲染为断点)。功率实体的中途 0 是真实待机读数,保留。 - 2026-08-29**`sensor_minute_refresh` 本地补丁(类比 tianqi 补丁,重跑上游 02 SQL 后需重打)**:值 CASE 的 `ELSE 0``ELSE NULL`。原因:scribe 对 unavailable 分钟 value 为 NULL,上游刷新过程兜底写 0;对差分模式的用电图,0→计数器回升会把插座的**生命周期累计值**(最高 1588 kWh)算进掉线那一小时。同日一次性清理既有脏 0:头部占位行 DELETE 505 行(各实体首次非零分钟之前的 value=0);`sensor.%_energy` 与温湿度实体的 value=0 → NULL(10+16 行,物理上不可能的真 0,图表渲染为断点)。功率实体的中途 0 是真实待机读数,保留。
+1 -1
View File
@@ -1,7 +1,7 @@
# Runbook: Home Assistant maintenance (hass.windy.lan) # Runbook: Home Assistant maintenance (hass.windy.lan)
Target: [hass.windy.lan](../hosts/hass.windy.lan.md) (physical x88 Pro box, HAOS `machine: green`) Target: [hass.windy.lan](../hosts/hass.windy.lan.md) (physical x88 Pro box, HAOS `machine: green`)
Upstream: HAOS 18.1 / Supervisor 2026.07.5 (verified 2026-08-14); Core 2026.8.3 (verified 2026-08-29) Upstream: HAOS 18.2 / Supervisor 2026.09.0 / Core 2026.9.1 (verified 2026-09-13)
This runbook covers routine Home Assistant maintenance through the **`ha` This runbook covers routine Home Assistant maintenance through the **`ha`
supervisor CLI**. All commands are wrapped by a single script supervisor CLI**. All commands are wrapped by a single script