docs(plane): 自托管 Plane 落地事实入仓库 + plane-health runbook + hardening 草稿

记录源 Linear→Plane (2026-09-03 起, Plane MCP) + plane.chans.xyz 服务行/upstream 段;
inventory + hosts/synapse.chans.xyz.md 补 Plane 部署事实 (Helm plane-ce-1.8.0 / app v1.4.1,
ns plane, IngressRoute/自有证书 issuer/PVC 5+5Gi local-path/无备份层);
新增 runbooks/plane-health.md (只读健康检查) 与 docs/plane-hardening/ 草稿
(values.hardened.yaml、secrets.yaml.example 占位、backup/ CronJob), 均为未应用设计稿;
.gitignore 增加 .tmp-* agent 临时文件。
This commit is contained in:
windyboy
2026-09-13 19:12:32 +08:00
parent c9dcde1274
commit c0c975584a
11 changed files with 512 additions and 5 deletions
+1
View File
@@ -32,3 +32,4 @@ skills-lock.json
*~ *~
# Agent working scratch (not repo content). # Agent working scratch (not repo content).
.agent-work/ .agent-work/
.tmp-*
+15 -4
View File
@@ -30,9 +30,12 @@ This repo is the **agent ops handbook + fact source** for maintaining personal V
> read-only unless a change is approved). > read-only unless a change is approved).
7. Record each material VPS operation, incident, configuration change, or 7. Record each material VPS operation, incident, configuration change, or
verification outcome in the corresponding **Linear `vps` project**. Include verification outcome in the corresponding **Plane `vps` project**
scope, action, verification, and remaining follow-up; never put passwords, (self-hosted `plane.chans.xyz`, Plane MCP `mcp__plane__*`). **Linear is
tokens, private keys, recovery keys, or private room IDs in Linear. retired as a record source (2026-09-03) — do not create Linear issues;**
existing W1N-* entries are read-only history. Include scope, action,
verification, and remaining follow-up; never put passwords, tokens, private
keys, recovery keys, or private room IDs in Plane or Linear.
### Runbook execution rules ### Runbook execution rules
@@ -67,6 +70,7 @@ not maintain a second copy of the machine table here.
| https://synapse.chans.xyz | synapse | Synapse Client-Server + Federation API | | https://synapse.chans.xyz | synapse | Synapse Client-Server + Federation API |
| https://account.chans.xyz | synapse | Matrix Authentication Service (local passwords) | | https://account.chans.xyz | synapse | Matrix Authentication Service (local passwords) |
| https://admin.chans.xyz | synapse | Element Admin console (MAS admin auth) | | https://admin.chans.xyz | synapse | Element Admin console (MAS admin auth) |
| https://plane.chans.xyz | synapse (`plane`, Helm `plane-ce` 1.8.0 / v1.4.1) | Plane project management (self-hosted, K3s) |
### Upstream docs ### Upstream docs
@@ -76,6 +80,8 @@ not maintain a second copy of the machine table here.
**Matrix (ESS on synapse):** Matrix homeserver running on `synapse.chans.xyz` via the official ESS (Element Server Suite) Helm chart with Synapse + MAS + Element Web + Admin. DNS zone `chans.xyz` managed by hk2 PowerDNS. Before changing config, read [docs/matrix-upstream.md](docs/matrix-upstream.md) and [hosts/synapse.chans.xyz.md](hosts/synapse.chans.xyz.md). K3s cluster on this node has hostPort 80/443 for Traefik (no ServiceLB). Health: [matrix-health](runbooks/matrix-health.md). **Matrix (ESS on synapse):** Matrix homeserver running on `synapse.chans.xyz` via the official ESS (Element Server Suite) Helm chart with Synapse + MAS + Element Web + Admin. DNS zone `chans.xyz` managed by hk2 PowerDNS. Before changing config, read [docs/matrix-upstream.md](docs/matrix-upstream.md) and [hosts/synapse.chans.xyz.md](hosts/synapse.chans.xyz.md). K3s cluster on this node has hostPort 80/443 for Traefik (no ServiceLB). Health: [matrix-health](runbooks/matrix-health.md).
**Plane (on synapse):** Self-hosted Plane project management at `plane.chans.xyz`, Helm release `plane-app` (chart `plane-ce-1.8.0`, app `v1.4.1`) in ns `plane` on the same K3s node as Matrix. Config from `/home/windy/plane-k3s/values.yaml`; workload/cert/ingress details in [hosts/synapse.chans.xyz.md](hosts/synapse.chans.xyz.md). Its Postgres/MinIO PVCs are **not** backed up.
**RustDesk:** Self-hosted RustDesk server on `hk2.chans.xyz` (`/opt/rustdesk`, containers `hbbs`/`hbbr`, image pinned `1.1.14`). The `hbbs -r` relay hostname must resolve to the host's public IP `154.36.174.161` — use `hk2.chans.xyz` (never `hk2.wsvc.info`, which has no DNS record). Health: [rustdesk-health](runbooks/rustdesk-health.md). **RustDesk:** Self-hosted RustDesk server on `hk2.chans.xyz` (`/opt/rustdesk`, containers `hbbs`/`hbbr`, image pinned `1.1.14`). The `hbbs -r` relay hostname must resolve to the host's public IP `154.36.174.161` — use `hk2.chans.xyz` (never `hk2.wsvc.info`, which has no DNS record). Health: [rustdesk-health](runbooks/rustdesk-health.md).
## Runbooks & scripts ## Runbooks & scripts
@@ -91,6 +97,7 @@ not maintain a second copy of the machine table here.
| PowerDNS health (hk2) | [runbooks/pdns-health.md](runbooks/pdns-health.md) | | PowerDNS health (hk2) | [runbooks/pdns-health.md](runbooks/pdns-health.md) |
| PowerDNS upstream refs | [docs/pdns-upstream.md](docs/pdns-upstream.md) | | PowerDNS upstream refs | [docs/pdns-upstream.md](docs/pdns-upstream.md) |
| Matrix health | [runbooks/matrix-health.md](runbooks/matrix-health.md) | | Matrix health | [runbooks/matrix-health.md](runbooks/matrix-health.md) |
| Plane health | [runbooks/plane-health.md](runbooks/plane-health.md) |
| RustDesk health (hk2) | [runbooks/rustdesk-health.md](runbooks/rustdesk-health.md) | | RustDesk health (hk2) | [runbooks/rustdesk-health.md](runbooks/rustdesk-health.md) |
| AdGuard Home health | [runbooks/adguard-home-health.md](runbooks/adguard-home-health.md) | | AdGuard Home health | [runbooks/adguard-home-health.md](runbooks/adguard-home-health.md) |
| Home Assistant maintenance | [runbooks/home-assistant-maintenance.md](runbooks/home-assistant-maintenance.md) + [scripts/ha-maintenance.sh](runbooks/scripts/ha-maintenance.sh) | | Home Assistant maintenance | [runbooks/home-assistant-maintenance.md](runbooks/home-assistant-maintenance.md) + [scripts/ha-maintenance.sh](runbooks/scripts/ha-maintenance.sh) |
@@ -116,7 +123,11 @@ Routine mailcow health: `cd ansible && ansible-playbook playbooks/health-report.
### Issue tracker ### Issue tracker
Issues are tracked in Linear and created/updated via the Linear MCP (`vps` project). See `docs/agents/issue-tracker.md`. Issues are tracked in **Plane** — self-hosted at `plane.chans.xyz`, project
`vps` — and created/updated via the Plane MCP (`mcp__plane__*`). **Linear is
retired as a record source (2026-09-03); do not create Linear issues.** Existing
W1N-* entries are read-only history. `docs/agents/issue-tracker.md` documents
the retired Linear workflow and is stale; treat this section as authoritative.
### Triage labels ### Triage labels
+45
View File
@@ -0,0 +1,45 @@
# Plane CE 加固草稿(docs/plane-hardening/
> **状态:草稿,未应用、未提交。** 对应追踪:Plane vps 项目条目(2026-09-03**记录源**Linear W1N-277 已取消,Linear 自 2026-09-03 起不再作为记录源)。
> 线上实例:`plane.chans.xyz`synapse K3sns `plane`release `plane-app` = chart `plane-ce-1.8.0` / app `v1.4.1`)。
> 依据:2026-09-03 只读核查(13 条审查意见中 11 条属实、#3 基本属实、#9 指标归属错误)+ 上游 chart 模板逐条核对。
## 文件
| 文件 | 内容 |
|------|------|
| `values.hardened.yaml` | 可选硬化 valuesexternal secrets 引用、requireExplicitSecrets、minio pin、上传限额对齐);含 HTTP→HTTPS `extraObjects` 示例 |
| `secrets.yaml.example` | 6 组外部 Secret 结构占位(只含 key 名,真实值仅存宿主机) |
| `backup/plane-backup.yaml` | **PostgreSQL 备份 CronJob**pg_dump `-Fc`hostPath `/var/backups/plane`MinIO 已按实际用量剔除) |
| `backup/README.md` | 备份方案说明(排程/容量/保留/还原/阻塞) |
## 应用顺序(每步先 diff 后执行,全部需用户逐项确认)
### 现在就值得做:DB 备份(P0,见 backup/
`plane-backup.yaml` 部署 + 手动触发验证一次即可;88 MB 库每日快照几乎零成本。
### 可选(顺手做一次,不是必须)
- **Phase A 密钥外部化**(零行为变化、无停机,约 15 分钟):按 `secrets.yaml.example`
在宿主机建 6 个 Secret(值先复制当前集群),用 `values.hardened.yaml`
`helm diff upgrade``helm upgrade`;验证后删除 chart 生成的旧 Secret。
价值:默认密钥不再落在 chart 公开常量上,作为保险。
- **MCP API Key 轮换**:若审查对话出过你的环境,Plane 后台重生成 + 更新
`/home/windy/plane-k3s/mcp/mcp.env`0600+ 重启 Cursor MCP。
- **/god-mode IP 白名单**:若在意管理后台被公网爆破。chart 1.8.0 的 IngressRoute
不支持给单条路由追加 middleware → 需 post-renderer 或 upgrade 后 `kubectl patch`
(升级会覆盖,需固化);源 IP 清单待提供。
### 明确暂缓/跳过(个人单节点,等出现症状再处理)
- SECRET_KEY 等轮换(Phase B):等真要配 SMTP/OAuth 前再做(避免旧密文不可解)。
- NetworkPolicy、有状态组件 resources limitschart 无 values 开关,需 post-render/patch)、
HTTP→HTTPS(草稿已给 `extraObjects` 示例)、metrics-server/Sentry。
## 关键限制(chart 1.8.0 模板已核对)
- `external_secrets.*_existingSecret` 设置后,对应 Secret **必须**包含模板所需全部 key
(缺失不自动补),见 `secrets.yaml.example` 注释。
- `app_keys_existingSecret` 的 envFrom 在所有 workload 上**最后注入**(后置生效),
保证 app/live 共享密钥一致——不要在其后再放同名 key 的 Secret。
- `DATABASE_URL`/`AMQP_URL`/`REDIS_URL` 是 chart 生成的派生 URL,内嵌明文密码;
外部化后轮换 DB/队列密码时必须同步更新 `plane-app-env`
- minio 的 `MINIO_ROOT_*``AWS_*` 同源于一个 Secret;升级时 bucket Job 会重跑
(需 admin 权限凭据)——换 svcacct 前先确认权限覆盖该 Job。
+48
View File
@@ -0,0 +1,48 @@
# Plane CE 备份方案(DB-only)— DRAFT (2026-09-03), 未应用
> 关联:`plane-backup.yaml`CronJob);追踪:Plane vps 项目条目(记录源,2026-09-03 起不用 Linear)。
> 现状(实测):pg 全库 **88 MB**310 issues / 1 user);MinIO uploads **264 KB**(几乎空)。
## 范围决策(2026-09-03,实际角度)
- **做:PostgreSQL 逻辑备份** —— 覆盖现实故障(误删、升级失败、磁盘坏、重装),成本≈0。
- **不做:MinIO/附件备份** —— 桶仅 264 KB,个人实例附件可接受丢失;不为它付日常维护。
日后附件明显变多再按原完整版思路加 `mc mirror`(历史版本见本目录 git 历史/Plane 条目评论)。
- 异机同步暂不启用(见下"局限/阻塞")。
## 方案
集群内 CronJobns `plane`,每天 **01:30 UTC = 03:30 本地**,控制器按 UTC 跑):
1. 单容器 `postgres:15.7-alpine``pg_dump -Fc`(自定义压缩格式)打 `plane`
`/var/backups/plane/pg/plane-<UTC时间戳>.dump`hostPath `DirectoryOrCreate`
2. 保留 7 天(`find -mtime +7 -delete`),成功/失败历史各留 3/2
3. 凭据:现 chart Secret `plane-app-pgdb-secrets`Phase A 外部化后改 `plane-pgdb-credentials`
## 容量
- 库 88 MB → `-Fc` 快照约 10–40 MB/天 × 7 天 ≈ **<300 MB**,对 83 G 可用盘可忽略。
## 还原(未演练;应用前先做一次隔离测试)
```bash
# 目标 PG15 实例(临时起一个 postgres:15.7-alpine 容器或另一台机):
# 先建空库: createdb plane (user=plane)
pg_restore -h <target> -U plane -d plane --clean --if-exists /var/backups/plane/pg/plane-<TS>.dump
# 还原后确认 310 issues 量级一致;附件为空属预期(未备份 MinIO)
```
## 验收(应用前逐项过)
- [ ] CronJob 建立后手动触发一次:`kubectl -n plane create job --from=cronjob/plane-backup plane-backup-manual-1`Job `Completed`
- [ ] `/var/backups/plane/pg/plane-*.dump` 可被 `pg_restore -l` 列出
- [ ] 备份 Job 只依赖 pgdb 服务,不依赖 Plane 应用 Pod(应用故障期间也能出备份)
- [ ] 保留清理 dry-run`find ... -print`)正确;`df -h /` 前后对比记录
## 局限 / 阻塞
- **本地方案不是离机备份**:单节点磁盘/整机故障即丢。如日后要离机,纳入
[Restic 异机 repository 决策与存取隔离](https://plane.chans.xyz/space/projects/56874283-7e1d-43a8-afa4-631cf1c4ad5b/issues/7825d564-ae15-446b-bced-be26b648346b/)
(与 Matrix 备份同一决策);恢复演练纪律见
[服务级 restore runbook 与隔离复元演练](https://plane.chans.xyz/space/projects/56874283-7e1d-43a8-afa4-631cf1c4ad5b/issues/a9bea3ba-c958-4a74-b2f1-6bbb653f21d3/)。
- 提醒:同一节点 **Matrix 数据价值远高于 Plane 且同样无备份** —— 若投入备份精力,顺序上 Matrix 优先。
@@ -0,0 +1,63 @@
# Plane CE PostgreSQL backup CronJob — DRAFT (2026-09-03), NOT applied.
# ns: plane (synapse K3s single node). Output: hostPath /var/backups/plane (root disk, auto-created).
#
# Scope decision (2026-09-03, practical): DB-only. MinIO dropped — uploads bucket
# measured at 264 KB / 444 KB total; attachments are acceptable loss for this
# personal 1-user instance (310 issues / 88 MB DB). Revisit only if usage grows.
#
# Credentials: read from the CURRENT chart-generated Secret (works today). After the
# optional external-secrets migration (docs/plane-hardening/README.md Phase A) switch
# the secretKeyRef name to plane-pgdb-credentials.
#
# Apply:
# ssh windy@synapse.chans.xyz 'sudo k3s kubectl apply -n plane -f -' < plane-backup.yaml
# Manual run + verify:
# sudo k3s kubectl -n plane create job --from=cronjob/plane-backup plane-backup-manual-1
# sudo k3s kubectl -n plane get cronjob,job,pods | grep plane-backup
# sudo ls -lh /var/backups/plane/pg
# Restore steps + tuning: see backup/README.md
apiVersion: batch/v1
kind: CronJob
metadata:
name: plane-backup
namespace: plane
spec:
# 01:30 UTC daily = 03:30 local (CEST). CronJob controller runs in UTC.
schedule: "30 1 * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 2
jobTemplate:
spec:
backoffLimit: 2
template:
spec:
restartPolicy: OnFailure
volumes:
- name: backup
hostPath:
path: /var/backups/plane
type: DirectoryOrCreate
containers:
- name: pg-dump
image: postgres:15.7-alpine
env:
- name: PGPASSWORD
valueFrom:
secretKeyRef:
name: plane-app-pgdb-secrets # -> plane-pgdb-credentials after Phase A
key: POSTGRES_PASSWORD
command: ["/bin/sh", "-c"]
args:
- |
set -euo pipefail
TS=$(date -u +%Y%m%dT%H%M%SZ)
mkdir -p /backup/pg
pg_dump -h plane-app-pgdb.plane.svc.cluster.local -U plane -d plane \
-Fc -f "/backup/pg/plane-${TS}.dump"
find /backup/pg -type f -name 'plane-*.dump' -mtime +7 -delete
echo "pg_dump done: /backup/pg/plane-${TS}.dump ($(du -h /backup/pg/plane-${TS}.dump | cut -f1))"
volumeMounts:
- name: backup
mountPath: /backup
+91
View File
@@ -0,0 +1,91 @@
# External Secret structure for Plane CE hardening — EXAMPLE ONLY.
# No real values here; this file is safe to commit. Real values live only on the
# host (/home/windy/plane-k3s, 0600/0700) and in the cluster.
#
# Phase A — create each Secret with the CURRENT cluster values first (zero change):
# # current source Secrets (chart-generated):
# kubectl -n plane get secret plane-app-app-secrets -o jsonpath='{.data.SECRET_KEY}' | base64 -d
# kubectl -n plane get secret plane-app-live-secrets -o jsonpath='{.data.REDIS_URL}' | base64 -d
# kubectl -n plane get secret plane-app-pgdb-secrets -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d
# kubectl -n plane get secret plane-app-rabbitmq-secrets -o jsonpath='{.data.RABBITMQ_DEFAULT_PASS}' | base64 -d
# kubectl -n plane get secret plane-app-doc-store-secrets -o jsonpath='{.data}' | base64 -d
#
# e.g. kubectl -n plane create secret generic plane-app-keys \
# --from-literal=SECRET_KEY="$(<copy from above>)" \
# --from-literal=LIVE_SERVER_SECRET_KEY="$(<copy from above>)"
#
# All keys below are REQUIRED by chart templates/plane-ce-1.8.0 (verified 2026-09-03):
# missing keys are NOT auto-filled once an existingSecret is referenced.
---
apiVersion: v1
kind: Secret
metadata:
name: plane-app-keys # external_secrets.app_keys_existingSecret
namespace: plane
type: Opaque
stringData:
SECRET_KEY: "" # current: copy from plane-app-app-secrets; rotate only in Phase B
LIVE_SERVER_SECRET_KEY: "" # current: same value as above / plane-app-live-secrets
---
apiVersion: v1
kind: Secret
metadata:
name: plane-app-env # external_secrets.app_env_existingSecret
namespace: plane
type: Opaque
stringData:
REDIS_URL: "" # redis://plane-app-redis.plane.svc.cluster.local:6379/
DATABASE_URL: "" # postgresql://plane:plane@plane-app-pgdb.plane.svc.cluster.local/plane
AMQP_URL: "" # amqp://plane:plane@plane-app-rabbitmq.plane.svc.cluster.local/
---
apiVersion: v1
kind: Secret
metadata:
name: plane-live-env # external_secrets.live_env_existingSecret
namespace: plane
type: Opaque
stringData:
REDIS_URL: "" # redis://plane-app-redis.plane.svc.cluster.local:6379/
---
apiVersion: v1
kind: Secret
metadata:
name: plane-pgdb-credentials # external_secrets.pgdb_existingSecret
namespace: plane
type: Opaque
stringData:
POSTGRES_PASSWORD: "" # Phase A: keep current ('plane'); Phase B: ALTER USER first, then sync
POSTGRES_DB: "plane"
POSTGRES_USER: "plane"
---
apiVersion: v1
kind: Secret
metadata:
name: plane-rabbitmq-credentials # external_secrets.rabbitmq_existingSecret
namespace: plane
type: Opaque
stringData:
RABBITMQ_DEFAULT_USER: "plane"
RABBITMQ_DEFAULT_PASS: "" # Phase A: keep current; Phase B: rabbitmqctl change_password first
---
apiVersion: v1
kind: Secret
metadata:
name: plane-minio-credentials # external_secrets.doc_store_existingSecret
namespace: plane
type: Opaque
stringData:
FILE_SIZE_LIMIT: "20971520" # must match env.doc_upload_size_limit
AWS_S3_BUCKET_NAME: "uploads"
USE_MINIO: "1"
MINIO_ROOT_USER: "admin"
MINIO_ROOT_PASSWORD: "" # root creds take effect on first init only
AWS_ACCESS_KEY_ID: "admin"
AWS_SECRET_ACCESS_KEY: "" # == MINIO_ROOT_PASSWORD while minio.local_setup
AWS_S3_ENDPOINT_URL: "http://plane-app-minio:9000"
+109
View File
@@ -0,0 +1,109 @@
# Plane CE hardened values — DRAFT (2026-09-03), NOT applied.
# Target file on host: /home/windy/plane-k3s/values.yaml (synapse.chans.xyz)
# Reference release: plane-app, chart plane-ce-1.8.0 (values.yaml L1-362 + templates verified 2026-09-03).
# No secrets in this file. Secret *values* live only in k8s Secrets (see secrets.yaml.example).
#
# Two phases:
# Phase A: externalize secrets (reference names below) with CURRENT values copied -> zero change.
# Phase B: rotate credentials one by one (see README.md). SECRET_KEY rotation is cheap only while
# SMTP/OAuth are unconfigured (no encrypted config rows yet).
planeVersion: v1.4.1
ingress:
enabled: true
appHost: plane.chans.xyz
ingressClass: traefik
traefik:
# 20 MiB (chart default). Keep aligned with env.doc_upload_size_limit below.
maxRequestBodyBytes: 20971520
ssl:
createIssuer: true
issuer: http # HTTP-01; ssl_token_existingSecret not needed
email: admin@chans.xyz
generateCerts: true
postgres:
storageClass: local-path
volumeSize: 5Gi
# NOTE: chart 1.8.0 exposes NO resources knob for the bundled datastores
# (stateful templates render no resources block). Add limits via
# --post-renderer/kustomize or `kubectl -n plane patch sts ...` re-applied on
# every upgrade (P2 task; see README.md).
redis:
storageClass: local-path
# image: valkey/valkey:7.2.11-alpine # already pinned by chart default; uncomment to make explicit
minio:
# P2: pin. Digest of the currently running :latest (2026-09-03, pod plane-app-minio-wl-0).
image: minio/minio@sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e
# image_mc: minio/mc@sha256:... # optional: pin one-shot bucket-init client the same way
storageClass: local-path
volumeSize: 5Gi
rabbitmq:
storageClass: local-path
env:
# Fail the render instead of ever falling back to the chart's PUBLIC constants
# (values.yaml L340-341 in chart 1.8.0). Requires external_secrets below.
requireExplicitSecrets: true
# SECRET_KEY / LIVE_SERVER_SECRET_KEY are deliberately OMITTED here.
# They live in k8s Secret `plane-app-keys` (referenced below). With
# requireExplicitSecrets=true and app_keys_existingSecret set, the chart renders
# neither key itself and app+live workloads both envFrom `plane-app-keys` LAST
# (later envFrom wins), which keeps the shared signing key consistent.
pgdb_name: plane
docstore_bucket: uploads
# Align app-side upload cap with the Traefik body limit (was 5242880/5MiB).
# Keep both at 20MiB, or lower both together.
doc_upload_size_limit: "20971520"
external_secrets:
# Shared signing keys (used by app + live). REQUIRED keys: SECRET_KEY, LIVE_SERVER_SECRET_KEY.
app_keys_existingSecret: plane-app-keys
# REQUIRED keys: REDIS_URL, DATABASE_URL, AMQP_URL (chart-derived URLs; update on DB/queue rotation).
app_env_existingSecret: plane-app-env
# REQUIRED keys: REDIS_URL.
live_env_existingSecret: plane-live-env
# REQUIRED keys: POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_USER.
pgdb_existingSecret: plane-pgdb-credentials
# REQUIRED keys: RABBITMQ_DEFAULT_USER, RABBITMQ_DEFAULT_PASS.
rabbitmq_existingSecret: plane-rabbitmq-credentials
# REQUIRED keys: FILE_SIZE_LIMIT, AWS_S3_BUCKET_NAME, USE_MINIO, MINIO_ROOT_USER,
# MINIO_ROOT_PASSWORD, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_S3_ENDPOINT_URL.
doc_store_existingSecret: plane-minio-credentials
# ssl_token_existingSecret: '' # DNS-01 only (cloudflare/digitalocean); unused with HTTP-01
# Optional, P2: HTTP -> HTTPS 301. The chart's own IngressRoute binds only
# 'websecure' (http:// currently 404s). extraObjects is rendered verbatim (toYaml).
# Uncomment and `helm upgrade` once reviewed:
# extraObjects:
# - apiVersion: traefik.io/v1alpha1
# kind: Middleware
# metadata:
# name: plane-https-redirect
# namespace: plane
# spec:
# redirectScheme:
# scheme: https
# permanent: true
# - apiVersion: traefik.io/v1alpha1
# kind: IngressRoute
# metadata:
# name: plane-http-to-https
# namespace: plane
# spec:
# entryPoints: [web]
# routes:
# - match: Host(`plane.chans.xyz`)
# kind: Rule
# middlewares:
# - name: plane-https-redirect
# services:
# - name: plane-app-web
# port: 3000
+45 -1
View File
@@ -24,6 +24,7 @@ ssh -4 windy@synapse.chans.xyz
| DB | ESS embedded PostgreSQL 17 (PVC 20Gi, local-path) | | DB | ESS embedded PostgreSQL 17 (PVC 20Gi, local-path) |
| Cache | ESS embedded Redis (PVC 2Gi) | | Cache | ESS embedded Redis (PVC 2Gi) |
| Chart | `oci://ghcr.io/element-hq/ess-helm/matrix-stack`, version `26.7.2` | | Chart | `oci://ghcr.io/element-hq/ess-helm/matrix-stack`, version `26.7.2` |
| Plane | Helm `plane-ce-1.8.0` (app `v1.4.1`), namespace `plane` — self-hosted Plane project management |
### Matrix service endpoints ### Matrix service endpoints
@@ -51,8 +52,50 @@ All other ports internal only (no K3s API, no database, no Redis exposed).
- `ess` — all ESS workloads (Synapse, MAS, Element, Postgres, Redis, HAProxy) - `ess` — all ESS workloads (Synapse, MAS, Element, Postgres, Redis, HAProxy)
- `matrix-system` — cluster base resources (ResourceQuota, LimitRange, mrtc-placeholder) - `matrix-system` — cluster base resources (ResourceQuota, LimitRange, mrtc-placeholder)
- `plane` — Plane project management (Helm release `plane-app`)
- `cert-manager` — cert-manager - `cert-manager` — cert-manager
## Plane (project management)
Self-hosted [Plane](https://github.com/makeplane/plane) on the same K3s node, deployed via the official `plane-ce` Helm chart.
| Item | Detail |
|------|--------|
| Release | `plane-app` (ns `plane`), chart `plane-ce-1.8.0`, app `v1.4.1`, revision 1 |
| URL | https://plane.chans.xyz |
| Install date | 2026-09-01 |
| Values source | `/home/windy/plane-k3s/values.yaml` (plain file, not a git repo) |
| Images | `artifacts.plane.so/makeplane/*` (`plane-frontend`, `plane-backend`, `plane-admin`, `plane-live`), pullPolicy `Always` |
| Ingress | Traefik `IngressRoute` `plane-app-ingress``/`→web, `/api` `/auth`→api, `/spaces`→space, `/god-mode`→admin, `/live`→live, `/uploads`→minio; `maxRequestBodyBytes` 20Mi |
| TLS | Own namespace `Issuer` `plane-app-cert-issuer` (HTTP-01, LE prod, `admin@chans.xyz`); cert `plane-app-ssl-cert` (CN `plane.chans.xyz`) |
| DB | Bundled Postgres `15.7-alpine` (PVC 5Gi, local-path) |
| Cache/queue | Bundled Redis (PVC 100Mi), RabbitMQ `3.13.6-management-alpine` (PVC 100Mi) |
| Storage | Bundled MinIO (`minio/minio:latest`, root user `admin`, PVC 5Gi) — S3 for uploads/docs |
| Resources | Every workload: cpu 50m/500m, mem 50Mi/1000Mi, replicas 1 |
| SMTP | Not configured (no `smtp` values) — Plane invites/password resets won't email yet |
Workloads (all 1/1 Running): 7 Deployments (`plane-app-{admin,api,beat-worker,live,space,web,worker}-wl`) + 4 StatefulSets (`plane-app-{minio,pgdb,rabbitmq,redis}-wl`); init Jobs `api-migrate-1` / `minio-bucket-1` Completed. All PVCs Bound on `local-path` (root disk).
### Plane configuration notes
- **`planeVersion: v1.4.1`** pinned in values.yaml; chart tracks Plane's own tags.
- **Secrets**: Helm-generated Opaque secrets (`plane-app-app-secrets`, `-doc-store-secrets`, `-pgdb-secrets`, `-rabbitmq-secrets`, `-live-secrets`); `requireExplicitSecrets: false`. Values live in `$SECRET_KEY`, `DATABASE_URL`, `AMQP_URL`, `REDIS_URL` etc.
- **Sentry / CORS**: `sentry_dsn` and `cors_allowed_origins` empty (defaults fine for single-host).
- **MinIO is `latest` tag** — pin a version for reproducibility.
- **Backup**: NOT covered by `/var/backups/matrix` (which is paused anyway) — Plane Postgres/MinIO PVCs have no backup tier yet.
### Plane verification
```bash
# Release + workloads
sudo helm list -A
sudo k3s kubectl -n plane get deploy,sts,pods -o wide
# Cert + ingress
sudo k3s kubectl -n plane get certificate,ingressroute
# Endpoint
curl -4 -s -o /dev/null -w '%{http_code}\n' https://plane.chans.xyz/
```
## Local backup ## Local backup
| Item | Detail | | Item | Detail |
@@ -62,7 +105,7 @@ All other ports internal only (no K3s API, no database, no Redis exposed).
| Retention | 7 days | | Retention | 7 days |
| Disk warning | 80% (healthcheck), 90% (backup stops) | | Disk warning | 80% (healthcheck), 90% (backup stops) |
| Content | Planned: PostgreSQL `synapse` + `mas` logical dumps, media store archive, `/etc/matrix-bootstrap` | | Content | Planned: PostgreSQL `synapse` + `mas` logical dumps, media store archive, `/etc/matrix-bootstrap` |
| Status | **Not operational** — no current Matrix backup or recovery tier | | Status | **Not operational** — no current Matrix backup or recovery tier. **Plane data (its own Postgres + MinIO PVCs in ns `plane`) is also not covered by any backup.** |
## Health checks ## Health checks
@@ -101,5 +144,6 @@ diagnosis and imperative recovery work.
- MatrixRTC / Element Call / LiveKit / Coturn not deployed (`mrtc.chans.xyz` reserved only) - MatrixRTC / Element Call / LiveKit / Coturn not deployed (`mrtc.chans.xyz` reserved only)
- SMTP email not yet configured (requires manual secret bootstrap followed by a - SMTP email not yet configured (requires manual secret bootstrap followed by a
reviewed Ansible stack deployment) reviewed Ansible stack deployment)
- Plane `minio` image uses `latest` tag (pin a version)
- No off-site Restic backup - No off-site Restic backup
- Single-node K3s (no HA for control plane) - Single-node K3s (no HA for control plane)
+1
View File
@@ -48,4 +48,5 @@ the software deployed there, see [the LAN overview](../docs/lan-overview.md).
| https://synapse.chans.xyz | Synapse API | Client-Server + Federation API | | https://synapse.chans.xyz | Synapse API | Client-Server + Federation API |
| https://account.chans.xyz | MAS | Matrix Authentication Service (local passwords) | | https://account.chans.xyz | MAS | Matrix Authentication Service (local passwords) |
| https://admin.chans.xyz | Element Admin | Admin console (MAS admin auth) | | https://admin.chans.xyz | Element Admin | Admin console (MAS admin auth) |
| https://plane.chans.xyz | Plane | Project management (Helm `plane-ce` v1.4.1, ns `plane`) |
| `mrtc.chans.xyz` | MatrixRTC | **Reserved** not deployed | | `mrtc.chans.xyz` | MatrixRTC | **Reserved** not deployed |
+1
View File
@@ -16,6 +16,7 @@ runbooks start from [`_template.md`](_template.md).
| PowerDNS health check | [pdns-health.md](pdns-health.md) | read-only | | PowerDNS health check | [pdns-health.md](pdns-health.md) | read-only |
| RustDesk health check | [rustdesk-health.md](rustdesk-health.md) | read-only | | RustDesk health check | [rustdesk-health.md](rustdesk-health.md) | read-only |
| Matrix health check | [matrix-health.md](matrix-health.md) | read-only | | Matrix health check | [matrix-health.md](matrix-health.md) | read-only |
| Plane health check | [plane-health.md](plane-health.md) | read-only |
| pgdb health check | [pgdb-health.md](pgdb-health.md) | read-only | | pgdb health check | [pgdb-health.md](pgdb-health.md) | read-only |
| pgdb DB restore (pg_restore) | [pgdb-restore.md](pgdb-restore.md) | change (procedure) | | pgdb DB restore (pg_restore) | [pgdb-restore.md](pgdb-restore.md) | change (procedure) |
| pgdb image/compose update | [pgdb-update.md](pgdb-update.md) | change (gated) | | pgdb image/compose update | [pgdb-update.md](pgdb-update.md) | change (gated) |
+93
View File
@@ -0,0 +1,93 @@
# Runbook: Plane Health Check
## Purpose
Read-only health check of the self-hosted Plane project-management instance
(`plane.chans.xyz`) running on the synapse K3s cluster.
## Scope
- Applicable: [synapse.chans.xyz](../hosts/synapse.chans.xyz.md), namespace `plane`.
- Read-only: does not change pods, ingress, certificates, secrets, or configuration.
- Not applicable: Plane upgrade, values changes, or data recovery — those need a
reviewed change (see `ansible-operations.md` / `release.md`).
## Ownership
- Owner: personal ops (Windy)
- Last reviewed: 2026-09-02
- Related systems: synapse.chans.xyz (Helm `plane-app`, chart `plane-ce-1.8.0`, app `v1.4.1`)
## Pass criteria
All of the following must hold; any conflict means `STOP` and record evidence.
- `sudo helm list -A` shows `plane-app` in ns `plane`, STATUS `deployed`.
- All 7 Deployments + 4 StatefulSets in ns `plane` are `1/1 Running` with 0 recent restarts.
- Init Jobs `api-migrate-*` / `minio-bucket-*` are `Completed`.
- Certificate `plane-app-ssl-cert` is `READY=True` (CN `plane.chans.xyz`).
- `https://plane.chans.xyz/` returns HTTP 200 with a valid Let's Encrypt cert.
- Root disk usage below the 80% warning threshold.
## Procedure
### 1. Release and workloads
```bash
ssh -4 windy@synapse.chans.xyz 'sudo helm list -A'
ssh -4 windy@synapse.chans.xyz 'sudo k3s kubectl -n plane get deploy,sts,pods -o wide'
```
Expected: `plane-app` deployed; all workloads `1/1 Running`, `RESTARTS` low;
no `CrashLoopBackOff` / `Evicted`. Otherwise `STOP` and record evidence.
### 2. TLS certificate and ingress
```bash
ssh -4 windy@synapse.chans.xyz 'sudo k3s kubectl -n plane get certificate,issuer,ingressroute'
```
Expected: `plane-app-ssl-cert` `READY=True`; `plane-app-ingress` present with
routes for `/`, `/api`, `/spaces`, `/god-mode`, `/live`, `/uploads`.
If `READY=False` with a pending HTTP-01 challenge, the usual cause is the node
DNS chain (coreDNS → systemd-resolved → uplink) failing to resolve
`plane.chans.xyz` — check `resolvectl query plane.chans.xyz` vs
`dig +short plane.chans.xyz @8.8.8.8`. If the record exists publicly but the
node fails, `sudo resolvectl flush-caches` and wait for the cert-manager retry;
do not mutate the issuer.
### 3. Endpoint verification
```bash
curl -4 -s -o /dev/null -w '%{http_code}\n' https://plane.chans.xyz/
echo | openssl s_client -connect plane.chans.xyz:443 -servername plane.chans.xyz 2>/dev/null | openssl x509 -noout -subject -issuer -dates
```
Expected: HTTP 200, cert `CN=plane.chans.xyz` issued by Let's Encrypt with a
future `notAfter`. `https://plane.chans.xyz/api/` may 404 — the API serves
under `/api/...` paths only, so a bare 404 there is not a failure.
### 4. Storage and disk
```bash
ssh -4 windy@synapse.chans.xyz 'sudo k3s kubectl -n plane get pvc'
ssh -4 windy@synapse.chans.xyz 'df -hP /'
```
Expected: all 4 PVCs `Bound` (minio 5Gi, pgdb 5Gi, rabbitmq 100Mi, redis 100Mi
on `local-path`); root disk < 80%.
## Safety
- Read-only: never mutate pods, ingress, certificates, or configuration during
this check.
- Plane data (ns `plane` Postgres + MinIO PVCs) has **no backup tier**; treat
the instance as at-risk until a backup design exists.
- If live state conflicts with the expected values above, `STOP` and record
evidence; do not "fix in passing".
## References
- [hosts/synapse.chans.xyz.md](../hosts/synapse.chans.xyz.md) — plane stack facts
- [matrix-health.md](matrix-health.md) — sibling service on the same cluster