2026-02-26 21:10:47 +08:00
|
|
|
|
---
|
|
|
|
|
|
title: 架构设计
|
|
|
|
|
|
created: 2026-02-25
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
# 架构设计
|
|
|
|
|
|
|
|
|
|
|
|
## 设计目标
|
|
|
|
|
|
|
|
|
|
|
|
1. 不使用本地 embedding 模型（避免常驻 Ollama）
|
|
|
|
|
|
2. 向量与元数据留在本地 PostgreSQL（数据可控）
|
|
|
|
|
|
3. 基于 Git 变更做增量 upsert/delete（避免全量重建）
|
|
|
|
|
|
4. 强一致：删除/重命名后不允许幽灵向量残留
|
|
|
|
|
|
5. 安全：敏感内容不入检索库，检索结果具备 Prompt 注入防护
|
|
|
|
|
|
|
|
|
|
|
|
## 双层记忆模型
|
|
|
|
|
|
|
2026-02-26 21:18:17 +08:00
|
|
|
|
```mermaid
|
|
|
|
|
|
graph TD
|
|
|
|
|
|
A["🗂️ Obsidian Vault (.md files)"]
|
|
|
|
|
|
B["🪝 git post-commit hook"]
|
|
|
|
|
|
C["⚙️ incremental_ingest.py"]
|
|
|
|
|
|
D[("🐘 PostgreSQL + pgvector")]
|
|
|
|
|
|
E["🔍 query_pgvector.py"]
|
|
|
|
|
|
F["📝 Claude Code / MEMORY.md"]
|
|
|
|
|
|
G["🤖 Claude Agent\nagent-with-memory.sh"]
|
|
|
|
|
|
H["Layer B: MEMORY.md\n偏好 / 事实 / 约束"]
|
|
|
|
|
|
I["Layer A: pgvector\n语义检索结果"]
|
|
|
|
|
|
|
|
|
|
|
|
A --> B --> C --> D --> E --> G
|
|
|
|
|
|
F --> G
|
|
|
|
|
|
G --> H
|
|
|
|
|
|
G --> I
|
2026-02-26 21:10:47 +08:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### Layer A：文档语义记忆
|
|
|
|
|
|
|
|
|
|
|
|
- Embedding：OpenRouter API（`openai/text-embedding-3-small`，dim=1536）
|
|
|
|
|
|
- 存储：本地 PostgreSQL + pgvector 扩展
|
|
|
|
|
|
- 数据源：`01_Projects/`、`02_Areas/`
|
|
|
|
|
|
- 排除：`00_Inbox/`、`04_Archive/`、`Infrastructure/`、`Home-Automation/`
|
|
|
|
|
|
- 触发：git `post-commit` hook → 异步 `incremental_ingest.py`
|
|
|
|
|
|
|
|
|
|
|
|
### Layer B：事实与偏好记忆
|
|
|
|
|
|
|
|
|
|
|
|
- 来源：Claude Code `MEMORY.md`（查询时动态读取）
|
|
|
|
|
|
- 用途：用户偏好、约束、近期状态
|
|
|
|
|
|
|
|
|
|
|
|
## 组件职责
|
|
|
|
|
|
|
|
|
|
|
|
| 文件 | 职责 |
|
|
|
|
|
|
|---|---|
|
|
|
|
|
|
| `index_common.py` | 共享工具：环境加载、DB 连接、Embedding API、排除逻辑、跨平台锁 |
|
|
|
|
|
|
| `blacklist.py` | 排除规则：路径模式、文件名关键词、敏感内容特征 |
|
|
|
|
|
|
| `ingest_vault.py` | 全量重建索引，清理过期条目 |
|
|
|
|
|
|
| `incremental_ingest.py` | 解析 `git diff-tree` 输出，处理 A/M/D/R 事件 |
|
|
|
|
|
|
| `query_pgvector.py` | 余弦相似度检索，返回 `<retrieved_context>` 块 |
|
|
|
|
|
|
| `install-hook.sh` | 向 `.git/hooks/post-commit` 追加异步索引触发器 |
|
|
|
|
|
|
| `agent-with-memory.sh` | 合并 A+B 层上下文，启动带记忆的 claude 会话 |
|
|
|
|
|
|
| `schema.sql` | DB schema：`memory_primary`（向量）、`memory_secure_audit`（隔离审计）|
|
|
|
|
|
|
|
|
|
|
|
|
## 数据库 Schema
|
|
|
|
|
|
|
|
|
|
|
|
```sql
|
|
|
|
|
|
-- 主检索表
|
|
|
|
|
|
memory_primary (
|
|
|
|
|
|
id TEXT PRIMARY KEY, -- vault 相对路径
|
|
|
|
|
|
source TEXT, -- 同 id，用于上下文注入显示
|
|
|
|
|
|
content TEXT, -- 完整 markdown 文本
|
|
|
|
|
|
content_hash TEXT, -- sha256，用于变更检测
|
|
|
|
|
|
embedding VECTOR(1536), -- ivfflat 余弦索引
|
|
|
|
|
|
updated_at TIMESTAMPTZ
|
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
-- 敏感文件审计表
|
|
|
|
|
|
memory_secure_audit (
|
|
|
|
|
|
id TEXT PRIMARY KEY,
|
|
|
|
|
|
source TEXT,
|
|
|
|
|
|
risk TEXT, -- 'excluded_or_sensitive'
|
|
|
|
|
|
updated_at TIMESTAMPTZ
|
|
|
|
|
|
)
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
索引：`ivfflat (embedding vector_cosine_ops) WITH (lists = 100)`
|
|
|
|
|
|
|
|
|
|
|
|
## 增量同步流程
|
|
|
|
|
|
|
2026-02-26 21:18:17 +08:00
|
|
|
|
```mermaid
|
|
|
|
|
|
flowchart TD
|
|
|
|
|
|
GC["git commit"]
|
|
|
|
|
|
HK["post-commit hook\nnohup，不阻塞提交"]
|
|
|
|
|
|
II["incremental_ingest.py\n--changes-file <tmp>"]
|
|
|
|
|
|
PD["解析 git diff-tree 输出\nA / M / T / D / R"]
|
|
|
|
|
|
LK["获取 index_lock\n跨平台文件锁"]
|
|
|
|
|
|
AMT["A/M/T → upsert_file()"]
|
|
|
|
|
|
D["D → DELETE from both tables"]
|
|
|
|
|
|
R["R → DELETE old\n+ upsert_file(new)"]
|
|
|
|
|
|
|
|
|
|
|
|
GC --> HK --> II --> PD --> LK
|
|
|
|
|
|
LK --> AMT
|
|
|
|
|
|
LK --> D
|
|
|
|
|
|
LK --> R
|
2026-02-26 21:10:47 +08:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## 安全策略
|
|
|
|
|
|
|
|
|
|
|
|
1. 路径精确匹配：按 `Path.parts` 做目录排除，非子串匹配
|
|
|
|
|
|
2. 单命中隔离：任一敏感特征命中即隔离到 `memory_secure_audit`
|
|
|
|
|
|
3. 强一致删除：D/R 事件先删旧 ID，再处理新路径
|
|
|
|
|
|
4. 单写者锁：跨平台文件锁串行化所有索引写操作
|
|
|
|
|
|
5. 只读上下文注入：检索结果包装为 `<retrieved_context>` 标签，明确标注为非指令
|
|
|
|
|
|
6. 长度截断：注入前全局 `max_chars=2500`
|
|
|
|
|
|
|
|
|
|
|
|
## 关键权衡
|
|
|
|
|
|
|
|
|
|
|
|
| 优点 | 代价 |
|
|
|
|
|
|
|---|---|
|
|
|
|
|
|
| 不跑本地模型，设备压力低 | Markdown 文本发送到 OpenRouter（非纯本地隐私）|
|
|
|
|
|
|
| 向量在本地 DB，数据控制力强 | 依赖网络与 API 可用性 |
|
|
|
|
|
|
| 与现有 Git 工作流兼容 | 需配置 API key 与限流/重试策略 |
|
