airport-wiki/concepts/knowledge-management/llm-wiki-v2-reference.md

---
title: LLM Wiki v2 参考文档
created: 2026-04-13
updated: 2026-04-15
type: core-component
tags: [llm-wiki, v2, reference, architecture]
sources_count: 1
confidence: 0.85
last_confirmed: 2026-04-15
status: active
relationships: 
- target: concepts/knowledge-management/hybrid-search.md
detail: "混合搜索系统"
---
# 📚 LLM Wiki v2 参考文档

**机场智能化工程知识库的架构与技术实现指南**

基于 Karpathy 的 LLM Wiki 理念，v2 版本引入了**动态知识管理**、**智能检索**和**自动化维护**三大核心能力。本文档详细说明架构设计、实现原理和配置方法。

> **v2 核心理念**：知识是动态的有机体，需要随时间衰减、冲突整合和持续验证，而非静态的文档集合。

---

## 🏗️ 架构总览

### 系统分层架构
```
应用层 (Application Layer)
├── 搜索引擎 (Hybrid Search Engine)
├── 关系图谱 (Knowledge Graph Browser)
└── 质量看板 (Quality Dashboard)

核心层 (Core Layer)
├── 自动化钩子系统 (Automation Hooks)
├── 置信度衰减引擎 (Confidence Decay Engine)
├── 冲突检测处理器 (Conflict Detection Processor)
└── 自我纠正机制 (Self-correction Mechanism)

存储层 (Storage Layer)
├── 向量数据库 (Vector Database)        # 嵌入存储
├── 文档存储 (Document Store)          # Markdown/YAML
└── 关系数据库 (Relational Database)    # 实体关系
```

### 数据流向
```
新来源 → 解析 → 实体提取 → 事实提取 → 冲突检测 → 置信度评估 → 存储
                 ↓           ↓           ↓           ↓          ↓
现有知识 ← 整合 ← 关系更新 ← 冲突解决 ← 置信度衰减 ← 质量验证 ← 定期任务
```

---

## 🔧 核心组件详解

### 1. **置信度系统 (Confidence System)**
#### 置信度字段结构
```yaml
---
confidence: 0.85                    # 当前置信度 (0.1-1.0)
sources_count: 3                    # 引用来源数量
last_confirmed: 2026-04-13         # 最后一次确认/更新
confidence_history:                 # 置信度变化历史
  - date: 2026-04-10
    value: 0.80
    reason: "new_source_added"
  - date: 2026-04-12
    value: 0.83
    reason: "conflict_resolved"
  - date: 2026-04-13
    value: 0.85
    reason: "weekly_decay_applied"
decay_factors:                     # 衰减因子权重
  time: 0.60
  usage: 0.20
  sources: 0.15
  conflicts: 0.05
---
```

#### 衰减算法
```python
def decay_confidence(current, age_days, usage_freq, sources_cnt, conflicts_cnt):
    # 基础时间衰减（每月5%）
    base_decay = 0.95 ** (age_days / 30)
    
    # 强化因子（使用频率和来源数量）
    reinforcement = (usage_freq * 0.3) + (min(sources_cnt, 5) * 0.05)
    
    # 应用衰减
    new_confidence = current * base_decay + (1 - base_decay) * reinforcement
    
    # 冲突惩罚
    new_confidence -= 0.05 * conflicts_cnt
    
    # 边界处理
    return max(0.05, min(1.0, new_confidence))
```

### 2. **实体图管理系统 (Entity Graph Management)**
#### 实体类型定义
```python
ENTITY_TYPES = {
    "airport": {
        "attributes": ["code", "name", "location", "capacity", "status"],
        "relationships": {
            "uses": ["technology", "system", "vendor"],
            "located_in": ["region", "country"],
            "implements": ["standard", "certification"]
        }
    },
    "technology": {
        "attributes": ["category", "vendor", "version", "specs"],
        "relationships": {
            "used_by": ["airport", "system"],
            "compatible_with": ["technology"],
            "replaces": ["technology"]
        }
    },
    "vendor": {
        "attributes": ["name", "country", "specialization", "market_share"],
        "relationships": {
            "provides": ["technology", "service"],
            "competes_with": ["vendor"],
            "partners_with": ["vendor"]
        }
    }
}
```

#### 关系类型
| 关系类型 | 语义 | 反向关系 | 示例 |
|----------|------|----------|------|
| **uses** | 使用 | used_by | 深圳机场 uses SITA AODB |
| **implements** | 实现 | implemented_by | JFK implements ACI EUROPE 2020 |
| **replaces** | 替换 | replaced_by | H100 replaces A100 |
| **based_on** | 基于 | basis_for | 数字孿生 based_on BIM 模型 |
| **compatible_with** | 兼容 | compatible_with | RoCE compatible_with InfiniBand |
| **partners_with** | 合作 | partners_with | SITA partners_with Huawei |

### 3. **自动化钩子系统 (Automation Hooks)**
#### 事件注册表
```python
HOOK_REGISTRY = {
    "source_added": [
        {"callback": "parse_source", "priority": 10},
        {"callback": "extract_entities", "priority": 9},
        {"callback": "detect_conflicts", "priority": 8},
        {"callback": "update_confidence", "priority": 7}
    ],
    "page_updated": [
        {"callback": "check_semantic_change", "priority": 10},
        {"callback": "update_embeddings", "priority": 9},
        {"callback": "propagate_relations", "priority": 8},
        {"callback": "log_change", "priority": 5}
    ],
    "query_executed": [
        {"callback": "record_query_pattern", "priority": 10},
        {"callback": "evaluate_results", "priority": 8},
        {"callback": "generate_suggestions", "priority": 5}
    ]
}
```

#### 定时任务调度
```python
SCHEDULE_CONFIG = {
    "daily": {
        "time": "02:30",
        "tasks": [
            "verify_recent_changes",
            "update_recommendations",
            "clean_temp_files",
            "backup_incremental"
        ]
    },
    "weekly": {
        "time": "03:00",
        "day": "sunday",
        "tasks": [
            "run_lint_check",
            "decay_confidence_scores",
            "regenerate_embeddings",
            "rebuild_search_index"
        ]
    },
    "monthly": {
        "time": "04:00",
        "day": 1,  # 每月1日
        "tasks": [
            "archive_stale_content",
            "evaluate_embedding_models",
            "analyze_growth_trends",
            "run_security_audit"
        ]
    }
}
```

### 4. **混合搜索系统 (Hybrid Search System)**
#### 搜索评分算法
```python
def calculate_search_score(page, query, user_context):
    # 1. 文本相关性 (BM25)
    text_relevance = bm25_score(page.content, query)
    
    # 2. 语义相关性 (嵌入相似度)
    semantic_relevance = embedding_similarity(page.embedding, query_embedding)
    
    # 3. 置信度调整
    confidence_adjustment = page.confidence ** 2
    
    # 4. 时效性调整
    recency_adjustment = 1.0 / (1 + page.age_days / 180)
    
    # 5. 个性化调整
    personalization = calculate_personalization_score(page, user_context)
    
    # 综合评分 (加权)
    score = (
        text_relevance * 0.4 +
        semantic_relevance * 0.4 +
        confidence_adjustment * 0.15 +
        recency_adjustment * 0.05 +
        personalization * 0.1
    )
    
    return score
```

#### 查询重写策略
```python
QUERY_REWRITE_RULES = [
    # 同义词扩展
    {"pattern": r"\bgpu\b", "expansion": "gpu OR graphics processing unit OR ai accelerator"},
    
    # 技术缩写扩展
    {"pattern": r"\baodb\b", "expansion": "aodb OR airport operational database"},
    
    # 机场代码映射
    {"pattern": r"\bSZX\b", "expansion": "SZX OR Shenzhen Bao'an International Airport"},
    {"pattern": r"\bJFK\b", "expansion": "JFK OR New York John F. Kennedy Airport"},
    
    # 单位标准化
    {"pattern": r"(\d+)\s*kw", "expansion": "$1 kW OR $1 kilowatt"},
    {"pattern": r"(\d+)\s*MW", "expansion": "$1 MW OR $1 megawatt"},
]
```

---

## ⚙️ 配置与部署

### 配置文件结构
```yaml
# ~/.hermes/airport-wiki/config.yaml

llm_wiki:
  version: "2.1.0"
  
  confidence:
    decay_rate: 0.05  # 每月衰减率
    min_confidence: 0.05
    max_confidence: 1.0
    usage_weight: 0.2
    sources_weight: 0.15
  
  automation:
    enabled: true
    check_interval_seconds: 15  # 文件监视间隔
    max_workers: 3
    
  search:
    hybrid_enabled: true
    vector_weight: 0.4
    keyword_weight: 0.4
    confidence_weight: 0.15
    recency_weight: 0.05
    query_expansion: true
    
  entities:
    types: ["airport", "technology", "vendor", "standard", "system"]
    relation_types: ["uses", "implements", "replaces", "based_on", "compatible_with"]
    
  storage:
    vector_db: "chromadb"
    doc_store: "filesystem"
    graph_db: "sqlite"
    
  monitoring:
    metrics_enabled: true
    alerting_enabled: true
    log_level: "info"
```

### 环境变量
```bash
# LLM Wiki 核心配置
export LLM_WIKI_HOME="/home/windy/.hermes/airport-wiki"
export EMBEDDING_MODEL="all-MiniLM-L6-v2"
export VECTOR_DB_HOST="localhost"
export VECTOR_DB_PORT=8000

# 自动化钩子
export HOOKS_ENABLED="true"
export HOOKS_CHECK_INTERVAL="15"
export HOOKS_MAX_WORKERS="3"

# 监控和日志
export LOG_LEVEL="info"
export METRICS_PORT="9090"
export ALERT_WEBHOOK="https://hooks.slack.com/services/..."
```

### 初始化脚本
```bash
#!/bin/bash
# init_llm_wiki_v2.sh

# 1. 检查依赖
check_dependencies() {
    echo "检查依赖..."
    python3 --version >/dev/null 2>&1 || { echo "需要 Python 3.8+"; exit 1; }
    pip --version >/dev/null 2>&1 || { echo "需要 pip"; exit 1; }
}

# 2. 安装 Python 包
install_packages() {
    echo "安装 Python 包..."
    pip install -r requirements.txt
}

# 3. 初始化数据库
init_databases() {
    echo "初始化数据库..."
    python -c "from storage import init_db; init_db()"
}

# 4. 生成初始嵌入
generate_initial_embeddings() {
    echo "生成初始嵌入..."
    python -c "from embeddings import generate_all_embeddings; generate_all_embeddings()"
}

# 5. 启动服务
start_services() {
    echo "启动服务..."
    
    # 启动文件监视服务
    python -m hooks.file_watcher &
    
    # 启动定时任务调度器
    python -m hooks.scheduler &
    
    # 启动监控服务
    python -m monitoring.metrics_server &
}

main() {
    echo "=== LLM Wiki v2 初始化 ==="
    
    check_dependencies
    install_packages
    init_databases
    generate_initial_embeddings
    start_services
    
    echo "✅ 初始化完成"
    echo "监控面板: http://localhost:9090"
    echo "搜索端点: http://localhost:8000/search"
}

main "$@"
```

---

## 🔄 升级与迁移

### 从 v1 升级到 v2
#### 步骤 1: 备份 v1 数据
```bash
# 备份整个 wiki 目录
tar -czf wiki_v1_backup_$(date +%Y%m%d).tar.gz airport-wiki/

# 导出实体关系
python -c "from v1_exporter import export_all; export_all('v1_export.json')"
```

#### 步骤 2: 安装 v2 组件
```bash
# 创建新配置目录
mkdir -p ~/.hermes/airport-wiki/concepts/knowledge-management

# 安装 v2 Python 包
pip install llm-wiki-v2

# 初始化 v2 数据库
python -m llm_wiki_v2.init --config config.yaml
```

#### 步骤 3: 迁移数据
```bash
# 运行迁移脚本
python -m llm_wiki_v2.migrate \
  --v1_path ./airport-wiki \
  --v2_path ./airport-wiki-v2 \
  --mode incremental
```

#### 步骤 4: 验证迁移
```bash
# 检查置信度字段
python -c "from validation import check_migration; check_migration('airport-wiki-v2')"

# 测试搜索功能
curl -X POST "http://localhost:8000/search" \
  -H "Content-Type: application/json" \
  -d '{"query": "GPU cluster power consumption", "limit": 5}'
```

### 数据迁移策略
| 数据类型 | v1 格式 | v2 格式 | 迁移方法 |
|----------|---------|---------|----------|
| **页面内容** | 纯 Markdown | Markdown + YAML frontmatter | 解析并添加置信度字段 |
| **实体关系** | 链接（无类型） | 类型化关系 | 提取文本关系并分类 |
| **嵌入向量** | 无 | 向量数据库 | 重新生成所有嵌入 |
| **搜索索引** | 文件搜索 | 混合搜索索引 | 重建索引 |

---

## 📊 监控与告警

### 关键性能指标 (KPIs)
```python
KPI_CONFIG = {
    "search": {
        "response_time_p95": {"threshold": 3000, "unit": "ms"},
        "success_rate": {"threshold": 0.95, "unit": "%"},
        "recall_at_5": {"threshold": 0.85, "unit": "%"}
    },
    "confidence": {
        "average_confidence": {"threshold": 0.7, "unit": "score"},
        "decay_rate": {"threshold": 0.1, "unit": "/month"},
        "conflict_resolution_rate": {"threshold": 0.9, "unit": "%"}
    },
    "automation": {
        "hook_success_rate": {"threshold": 0.95, "unit": "%"},
        "processing_time_p95": {"threshold": 5000, "unit": "ms"},
        "backlog_size": {"threshold": 100, "unit": "items"}
    }
}
```

### 告警规则
```yaml
alerts:
  - name: "search_degradation"
    condition: "search_response_time_p95 > 3000 OR search_success_rate < 0.95"
    severity: "critical"
    actions: ["page_oncall", "rollback_search_config"]
    
  - name: "confidence_anomaly"
    condition: "average_confidence < 0.6 OR decay_rate > 0.15"
    severity: "high"
    actions: ["notify_maintainer", "run_verification"]
    
  - name: "automation_failure"
    condition: "hook_success_rate < 0.9 OR backlog_size > 200"
    severity: "medium"
    actions: ["log_incident", "restart_workers"]
```

### 监控仪表板
- **搜索性能仪表板**：响应时间、命中率、用户满意度
- **知识质量仪表板**：平均置信度、冲突数量、更新频率
- **系统健康仪表板**：自动化成功率、存储使用率、错误率

---

## 🛠️ 故障排除

### 常见问题及解决方法
| 问题 | 症状 | 解决方案 |
|------|------|----------|
| **置信度不衰减** | 页面置信度长期不变 | 检查定时任务是否运行；验证衰减算法参数 |
| **搜索结果差** | 相关页面排名靠后 | 调整搜索权重；重新生成嵌入；检查索引 |
| **自动化钩子失败** | 文件变更未触发处理 | 验证文件监视配置；检查权限；查看日志 |
| **实体关系缺失** | 页面无关系链接 | 运行实体提取；检查关系检测规则 |
| **嵌入生成失败** | 页面无嵌入向量 | 检查模型加载；验证文本编码；查看错误日志 |

### 诊断命令
```bash
# 检查系统状态
python -m llm_wiki_v2.status --full

# 查看日志
tail -f ~/.hermes/logs/llm_wiki.log

# 手动触发维护任务
python -m hooks.runner --task weekly_maintenance

# 检查数据库完整性
python -c "from storage import verify_integrity; verify_integrity()"

# 重置错误状态
python -m llm_wiki_v2.reset --component hooks
```

---

## 🔮 未来发展方向

### 近期计划 (1-3个月)
- **智能答案生成**：基于查询自动生成综合答案
- **预测性维护**：基于历史模式预测知识老化
- **多模态支持**：图像、图表等非文本内容处理
- **用户行为分析**：优化搜索和推荐系统

### 中期计划 (3-12个月)
- **跨wiki知识同步**：多个wiki之间的知识共享
- **自适应学习**：系统自动调整参数和规则
- **自然语言更新**：用户用自然语言编辑知识
- **实时协作**：多用户同时编辑和注释

### 长期愿景 (1年以上)
- **自主知识管理**：系统完全自主维护和优化知识库
- **预测性内容创建**：基于趋势预测自动创建新内容
- **智能决策支持**：基于知识库提供决策建议
- **认知增强**：与人类思维深度协同的知识系统

---

## 📚 相关资源

### 官方文档
- [[SCHEMA.md]] - 架构定义和设计规范
- [[concepts/knowledge-management/automation-hooks.md]] - 自动化钩子详细实现
- [[concepts/knowledge-management/knowledge-lifecycle.md]] - 知识生命周期管理
- [[concepts/knowledge-management/quality-control.md]] - 质量控制机制
- [[concepts/knowledge-management/knowledge-graph.md]] - 实体图管理
- [[concepts/knowledge-management/hybrid-search.md]] - 混合搜索系统

### 工具和库
- **向量数据库**: ChromaDB, Qdrant, Weaviate
- **嵌入模型**: all-MiniLM-L6-v2, BGE, OpenAI embeddings
- **搜索引擎**: Elasticsearch, Meilisearch, Typesense
- **监控**: Prometheus, Grafana, OpenTelemetry

### 参考文献
1. Karpathy, A. "LLM: A Personal Knowledge Base"
2. Luhmann, N. "Zettelkasten Method"
3. Ahrens, S. "How to Take Smart Notes"
4. Vannevar Bush, "As We May Think"

---

> **版本**: v2.1.0 | **最后更新**: 2026-04-13  
> **维护状态**: 活跃 | **支持**: 用户文档 + 技术支持论坛  
> **注意**: 本系统持续演进，建议定期查看相关文档获取最新信息。