100 lines
3.1 KiB
Markdown
100 lines
3.1 KiB
Markdown
# LLM Wiki v2 — Comprehensive Summary
|
|
|
|
## Source
|
|
|
|
https://gist.github.com/rohitg00/2067ab416f7bbe447c1977edaaa681e2
|
|
Author: rohitg00
|
|
Forked from: karpathy/llm-wiki.md
|
|
Last active: 2026-04-13
|
|
|
|
## Overview
|
|
|
|
A pattern for building personal knowledge bases using LLMs, extending Karpathy's original LLM Wiki idea with lessons from building agentmemory. Addresses what breaks at scale, what's missing, and what separates a useful wiki from one that rots.
|
|
|
|
---
|
|
|
|
## What the Original Gets Right
|
|
|
|
> **Stop re-deriving, start compiling.** RAG retrieves and forgets. A wiki accumulates and compounds.
|
|
|
|
- Three-layer architecture works: raw sources → wiki → schema
|
|
- Basic operations (ingest, query, lint) cover the basics
|
|
|
|
---
|
|
|
|
## Missing Layer: Memory Lifecycle
|
|
|
|
### Confidence Scoring
|
|
Every fact should carry a confidence score indicating:
|
|
- How many sources support it
|
|
- How recently it was confirmed
|
|
- Whether anything contradicts it
|
|
|
|
### Supersession
|
|
When new information contradicts existing claims:
|
|
- Old claim explicitly superseded, not just noted
|
|
- Linked and timestamped
|
|
- Old version preserved but marked stale
|
|
|
|
### Forgetting
|
|
- Wikis that never forget become noisy
|
|
- Implement a retention curve based on Ebbinghaus's forgetting curve
|
|
- Architecture decisions decay slowly. Transient bugs decay fast.
|
|
|
|
### Consolidation Tiers
|
|
|
|
| Tier | Description | Characteristics |
|
|
|------|-------------|------------------|
|
|
| Working memory | Recent observations | Not yet processed |
|
|
| Episodic memory | Session summaries | Compressed from raw |
|
|
| Semantic memory | Cross-session facts | Consolidated from episodes |
|
|
| Procedural memory | Workflows and patterns | Extracted from repeated semantics |
|
|
|
|
---
|
|
|
|
## Beyond Flat Pages: Knowledge Graph
|
|
|
|
### Entity Extraction
|
|
Extract structured entities: People, projects, libraries, concepts, files, decisions
|
|
|
|
### Typed Relationships
|
|
Not all connections are equal: uses, depends_on, contradicts, caused, fixed, supersedes
|
|
|
|
### Graph Traversal for Queries
|
|
Instead of keyword search: walk outward through typed edges to find all related nodes.
|
|
|
|
---
|
|
|
|
## Search That Actually Scales
|
|
|
|
### When index.md Breaks
|
|
Works up to ~100-200 pages. Beyond that, becomes too long for LLM.
|
|
|
|
### Hybrid Search Architecture
|
|
|
|
| Stream | Catches | Method |
|
|
|--------|---------|--------|
|
|
| BM25 | Exact terms | Keyword matching |
|
|
| Vector search | Semantic similarity | Embeddings |
|
|
| Graph traversal | Structural connections | Entity-aware relationship walking |
|
|
|
|
---
|
|
|
|
## Automation: Event-Driven Operations
|
|
|
|
| Event | Action |
|
|
|-------|--------|
|
|
| On new source | Auto-ingest, extract entities, update graph, update index |
|
|
| On session start | Load relevant context based on recent activity |
|
|
| On session end | Compress session into observations, file insights |
|
|
| On query | Check if answer is worth filing back (quality score > threshold) |
|
|
| On memory write | Check for contradictions, trigger supersession |
|
|
| On schedule | Periodic lint, consolidation, retention decay |
|
|
|
|
---
|
|
|
|
## Quality and Self-Correction
|
|
|
|
### Score Everything
|
|
Every piece of LLM-generated content gets a quality score based on structure, citations, wikilink density, length, and fact consistency.
|