3.1 KiB
LLM Wiki v2 — Comprehensive Summary
Source
https://gist.github.com/rohitg00/2067ab416f7bbe447c1977edaaa681e2 Author: rohitg00 Forked from: karpathy/llm-wiki.md Last active: 2026-04-13
Overview
A pattern for building personal knowledge bases using LLMs, extending Karpathy's original LLM Wiki idea with lessons from building agentmemory. Addresses what breaks at scale, what's missing, and what separates a useful wiki from one that rots.
What the Original Gets Right
Stop re-deriving, start compiling. RAG retrieves and forgets. A wiki accumulates and compounds.
- Three-layer architecture works: raw sources → wiki → schema
- Basic operations (ingest, query, lint) cover the basics
Missing Layer: Memory Lifecycle
Confidence Scoring
Every fact should carry a confidence score indicating:
- How many sources support it
- How recently it was confirmed
- Whether anything contradicts it
Supersession
When new information contradicts existing claims:
- Old claim explicitly superseded, not just noted
- Linked and timestamped
- Old version preserved but marked stale
Forgetting
- Wikis that never forget become noisy
- Implement a retention curve based on Ebbinghaus's forgetting curve
- Architecture decisions decay slowly. Transient bugs decay fast.
Consolidation Tiers
| Tier | Description | Characteristics |
|---|---|---|
| Working memory | Recent observations | Not yet processed |
| Episodic memory | Session summaries | Compressed from raw |
| Semantic memory | Cross-session facts | Consolidated from episodes |
| Procedural memory | Workflows and patterns | Extracted from repeated semantics |
Beyond Flat Pages: Knowledge Graph
Entity Extraction
Extract structured entities: People, projects, libraries, concepts, files, decisions
Typed Relationships
Not all connections are equal: uses, depends_on, contradicts, caused, fixed, supersedes
Graph Traversal for Queries
Instead of keyword search: walk outward through typed edges to find all related nodes.
Search That Actually Scales
When index.md Breaks
Works up to ~100-200 pages. Beyond that, becomes too long for LLM.
Hybrid Search Architecture
| Stream | Catches | Method |
|---|---|---|
| BM25 | Exact terms | Keyword matching |
| Vector search | Semantic similarity | Embeddings |
| Graph traversal | Structural connections | Entity-aware relationship walking |
Automation: Event-Driven Operations
| Event | Action |
|---|---|
| On new source | Auto-ingest, extract entities, update graph, update index |
| On session start | Load relevant context based on recent activity |
| On session end | Compress session into observations, file insights |
| On query | Check if answer is worth filing back (quality score > threshold) |
| On memory write | Check for contradictions, trigger supersession |
| On schedule | Periodic lint, consolidation, retention decay |
Quality and Self-Correction
Score Everything
Every piece of LLM-generated content gets a quality score based on structure, citations, wikilink density, length, and fact consistency.