vault backup: 2026-02-26 21:10:47

This commit is contained in:
windyboy
2026-02-26 21:10:47 +08:00
parent 1e3905d304
commit 410167d3b4
38 changed files with 3890 additions and 70 deletions
-10
View File
@@ -54,16 +54,6 @@ Create a comprehensive synthesis of the week's work.
Best for: Weekly reviews, pattern recognition Best for: Weekly reviews, pattern recognition
### 💬 git-commit-msg
AI-generated conventional commit messages from staged changes.
```
/git-commit-msg
```
Best for: Creating meaningful commit messages, maintaining clean git history
## Creating Custom Commands ## Creating Custom Commands
1. Create a new `.md` file in this directory 1. Create a new `.md` file in this directory
+367
View File
@@ -0,0 +1,367 @@
---
name: pragmatic-review
description:
'Interactive pragmatic code review focusing on YAGNI and KISS principles'
version: 1.0.0
argument-hint:
'[--auto] [--ci] [--deep (6-pass)] [--branch branch-name] [--base base-branch]'
allowed-tools:
[
Read,
Grep,
Glob,
Bash(test:*),
Bash(git:*),
Bash(echo:*),
Bash(head:*),
Bash(wc:*),
Bash(tr:*),
]
---
# Pragmatic Code Review: YAGNI & KISS Focus
You will perform an interactive code review with laser focus on **YAGNI** (You
Aren't Gonna Need It) and **KISS** (Keep It Simple, Stupid) principles.
## Review Modes
**Default mode**: Fast YAGNI/KISS-focused review
- Scans for over-engineering, unused abstractions, unnecessary complexity
- Quick security and performance checks (OWASP basics, obvious N+1 queries)
- Self-reflection to validate findings with evidence
**Deep mode** (`--deep` flag): Multi-pass comprehensive review
- Pass 1: Security (OWASP Top 10, input validation, auth issues)
- Pass 2: Architecture (SOLID principles, separation of concerns)
- Pass 3: Logic (edge cases, error handling, correctness)
- Pass 4: Performance (algorithm complexity, resource leaks)
- Pass 5: YAGNI/KISS (over-engineering, unnecessary abstractions)
- Pass 6: Maintainability (readability, tests, documentation)
- Self-reflection after all passes
Use `--deep` when:
- Security-critical changes (auth, payment, data handling)
- Core architecture modifications
- Complex logic changes with many edge cases
- Performance-sensitive code paths
Use default mode when:
- Feature additions
- Bug fixes
- Refactoring
- Documentation changes
**CI mode** (`--ci` flag): Non-interactive mode for GitHub Actions
- Skips ALL interactive prompts
- Auto-selects: all branch changes vs base branch
- Uses `$GITHUB_BASE_REF` environment variable if available
- Outputs all findings at once as markdown (summary view)
## Step 1: Determine Review Scope
### Check Current Git State
First, verify we're in a git repository by running:
- `test -d .git` to check if .git directory exists
If not in a git repository, ask the user to specify files to review manually.
If in a git repository, gather information:
#### Current branch:
Run: `git rev-parse --abbrev-ref HEAD`
#### Default branch detection:
1. Try: `git rev-parse --verify main`
2. If that fails, try: `git rev-parse --verify master`
3. If that fails, try: `git rev-parse --verify develop`
If user specified `--base [branch]` in arguments, use that instead.
#### Working directory status:
Run: `git status --short | head -20`
### Present Options to User
**If `--ci` flag is present:** Skip all interactive prompts and auto-select
option 2: Review all changes on current branch vs base.
Unless `--auto` or `--ci` flag is present, ask the user:
```
📋 CODE REVIEW SCOPE SELECTION
════════════════════════════════
What would you like to review?
1️⃣ Current uncommitted changes
2️⃣ All changes on current branch (compared to [detected default branch])
3️⃣ Specific files or directory
4️⃣ Last N commits
5️⃣ Staged changes only
Please enter your choice (1-5):
```
## Step 2: YAGNI/KISS Analysis Framework
For each file identified, analyze for these patterns:
### YAGNI Detection Patterns
1. **Unused abstractions**
- Interfaces/protocols with single implementations
- Abstract base classes with one concrete subclass
- Generic types that are always the same
2. **Premature flexibility**
- Configuration for things that never change
- Plugin systems with no plugins
- Feature flags that are always on/off
3. **Over-engineering indicators**
- Factory classes for simple objects
- Builder patterns for objects with 2-3 fields
- Event systems with single listeners
4. **Speculative code**
- "TODO: might need this" comments
- Commented-out code "just in case"
- Unreachable code paths
- Methods that are never called
5. **The GenericButton Anti-Pattern**
- Components with 8+ optional parameters serving different use cases
- So many props that using it is as complex as writing from scratch
6. **Premature Abstraction - Rule of Three**
- Abstraction created at 1st or 2nd duplication (wait for 3rd!)
- Reference: Martin Fowler - "Tolerate duplication twice, refactor on the third"
### KISS Violation Patterns
1. **Verbose implementations**
- Can be reduced by >50% lines
- Reimplements standard library functions
- Complex regex when simple string operations work
2. **Abstraction addiction**
- More than 3 levels of inheritance/wrapping
- Interfaces between every layer
3. **Clever code**
- Needs extensive comments to explain
- Uses obscure language features unnecessarily
- One-liners that should be 5 clear lines
4. **Catch-Log-Exit Anti-Pattern**
- Catching exceptions just to log and exit
- Replaces actual error with a guess about what went wrong
```typescript
// TERRIBLE: replaces actual error with a guess
try {
await createNewBranch({ branchName, cwd })
} catch (error) {
console.error('Error: Not in a git repository') // Maybe wrong!
process.exit(1)
}
// CORRECT: let it throw naturally
await createNewBranch({ branchName, cwd })
```
### Security Patterns to Check
Even in a YAGNI/KISS review, flag critical security issues:
1. **SQL Injection**
- String concatenation in SQL queries
- Missing parameterized queries
2. **Authentication/Authorization**
- Hardcoded secrets
- Weak defaults: `SECRET = os.getenv('KEY', 'default')`
- JWT without expiration
3. **Unvalidated External Inputs**
- URL parameters used directly without validation
- API response data trusted without schema validation
### Performance Patterns to Check
Flag obvious performance issues:
1. **N+1 Query Problems**
- Loops that make database calls
- Missing eager loading
2. **Inefficient Algorithms**
- O(n²) where O(n) or O(n log n) would work
- Unnecessary nested loops
## Step 3: Perform Analysis
**Check for `--deep` flag**: If present, use Multi-Pass Deep Mode with 6
sequential passes. Otherwise, use Fast YAGNI/KISS Mode.
**IMPORTANT**: Only analyze code that was actually changed in this review scope.
Do not flag pre-existing issues.
## Step 3.5: Self-Review Pass
**Before presenting findings, validate each issue:**
1. **Evidence Check:**
- Can I provide a link/reference supporting this criticism?
- Have I explained WHY this matters?
2. **Severity Validation:**
- Is this rating accurate (High/Medium/Low)?
- Would this issue actually cause problems?
3. **YAGNI-Specific Checks:**
- If flagging duplication: Is this the 3rd+ occurrence?
- Can this be refactored later when we have more information?
**Remove or downgrade any issues that fail these checks.**
## Step 4: Interactive Review Process
### Issue Severity Prefixes
Use these prefixes to communicate priority:
| Prefix | Meaning | Action Required |
| ------------- | ---------------------------------- | --------------------- |
| `issue:` | Bug, correctness problem | Must fix before merge |
| `nit:` | Minor improvement, style | Optional, don't block |
| `thought:` | Design consideration | Discuss, may defer |
| `suggestion:` | Specific improvement with code | Consider seriously |
### Interactive Walkthrough
For each issue, present:
```
═══════════════════════════════════════
Issue [current] of [total]
═══════════════════════════════════════
📁 File: [filename]
📍 Lines: [start-end]
🏷️ Type: [YAGNI | KISS | Both]
🎯 Severity: [High | Medium | Low]
CURRENT CODE:
[show actual code snippet]
ISSUE DETECTED: [Specific description]
WHY THIS MATTERS: [Explain the real cost/problem]
SUGGESTED SIMPLIFICATION:
[Show the simpler alternative code]
═══════════════════════════════════════
What would you like to do?
1. ✅ Accept - Add to fix list
2. ❌ Skip - Keep current code
3. 💬 Discuss - Mark for team review
4. 👀 Context - See more surrounding code
5. ⏹️ Stop - End review here
```
## Step 5: Core Review Rules
### ALWAYS Flag These YAGNI Issues:
1. **Interfaces with single implementation**
2. **Unused code** - functions/methods with zero callers
3. **Speculative database fields** - columns always NULL
4. **Premature optimization** - caching before measuring
### ALWAYS Flag These KISS Violations:
1. **Standard library reimplementation**
2. **Excessive abstraction layers**
3. **Configuration over convention** - 100 lines config for 50 lines code
### DON'T Flag These:
1. **Necessary complexity** - error handling, security measures
2. **Domain complexity** - business rules that ARE complex
3. **Team conventions** - agreed-upon patterns
## Step 6: Final Summary
```
📝 PRAGMATIC REVIEW COMPLETE
═══════════════════════════════
Review Statistics:
• Files reviewed: [X]
• Lines changed: [Y]
Issues Found: [Y total]
• Critical (blocking): [count]
• High priority: [count]
• Medium: [count]
• Low: [count]
COMPLEXITY REDUCTION POTENTIAL:
• Lines removable: ~[total] (-X%)
• Unnecessary abstractions: [count]
TOP 3 QUICK WINS:
1. [Biggest impact, easiest change]
2. [Second biggest impact]
3. [Third biggest impact]
RECOMMENDATION: [Clear ship/don't ship with reasoning]
═══════════════════════════════
```
## Command Parameters Reference
- `--auto` : Skip interactive prompts, use defaults (uncommitted changes)
- `--ci` : CI mode - skip ALL prompts, review branch vs base
- `--deep` : Enable 6-pass comprehensive review
- `--branch [name]` : Review specific branch
- `--base [branch]` : Compare against this base branch
Examples:
- `/pragmatic-review` - Interactive mode
- `/pragmatic-review --auto` - Review current changes automatically
- `/pragmatic-review --ci` - CI mode for GitHub Actions
- `/pragmatic-review --deep` - Comprehensive 6-pass review
## Core Philosophy
When in doubt, remember:
1. **YAGNI**: Features cost 4x: build time, carry cost, repair cost, opportunity cost
2. **KISS**: Debugging is twice as hard as writing - if you write the cleverest
code possible, you're by definition not smart enough to debug it
3. **Rule of Three**: Tolerate duplication twice, refactor on the third
4. **Pragmatic**: Ship working software today, perfect it tomorrow
Your role is to be the champion of simplicity. Every line deleted is a victory.
## References
- Martin Fowler - YAGNI: https://martinfowler.com/bliki/Yagni.html
- KISS principle: https://en.wikipedia.org/wiki/KISS_principle
- OWASP Top 10: https://owasp.org/www-project-top-ten/
- Addy Osmani - "Avoid Large Pull Requests"
- Jeff Atwood - "Curly's Law: Do One Thing"
+3 -2
View File
@@ -32,8 +32,9 @@ capabilities.
- Check if already on latest version: - Check if already on latest version:
```bash ```bash
CURRENT=$(grep '"version"' package.json | sed 's/.*: "\(.*\)".*/\1/') # Use cut instead of sed to avoid zsh parentheses escaping issues
LATEST=$(curl -s https://raw.githubusercontent.com/heyitsnoah/claudesidian/main/package.json | grep '"version"' | sed 's/.*: "\(.*\)".*/\1/') CURRENT=$(grep '"version"' package.json | head -1 | cut -d'"' -f4)
LATEST=$(curl -s https://raw.githubusercontent.com/heyitsnoah/claudesidian/main/package.json | grep '"version"' | head -1 | cut -d'"' -f4)
if [ "$CURRENT" = "$LATEST" ]; then if [ "$CURRENT" = "$LATEST" ]; then
echo "✅ You're already on the latest version ($CURRENT)" echo "✅ You're already on the latest version ($CURRENT)"
+48
View File
@@ -0,0 +1,48 @@
#!/bin/bash
# Skill discovery hook - suggests relevant skills when user mentions "skill"
# Exit 0 stdout → added as context for Claude
# No dependencies - pure bash
# Read all stdin
INPUT=$(cat)
# Extract just the prompt field value (pure bash, no jq)
# Input format: {"prompt": "user text here", ...}
# Pattern handles escaped quotes: matches [^\\"] or \\. sequences
# Use -n and /p so non-matching input produces empty output (not original input)
PROMPT=$(echo "$INPUT" | sed -nE 's/.*"prompt"[[:space:]]*:[[:space:]]*"(([^\\"]|\\.)*)".*/\1/p')
# Match: skill, skills (case-insensitive, word boundary) in prompt only
if echo "$PROMPT" | grep -iqE '\bskills?\b'; then
# Get skill names from .claude/skills/ directory
SKILLS_DIR="${CLAUDE_PROJECT_DIR:-.}/.claude/skills"
if [ -d "$SKILLS_DIR" ]; then
# Build list of skills with descriptions
output=""
for d in "$SKILLS_DIR"/*/; do
[ -d "$d" ] || continue
name=$(basename "$d")
skill_file="$d/SKILL.md"
if [ -f "$skill_file" ]; then
# Extract description from YAML frontmatter
desc=$(grep -m1 '^description:' "$skill_file" | sed 's/^description: *//')
output="$output$name: $desc"$'\n'
else
output="$output$name"$'\n'
fi
done
if [ -n "$output" ]; then
echo "<skill-discovery>"
echo "The user mentioned 'skill'. Available skills in this project:"
echo ""
echo "$output" | sort
echo "If relevant to the user's request, read the SKILL.md file to load the skill instructions."
echo "</skill-discovery>"
fi
fi
fi
# Always exit 0 - never block user prompts
exit 0
+28 -7
View File
@@ -8,16 +8,37 @@
"Edit(.obsidian/plugins/quickadd/data.json)", "Edit(.obsidian/plugins/quickadd/data.json)",
"Edit(04_Archive/Projects/Airport/Chengdu/Office Test Env.md)", "Edit(04_Archive/Projects/Airport/Chengdu/Office Test Env.md)",
"Bash(wc:*)", "Bash(wc:*)",
"Bash(cat > .replace_urls.py << 'PYEOF'\n#!/usr/bin/env python3\nimport re\n\n# Read the mapping\nmappings = []\nwith open('.url_mapping.txt', 'r') as f:\n for line in f:\n if '|' in line:\n url, filename = line.strip().split('|', 1)\n mappings.append((url, filename))\n\n# Read the markdown file\nnote_path = \"00_Inbox/Clippings/2026/01/ZeroLuawesome-nanobanana-pro 🚀 An awesome list of curated Nano Banana pro prompts and examples. Your go-to resource for mastering prompt engineering and exploring the creative potential of the Nano banana pro(Nano banana 2) AI image model..md\"\n\nwith open(note_path, 'r', encoding='utf-8') as f:\n content = f.read()\n\n# Perform replacements\nreplacements_made = 0\nfor url, filename in mappings:\n # Escape special regex characters in URL\n escaped_url = re.escape(url)\n # Replace markdown image syntax: ![alt](url) -> ![[path/filename]]\n pattern = r'!\\[([^\\]]*)\\]\\(' + escaped_url + r'\\)'\n replacement = f'![[05_Attachments/nanobanana-pro/{filename}]]'\n new_content, count = re.subn(pattern, replacement, content)\n if count > 0:\n content = new_content\n replacements_made += count\n print(f\"✓ Replaced {count}x: {filename}\")\n\n# Write back\nwith open(note_path, 'w', encoding='utf-8') as f:\n f.write(content)\n\nprint(f\"\\nTotal replacements: {replacements_made}\")\nPYEOF\npython3 .replace_urls.py)",
"Bash(cat > .fix_links.py << 'PYEOF'\n#!/usr/bin/env python3\nimport re\n\nnote_path = \"00_Inbox/Clippings/2026/01/ZeroLuawesome-nanobanana-pro 🚀 An awesome list of curated Nano Banana pro prompts and examples. Your go-to resource for mastering prompt engineering and exploring the creative potential of the Nano banana pro(Nano banana 2) AI image model..md\"\n\nwith open(note_path, 'r', encoding='utf-8') as f:\n content = f.read()\n\n# Pattern: [![[wikilink]]](url) -> ![[wikilink]]\n# Remove the clickable link wrapper around wikilinks\npattern = r'\\[(!\\[\\[05_Attachments/nanobanana-pro/[^\\]]+\\]\\])\\]\\([^)]+\\)'\nreplacement = r'\\1'\nnew_content, count = re.subn(pattern, replacement, content)\n\nprint(f\"Fixed {count} clickable link wrappers\")\n\nwith open(note_path, 'w', encoding='utf-8') as f:\n f.write(new_content)\nPYEOF\npython3 .fix_links.py)",
"Bash(ls -lh 05_Attachments/nanobanana-pro/ | awk '{if ($5 ~ /^[0-9]+$/ && $5 < 1024) print $9, $5}' | head -20)",
"Bash(find:*)", "Bash(find:*)",
"Bash(find 05_Attachments/nanobanana-pro/ -type f \\( -size 0 -o -size -100c \\) | sort > .failed_images.txt && head -10 .failed_images.txt)", "Bash(git add:*)"
"Bash(cat > .fix_conflicts.py << 'EOF'\n#!/usr/bin/env python3\nimport re\n\n# Read the file\nwith open\\('.obsidian/plugins/claudian/main.js', 'r'\\) as f:\n content = f.read\\(\\)\n\n# Fix first conflict \\(line ~2076\\) - use origin/main version \\(simpler path\\)\npattern1 = r'<<<<<<< HEAD\\\\n// node_modules/@modelcontextprotocol/sdk/node_modules/ajv/node_modules/json-schema-traverse/index\\\\.js\\\\nvar require_json_schema_traverse = __commonJS\\\\\\(\\\\{\\\\n \"node_modules/@modelcontextprotocol/sdk/node_modules/ajv/node_modules/json-schema-traverse/index\\\\.js\"\\\\\\(exports, module2\\\\\\) \\\\{\\\\n=======\\\\n// node_modules/@modelcontextprotocol/sdk/node_modules/json-schema-traverse/index\\\\.js\\\\nvar require_json_schema_traverse = __commonJS\\\\\\(\\\\{\\\\n \"node_modules/@modelcontextprotocol/sdk/node_modules/json-schema-traverse/index\\\\.js\"\\\\\\(exports, module2\\\\\\) \\\\{\\\\n>>>>>>> origin/main'\n\nreplacement1 = '''// node_modules/@modelcontextprotocol/sdk/node_modules/json-schema-traverse/index.js\nvar require_json_schema_traverse = __commonJS\\({\n \"node_modules/@modelcontextprotocol/sdk/node_modules/json-schema-traverse/index.js\"\\(exports, module2\\) {'''\n\ncontent = re.sub\\(pattern1, replacement1, content\\)\n\n# Fix second conflict \\(line ~8693\\) - use origin/main version \\(simpler path\\)\npattern2 = r'<<<<<<< HEAD\\\\n// node_modules/ajv-formats/node_modules/ajv/node_modules/json-schema-traverse/index\\\\.js\\\\nvar require_json_schema_traverse2 = __commonJS\\\\\\(\\\\{\\\\n \"node_modules/ajv-formats/node_modules/ajv/node_modules/json-schema-traverse/index\\\\.js\"\\\\\\(exports, module2\\\\\\) \\\\{\\\\n=======\\\\n// node_modules/ajv-formats/node_modules/json-schema-traverse/index\\\\.js\\\\nvar require_json_schema_traverse2 = __commonJS\\\\\\(\\\\{\\\\n \"node_modules/ajv-formats/node_modules/json-schema-traverse/index\\\\.js\"\\\\\\(exports, module2\\\\\\) \\\\{\\\\n>>>>>>> origin/main'\n\nreplacement2 = '''// node_modules/ajv-formats/node_modules/json-schema-traverse/index.js\nvar require_json_schema_traverse2 = __commonJS\\({\n \"node_modules/ajv-formats/node_modules/json-schema-traverse/index.js\"\\(exports, module2\\) {'''\n\ncontent = re.sub\\(pattern2, replacement2, content\\)\n\n# Write back\nwith open\\('.obsidian/plugins/claudian/main.js', 'w'\\) as f:\n f.write\\(content\\)\n\nprint\\(\"✓ Fixed both conflicts in main.js\"\\)\nEOF\npython3 .fix_conflicts.py)",
"Bash(git add:*)",
"Bash(git commit -m \"$\\(cat <<''EOF''\nMerge branch ''origin/main'' into main\n\nResolved conflicts in:\n- .claude/settings.json \\(kept local permissions\\)\n- .obsidian/community-plugins.json \\(merged both plugin lists\\)\n- .obsidian/plugins/obsidian-excalidraw-plugin/manifest.json \\(kept newer version 2.20.2\\)\n- .obsidian/plugins/claudian/main.js \\(used simpler dependency paths\\)\n\nCo-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>\nEOF\n\\)\")"
], ],
"deny": [], "deny": [],
"ask": [] "ask": []
},
"hooks": {
"SessionStart": [
{
"hooks": [
{
"type": "command",
"command": "[ -f FIRST_RUN ] && echo '{\"hookSpecificOutput\":{\"hookEventName\":\"SessionStart\",\"additionalContext\":\"\\n\\n# 🚀 Welcome to Claudesidian!\\n\\n**This appears to be your first time using this vault.**\\n\\n## Quick Start\\n\\nRun the setup wizard:\\n\\n⬇\\n/init-bootstrap\\n⬆\\n\\n## What this will do:\\n\\n✅ Set up your personalized configuration\\n✅ Disconnect from the original repository\\n✅ Help you import any existing Obsidian vault\\n✅ Configure your preferred workflow\\n✅ Create your PARA folder structure\\n\\nThe setup wizard will guide you through everything!\\n\\n\"}}' || true"
},
{
"type": "command",
"command": "npm run check-updates --silent 2>/dev/null || true"
}
]
}
],
"UserPromptSubmit": [
{
"hooks": [
{
"command": "${CLAUDE_PROJECT_DIR}/.claude/hooks/skill-discovery.sh",
"timeout": 5000,
"type": "command"
}
]
}
]
} }
} }
+41
View File
@@ -0,0 +1,41 @@
---
name: defuddle
description: Extract clean markdown content from web pages using Defuddle CLI, removing clutter and navigation to save tokens. Use instead of WebFetch when the user provides a URL to read or analyze, for online documentation, articles, blog posts, or any standard web page.
---
# Defuddle
Use Defuddle CLI to extract clean readable content from web pages. Prefer over WebFetch for standard web pages — it removes navigation, ads, and clutter, reducing token usage.
If not installed: `npm install -g defuddle-cli`
## Usage
Always use `--md` for markdown output:
```bash
defuddle parse <url> --md
```
Save to file:
```bash
defuddle parse <url> --md -o content.md
```
Extract specific metadata:
```bash
defuddle parse <url> -p title
defuddle parse <url> -p description
defuddle parse <url> -p domain
```
## Output formats
| Flag | Format |
|------|--------|
| `--md` | Markdown (default choice) |
| `--json` | JSON with both HTML and markdown |
| (none) | HTML |
| `-p <name>` | Specific metadata property |
+656
View File
@@ -0,0 +1,656 @@
---
name: json-canvas
description: Create and edit JSON Canvas files (.canvas) with nodes, edges, groups, and connections. Use when working with .canvas files, creating visual canvases, mind maps, flowcharts, or when the user mentions Canvas files in Obsidian.
---
# JSON Canvas Skill
This skill enables skills-compatible agents to create and edit valid JSON Canvas files (`.canvas`) used in Obsidian and other applications.
## Overview
JSON Canvas is an open file format for infinite canvas data. Canvas files use the `.canvas` extension and contain valid JSON following the [JSON Canvas Spec 1.0](https://jsoncanvas.org/spec/1.0/).
## File Structure
A canvas file contains two top-level arrays:
```json
{
"nodes": [],
"edges": []
}
```
- `nodes` (optional): Array of node objects
- `edges` (optional): Array of edge objects connecting nodes
## Nodes
Nodes are objects placed on the canvas. There are four node types:
- `text` - Text content with Markdown
- `file` - Reference to files/attachments
- `link` - External URL
- `group` - Visual container for other nodes
### Z-Index Ordering
Nodes are ordered by z-index in the array:
- First node = bottom layer (displayed below others)
- Last node = top layer (displayed above others)
### Generic Node Attributes
All nodes share these attributes:
| Attribute | Required | Type | Description |
|-----------|----------|------|-------------|
| `id` | Yes | string | Unique identifier for the node |
| `type` | Yes | string | Node type: `text`, `file`, `link`, or `group` |
| `x` | Yes | integer | X position in pixels |
| `y` | Yes | integer | Y position in pixels |
| `width` | Yes | integer | Width in pixels |
| `height` | Yes | integer | Height in pixels |
| `color` | No | canvasColor | Node color (see Color section) |
### Text Nodes
Text nodes contain Markdown content.
```json
{
"id": "6f0ad84f44ce9c17",
"type": "text",
"x": 0,
"y": 0,
"width": 400,
"height": 200,
"text": "# Hello World\n\nThis is **Markdown** content."
}
```
#### Newline Escaping (Common Pitfall)
In JSON, newline characters inside strings **must** be represented as `\n`. Do **not** use the literal sequence `\\n` in a `.canvas` file—Obsidian will render it as the characters `\` and `n` instead of a line break.
Examples:
```json
{ "type": "text", "text": "Line 1\nLine 2" }
```
```json
{ "type": "text", "text": "Line 1\\nLine 2" }
```
| Attribute | Required | Type | Description |
|-----------|----------|------|-------------|
| `text` | Yes | string | Plain text with Markdown syntax |
### File Nodes
File nodes reference files or attachments (images, videos, PDFs, notes, etc.).
```json
{
"id": "a1b2c3d4e5f67890",
"type": "file",
"x": 500,
"y": 0,
"width": 400,
"height": 300,
"file": "Attachments/diagram.png"
}
```
```json
{
"id": "b2c3d4e5f6789012",
"type": "file",
"x": 500,
"y": 400,
"width": 400,
"height": 300,
"file": "Notes/Project Overview.md",
"subpath": "#Implementation"
}
```
| Attribute | Required | Type | Description |
|-----------|----------|------|-------------|
| `file` | Yes | string | Path to file within the system |
| `subpath` | No | string | Link to heading or block (starts with `#`) |
### Link Nodes
Link nodes display external URLs.
```json
{
"id": "c3d4e5f678901234",
"type": "link",
"x": 1000,
"y": 0,
"width": 400,
"height": 200,
"url": "https://obsidian.md"
}
```
| Attribute | Required | Type | Description |
|-----------|----------|------|-------------|
| `url` | Yes | string | External URL |
### Group Nodes
Group nodes are visual containers for organizing other nodes.
```json
{
"id": "d4e5f6789012345a",
"type": "group",
"x": -50,
"y": -50,
"width": 1000,
"height": 600,
"label": "Project Overview",
"color": "4"
}
```
```json
{
"id": "e5f67890123456ab",
"type": "group",
"x": 0,
"y": 700,
"width": 800,
"height": 500,
"label": "Resources",
"background": "Attachments/background.png",
"backgroundStyle": "cover"
}
```
| Attribute | Required | Type | Description |
|-----------|----------|------|-------------|
| `label` | No | string | Text label for the group |
| `background` | No | string | Path to background image |
| `backgroundStyle` | No | string | Background rendering style |
#### Background Styles
| Value | Description |
|-------|-------------|
| `cover` | Fills entire width and height of node |
| `ratio` | Maintains aspect ratio of background image |
| `repeat` | Repeats image as pattern in both directions |
## Edges
Edges are lines connecting nodes.
```json
{
"id": "f67890123456789a",
"fromNode": "6f0ad84f44ce9c17",
"toNode": "a1b2c3d4e5f67890"
}
```
```json
{
"id": "0123456789abcdef",
"fromNode": "6f0ad84f44ce9c17",
"fromSide": "right",
"fromEnd": "none",
"toNode": "b2c3d4e5f6789012",
"toSide": "left",
"toEnd": "arrow",
"color": "1",
"label": "leads to"
}
```
| Attribute | Required | Type | Default | Description |
|-----------|----------|------|---------|-------------|
| `id` | Yes | string | - | Unique identifier for the edge |
| `fromNode` | Yes | string | - | Node ID where connection starts |
| `fromSide` | No | string | - | Side where edge starts |
| `fromEnd` | No | string | `none` | Shape at edge start |
| `toNode` | Yes | string | - | Node ID where connection ends |
| `toSide` | No | string | - | Side where edge ends |
| `toEnd` | No | string | `arrow` | Shape at edge end |
| `color` | No | canvasColor | - | Line color |
| `label` | No | string | - | Text label for the edge |
### Side Values
| Value | Description |
|-------|-------------|
| `top` | Top edge of node |
| `right` | Right edge of node |
| `bottom` | Bottom edge of node |
| `left` | Left edge of node |
### End Shapes
| Value | Description |
|-------|-------------|
| `none` | No endpoint shape |
| `arrow` | Arrow endpoint |
## Colors
The `canvasColor` type can be specified in two ways:
### Hex Colors
```json
{
"color": "#FF0000"
}
```
### Preset Colors
```json
{
"color": "1"
}
```
| Preset | Color |
|--------|-------|
| `"1"` | Red |
| `"2"` | Orange |
| `"3"` | Yellow |
| `"4"` | Green |
| `"5"` | Cyan |
| `"6"` | Purple |
Note: Specific color values for presets are intentionally undefined, allowing applications to use their own brand colors.
## Complete Examples
### Simple Canvas with Text and Connections
```json
{
"nodes": [
{
"id": "8a9b0c1d2e3f4a5b",
"type": "text",
"x": 0,
"y": 0,
"width": 300,
"height": 150,
"text": "# Main Idea\n\nThis is the central concept."
},
{
"id": "1a2b3c4d5e6f7a8b",
"type": "text",
"x": 400,
"y": -100,
"width": 250,
"height": 100,
"text": "## Supporting Point A\n\nDetails here."
},
{
"id": "2b3c4d5e6f7a8b9c",
"type": "text",
"x": 400,
"y": 100,
"width": 250,
"height": 100,
"text": "## Supporting Point B\n\nMore details."
}
],
"edges": [
{
"id": "3c4d5e6f7a8b9c0d",
"fromNode": "8a9b0c1d2e3f4a5b",
"fromSide": "right",
"toNode": "1a2b3c4d5e6f7a8b",
"toSide": "left"
},
{
"id": "4d5e6f7a8b9c0d1e",
"fromNode": "8a9b0c1d2e3f4a5b",
"fromSide": "right",
"toNode": "2b3c4d5e6f7a8b9c",
"toSide": "left"
}
]
}
```
### Project Board with Groups
```json
{
"nodes": [
{
"id": "5e6f7a8b9c0d1e2f",
"type": "group",
"x": 0,
"y": 0,
"width": 300,
"height": 500,
"label": "To Do",
"color": "1"
},
{
"id": "6f7a8b9c0d1e2f3a",
"type": "group",
"x": 350,
"y": 0,
"width": 300,
"height": 500,
"label": "In Progress",
"color": "3"
},
{
"id": "7a8b9c0d1e2f3a4b",
"type": "group",
"x": 700,
"y": 0,
"width": 300,
"height": 500,
"label": "Done",
"color": "4"
},
{
"id": "8b9c0d1e2f3a4b5c",
"type": "text",
"x": 20,
"y": 50,
"width": 260,
"height": 80,
"text": "## Task 1\n\nImplement feature X"
},
{
"id": "9c0d1e2f3a4b5c6d",
"type": "text",
"x": 370,
"y": 50,
"width": 260,
"height": 80,
"text": "## Task 2\n\nReview PR #123",
"color": "2"
},
{
"id": "0d1e2f3a4b5c6d7e",
"type": "text",
"x": 720,
"y": 50,
"width": 260,
"height": 80,
"text": "## Task 3\n\n~~Setup CI/CD~~"
}
],
"edges": []
}
```
### Research Canvas with Files and Links
```json
{
"nodes": [
{
"id": "1e2f3a4b5c6d7e8f",
"type": "text",
"x": 300,
"y": 200,
"width": 400,
"height": 200,
"text": "# Research Topic\n\n## Key Questions\n\n- How does X affect Y?\n- What are the implications?",
"color": "5"
},
{
"id": "2f3a4b5c6d7e8f9a",
"type": "file",
"x": 0,
"y": 0,
"width": 250,
"height": 150,
"file": "Literature/Paper A.pdf"
},
{
"id": "3a4b5c6d7e8f9a0b",
"type": "file",
"x": 0,
"y": 200,
"width": 250,
"height": 150,
"file": "Notes/Meeting Notes.md",
"subpath": "#Key Insights"
},
{
"id": "4b5c6d7e8f9a0b1c",
"type": "link",
"x": 0,
"y": 400,
"width": 250,
"height": 100,
"url": "https://example.com/research"
},
{
"id": "5c6d7e8f9a0b1c2d",
"type": "file",
"x": 750,
"y": 150,
"width": 300,
"height": 250,
"file": "Attachments/diagram.png"
}
],
"edges": [
{
"id": "6d7e8f9a0b1c2d3e",
"fromNode": "2f3a4b5c6d7e8f9a",
"fromSide": "right",
"toNode": "1e2f3a4b5c6d7e8f",
"toSide": "left",
"label": "supports"
},
{
"id": "7e8f9a0b1c2d3e4f",
"fromNode": "3a4b5c6d7e8f9a0b",
"fromSide": "right",
"toNode": "1e2f3a4b5c6d7e8f",
"toSide": "left",
"label": "informs"
},
{
"id": "8f9a0b1c2d3e4f5a",
"fromNode": "4b5c6d7e8f9a0b1c",
"fromSide": "right",
"toNode": "1e2f3a4b5c6d7e8f",
"toSide": "left",
"toEnd": "arrow",
"color": "6"
},
{
"id": "9a0b1c2d3e4f5a6b",
"fromNode": "1e2f3a4b5c6d7e8f",
"fromSide": "right",
"toNode": "5c6d7e8f9a0b1c2d",
"toSide": "left",
"label": "visualized by"
}
]
}
```
### Flowchart
```json
{
"nodes": [
{
"id": "a0b1c2d3e4f5a6b7",
"type": "text",
"x": 200,
"y": 0,
"width": 150,
"height": 60,
"text": "**Start**",
"color": "4"
},
{
"id": "b1c2d3e4f5a6b7c8",
"type": "text",
"x": 200,
"y": 100,
"width": 150,
"height": 60,
"text": "Step 1:\nGather data"
},
{
"id": "c2d3e4f5a6b7c8d9",
"type": "text",
"x": 200,
"y": 200,
"width": 150,
"height": 80,
"text": "**Decision**\n\nIs data valid?",
"color": "3"
},
{
"id": "d3e4f5a6b7c8d9e0",
"type": "text",
"x": 400,
"y": 200,
"width": 150,
"height": 60,
"text": "Process data"
},
{
"id": "e4f5a6b7c8d9e0f1",
"type": "text",
"x": 0,
"y": 200,
"width": 150,
"height": 60,
"text": "Request new data",
"color": "1"
},
{
"id": "f5a6b7c8d9e0f1a2",
"type": "text",
"x": 400,
"y": 320,
"width": 150,
"height": 60,
"text": "**End**",
"color": "4"
}
],
"edges": [
{
"id": "a6b7c8d9e0f1a2b3",
"fromNode": "a0b1c2d3e4f5a6b7",
"fromSide": "bottom",
"toNode": "b1c2d3e4f5a6b7c8",
"toSide": "top"
},
{
"id": "b7c8d9e0f1a2b3c4",
"fromNode": "b1c2d3e4f5a6b7c8",
"fromSide": "bottom",
"toNode": "c2d3e4f5a6b7c8d9",
"toSide": "top"
},
{
"id": "c8d9e0f1a2b3c4d5",
"fromNode": "c2d3e4f5a6b7c8d9",
"fromSide": "right",
"toNode": "d3e4f5a6b7c8d9e0",
"toSide": "left",
"label": "Yes",
"color": "4"
},
{
"id": "d9e0f1a2b3c4d5e6",
"fromNode": "c2d3e4f5a6b7c8d9",
"fromSide": "left",
"toNode": "e4f5a6b7c8d9e0f1",
"toSide": "right",
"label": "No",
"color": "1"
},
{
"id": "e0f1a2b3c4d5e6f7",
"fromNode": "e4f5a6b7c8d9e0f1",
"fromSide": "top",
"fromEnd": "none",
"toNode": "b1c2d3e4f5a6b7c8",
"toSide": "left",
"toEnd": "arrow"
},
{
"id": "f1a2b3c4d5e6f7a8",
"fromNode": "d3e4f5a6b7c8d9e0",
"fromSide": "bottom",
"toNode": "f5a6b7c8d9e0f1a2",
"toSide": "top"
}
]
}
```
## ID Generation
Node and edge IDs must be unique strings. Obsidian generates 16-character hexadecimal IDs:
```json
"id": "6f0ad84f44ce9c17"
"id": "a3b2c1d0e9f8g7h6"
"id": "1234567890abcdef"
```
This format is a 16-character lowercase hex string (64-bit random value).
## Layout Guidelines
### Positioning
- Coordinates can be negative (canvas extends infinitely)
- `x` increases to the right
- `y` increases downward
- Position refers to top-left corner of node
### Recommended Sizes
| Node Type | Suggested Width | Suggested Height |
|-----------|-----------------|------------------|
| Small text | 200-300 | 80-150 |
| Medium text | 300-450 | 150-300 |
| Large text | 400-600 | 300-500 |
| File preview | 300-500 | 200-400 |
| Link preview | 250-400 | 100-200 |
| Group | Varies | Varies |
### Spacing
- Leave 20-50px padding inside groups
- Space nodes 50-100px apart for readability
- Align nodes to grid (multiples of 10 or 20) for cleaner layouts
## Validation Rules
1. All `id` values must be unique across nodes and edges
2. `fromNode` and `toNode` must reference existing node IDs
3. Required fields must be present for each node type
4. `type` must be one of: `text`, `file`, `link`, `group`
5. `backgroundStyle` must be one of: `cover`, `ratio`, `repeat`
6. `fromSide`, `toSide` must be one of: `top`, `right`, `bottom`, `left`
7. `fromEnd`, `toEnd` must be one of: `none`, `arrow`
8. Color presets must be `"1"` through `"6"` or valid hex color
## References
- [JSON Canvas Spec 1.0](https://jsoncanvas.org/spec/1.0/)
- [JSON Canvas GitHub](https://github.com/obsidianmd/jsoncanvas)
+651
View File
@@ -0,0 +1,651 @@
---
name: obsidian-bases
description: Create and edit Obsidian Bases (.base files) with views, filters, formulas, and summaries. Use when working with .base files, creating database-like views of notes, or when the user mentions Bases, table views, card views, filters, or formulas in Obsidian.
---
# Obsidian Bases Skill
This skill enables skills-compatible agents to create and edit valid Obsidian Bases (`.base` files) including views, filters, formulas, and all related configurations.
## Overview
Obsidian Bases are YAML-based files that define dynamic views of notes in an Obsidian vault. A Base file can contain multiple views, global filters, formulas, property configurations, and custom summaries.
## File Format
Base files use the `.base` extension and contain valid YAML. They can also be embedded in Markdown code blocks.
## Complete Schema
```yaml
# Global filters apply to ALL views in the base
filters:
# Can be a single filter string
# OR a recursive filter object with and/or/not
and: []
or: []
not: []
# Define formula properties that can be used across all views
formulas:
formula_name: 'expression'
# Configure display names and settings for properties
properties:
property_name:
displayName: "Display Name"
formula.formula_name:
displayName: "Formula Display Name"
file.ext:
displayName: "Extension"
# Define custom summary formulas
summaries:
custom_summary_name: 'values.mean().round(3)'
# Define one or more views
views:
- type: table | cards | list | map
name: "View Name"
limit: 10 # Optional: limit results
groupBy: # Optional: group results
property: property_name
direction: ASC | DESC
filters: # View-specific filters
and: []
order: # Properties to display in order
- file.name
- property_name
- formula.formula_name
summaries: # Map properties to summary formulas
property_name: Average
```
## Filter Syntax
Filters narrow down results. They can be applied globally or per-view.
### Filter Structure
```yaml
# Single filter
filters: 'status == "done"'
# AND - all conditions must be true
filters:
and:
- 'status == "done"'
- 'priority > 3'
# OR - any condition can be true
filters:
or:
- 'file.hasTag("book")'
- 'file.hasTag("article")'
# NOT - exclude matching items
filters:
not:
- 'file.hasTag("archived")'
# Nested filters
filters:
or:
- file.hasTag("tag")
- and:
- file.hasTag("book")
- file.hasLink("Textbook")
- not:
- file.hasTag("book")
- file.inFolder("Required Reading")
```
### Filter Operators
| Operator | Description |
|----------|-------------|
| `==` | equals |
| `!=` | not equal |
| `>` | greater than |
| `<` | less than |
| `>=` | greater than or equal |
| `<=` | less than or equal |
| `&&` | logical and |
| `\|\|` | logical or |
| <code>!</code> | logical not |
## Properties
### Three Types of Properties
1. **Note properties** - From frontmatter: `note.author` or just `author`
2. **File properties** - File metadata: `file.name`, `file.mtime`, etc.
3. **Formula properties** - Computed values: `formula.my_formula`
### File Properties Reference
| Property | Type | Description |
|----------|------|-------------|
| `file.name` | String | File name |
| `file.basename` | String | File name without extension |
| `file.path` | String | Full path to file |
| `file.folder` | String | Parent folder path |
| `file.ext` | String | File extension |
| `file.size` | Number | File size in bytes |
| `file.ctime` | Date | Created time |
| `file.mtime` | Date | Modified time |
| `file.tags` | List | All tags in file |
| `file.links` | List | Internal links in file |
| `file.backlinks` | List | Files linking to this file |
| `file.embeds` | List | Embeds in the note |
| `file.properties` | Object | All frontmatter properties |
### The `this` Keyword
- In main content area: refers to the base file itself
- When embedded: refers to the embedding file
- In sidebar: refers to the active file in main content
## Formula Syntax
Formulas compute values from properties. Defined in the `formulas` section.
```yaml
formulas:
# Simple arithmetic
total: "price * quantity"
# Conditional logic
status_icon: 'if(done, "✅", "⏳")'
# String formatting
formatted_price: 'if(price, price.toFixed(2) + " dollars")'
# Date formatting
created: 'file.ctime.format("YYYY-MM-DD")'
# Calculate days since created (use .days for Duration)
days_old: '(now() - file.ctime).days'
# Calculate days until due date
days_until_due: 'if(due_date, (date(due_date) - today()).days, "")'
```
## Functions Reference
### Global Functions
| Function | Signature | Description |
|----------|-----------|-------------|
| `date()` | `date(string): date` | Parse string to date. Format: `YYYY-MM-DD HH:mm:ss` |
| `duration()` | `duration(string): duration` | Parse duration string |
| `now()` | `now(): date` | Current date and time |
| `today()` | `today(): date` | Current date (time = 00:00:00) |
| `if()` | `if(condition, trueResult, falseResult?)` | Conditional |
| `min()` | `min(n1, n2, ...): number` | Smallest number |
| `max()` | `max(n1, n2, ...): number` | Largest number |
| `number()` | `number(any): number` | Convert to number |
| `link()` | `link(path, display?): Link` | Create a link |
| `list()` | `list(element): List` | Wrap in list if not already |
| `file()` | `file(path): file` | Get file object |
| `image()` | `image(path): image` | Create image for rendering |
| `icon()` | `icon(name): icon` | Lucide icon by name |
| `html()` | `html(string): html` | Render as HTML |
| `escapeHTML()` | `escapeHTML(string): string` | Escape HTML characters |
### Any Type Functions
| Function | Signature | Description |
|----------|-----------|-------------|
| `isTruthy()` | `any.isTruthy(): boolean` | Coerce to boolean |
| `isType()` | `any.isType(type): boolean` | Check type |
| `toString()` | `any.toString(): string` | Convert to string |
### Date Functions & Fields
**Fields:** `date.year`, `date.month`, `date.day`, `date.hour`, `date.minute`, `date.second`, `date.millisecond`
| Function | Signature | Description |
|----------|-----------|-------------|
| `date()` | `date.date(): date` | Remove time portion |
| `format()` | `date.format(string): string` | Format with Moment.js pattern |
| `time()` | `date.time(): string` | Get time as string |
| `relative()` | `date.relative(): string` | Human-readable relative time |
| `isEmpty()` | `date.isEmpty(): boolean` | Always false for dates |
### Duration Type
When subtracting two dates, the result is a **Duration** type (not a number). Duration has its own properties and methods.
**Duration Fields:**
| Field | Type | Description |
|-------|------|-------------|
| `duration.days` | Number | Total days in duration |
| `duration.hours` | Number | Total hours in duration |
| `duration.minutes` | Number | Total minutes in duration |
| `duration.seconds` | Number | Total seconds in duration |
| `duration.milliseconds` | Number | Total milliseconds in duration |
**IMPORTANT:** Duration does NOT support `.round()`, `.floor()`, `.ceil()` directly. You must access a numeric field first (like `.days`), then apply number functions.
```yaml
# CORRECT: Calculate days between dates
"(date(due_date) - today()).days" # Returns number of days
"(now() - file.ctime).days" # Days since created
# CORRECT: Round the numeric result if needed
"(date(due_date) - today()).days.round(0)" # Rounded days
"(now() - file.ctime).hours.round(0)" # Rounded hours
# WRONG - will cause error:
# "((date(due) - today()) / 86400000).round(0)" # Duration doesn't support division then round
```
### Date Arithmetic
```yaml
# Duration units: y/year/years, M/month/months, d/day/days,
# w/week/weeks, h/hour/hours, m/minute/minutes, s/second/seconds
# Add/subtract durations
"date + \"1M\"" # Add 1 month
"date - \"2h\"" # Subtract 2 hours
"now() + \"1 day\"" # Tomorrow
"today() + \"7d\"" # A week from today
# Subtract dates returns Duration type
"now() - file.ctime" # Returns Duration
"(now() - file.ctime).days" # Get days as number
"(now() - file.ctime).hours" # Get hours as number
# Complex duration arithmetic
"now() + (duration('1d') * 2)"
```
### String Functions
**Field:** `string.length`
| Function | Signature | Description |
|----------|-----------|-------------|
| `contains()` | `string.contains(value): boolean` | Check substring |
| `containsAll()` | `string.containsAll(...values): boolean` | All substrings present |
| `containsAny()` | `string.containsAny(...values): boolean` | Any substring present |
| `startsWith()` | `string.startsWith(query): boolean` | Starts with query |
| `endsWith()` | `string.endsWith(query): boolean` | Ends with query |
| `isEmpty()` | `string.isEmpty(): boolean` | Empty or not present |
| `lower()` | `string.lower(): string` | To lowercase |
| `title()` | `string.title(): string` | To Title Case |
| `trim()` | `string.trim(): string` | Remove whitespace |
| `replace()` | `string.replace(pattern, replacement): string` | Replace pattern |
| `repeat()` | `string.repeat(count): string` | Repeat string |
| `reverse()` | `string.reverse(): string` | Reverse string |
| `slice()` | `string.slice(start, end?): string` | Substring |
| `split()` | `string.split(separator, n?): list` | Split to list |
### Number Functions
| Function | Signature | Description |
|----------|-----------|-------------|
| `abs()` | `number.abs(): number` | Absolute value |
| `ceil()` | `number.ceil(): number` | Round up |
| `floor()` | `number.floor(): number` | Round down |
| `round()` | `number.round(digits?): number` | Round to digits |
| `toFixed()` | `number.toFixed(precision): string` | Fixed-point notation |
| `isEmpty()` | `number.isEmpty(): boolean` | Not present |
### List Functions
**Field:** `list.length`
| Function | Signature | Description |
|----------|-----------|-------------|
| `contains()` | `list.contains(value): boolean` | Element exists |
| `containsAll()` | `list.containsAll(...values): boolean` | All elements exist |
| `containsAny()` | `list.containsAny(...values): boolean` | Any element exists |
| `filter()` | `list.filter(expression): list` | Filter by condition (uses `value`, `index`) |
| `map()` | `list.map(expression): list` | Transform elements (uses `value`, `index`) |
| `reduce()` | `list.reduce(expression, initial): any` | Reduce to single value (uses `value`, `index`, `acc`) |
| `flat()` | `list.flat(): list` | Flatten nested lists |
| `join()` | `list.join(separator): string` | Join to string |
| `reverse()` | `list.reverse(): list` | Reverse order |
| `slice()` | `list.slice(start, end?): list` | Sublist |
| `sort()` | `list.sort(): list` | Sort ascending |
| `unique()` | `list.unique(): list` | Remove duplicates |
| `isEmpty()` | `list.isEmpty(): boolean` | No elements |
### File Functions
| Function | Signature | Description |
|----------|-----------|-------------|
| `asLink()` | `file.asLink(display?): Link` | Convert to link |
| `hasLink()` | `file.hasLink(otherFile): boolean` | Has link to file |
| `hasTag()` | `file.hasTag(...tags): boolean` | Has any of the tags |
| `hasProperty()` | `file.hasProperty(name): boolean` | Has property |
| `inFolder()` | `file.inFolder(folder): boolean` | In folder or subfolder |
### Link Functions
| Function | Signature | Description |
|----------|-----------|-------------|
| `asFile()` | `link.asFile(): file` | Get file object |
| `linksTo()` | `link.linksTo(file): boolean` | Links to file |
### Object Functions
| Function | Signature | Description |
|----------|-----------|-------------|
| `isEmpty()` | `object.isEmpty(): boolean` | No properties |
| `keys()` | `object.keys(): list` | List of keys |
| `values()` | `object.values(): list` | List of values |
### Regular Expression Functions
| Function | Signature | Description |
|----------|-----------|-------------|
| `matches()` | `regexp.matches(string): boolean` | Test if matches |
## View Types
### Table View
```yaml
views:
- type: table
name: "My Table"
order:
- file.name
- status
- due_date
summaries:
price: Sum
count: Average
```
### Cards View
```yaml
views:
- type: cards
name: "Gallery"
order:
- file.name
- cover_image
- description
```
### List View
```yaml
views:
- type: list
name: "Simple List"
order:
- file.name
- status
```
### Map View
Requires latitude/longitude properties and the Maps community plugin.
```yaml
views:
- type: map
name: "Locations"
# Map-specific settings for lat/lng properties
```
## Default Summary Formulas
| Name | Input Type | Description |
|------|------------|-------------|
| `Average` | Number | Mathematical mean |
| `Min` | Number | Smallest number |
| `Max` | Number | Largest number |
| `Sum` | Number | Sum of all numbers |
| `Range` | Number | Max - Min |
| `Median` | Number | Mathematical median |
| `Stddev` | Number | Standard deviation |
| `Earliest` | Date | Earliest date |
| `Latest` | Date | Latest date |
| `Range` | Date | Latest - Earliest |
| `Checked` | Boolean | Count of true values |
| `Unchecked` | Boolean | Count of false values |
| `Empty` | Any | Count of empty values |
| `Filled` | Any | Count of non-empty values |
| `Unique` | Any | Count of unique values |
## Complete Examples
### Task Tracker Base
```yaml
filters:
and:
- file.hasTag("task")
- 'file.ext == "md"'
formulas:
days_until_due: 'if(due, (date(due) - today()).days, "")'
is_overdue: 'if(due, date(due) < today() && status != "done", false)'
priority_label: 'if(priority == 1, "🔴 High", if(priority == 2, "🟡 Medium", "🟢 Low"))'
properties:
status:
displayName: Status
formula.days_until_due:
displayName: "Days Until Due"
formula.priority_label:
displayName: Priority
views:
- type: table
name: "Active Tasks"
filters:
and:
- 'status != "done"'
order:
- file.name
- status
- formula.priority_label
- due
- formula.days_until_due
groupBy:
property: status
direction: ASC
summaries:
formula.days_until_due: Average
- type: table
name: "Completed"
filters:
and:
- 'status == "done"'
order:
- file.name
- completed_date
```
### Reading List Base
```yaml
filters:
or:
- file.hasTag("book")
- file.hasTag("article")
formulas:
reading_time: 'if(pages, (pages * 2).toString() + " min", "")'
status_icon: 'if(status == "reading", "📖", if(status == "done", "✅", "📚"))'
year_read: 'if(finished_date, date(finished_date).year, "")'
properties:
author:
displayName: Author
formula.status_icon:
displayName: ""
formula.reading_time:
displayName: "Est. Time"
views:
- type: cards
name: "Library"
order:
- cover
- file.name
- author
- formula.status_icon
filters:
not:
- 'status == "dropped"'
- type: table
name: "Reading List"
filters:
and:
- 'status == "to-read"'
order:
- file.name
- author
- pages
- formula.reading_time
```
### Project Notes Base
```yaml
filters:
and:
- file.inFolder("Projects")
- 'file.ext == "md"'
formulas:
last_updated: 'file.mtime.relative()'
link_count: 'file.links.length'
summaries:
avgLinks: 'values.filter(value.isType("number")).mean().round(1)'
properties:
formula.last_updated:
displayName: "Updated"
formula.link_count:
displayName: "Links"
views:
- type: table
name: "All Projects"
order:
- file.name
- status
- formula.last_updated
- formula.link_count
summaries:
formula.link_count: avgLinks
groupBy:
property: status
direction: ASC
- type: list
name: "Quick List"
order:
- file.name
- status
```
### Daily Notes Index
```yaml
filters:
and:
- file.inFolder("Daily Notes")
- '/^\d{4}-\d{2}-\d{2}$/.matches(file.basename)'
formulas:
word_estimate: '(file.size / 5).round(0)'
day_of_week: 'date(file.basename).format("dddd")'
properties:
formula.day_of_week:
displayName: "Day"
formula.word_estimate:
displayName: "~Words"
views:
- type: table
name: "Recent Notes"
limit: 30
order:
- file.name
- formula.day_of_week
- formula.word_estimate
- file.mtime
```
## Embedding Bases
Embed in Markdown files:
```markdown
![[MyBase.base]]
<!-- Specific view -->
![[MyBase.base#View Name]]
```
## YAML Quoting Rules
- Use single quotes for formulas containing double quotes: `'if(done, "Yes", "No")'`
- Use double quotes for simple strings: `"My View Name"`
- Escape nested quotes properly in complex expressions
## Common Patterns
### Filter by Tag
```yaml
filters:
and:
- file.hasTag("project")
```
### Filter by Folder
```yaml
filters:
and:
- file.inFolder("Notes")
```
### Filter by Date Range
```yaml
filters:
and:
- 'file.mtime > now() - "7d"'
```
### Filter by Property Value
```yaml
filters:
and:
- 'status == "active"'
- 'priority >= 3'
```
### Combine Multiple Conditions
```yaml
filters:
or:
- and:
- file.hasTag("important")
- 'status != "done"'
- and:
- 'priority == 1'
- 'due != ""'
```
## References
- [Bases Syntax](https://help.obsidian.md/bases/syntax)
- [Functions](https://help.obsidian.md/bases/functions)
- [Views](https://help.obsidian.md/bases/views)
- [Formulas](https://help.obsidian.md/formulas)
+103
View File
@@ -0,0 +1,103 @@
---
name: obsidian-cli
description: Interact with Obsidian vaults using the Obsidian CLI to read, create, search, and manage notes, tasks, properties, and more. Also supports plugin and theme development with commands to reload plugins, run JavaScript, capture errors, take screenshots, and inspect the DOM. Use when the user asks to interact with their Obsidian vault, manage notes, search vault content, perform vault operations from the command line, or develop and debug Obsidian plugins and themes.
---
# Obsidian CLI
Use the `obsidian` CLI to interact with a running Obsidian instance. Requires Obsidian to be open.
## Command reference
Run `obsidian help` to see all available commands. This is always up to date. Full docs: https://help.obsidian.md/cli
## Syntax
**Parameters** take a value with `=`. Quote values with spaces:
```bash
obsidian create name="My Note" content="Hello world"
```
**Flags** are boolean switches with no value:
```bash
obsidian create name="My Note" silent overwrite
```
For multiline content use `\n` for newline and `\t` for tab.
## File targeting
Many commands accept `file` or `path` to target a file. Without either, the active file is used.
- `file=<name>` — resolves like a wikilink (name only, no path or extension needed)
- `path=<path>` — exact path from vault root, e.g. `folder/note.md`
## Vault targeting
Commands target the most recently focused vault by default. Use `vault=<name>` as the first parameter to target a specific vault:
```bash
obsidian vault="My Vault" search query="test"
```
## Common patterns
```bash
obsidian read file="My Note"
obsidian create name="New Note" content="# Hello" template="Template" silent
obsidian append file="My Note" content="New line"
obsidian search query="search term" limit=10
obsidian daily:read
obsidian daily:append content="- [ ] New task"
obsidian property:set name="status" value="done" file="My Note"
obsidian tasks daily todo
obsidian tags sort=count counts
obsidian backlinks file="My Note"
```
Use `--copy` on any command to copy output to clipboard. Use `silent` to prevent files from opening. Use `total` on list commands to get a count.
## Plugin development
Reload a plugin after code changes — essential for the develop/test cycle:
```bash
obsidian plugin:reload id=my-plugin
```
Run JavaScript in the app context:
```bash
obsidian eval code="app.vault.getFiles().length"
```
Check for errors and console output:
```bash
obsidian dev:errors
obsidian dev:console
obsidian dev:console level=error
```
Take a screenshot for visual testing:
```bash
obsidian dev:screenshot path=screenshot.png
```
Inspect DOM and CSS:
```bash
obsidian dev:dom selector=".workspace-leaf" text
obsidian dev:css selector=".workspace-leaf" prop=background-color
```
Toggle mobile emulation:
```bash
obsidian dev:mobile on
```
Run `obsidian help` to see additional developer commands including CDP and debugger controls.
+620
View File
@@ -0,0 +1,620 @@
---
name: obsidian-markdown
description: Create and edit Obsidian Flavored Markdown with wikilinks, embeds, callouts, properties, and other Obsidian-specific syntax. Use when working with .md files in Obsidian, or when the user mentions wikilinks, callouts, frontmatter, tags, embeds, or Obsidian notes.
---
# Obsidian Flavored Markdown Skill
This skill enables skills-compatible agents to create and edit valid Obsidian Flavored Markdown, including all Obsidian-specific syntax extensions.
## Overview
Obsidian uses a combination of Markdown flavors:
- [CommonMark](https://commonmark.org/)
- [GitHub Flavored Markdown](https://github.github.com/gfm/)
- [LaTeX](https://www.latex-project.org/) for math
- Obsidian-specific extensions (wikilinks, callouts, embeds, etc.)
## Basic Formatting
### Paragraphs and Line Breaks
```markdown
This is a paragraph.
This is another paragraph (blank line between creates separate paragraphs).
For a line break within a paragraph, add two spaces at the end
or use Shift+Enter.
```
### Headings
```markdown
# Heading 1
## Heading 2
### Heading 3
#### Heading 4
##### Heading 5
###### Heading 6
```
### Text Formatting
| Style | Syntax | Example | Output |
|-------|--------|---------|--------|
| Bold | `**text**` or `__text__` | `**Bold**` | **Bold** |
| Italic | `*text*` or `_text_` | `*Italic*` | *Italic* |
| Bold + Italic | `***text***` | `***Both***` | ***Both*** |
| Strikethrough | `~~text~~` | `~~Striked~~` | ~~Striked~~ |
| Highlight | `==text==` | `==Highlighted==` | ==Highlighted== |
| Inline code | `` `code` `` | `` `code` `` | `code` |
### Escaping Formatting
Use backslash to escape special characters:
```markdown
\*This won't be italic\*
\#This won't be a heading
1\. This won't be a list item
```
Common characters to escape: `\*`, `\_`, `\#`, `` \` ``, `\|`, `\~`
## Internal Links (Wikilinks)
### Basic Links
```markdown
[[Note Name]]
[[Note Name.md]]
[[Note Name|Display Text]]
```
### Link to Headings
```markdown
[[Note Name#Heading]]
[[Note Name#Heading|Custom Text]]
[[#Heading in same note]]
[[##Search all headings in vault]]
```
### Link to Blocks
```markdown
[[Note Name#^block-id]]
[[Note Name#^block-id|Custom Text]]
```
Define a block ID by adding `^block-id` at the end of a paragraph:
```markdown
This is a paragraph that can be linked to. ^my-block-id
```
For lists and quotes, add the block ID on a separate line:
```markdown
> This is a quote
> With multiple lines
^quote-id
```
### Search Links
```markdown
[[##heading]] Search for headings containing "heading"
[[^^block]] Search for blocks containing "block"
```
## Markdown-Style Links
```markdown
[Display Text](Note%20Name.md)
[Display Text](Note%20Name.md#Heading)
[Display Text](https://example.com)
[Note](obsidian://open?vault=VaultName&file=Note.md)
```
Note: Spaces must be URL-encoded as `%20` in Markdown links.
## Embeds
### Embed Notes
```markdown
![[Note Name]]
![[Note Name#Heading]]
![[Note Name#^block-id]]
```
### Embed Images
```markdown
![[image.png]]
![[image.png|640x480]] Width x Height
![[image.png|300]] Width only (maintains aspect ratio)
```
### External Images
```markdown
![Alt text](https://example.com/image.png)
![Alt text|300](https://example.com/image.png)
```
### Embed Audio
```markdown
![[audio.mp3]]
![[audio.ogg]]
```
### Embed PDF
```markdown
![[document.pdf]]
![[document.pdf#page=3]]
![[document.pdf#height=400]]
```
### Embed Lists
```markdown
![[Note#^list-id]]
```
Where the list has been defined with a block ID:
```markdown
- Item 1
- Item 2
- Item 3
^list-id
```
### Embed Search Results
````markdown
```query
tag:#project status:done
```
````
## Callouts
### Basic Callout
```markdown
> [!note]
> This is a note callout.
> [!info] Custom Title
> This callout has a custom title.
> [!tip] Title Only
```
### Foldable Callouts
```markdown
> [!faq]- Collapsed by default
> This content is hidden until expanded.
> [!faq]+ Expanded by default
> This content is visible but can be collapsed.
```
### Nested Callouts
```markdown
> [!question] Outer callout
> > [!note] Inner callout
> > Nested content
```
### Supported Callout Types
| Type | Aliases | Description |
|------|---------|-------------|
| `note` | - | Blue, pencil icon |
| `abstract` | `summary`, `tldr` | Teal, clipboard icon |
| `info` | - | Blue, info icon |
| `todo` | - | Blue, checkbox icon |
| `tip` | `hint`, `important` | Cyan, flame icon |
| `success` | `check`, `done` | Green, checkmark icon |
| `question` | `help`, `faq` | Yellow, question mark |
| `warning` | `caution`, `attention` | Orange, warning icon |
| `failure` | `fail`, `missing` | Red, X icon |
| `danger` | `error` | Red, zap icon |
| `bug` | - | Red, bug icon |
| `example` | - | Purple, list icon |
| `quote` | `cite` | Gray, quote icon |
### Custom Callouts (CSS)
```css
.callout[data-callout="custom-type"] {
--callout-color: 255, 0, 0;
--callout-icon: lucide-alert-circle;
}
```
## Lists
### Unordered Lists
```markdown
- Item 1
- Item 2
- Nested item
- Another nested
- Item 3
* Also works with asterisks
+ Or plus signs
```
### Ordered Lists
```markdown
1. First item
2. Second item
1. Nested numbered
2. Another nested
3. Third item
1) Alternative syntax
2) With parentheses
```
### Task Lists
```markdown
- [ ] Incomplete task
- [x] Completed task
- [ ] Task with sub-tasks
- [ ] Subtask 1
- [x] Subtask 2
```
## Quotes
```markdown
> This is a blockquote.
> It can span multiple lines.
>
> And include multiple paragraphs.
>
> > Nested quotes work too.
```
## Code
### Inline Code
```markdown
Use `backticks` for inline code.
Use double backticks for ``code with a ` backtick inside``.
```
### Code Blocks
````markdown
```
Plain code block
```
```javascript
// Syntax highlighted code block
function hello() {
console.log("Hello, world!");
}
```
```python
# Python example
def greet(name):
print(f"Hello, {name}!")
```
````
### Nesting Code Blocks
Use more backticks or tildes for the outer block:
`````markdown
````markdown
Here's how to create a code block:
```js
console.log("Hello")
```
````
`````
## Tables
```markdown
| Header 1 | Header 2 | Header 3 |
|----------|----------|----------|
| Cell 1 | Cell 2 | Cell 3 |
| Cell 4 | Cell 5 | Cell 6 |
```
### Alignment
```markdown
| Left | Center | Right |
|:---------|:--------:|---------:|
| Left | Center | Right |
```
### Using Pipes in Tables
Escape pipes with backslash:
```markdown
| Column 1 | Column 2 |
|----------|----------|
| [[Link\|Display]] | ![[Image\|100]] |
```
## Math (LaTeX)
### Inline Math
```markdown
This is inline math: $e^{i\pi} + 1 = 0$
```
### Block Math
```markdown
$$
\begin{vmatrix}
a & b \\
c & d
\end{vmatrix} = ad - bc
$$
```
### Common Math Syntax
```markdown
$x^2$ Superscript
$x_i$ Subscript
$\frac{a}{b}$ Fraction
$\sqrt{x}$ Square root
$\sum_{i=1}^{n}$ Summation
$\int_a^b$ Integral
$\alpha, \beta$ Greek letters
```
## Diagrams (Mermaid)
````markdown
```mermaid
graph TD
A[Start] --> B{Decision}
B -->|Yes| C[Do this]
B -->|No| D[Do that]
C --> E[End]
D --> E
```
````
### Sequence Diagrams
````markdown
```mermaid
sequenceDiagram
Alice->>Bob: Hello Bob
Bob-->>Alice: Hi Alice
```
````
### Linking in Diagrams
````markdown
```mermaid
graph TD
A[Biology]
B[Chemistry]
A --> B
class A,B internal-link;
```
````
## Footnotes
```markdown
This sentence has a footnote[^1].
[^1]: This is the footnote content.
You can also use named footnotes[^note].
[^note]: Named footnotes still appear as numbers.
Inline footnotes are also supported.^[This is an inline footnote.]
```
## Comments
```markdown
This is visible %%but this is hidden%% text.
%%
This entire block is hidden.
It won't appear in reading view.
%%
```
## Horizontal Rules
```markdown
---
***
___
- - -
* * *
```
## Properties (Frontmatter)
Properties use YAML frontmatter at the start of a note:
```yaml
---
title: My Note Title
date: 2024-01-15
tags:
- project
- important
aliases:
- My Note
- Alternative Name
cssclasses:
- custom-class
status: in-progress
rating: 4.5
completed: false
due: 2024-02-01T14:30:00
---
```
### Property Types
| Type | Example |
|------|---------|
| Text | `title: My Title` |
| Number | `rating: 4.5` |
| Checkbox | `completed: true` |
| Date | `date: 2024-01-15` |
| Date & Time | `due: 2024-01-15T14:30:00` |
| List | `tags: [one, two]` or YAML list |
| Links | `related: "[[Other Note]]"` |
### Default Properties
- `tags` - Note tags
- `aliases` - Alternative names for the note
- `cssclasses` - CSS classes applied to the note
## Tags
```markdown
#tag
#nested/tag
#tag-with-dashes
#tag_with_underscores
In frontmatter:
---
tags:
- tag1
- nested/tag2
---
```
Tags can contain:
- Letters (any language)
- Numbers (not as first character)
- Underscores `_`
- Hyphens `-`
- Forward slashes `/` (for nesting)
## HTML Content
Obsidian supports HTML within Markdown:
```markdown
<div class="custom-container">
<span style="color: red;">Colored text</span>
</div>
<details>
<summary>Click to expand</summary>
Hidden content here.
</details>
<kbd>Ctrl</kbd> + <kbd>C</kbd>
```
## Complete Example
````markdown
---
title: Project Alpha
date: 2024-01-15
tags:
- project
- active
status: in-progress
priority: high
---
# Project Alpha
## Overview
This project aims to [[improve workflow]] using modern techniques.
> [!important] Key Deadline
> The first milestone is due on ==January 30th==.
## Tasks
- [x] Initial planning
- [x] Resource allocation
- [ ] Development phase
- [ ] Backend implementation
- [ ] Frontend design
- [ ] Testing
- [ ] Deployment
## Technical Notes
The main algorithm uses the formula $O(n \log n)$ for sorting.
```python
def process_data(items):
return sorted(items, key=lambda x: x.priority)
```
## Architecture
```mermaid
graph LR
A[Input] --> B[Process]
B --> C[Output]
B --> D[Cache]
```
## Related Documents
- ![[Meeting Notes 2024-01-10#Decisions]]
- [[Budget Allocation|Budget]]
- [[Team Members]]
## References
For more details, see the official documentation[^1].
[^1]: https://example.com/docs
%%
Internal notes:
- Review with team on Friday
- Consider alternative approaches
%%
````
## References
- [Basic formatting syntax](https://help.obsidian.md/syntax)
- [Advanced formatting syntax](https://help.obsidian.md/advanced-syntax)
- [Obsidian Flavored Markdown](https://help.obsidian.md/obsidian-flavored-markdown)
- [Internal links](https://help.obsidian.md/links)
- [Embed files](https://help.obsidian.md/embeds)
- [Callouts](https://help.obsidian.md/callouts)
- [Properties](https://help.obsidian.md/properties)
+2 -2
View File
@@ -44,9 +44,9 @@ CLIPPINGS_DIR="$OUTPUT_DIR"
# Create clippings directory if it doesn't exist # Create clippings directory if it doesn't exist
mkdir -p "$CLIPPINGS_DIR" mkdir -p "$CLIPPINGS_DIR"
# Function to sanitize filename # Function to sanitize filename (preserve CJK characters, only remove filesystem-illegal chars)
sanitize_filename() { sanitize_filename() {
echo "$1" | sed 's/[^a-zA-Z0-9 -]//g' | sed 's/ \+/ /g' | sed 's/^ *//;s/ *$//' echo "$1" | sed 's/[\/\\:*?"<>|]//g' | sed 's/ \+/ /g' | sed 's/^ *//;s/ *$//'
} }
# Function to extract domain name for fallback # Function to extract domain name for fallback
View File
+7 -1
View File
@@ -2,7 +2,13 @@
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VAULT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)" ENV_FILE="$SCRIPT_DIR/.env.memory"
if [[ -f "$ENV_FILE" ]]; then
VAULT_DIR="$(grep '^VAULT_DIR=' "$ENV_FILE" | cut -d= -f2- | tr -d '"' | tr -d "'")"
fi
if [[ -z "${VAULT_DIR:-}" ]]; then
VAULT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
fi
QUERY="${*:-}" QUERY="${*:-}"
if [[ -z "$QUERY" ]]; then if [[ -z "$QUERY" ]]; then
-2
View File
@@ -14,8 +14,6 @@ SENSITIVE_LITERAL_MARKERS = [
'aws_access_key_id', 'aws_access_key_id',
'password:', 'password:',
'passwd:', 'passwd:',
'api_key',
'access_key',
'账号:', '账号:',
'账号:', '账号:',
'用户名:', '用户名:',
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="$SCRIPT_DIR/../../.env.memory"
[[ -f "$ENV_FILE" ]] && source <(grep -E '^[A-Z_]+=.+' "$ENV_FILE" | sed 's/^/export /')
PASS=0
FAIL=0
check() {
local label="$1" result="$2"
if [[ "$result" == "ok" ]]; then
echo "$label"
((PASS++)) || true
else
echo "$label: $result"
((FAIL++)) || true
fi
}
# 1. .env.memory 存在且关键字段非空
if [[ -f "$ENV_FILE" ]]; then
missing=""
for key in PG_DSN VAULT_DIR OPENROUTER_API_KEY OPENROUTER_EMBED_MODEL; do
val="$(grep "^${key}=" "$ENV_FILE" | cut -d= -f2- | tr -d '"' | tr -d "'" | xargs)"
[[ -z "$val" ]] && missing="$missing $key"
done
[[ -z "$missing" ]] && check ".env.memory 关键字段" "ok" || check ".env.memory 关键字段" "缺少:$missing"
else
check ".env.memory 存在" "文件不存在: $ENV_FILE"
fi
# 2. PostgreSQL 可连接
if command -v psql &>/dev/null; then
result="$(psql "$PG_DSN" -c "SELECT 1;" -t 2>&1 | xargs)"
[[ "$result" == "1" ]] && check "PostgreSQL 连通" "ok" || check "PostgreSQL 连通" "$result"
elif command -v docker &>/dev/null; then
result="$(docker exec pgvector psql -U postgres -d memory -c "SELECT 1;" -t 2>&1 | xargs)"
[[ "$result" == "1" ]] && check "PostgreSQL 连通" "ok" || check "PostgreSQL 连通" "$result"
else
check "PostgreSQL 连通" "psql/docker 均不可用"
fi
# 3. memory_primary 条数 > 0
if command -v psql &>/dev/null; then
count="$(psql "$PG_DSN" -c "SELECT count(*) FROM memory_primary;" -t 2>&1 | xargs)"
elif command -v docker &>/dev/null; then
count="$(docker exec pgvector psql -U postgres -d memory -c "SELECT count(*) FROM memory_primary;" -t 2>&1 | xargs)"
else
count="0"
fi
[[ "$count" -gt 0 ]] 2>/dev/null && check "memory_primary 条数 ($count)" "ok" || check "memory_primary 条数" "为空或查询失败: $count"
# 4. 最近 updated_at 在 24h 内
if command -v psql &>/dev/null; then
fresh="$(psql "$PG_DSN" -c "SELECT count(*) FROM memory_primary WHERE updated_at > now() - interval '24 hours';" -t 2>&1 | xargs)"
elif command -v docker &>/dev/null; then
fresh="$(docker exec pgvector psql -U postgres -d memory -c "SELECT count(*) FROM memory_primary WHERE updated_at > now() - interval '24 hours';" -t 2>&1 | xargs)"
else
fresh="0"
fi
[[ "$fresh" -gt 0 ]] 2>/dev/null && check "24h 内有更新 ($fresh 条)" "ok" || check "24h 内有更新" "无近期更新"
echo ""
echo "结果: ${PASS} 通过 / ${FAIL} 失败"
[[ "$FAIL" -eq 0 ]] && exit 0 || exit 1
+16 -2
View File
@@ -14,6 +14,7 @@ from index_common import (
) )
MIN_TEXT_LEN = 50 MIN_TEXT_LEN = 50
PRIMARY_DIRS = {'01_Projects', '02_Areas'}
def parse_changes(changes_file: Path) -> list[dict]: def parse_changes(changes_file: Path) -> list[dict]:
@@ -35,11 +36,17 @@ def parse_changes(changes_file: Path) -> list[dict]:
def upsert_file(cur, rel_path: str) -> None: def upsert_file(cur, rel_path: str) -> None:
top_dir = Path(rel_path).parts[0] if Path(rel_path).parts else ''
if top_dir not in PRIMARY_DIRS:
print(f'[SKIPPED-OUT-OF-SCOPE] {rel_path}')
return
abs_path = VAULT_ROOT / rel_path abs_path = VAULT_ROOT / rel_path
if not abs_path.exists() or abs_path.suffix.lower() != '.md': if not abs_path.exists() or abs_path.suffix.lower() != '.md':
return return
if is_excluded(abs_path): reason = is_excluded(abs_path)
if reason:
cur.execute('DELETE FROM memory_primary WHERE id=%s', (rel_path,)) cur.execute('DELETE FROM memory_primary WHERE id=%s', (rel_path,))
cur.execute( cur.execute(
''' '''
@@ -49,7 +56,7 @@ def upsert_file(cur, rel_path: str) -> None:
risk = EXCLUDED.risk, risk = EXCLUDED.risk,
updated_at = now() updated_at = now()
''', ''',
(rel_path, rel_path, 'excluded_or_sensitive'), (rel_path, rel_path, reason),
) )
print(f'[QUARANTINED] {rel_path}') print(f'[QUARANTINED] {rel_path}')
return return
@@ -61,6 +68,13 @@ def upsert_file(cur, rel_path: str) -> None:
print(f'[PRUNED] {rel_path}') print(f'[PRUNED] {rel_path}')
return return
new_hash = sha256_text(text)
cur.execute('SELECT content_hash FROM memory_primary WHERE id=%s', (rel_path,))
row = cur.fetchone()
if row and row[0] == new_hash:
print(f'[SKIPPED] {rel_path}')
return
try: try:
embedding = embed_text(text) embedding = embed_text(text)
except Exception as error: except Exception as error:
+34 -21
View File
@@ -1,8 +1,8 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
import fcntl
import hashlib import hashlib
import os import os
import sys
import time import time
from contextlib import contextmanager from contextlib import contextmanager
from pathlib import Path from pathlib import Path
@@ -62,29 +62,33 @@ def _sample_head_mid_tail(content: bytes, span: int = 1200) -> str:
return sampled.decode('utf-8', errors='ignore').lower() return sampled.decode('utf-8', errors='ignore').lower()
def is_excluded(file_path: Path) -> bool: def is_excluded(file_path: Path) -> str | None:
"""返回排除原因字符串,未排除则返回 None。"""
rel = safe_rel(file_path) rel = safe_rel(file_path)
if rel is None: if rel is None:
return True return 'path:unresolvable'
parts = set(Path(rel).parts) parts = set(Path(rel).parts)
if parts.intersection(EXCLUDE_DIR_NAMES): for name in parts.intersection(EXCLUDE_DIR_NAMES):
return True return f'dir:{name}'
if parts.intersection(EXCLUDE_PATH_PARTS): for name in parts.intersection(EXCLUDE_PATH_PARTS):
return True return f'path:{name}'
if any(kw in file_path.name.lower() for kw in EXCLUDE_FILENAME_KEYWORDS): for kw in EXCLUDE_FILENAME_KEYWORDS:
return True if kw in file_path.name.lower():
return f'filename:{kw}'
try: try:
snippet = _sample_head_mid_tail(file_path.read_bytes()) snippet = _sample_head_mid_tail(file_path.read_bytes())
if any(marker in snippet for marker in SENSITIVE_LITERAL_MARKERS): for marker in SENSITIVE_LITERAL_MARKERS:
return True if marker in snippet:
if any(pattern.search(snippet) for pattern in SENSITIVE_REGEX_PATTERNS): return f'content:literal:{marker[:20]}'
return True for pattern in SENSITIVE_REGEX_PATTERNS:
if pattern.search(snippet):
return f'content:regex:{pattern.pattern[:30]}'
except Exception: except Exception:
return True return 'content:read_error'
return False return None
@contextmanager @contextmanager
@@ -92,11 +96,20 @@ def index_lock():
lock_file = Path(os.getenv('INDEX_LOCK_FILE', str(VAULT_ROOT / '.memory-index.lock'))) lock_file = Path(os.getenv('INDEX_LOCK_FILE', str(VAULT_ROOT / '.memory-index.lock')))
lock_file.parent.mkdir(parents=True, exist_ok=True) lock_file.parent.mkdir(parents=True, exist_ok=True)
with open(lock_file, 'w', encoding='utf-8') as fh: with open(lock_file, 'w', encoding='utf-8') as fh:
fcntl.flock(fh, fcntl.LOCK_EX) if sys.platform == 'win32':
try: import msvcrt
yield msvcrt.locking(fh.fileno(), msvcrt.LK_LOCK, 1)
finally: try:
fcntl.flock(fh, fcntl.LOCK_UN) yield
finally:
msvcrt.locking(fh.fileno(), msvcrt.LK_UNLCK, 1)
else:
import fcntl as _fcntl
_fcntl.flock(fh, _fcntl.LOCK_EX)
try:
yield
finally:
_fcntl.flock(fh, _fcntl.LOCK_UN)
def embed_text(text: str) -> list[float]: def embed_text(text: str) -> list[float]:
@@ -118,7 +131,7 @@ def embed_text(text: str) -> list[float]:
if title: if title:
headers['X-OpenRouter-Title'] = title headers['X-OpenRouter-Title'] = title
payload = {'model': model, 'input': text} payload = {'model': model, 'input': text, 'encoding_format': 'float'}
last_error: Exception | None = None last_error: Exception | None = None
for attempt in range(1, 4): for attempt in range(1, 4):
+4 -3
View File
@@ -49,7 +49,8 @@ def run() -> None:
for md_file in target.rglob('*.md'): for md_file in target.rglob('*.md'):
rel = normalize_rel(md_file) rel = normalize_rel(md_file)
if is_excluded(md_file): reason = is_excluded(md_file)
if reason:
secure_ids.add(rel) secure_ids.add(rel)
cur.execute( cur.execute(
''' '''
@@ -59,7 +60,7 @@ def run() -> None:
risk = EXCLUDED.risk, risk = EXCLUDED.risk,
updated_at = now() updated_at = now()
''', ''',
(rel, rel, 'excluded_or_sensitive'), (rel, rel, reason),
) )
cur.execute('DELETE FROM memory_primary WHERE id=%s', (rel,)) cur.execute('DELETE FROM memory_primary WHERE id=%s', (rel,))
print(f'[QUARANTINED] {rel}') print(f'[QUARANTINED] {rel}')
@@ -85,7 +86,7 @@ def run() -> None:
cur.execute('SELECT id FROM memory_primary') cur.execute('SELECT id FROM memory_primary')
db_primary_ids = {row[0] for row in cur.fetchall()} db_primary_ids = {row[0] for row in cur.fetchall()}
stale_primary_ids = sorted(db_primary_ids - valid_ids) stale_primary_ids = sorted(db_primary_ids - valid_ids - set(failed_ids))
for stale_id in stale_primary_ids: for stale_id in stale_primary_ids:
cur.execute('DELETE FROM memory_primary WHERE id=%s', (stale_id,)) cur.execute('DELETE FROM memory_primary WHERE id=%s', (stale_id,))
+13 -2
View File
@@ -26,8 +26,19 @@ run_memory_async_index() {
git diff-tree --no-commit-id --name-status -r -M --diff-filter=ACDMRT HEAD -- '*.md' > "$changes_file" 2>/dev/null || true git diff-tree --no-commit-id --name-status -r -M --diff-filter=ACDMRT HEAD -- '*.md' > "$changes_file" 2>/dev/null || true
[[ -s "$changes_file" ]] || { rm -f "$changes_file"; return 0; } [[ -s "$changes_file" ]] || { rm -f "$changes_file"; return 0; }
nohup uv run --project "$vault_dir/.scripts/memory" python "$vault_dir/.scripts/memory/incremental_ingest.py" \ nohup bash -c "
--changes-file "$changes_file" >> "$log_file" 2>&1 & # 日志轮转:超过 5MB 保留最后 1000 行
if [[ -f '$log_file' ]] && [[ \$(wc -c < '$log_file') -gt 5242880 ]]; then
tail -n 1000 '$log_file' > '$log_file.tmp' && mv '$log_file.tmp' '$log_file'
fi
uv run --project '$vault_dir/.scripts/memory' python '$vault_dir/.scripts/memory/incremental_ingest.py' \
--changes-file '$changes_file' >> '$log_file' 2>&1
exit_code=\$?
if [[ \$exit_code -ne 0 ]]; then
echo \"\$(date '+%Y-%m-%d %H:%M:%S') [ERROR] incremental_ingest 退出码=\$exit_code\" >> '$log_file'
fi
rm -f '$changes_file'
" &
} }
run_memory_async_index run_memory_async_index
# --- memory async index hook end --- # --- memory async index hook end ---
+3 -2
View File
@@ -9,7 +9,7 @@ def sanitize(text: str) -> str:
return text.replace('```', '` ` `').strip() return text.replace('```', '` ` `').strip()
def query(text: str, top_k: int = 5, max_chars: int = 2500) -> str: def query(text: str, top_k: int = 5, max_chars: int = 2500, threshold: float = 0.5) -> str:
embedding = embed_text(text) embedding = embed_text(text)
vector = vector_literal(embedding) vector = vector_literal(embedding)
@@ -19,10 +19,11 @@ def query(text: str, top_k: int = 5, max_chars: int = 2500) -> str:
''' '''
SELECT source, content SELECT source, content
FROM memory_primary FROM memory_primary
WHERE embedding <=> %s::vector < %s
ORDER BY embedding <=> %s::vector ORDER BY embedding <=> %s::vector
LIMIT %s LIMIT %s
''', ''',
(vector, top_k), (vector, threshold, vector, top_k),
) )
rows = cur.fetchall() rows = cur.fetchall()
+2 -2
View File
@@ -5,7 +5,7 @@ CREATE TABLE IF NOT EXISTS memory_primary (
source TEXT NOT NULL, source TEXT NOT NULL,
content TEXT NOT NULL, content TEXT NOT NULL,
content_hash TEXT NOT NULL, content_hash TEXT NOT NULL,
embedding VECTOR(1536) NOT NULL, embedding VECTOR(4096) NOT NULL,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now() updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
); );
@@ -17,4 +17,4 @@ CREATE TABLE IF NOT EXISTS memory_secure_audit (
); );
CREATE INDEX IF NOT EXISTS memory_primary_embedding_idx CREATE INDEX IF NOT EXISTS memory_primary_embedding_idx
ON memory_primary USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100); ON memory_primary USING hnsw (embedding vector_cosine_ops);
+22
View File
@@ -0,0 +1,22 @@
# Upgrade Checklist: 0.13.1 → 0.14.2
Generated: 2026-02-26
## Commands (2 changed, 1 new in upstream, 2 local-only)
[x] .claude/commands/README.md (changed) → applied upstream
[x] .claude/commands/upgrade.md (changed) → applied upstream
[x] .claude/commands/pragmatic-review.md (NEW) → added
[-] .claude/commands/compact-sessions.md (local-only, kept)
[-] .claude/commands/git-commit-msg.md (local-only, kept)
## Scripts (1 changed)
[x] .scripts/firecrawl-batch.sh (changed) → applied upstream (CJK fix)
## Settings (1 changed)
[x] .claude/settings.json (changed) → merged (kept permissions + added hooks)
## Core Files (1 changed)
[x] package.json (changed) → merged (kept local scripts + bumped deps to 0.14.2)
@@ -0,0 +1,115 @@
---
title: 架构设计
created: 2026-02-25
---
# 架构设计
## 设计目标
1. 不使用本地 embedding 模型(避免常驻 Ollama
2. 向量与元数据留在本地 PostgreSQL(数据可控)
3. 基于 Git 变更做增量 upsert/delete(避免全量重建)
4. 强一致:删除/重命名后不允许幽灵向量残留
5. 安全:敏感内容不入检索库,检索结果具备 Prompt 注入防护
## 双层记忆模型
```
┌─────────────────────────────────────────────┐
│ Claude Agent (agent-with-memory.sh) │
│ │
│ Layer B: MEMORY.md (偏好/事实/约束) │
│ Layer A: pgvector 语义检索结果 │
└─────────────────────────────────────────────┘
↑ ↑
Claude Code query_pgvector.py
MEMORY.md PostgreSQL + pgvector
incremental_ingest.py
git post-commit hook
Obsidian Vault (.md files)
```
### Layer A:文档语义记忆
- EmbeddingOpenRouter API`openai/text-embedding-3-small`dim=1536
- 存储:本地 PostgreSQL + pgvector 扩展
- 数据源:`01_Projects/``02_Areas/`
- 排除:`00_Inbox/``04_Archive/``Infrastructure/``Home-Automation/`
- 触发:git `post-commit` hook → 异步 `incremental_ingest.py`
### Layer B:事实与偏好记忆
- 来源:Claude Code `MEMORY.md`(查询时动态读取)
- 用途:用户偏好、约束、近期状态
## 组件职责
| 文件 | 职责 |
|---|---|
| `index_common.py` | 共享工具:环境加载、DB 连接、Embedding API、排除逻辑、跨平台锁 |
| `blacklist.py` | 排除规则:路径模式、文件名关键词、敏感内容特征 |
| `ingest_vault.py` | 全量重建索引,清理过期条目 |
| `incremental_ingest.py` | 解析 `git diff-tree` 输出,处理 A/M/D/R 事件 |
| `query_pgvector.py` | 余弦相似度检索,返回 `<retrieved_context>` 块 |
| `install-hook.sh` | 向 `.git/hooks/post-commit` 追加异步索引触发器 |
| `agent-with-memory.sh` | 合并 A+B 层上下文,启动带记忆的 claude 会话 |
| `schema.sql` | DB schema`memory_primary`(向量)、`memory_secure_audit`(隔离审计)|
## 数据库 Schema
```sql
-- 主检索表
memory_primary (
id TEXT PRIMARY KEY, -- vault 相对路径
source TEXT, -- 同 id,用于上下文注入显示
content TEXT, -- 完整 markdown 文本
content_hash TEXT, -- sha256,用于变更检测
embedding VECTOR(1536), -- ivfflat 余弦索引
updated_at TIMESTAMPTZ
)
-- 敏感文件审计表
memory_secure_audit (
id TEXT PRIMARY KEY,
source TEXT,
risk TEXT, -- 'excluded_or_sensitive'
updated_at TIMESTAMPTZ
)
```
索引:`ivfflat (embedding vector_cosine_ops) WITH (lists = 100)`
## 增量同步流程
```
git commit
└── post-commit hooknohup,不阻塞提交)
└── incremental_ingest.py --changes-file <tmp>
├── 解析 git diff-tree 输出(A/M/T/D/R
├── 获取 index_lock(跨平台文件锁)
└── 逐事件处理:
A/M/T → upsert_file()
D → DELETE from both tables
R → DELETE old, upsert_file(new)
```
## 安全策略
1. 路径精确匹配:按 `Path.parts` 做目录排除,非子串匹配
2. 单命中隔离:任一敏感特征命中即隔离到 `memory_secure_audit`
3. 强一致删除:D/R 事件先删旧 ID,再处理新路径
4. 单写者锁:跨平台文件锁串行化所有索引写操作
5. 只读上下文注入:检索结果包装为 `<retrieved_context>` 标签,明确标注为非指令
6. 长度截断:注入前全局 `max_chars=2500`
## 关键权衡
| 优点 | 代价 |
|---|---|
| 不跑本地模型,设备压力低 | Markdown 文本发送到 OpenRouter(非纯本地隐私)|
| 向量在本地 DB,数据控制力强 | 依赖网络与 API 可用性 |
| 与现有 Git 工作流兼容 | 需配置 API key 与限流/重试策略 |
@@ -0,0 +1,122 @@
---
title: 部署指南
created: 2026-02-25
---
# 部署指南
## 前置条件
- Python 3.10+,已安装 [uv](https://github.com/astral-sh/uv)
- Docker + Docker Compose(含 pgvector 扩展,推荐)或本机 PostgreSQL 15+
- OpenRouter API key
## 步骤
### 1. 启动 pgvector
`.scripts/memory/` 目录下创建 `docker-compose.yml`
```yaml
services:
postgres:
image: pgvector/pgvector:pg16
container_name: pgvector
restart: unless-stopped
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: memory
ports:
- "5432:5432"
volumes:
- pg_data:/var/lib/postgresql/data
- ./initdb:/docker-entrypoint-initdb.d:ro
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d memory"]
interval: 5s
timeout: 3s
retries: 20
volumes:
pg_data:
```
启动:
```bash
docker compose up -d
# 等待 healthy 后再继续
docker compose ps
```
### 2. 初始化 Schema
```bash
psql postgresql://postgres:postgres@localhost:5432/memory -f schema.sql
```
### 3. 配置环境变量
```bash
cp .env.memory.example .env.memory
# 编辑 .env.memory,填入 VAULT_DIR 和 OPENROUTER_API_KEY
```
必填项:
```ini
VAULT_DIR=/path/to/your/obsidian-vault
PG_DSN=postgresql://postgres:postgres@localhost:5432/memory
OPENROUTER_API_KEY=sk-or-v1-...
OPENROUTER_EMBED_MODEL=openai/text-embedding-3-small
OPENROUTER_EMBED_DIM=1536
```
### 4. 安装依赖
```bash
uv sync
```
### 5. 全量索引
```bash
uv run python ingest_vault.py
```
预期输出:`[DONE] 全量索引完成 primary=N secure=N ...`
### 6. 安装 Git Hook(在 vault 目录执行)
```bash
bash /path/to/vault-memory-pgvector/install-hook.sh
```
验证:`.git/hooks/post-commit` 中出现 `memory async index hook begin`
## 验收检查
| 检查项 | 命令 | 预期 |
|---|---|---|
| DB 连通 | `psql $PG_DSN -c "SELECT 1;"` | 返回 `1` |
| 表存在 | `psql $PG_DSN -c "\dt memory_*"` | 两张表可见 |
| 索引条数 | `psql $PG_DSN -c "SELECT count(*) FROM memory_primary;"` | > 0 |
| 查询链路 | `uv run python query_pgvector.py "测试查询"` | 返回 `<retrieved_context>` 块 |
| Hook 生效 | 提交一个 `.md` 文件后查看 `.memory-sync.log` | 出现 `[UPSERTED]` |
## 回滚
禁用记忆同步(不删数据):
```bash
# 编辑 vault 的 .git/hooks/post-commit
# 删除 --- memory async index hook begin --- 到 --- memory async index hook end --- 之间的内容
```
清空数据库:
```bash
psql postgresql://postgres:postgres@localhost:5432/memory \
-c "TRUNCATE memory_primary, memory_secure_audit;"
```
@@ -0,0 +1,50 @@
---
title: Vault Memory System
status: active
created: 2026-02-25
project: vault-memory-pgvector
---
# Vault Memory System
为 Obsidian Vault 提供语义记忆能力的独立工具项目。通过 OpenRouter Embedding + 本地 pgvector,让 Claude Agent 在跨 Session 时能召回相关笔记上下文。
## 项目位置
- 代码仓库:`D:/tmp/vault-memory-pgvector/`
- Vault 脚本(旧):`.scripts/memory/`(已迁移,可删除)
## 文档导航
**系统文档**
- [[architecture]] — 系统架构与设计决策
- [[deployment]] — 部署与配置步骤
- [[status]] — 当前运行状态与已知问题
**审查与重构**
- [[review-2026-02-26]] — 代码审查报告(2026-02-26
- [[refactor-plan]] — 重构计划(3 阶段)
- [[refactor-board]] — 重构看板(任务进度跟踪)
## 快速命令
```bash
# 查询记忆
uv run python query_pgvector.py "最近的重点项目"
# 带记忆启动 Claude
bash agent-with-memory.sh "帮我整理本周进展"
# 重建全量索引
uv run python ingest_vault.py
```
## 关键指标(2026-02-25
| 指标 | 目标 | 实测 | 状态 |
|---|---|---|---|
| Recall@5 | ≥ 70% | 90% | ✅ |
| 增量 P95 延迟 | < 1s | 1.4s | ⚠️ |
| 冷启动 | ≤ 4.5s | 未测 | 🔲 |
| 敏感内容隔离 | 红线 | 通过 | ✅ |
| 删除/重命名一致性 | 红线 | 通过 | ✅ |
@@ -0,0 +1,33 @@
---
kanban-plugin: board
---
## Phase 1 · 数据安全与正确性
_(全部完成)_
## Phase 2 · 性能与一致性
_(代码改动已完成,待验收)_
## Phase 3 · 可观测性与运营
- [ ] **P3-1** audit 表记录触发原因 · `blacklist.py` · 🟡 Nice
- [ ] **P3-2** hook 失败写入可见日志 · `install-hook.sh` · 🟡 Nice
- [ ] **P3-3** 添加 health-check 脚本 · 新增文件 · 🟡 Nice
- [ ] **P3-4** 日志轮转 · `install-hook.sh` · 🟡 Nice
## 已完成
- [x] **P1-1** 删除 blacklist 过宽字面量 · `blacklist.py` · 🔴 Critical
- [x] **P1-2** stale 清理排除 failed_ids · `ingest_vault.py`:88 · 🔴 Critical
- [x] **P1-4** 重建全量索引验收 · primary=96, secure=85 · 🔴 Critical
- [x] **P2-1** 增量索引加 hash 检查 · `incremental_ingest.py` · 🟠 Important
- [x] **P2-2** 统一 ingest 目录范围 · `incremental_ingest.py` · 🟠 Important
- [x] **P2-3** 换用 HNSW 索引 · `schema.sql` · 🟠 Important
- [x] **P2-4** 修复 VAULT_DIR 推导 · `agent-with-memory.sh` · 🟠 Important
- [x] **P2-5** 查询加相似度阈值 · `query_pgvector.py` · 🟠 Important
%% kanban:settings
{"kanban-plugin":"board","list-collapse":[false,false,false,false]}
%%
@@ -0,0 +1,273 @@
---
title: 重构计划
date: 2026-02-26
based-on: "[[review-2026-02-26]]"
status: 待执行
---
# 重构计划
基于 [[review-2026-02-26]] 的审查结果,按优先级分三个阶段执行。
---
## Phase 1 · 数据安全与正确性(Critical)
> 目标:消除静默数据损坏和误隔离问题。不改变接口,不影响现有功能。
### P1-1 修复 blacklist.py — 删除过宽字面量
**文件**`blacklist.py`
**问题**C449% 误隔离率
**改动**:从 `SENSITIVE_LITERAL_MARKERS` 删除 `'api_key'``'access_key'`,这两个已被第 33 行的正则覆盖(要求后跟赋值符号)。
**验收**:重建索引后 `memory_primary` 条数应显著增加,`memory_secure_audit` 条数应下降。
```python
# 删除这两行
'api_key',
'access_key',
```
---
### P1-2 修复 ingest_vault.py — stale 清理排除 failed_ids
**文件**`ingest_vault.py` 第 88 行
**问题**:C1,embed 失败的文档在下次全量重建时被删除
**改动**:一行修改
```python
# 修改前
stale_primary_ids = sorted(db_primary_ids - valid_ids)
# 修改后
stale_primary_ids = sorted(db_primary_ids - valid_ids - set(failed_ids))
```
---
### P1-3 修复 install-hook.sh — 清理 changes 临时文件
**文件**`install-hook.sh`(以及 vault 里已安装的 `.git/hooks/post-commit`
**问题**:C3,每次 commit 留下临时文件
**改动**:用子 shell 包装 nohup 调用,处理完后删除临时文件
```bash
# 修改前
nohup uv run --project "$vault_dir/.scripts/memory" python \
"$vault_dir/.scripts/memory/incremental_ingest.py" \
--changes-file "$changes_file" >> "$log_file" 2>&1 &
# 修改后
nohup bash -c "uv run --project '$vault_dir/.scripts/memory' python \
'$vault_dir/.scripts/memory/incremental_ingest.py' \
--changes-file '$changes_file' >> '$log_file' 2>&1; \
rm -f '$changes_file'" &
```
修改后需重新运行 `install-hook.sh` 更新已安装的 hook。
---
### P1-4 重建全量索引验收
完成 P1-1 后必须重建索引,验收隔离率是否恢复正常:
```bash
# 清空现有数据
psql $PG_DSN -c "TRUNCATE memory_primary, memory_secure_audit;"
# 重建
uv run python ingest_vault.py
# 检查结果
psql $PG_DSN -c "SELECT count(*) FROM memory_primary;"
psql $PG_DSN -c "SELECT count(*) FROM memory_secure_audit;"
```
预期:`memory_primary` 应接近 81+`memory_secure_audit` 应大幅下降。
---
## Phase 2 · 性能与一致性(Important
> 目标:修复 P95 延迟超标问题,统一两个 ingest 脚本的行为。
### P2-1 增量索引加入 hash 检查
**文件**`incremental_ingest.py``upsert_file()` 函数
**问题**:C2,每次 M 事件都调 APIP95=1.4s 的直接原因
**改动**:在调用 `embed_text` 前查询现有 hash,匹配则跳过
```python
def upsert_file(cur, rel_path: str) -> None:
# ... 现有的 excluded / pruned 检查 ...
text = abs_path.read_text(encoding='utf-8', errors='ignore')
new_hash = sha256_text(text)
# 新增:hash 未变则跳过 embed
cur.execute('SELECT content_hash FROM memory_primary WHERE id=%s', (rel_path,))
row = cur.fetchone()
if row and row[0] == new_hash:
print(f'[SKIPPED] {rel_path}')
return
# 继续 embed ...
```
**预期效果**:纯格式调整或无关文件的 commit 不再触发 API 调用,P95 应降至网络延迟本身(~0.3-0.5s)。
---
### P2-2 统一两个 ingest 脚本的目录范围
**文件**`incremental_ingest.py`
**问题**:I2,增量索引处理任意目录,全量索引只处理 PRIMARY_DIRS
**改动**:在 `upsert_file()` 入口加目录过滤
```python
PRIMARY_DIRS = {'01_Projects', '02_Areas'} # 与 ingest_vault.py 保持一致
def upsert_file(cur, rel_path: str) -> None:
# 新增:只处理 PRIMARY_DIRS 内的文件
top_dir = Path(rel_path).parts[0] if Path(rel_path).parts else ''
if top_dir not in PRIMARY_DIRS:
print(f'[SKIPPED-OUT-OF-SCOPE] {rel_path}')
return
# ... 其余逻辑不变 ...
```
---
### P2-3 修复 ivfflat index lists 参数
**文件**`schema.sql`
**问题**I1lists=100 对 81 行数据无效
**改动**:重建索引时使用动态 lists 值,或改用 HNSW(更适合小数据集)
```sql
-- 方案 A:删除旧索引,改用 HNSW(推荐,无需调参)
DROP INDEX IF EXISTS memory_primary_embedding_idx;
CREATE INDEX memory_primary_embedding_idx
ON memory_primary USING hnsw (embedding vector_cosine_ops);
-- 方案 B:保留 ivfflat,修正 lists
DROP INDEX IF EXISTS memory_primary_embedding_idx;
CREATE INDEX memory_primary_embedding_idx
ON memory_primary USING ivfflat (embedding vector_cosine_ops) WITH (lists = 1);
```
推荐方案 A:HNSW 在小数据集上性能更好,且不需要手动维护 lists 参数。
---
### P2-4 修复 agent-with-memory.sh 的 VAULT_DIR 推导
**文件**`agent-with-memory.sh`
**问题**:I4,路径推导假设脚本在特定目录层级
**改动**:从 `.env.memory` 读取 VAULT_DIR,同时修正用法提示
```bash
# 修改前
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VAULT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
# 修改后
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="$SCRIPT_DIR/.env.memory"
if [[ -f "$ENV_FILE" ]]; then
VAULT_DIR="$(grep '^VAULT_DIR=' "$ENV_FILE" | cut -d= -f2- | tr -d '"' | tr -d "'")"
fi
if [[ -z "${VAULT_DIR:-}" ]]; then
echo "错误: 未找到 VAULT_DIR,请在 .env.memory 中配置" >&2
exit 1
fi
```
---
### P2-5 查询加入相似度阈值
**文件**`query_pgvector.py`
**问题**:I5,无关查询仍返回 top_k 结果污染上下文
**改动**:加 `--threshold` 参数,默认 0.5(余弦距离,越小越相似)
```python
def query(text: str, top_k: int = 5, max_chars: int = 2500, threshold: float = 0.5) -> str:
# ...
cur.execute(
'''
SELECT source, content
FROM memory_primary
WHERE embedding <=> %s::vector < %s
ORDER BY embedding <=> %s::vector
LIMIT %s
''',
(vector, threshold, vector, top_k),
)
```
阈值 0.5 为初始值,需根据实际 Recall@5 评测结果调整。
---
## Phase 3 · 可观测性与运营(Nice to have
> 目标:让系统状态可见,减少静默失败。
### P3-1 audit 表记录触发原因
**文件**`schema.sql``blacklist.py``index_common.py`
**问题**I6,无法区分隔离原因
**改动**`is_excluded` 返回触发原因字符串而非布尔值,audit 表 `risk` 列存具体规则名
---
### P3-2 hook 失败时写入可见日志
**文件**`install-hook.sh`
**问题**I7,所有错误静默
**改动**:在 `.memory-sync.log` 里写入带时间戳的错误行,并在 hook 末尾检查日志最后一行是否为错误
---
### P3-3 添加 health-check 脚本
新增 `health-check.sh`,检查:
- PostgreSQL 是否可连接
- `memory_primary` 条数是否 > 0
- 最近一次 `updated_at` 是否在 24h 内
- `.env.memory` 是否存在且关键字段非空
---
### P3-4 日志轮转
**文件**`install-hook.sh`
**问题**O1`.memory-sync.log` 无限增长
**改动**:hook 里加简单的大小检查,超过 5MB 时截断旧内容
---
## 执行顺序
```
Phase 1(必须先做)
P1-1 → P1-2 → P1-3 → P1-4(验收)
Phase 2P1 完成后)
P2-1 → P2-2 → P2-3 → P2-4 → P2-5
Phase 3(可选,按需)
P3-1 → P3-2 → P3-3 → P3-4
```
## 预期收益
| 问题 | 修复后预期 |
|---|---|
| 49% 误隔离 | `memory_primary` 恢复到 ~140+ 文档 |
| P95=1.4s | hash 命中时降至 <0.1s,实际 embed 时降至 ~0.8sHNSW 加速)|
| 临时文件积累 | 每次 commit 后自动清理 |
| 数据静默丢失 | embed 失败不再删除已有向量 |
@@ -0,0 +1,367 @@
---
title: 重构任务清单
date: 2026-02-26
based-on: "[[refactor-plan]]"
kanban-board: "[[refactor-board]]"
status: 进行中
---
# 重构任务清单
> 看板视图见 [[refactor-board]]。完成任务后勾选 checkbox,填写完成时间和实际结果。
---
## Phase 1 · 数据安全与正确性
> 必须按顺序执行,P1-4 是验收步骤。
### P1-1 · 删除 blacklist 过宽字面量
| 字段 | 内容 |
|---|---|
| 状态 | ✅ DONE |
| 优先级 | 🔴 Critical |
| 文件 | `blacklist.py` |
| 关联问题 | C4(49% 误隔离率)|
| 完成时间 | 2026-02-26 |
| 实际结果 | 已删除 `api_key``access_key` 两行字面量 |
改动:从 `SENSITIVE_LITERAL_MARKERS` 删除以下两行(已被正则覆盖)
```python
'api_key',
'access_key',
```
验收:重建索引后 `memory_primary` 条数显著增加,`memory_secure_audit` 条数下降。
- [x] 完成改动
- [ ] 验收通过
---
### P1-2 · stale 清理排除 failed_ids
| 字段 | 内容 |
|---|---|
| 状态 | ✅ DONE |
| 优先级 | 🔴 Critical |
| 文件 | `ingest_vault.py` 第 88 行 |
| 关联问题 | C1(embed 失败文档被静默删除)|
| 完成时间 | 2026-02-26 |
| 实际结果 | 已在 stale 计算中排除 failed_ids |
改动:
```python
# 修改前
stale_primary_ids = sorted(db_primary_ids - valid_ids)
# 修改后
stale_primary_ids = sorted(db_primary_ids - valid_ids - set(failed_ids))
```
验收:全量重建时模拟 embed 失败,确认失败文档的旧向量保留。
- [x] 完成改动
- [ ] 验收通过
---
### P1-3 · hook 清理 changes 临时文件
| 字段 | 内容 |
|---|---|
| 状态 | ✅ DONE |
| 优先级 | 🔴 Critical |
| 文件 | `install-hook.sh`(修改后需重新安装 hook|
| 关联问题 | C3(每次 commit 留下临时文件)|
| 完成时间 | 2026-02-26 |
| 实际结果 | 已加 `rm -f '$changes_file'`hook 已重新安装 |
改动:
```bash
# 修改前
nohup uv run ... --changes-file "$changes_file" >> "$log_file" 2>&1 &
# 修改后
nohup bash -c "uv run ... --changes-file '$changes_file' >> '$log_file' 2>&1; rm -f '$changes_file'" &
```
验收:commit 后确认 vault 根目录无 `.memory-changes-*.txt` 残留。
- [x] 完成改动
- [x] 重新安装 hook
- [ ] 验收通过
---
### P1-4 · 重建全量索引验收
| 字段 | 内容 |
|---|---|
| 状态 | ✅ DONE |
| 优先级 | 🔴 Critical |
| 依赖 | P1-1 ~ P1-3 全部完成后执行 |
| 完成时间 | 2026-02-26 |
| 实测 primary | 96 |
| 实测 secure | 85 |
操作:
```bash
psql $PG_DSN -c "TRUNCATE memory_primary, memory_secure_audit;"
uv run python ingest_vault.py
psql $PG_DSN -c "SELECT count(*) FROM memory_primary;"
psql $PG_DSN -c "SELECT count(*) FROM memory_secure_audit;"
```
验收标准:`memory_primary` > 100`memory_secure_audit` < 20。
- [x] 执行全量重建
- [x] `memory_primary` > 100 ⚠️ 实测 96Infrastructure/Home-Automation 被路径规则整体隔离)
- [x] `memory_secure_audit` < 20 ⚠️ 实测 85(同上原因)
---
## Phase 2 · 性能与一致性
> 依赖 Phase 1 全部完成后执行。
### P2-1 · 增量索引加 hash 检查
| 字段 | 内容 |
|---|---|
| 状态 | TODO |
| 优先级 | 🟠 Important |
| 文件 | `incremental_ingest.py` |
| 关联问题 | C2P95=1.4s 的直接原因)|
| 完成时间 | — |
| 实际结果 | — |
改动:调用 `embed_text` 前先查 `content_hash`,匹配则跳过
```python
new_hash = sha256_text(text)
cur.execute('SELECT content_hash FROM memory_primary WHERE id=%s', (rel_path,))
row = cur.fetchone()
if row and row[0] == new_hash:
print(f'[SKIPPED] {rel_path}')
return
```
验收:对未修改文件触发 commit,日志出现 `[SKIPPED]`,无 API 调用。预期 P95 < 0.1shash 命中时)。
- [ ] 完成改动
- [ ] 验收通过(日志出现 `[SKIPPED]`
---
### P2-2 · 统一 ingest 目录范围
| 字段 | 内容 |
|---|---|
| 状态 | TODO |
| 优先级 | 🟠 Important |
| 文件 | `incremental_ingest.py` |
| 关联问题 | I2(增量处理任意目录,全量只处理 PRIMARY_DIRS|
| 完成时间 | — |
| 实际结果 | — |
改动:`upsert_file()` 入口加目录过滤
```python
PRIMARY_DIRS = {'01_Projects', '02_Areas'}
top_dir = Path(rel_path).parts[0] if Path(rel_path).parts else ''
if top_dir not in PRIMARY_DIRS:
print(f'[SKIPPED-OUT-OF-SCOPE] {rel_path}')
return
```
验收:提交 `03_Resources/` 下的文件,日志出现 `[SKIPPED-OUT-OF-SCOPE]`
- [ ] 完成改动
- [ ] 验收通过
---
### P2-3 · 换用 HNSW 索引
| 字段 | 内容 |
|---|---|
| 状态 | TODO |
| 优先级 | 🟠 Important |
| 文件 | `schema.sql` |
| 关联问题 | I1ivfflat lists=100 对 81 行无效)|
| 完成时间 | — |
| 实际结果 | — |
改动:
```sql
DROP INDEX IF EXISTS memory_primary_embedding_idx;
CREATE INDEX memory_primary_embedding_idx
ON memory_primary USING hnsw (embedding vector_cosine_ops);
```
验收:`EXPLAIN SELECT ... ORDER BY embedding <=> ...` 显示使用新索引。
- [ ] 执行 SQL 变更
- [ ] EXPLAIN 确认使用 HNSW 索引
---
### P2-4 · 修复 VAULT_DIR 推导
| 字段 | 内容 |
|---|---|
| 状态 | TODO |
| 优先级 | 🟠 Important |
| 文件 | `agent-with-memory.sh` |
| 关联问题 | I4(路径推导假设脚本在特定目录层级)|
| 完成时间 | — |
| 实际结果 | — |
改动:从 `.env.memory` 读取 `VAULT_DIR`,加校验
```bash
ENV_FILE="$SCRIPT_DIR/.env.memory"
if [[ -f "$ENV_FILE" ]]; then
VAULT_DIR="$(grep '^VAULT_DIR=' "$ENV_FILE" | cut -d= -f2- | tr -d '"' | tr -d "'")"
fi
if [[ -z "${VAULT_DIR:-}" ]]; then
echo "错误: 未找到 VAULT_DIR,请在 .env.memory 中配置" >&2
exit 1
fi
```
验收:从任意目录执行脚本,`VAULT_DIR` 正确解析。
- [ ] 完成改动
- [ ] 验收通过
---
### P2-5 · 查询加相似度阈值
| 字段 | 内容 |
|---|---|
| 状态 | TODO |
| 优先级 | 🟠 Important |
| 文件 | `query_pgvector.py` |
| 关联问题 | I5(无关查询仍返回 top_k 结果)|
| 阈值(初始) | 0.5(需实测调整)|
| 完成时间 | — |
| 实际结果 | — |
改动:加 `threshold` 参数
```python
def query(text: str, top_k: int = 5, max_chars: int = 2500, threshold: float = 0.5) -> str:
cur.execute(
'''
SELECT source, content
FROM memory_primary
WHERE embedding <=> %s::vector < %s
ORDER BY embedding <=> %s::vector
LIMIT %s
''',
(vector, threshold, vector, top_k),
)
```
验收:用完全不相关的查询测试,确认返回空结果而非噪音。
- [ ] 完成改动
- [ ] 验收通过(无关查询返回空)
---
## Phase 3 · 可观测性与运营
> 可选,按需执行。
### P3-1 · audit 表记录触发原因
| 字段 | 内容 |
|---|---|
| 状态 | TODO |
| 优先级 | 🟡 Nice |
| 文件 | `schema.sql``blacklist.py``index_common.py` |
| 关联问题 | I6`risk` 列永远是同一个值)|
| 完成时间 | — |
| 实际结果 | — |
改动:`is_excluded()` 返回触发原因字符串(如 `path:00_Inbox``content:api_key_regex`),写入 `risk` 列。
- [ ] 完成改动
- [ ] 验收通过
---
### P3-2 · hook 失败写入可见日志
| 字段 | 内容 |
|---|---|
| 状态 | TODO |
| 优先级 | 🟡 Nice |
| 文件 | `install-hook.sh` |
| 关联问题 | I7(所有错误静默)|
| 完成时间 | — |
| 实际结果 | — |
改动:Python 进程退出码非 0 时,向 `.memory-sync.log` 写入带时间戳的 `[ERROR]` 行。
- [ ] 完成改动
- [ ] 验收通过
---
### P3-3 · 添加 health-check 脚本
| 字段 | 内容 |
|---|---|
| 状态 | TODO |
| 优先级 | 🟡 Nice |
| 文件 | 新增 `health-check.sh` |
| 完成时间 | — |
| 实际结果 | — |
检查项:
- [ ] PostgreSQL 可连接(`SELECT 1`
- [ ] `memory_primary` 条数 > 0
- [ ] 最近 `updated_at` 在 24h 内
- [ ] `.env.memory` 存在且关键字段非空
---
### P3-4 · 日志轮转
| 字段 | 内容 |
|---|---|
| 状态 | TODO |
| 优先级 | 🟡 Nice |
| 文件 | `install-hook.sh` |
| 关联问题 | O1`.memory-sync.log` 无限增长)|
| 完成时间 | — |
| 实际结果 | — |
改动:hook 里检查日志大小,超过 5MB 时保留最后 1000 行。
- [ ] 完成改动
- [ ] 验收通过
---
## 验收基线
| 指标 | 重构前 | 目标 | 实测结果 |
|---|---|---|---|
| `memory_primary` 条数 | 81 | > 120 | 96 ⚠️ |
| `memory_secure_audit` 条数 | 79 | < 20 | 85 ⚠️(路径规则隔离)|
| 增量 P95hash 命中) | 1.4s | < 0.1s | 待测 |
| 增量 P95(实际 embed | 1.4s | < 0.8s | 待测 |
| Recall@5 | 90% | ≥ 90% | 待测 |
@@ -0,0 +1,171 @@
---
title: 代码审查报告
date: 2026-02-26
reviewer: Claude Sonnet 4.6
status: 完成
---
# 代码审查报告(2026-02-26
## 总体评价
架构设计合理,双层记忆模型思路清晰,核心链路(embedding → pgvector → 检索注入)实现正确。`ON CONFLICT DO UPDATE` 幂等 upsert、异步 post-commit hook、`<retrieved_context>` 注入防护均为正确决策。
主要问题集中在:**数据安全**(静默删除好数据)、**性能浪费**(每次都重新 embed)、**误隔离**blacklist 过宽导致 49% 文档未被索引)三个方向。
---
## Critical 问题
### C1 · 全量索引会静默删除 embedding 失败的文档
**文件**`ingest_vault.py` 第 86-90 行
stale 清理逻辑 `db_primary_ids - valid_ids` 未排除 `failed_ids`。一次网络抖动导致 embedding 失败,下次全量重建就把之前已有的好向量删掉。数据静默丢失。
```python
# 当前(有 bug
stale_primary_ids = sorted(db_primary_ids - valid_ids)
# 修复
stale_primary_ids = sorted(db_primary_ids - valid_ids - set(failed_ids))
```
---
### C2 · 增量索引从不检查 content_hash,每次都重新 embed
**文件**`incremental_ingest.py``upsert_file()` 函数
`content_hash` 列已存在于 schema 并由全量索引填充,但增量索引完全忽略它。每个 `M` 事件都调一次 OpenRouter API,即使文件内容没有变化。这是 P95=1.4s 的直接原因,也在浪费 API 额度。
修复:在调用 `embed_text` 前,先查 `memory_primary``content_hash`,若匹配则跳过。
---
### C3 · changes 临时文件永远不删除
**文件**`install-hook.sh` 第 24、29-30 行
hook 通过 `nohup` 后台运行 Python 进程,没有任何机制在处理完成后删除 `.memory-changes-<timestamp>-<pid>.txt`。每次涉及 `.md` 文件的 commit 都在 vault 根目录留下一个文件,长期无限积累。
```bash
# 修复:用子 shell 包装,处理完后清理
nohup bash -c "uv run ... --changes-file '$changes_file' >> '$log_file' 2>&1; rm -f '$changes_file'" &
```
---
### C4 · `api_key` 字面量过宽,导致 49% 文档被误隔离
**文件**`blacklist.py` 第 17 行
字面量 `'api_key'` 会匹配任何包含该字符串的笔记,包括架构文档、项目笔记、本项目自身的文档。实测结果:79/160 文档被隔离(49%),接近一半的 vault 内容未被索引。
同文件第 33 行的正则 `(?i)\b(password|...api[_-]?key)\b\s*[:=]\s*\S{4,}` 才是正确做法(要求后面跟赋值符号)。应删除 `'api_key'` 字面量,或改为 `'api_key='``'api_key:'`
---
### C5 · Windows 文件锁可能是非阻塞的
**文件**`index_common.py` 第 97、101 行
`msvcrt.LK_LOCK` 在部分 Python/Windows 版本下行为不一致,可能不阻塞直接抛 `OSError`。两个并发 ingest 进程可能同时通过锁,导致数据竞争。需要用带重试的循环或换用更可靠的 Windows 锁原语。
---
## Important 问题
### I1 · ivfflat lists=100 对当前数据量完全无效
**文件**`schema.sql` 第 20 行
pgvector 建议 `lists = rows / 1000`81 行数据应用 `lists=1`。当前 lists 数量多于行数,查询规划器会忽略索引直接走全表扫描,索引只有写开销没有读收益。
---
### I2 · 两个 ingest 脚本的目录范围不一致
`ingest_vault.py` 只扫 `01_Projects``02_Areas`,但 `incremental_ingest.py` 处理 git diff 里任何 `.md` 文件。`03_Resources/` 里的文件会被增量索引,但下次全量重建时被删掉,造成数据不一致。
---
### I3 · 大文件被 API 静默截断
`text-embedding-3-small` 上限 8191 tokens,超长笔记的后半部分永远不会被 embed,且没有任何警告。`_sample_head_mid_tail` 函数已存在于 `index_common.py`(用于敏感扫描),但未用于 embedding。
---
### I4 · `agent-with-memory.sh` 的 VAULT_DIR 推导方式脆弱
脚本假设自己在 vault 根目录下两层(`.scripts/memory/`),迁移到独立项目后这个假设已不成立。应从 `.env.memory` 读取 `VAULT_DIR` 作为权威来源。
---
### I5 · 查询没有相似度阈值
无论相关性多低,始终返回 top_k 结果。查询完全不相关的内容时,会把最不相关的 5 个文档注入 Claude 上下文,产生噪音甚至误导。
建议加 `WHERE embedding <=> %s::vector < 0.5`(阈值需实测调整)。
---
### I6 · `memory_secure_audit` 不记录触发原因
`risk` 列永远是 `'excluded_or_sensitive'`,无法区分是路径规则、文件名规则还是内容规则触发的。调整 blacklist 时完全没有依据。
---
### I7 · hook 静默失败,用户无感知
PostgreSQL 挂了、`.env.memory` 不存在、Python 环境损坏,全部静默失败。用户不知道索引已经落后,只能手动查看 `.memory-sync.log`
---
### I8 · 全量索引持有超长 DB 事务
`ingest_vault.py` 在单个事务内完成所有 embedding API 调用(81 次 HTTP 请求,可能数分钟)。事务期间持有连接和行锁,进程被杀时虽然回滚干净,但锁文件同时释放,可能导致并发 ingest 在部分更新状态下运行。
---
## 文档问题
| # | 文件 | 问题 |
|---|---|---|
| D1 | `agent-with-memory.sh` | 用法提示仍写旧路径 `.scripts/memory/` |
| D2 | `README.md` step 6 | `install-hook.sh` 路径是占位符,未说明如何确定实际路径 |
| D3 | `index.md` vs `status.md` | 冷启动状态描述不一致("未测" vs "未完成量化"|
| D4 | `README.md` | `OPENROUTER_EMBED_DIM` 标为必填,但代码有默认值 1536 |
| D5 | 所有文档 | Docker 示例使用默认密码 `postgres:postgres`,未提示修改 |
| D6 | 所有文档 | `eval_cold_start.py` 完全未被文档化 |
| D7 | `status.md` | 49% 隔离率记录为已知问题,但未分析根因(实为 C4 的直接证据)|
---
## 运营问题
| # | 问题 |
|---|---|
| O1 | `.memory-sync.log` 无限增长,无轮转机制 |
| O2 | 无健康检查命令,无法快速验证系统是否正常运行 |
| O3 | 无索引漂移检测机制,hook 连续失败时用户无感知 |
---
## 问题汇总
| 编号 | 严重度 | 文件 | 问题 |
|---|---|---|---|
| C1 | Critical | `ingest_vault.py` | stale 清理删除 embed 失败的文档 |
| C2 | Critical | `incremental_ingest.py` | 未用 content_hash,每次都重新 embed |
| C3 | Critical | `install-hook.sh` | changes 临时文件永远不删 |
| C4 | Critical | `blacklist.py` | `api_key` 字面量过宽,49% 误隔离 |
| C5 | Critical | `index_common.py` | Windows 锁可能非阻塞 |
| I1 | Important | `schema.sql` | ivfflat lists=100 对 81 行无效 |
| I2 | Important | 两个 ingest 脚本 | 目录范围不一致 |
| I3 | Important | 两个 ingest 脚本 | 大文件静默截断 |
| I4 | Important | `agent-with-memory.sh` | VAULT_DIR 推导脆弱 |
| I5 | Important | `query_pgvector.py` | 无相似度阈值 |
| I6 | Important | `schema.sql` | audit 表不记录触发原因 |
| I7 | Important | `install-hook.sh` | hook 静默失败 |
| I8 | Important | `ingest_vault.py` | 全量索引持有超长事务 |
@@ -0,0 +1,52 @@
---
title: 运行状态
updated: 2026-02-25
---
# 运行状态
## 当前状态:已上线 ✅
系统于 2026-02-25 完成部署,核心链路全部验收通过。
## 指标
| 指标 | 目标 | 实测(2026-02-25| 状态 |
|---|---|---|---|
| Recall@5 | ≥ 70% | 90%9/10 样本)| ✅ 达标 |
| 增量 P95 延迟 | < 1s | 1.4s35 次样本)| ⚠️ 未达标 |
| 冷启动端到端 | ≤ 4.5s | 未完成量化 | 🔲 BLOCKED |
| 敏感内容隔离 | 红线 | 通过 | ✅ |
| 删除一致性 | 红线 | 通过 | ✅ |
| 重命名一致性 | 红线 | 通过 | ✅ |
| Prompt 注入防护 | 红线 | 通过 | ✅ |
索引规模:`memory_primary=96``memory_secure_audit=85`Infrastructure/Home-Automation 被路径规则整体隔离)
## 已知问题
### ⚠️ 增量 P95 = 1.4s(目标 < 1s
- 原因:OpenRouter API 网络延迟为主要瓶颈,P50 = 1.15s
- 影响:post-commit hook 异步执行,不阻塞提交,用户无感知
- 方向:可考虑批量 embedding 或换更快的 embedding 端点
### 🔲 冷启动延迟未量化(T6.3 BLOCKED
- 原因:`claude -p` 在自动化测量中超时(>60s),无法稳定完成 20 次采样
- 影响:无法验证 ≤ 4.5s 目标
- 方向:手动计时或换用非交互式测量方式
### ⚠️ 敏感内容漏检样例存在
- 现象:T5.3 验收时发现部分边缘样例未被 `blacklist.py` 捕获
- 影响:极少数敏感文档可能进入 `memory_primary`
- 方向:持续加固 `blacklist.py` 的正则规则
## 变更记录
| 日期 | 变更 |
|---|---|
| 2026-02-25 | 初始部署,完成全量索引(81 docs),安装 git hook,完成验收 |
| 2026-02-25 | 代码迁移至独立项目 `vault-memory-pgvector`,修复 Windows 跨平台锁 |
| 2026-02-26 | 重构 P1-1~3:删除 blacklist 过宽字面量、修复 stale 清理误删 embed 失败文档、hook 自动清理临时文件;部署方式更新为 docker compose |
+6
View File
@@ -0,0 +1,6 @@
---
kanban-plugin: board
---
+13 -13
View File
@@ -1,6 +1,6 @@
{ {
"name": "claudesidian", "name": "claudesidian",
"version": "0.13.1", "version": "0.14.2",
"description": "Claude Code + Obsidian Starter Kit - AI-powered second brain", "description": "Claude Code + Obsidian Starter Kit - AI-powered second brain",
"type": "module", "type": "module",
"scripts": { "scripts": {
@@ -41,21 +41,21 @@
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@google/generative-ai": "^0.21.0", "@google/generative-ai": "^0.21.0",
"@modelcontextprotocol/sdk": "^1.0.0" "@modelcontextprotocol/sdk": "^1.25.2"
}, },
"devDependencies": { "devDependencies": {
"@eslint/compat": "^1.3.0", "@eslint/compat": "^1.4.1",
"@typescript-eslint/eslint-plugin": "^8.35.0", "@typescript-eslint/eslint-plugin": "^8.53.0",
"@typescript-eslint/parser": "^8.35.0", "@typescript-eslint/parser": "^8.53.0",
"eslint": "^9.29.0", "eslint": "^9.39.2",
"eslint-plugin-check-file": "^3.3.0", "eslint-plugin-check-file": "^3.3.1",
"eslint-plugin-import": "^2.32.0", "eslint-plugin-import": "^2.32.0",
"eslint-plugin-perfectionist": "^4.15.0", "eslint-plugin-perfectionist": "^4.15.1",
"globals": "^16.2.0", "globals": "^16.5.0",
"prettier": "^3.6.0", "prettier": "^3.7.4",
"prettier-plugin-packagejson": "^2.5.15", "prettier-plugin-packagejson": "^2.5.21",
"prettier-plugin-sort-json": "^4.1.1", "prettier-plugin-sort-json": "^4.2.0",
"typescript-eslint": "^8.35.0" "typescript-eslint": "^8.53.0"
}, },
"packageManager": "pnpm@9.15.4" "packageManager": "pnpm@9.15.4"
} }