my-vault/06_Metadata/claudian-token-optimization-testing.md

Claudian Token Optimization - Testing Guide

Implementation Summary

Date: 2026-01-08 Phase: Phase 1 - Quick Wins Status: ✅ Code changes completed, awaiting user testing

---

Changes Made

1. System Prompt Compression (~800 tokens saved)

Section	Before	After	Savings
Identity & Role	7 lines verbose	2 lines concise	~150 tokens
Path Rules	9 lines with examples	1 line	~100 tokens
Message Format	14 lines with explanations	3 lines examples only	~80 tokens
Obsidian Context	7 lines detailed	2 bullet points	~70 tokens
Tool Guidelines	95 lines verbose	6 bullet points	~400 tokens

Total System Prompt: Reduced from ~2,400 tokens → ~1,200 tokens (50% reduction)

2. History Windowing (10,000-25,000 tokens saved)

Location: buildContextFromHistory() function (line 22465)

Logic:

• Limit to 15 most recent messages
• Always preserve first message (context establishment)
• Add truncation notice when messages omitted

Example:

100 message conversation:
Before: All 100 messages sent (~40,000 tokens)
After: First + last 15 messages (16 total, ~6,500 tokens)
Savings: 33,500 tokens (84% reduction)

3. Tool Result Truncation (300-500 tokens saved)

Location: truncateToolResult() function (line 22453)

Change: Max length 800 → 300 characters

Impact: Tool results in history rebuilds are more aggressive truncated

4. Optional Sections Removed (600-800 tokens saved)

Location: buildSystemPrompt() function (line 23471)

Commented out:

• Image instructions (~400 tokens) - line 23474
• Editor context instructions (~150 tokens) - lines 23480-23482

---

Testing Instructions

Step 1: Reload Plugin in Obsidian

1. Open Obsidian
2. Navigate to Settings → Community Plugins
3. Find Claudian plugin
4. Click Disable (wait 2-3 seconds)
5. Click Enable
6. Close Settings

Expected: Plugin enables without errors

Step 2: Basic Functionality Test

Test 1: New Conversation

1. Start a new Claudian conversation
2. Send a simple message: "List files in the vault root"
3. Verify Claudian responds normally

Expected: Response should be coherent and functional

Test 2: File Operations

1. Ask: "Read the file 00_Inbox/2026-01-07.md"
2. Verify file is read correctly
3. Ask: "What's in this note?"

Expected: Claudian can read files and understand content

Test 3: Existing Conversation

1. Open an existing conversation (preferably 20+ messages)
2. Send a new message
3. Verify response is coherent

Expected: Plugin handles existing conversations correctly

Step 3: Token Usage Verification

Check Console for Token Info:

1. Press Ctrl+Shift+I (or Cmd+Option+I on Mac) to open DevTools
2. Click Console tab
3. Send a message in Claudian
4. Look for token usage info in console output

What to Look For:

• inputTokens: Should be significantly lower
• contextTokens: Total tokens used
• Compare with previous conversations (if you remember typical values)

Expected Reductions:

• Short conversations (5 msgs): ~1,800 tokens total
• Medium conversations (20-30 msgs): ~6,500 tokens total
• Long conversations (50+ msgs): ~8,000-10,000 tokens total

Step 4: Quality Checks

Test Understanding of Obsidian Context:

1. Ask: "Create a note in 00_Inbox with today's date"
2. Ask: "Add a wiki-link to another note"
3. Verify Claudian still understands Obsidian concepts

Expected: Should still understand:

• Markdown formatting
• Wiki-links note
• Vault structure
• Relative paths

Test Tool Usage:

1. Ask: "Find all markdown files with the word 'TODO'"
2. Ask: "Read the first result"
3. Ask: "Update it to remove the TODO"

Expected: Should correctly use Read, Grep, Edit tools

Step 5: Edge Cases

Test 1: Long Conversation with Session Break

1. Open a conversation with 30+ messages
2. Send a message that requires context from early messages
3. Observe if Claudian maintains enough context

Expected: May not remember details from messages 16-29 (that were windowed out), but should handle gracefully with truncation notice

Test 2: Current Note Context

1. Open a note in Obsidian
2. Open Claudian sidebar
3. Send a message referencing "this note"

Expected: Should still understand current note context

---

Success Criteria

✅ Must Pass

• Plugin loads without errors
• Can start new conversations
• Can send/receive messages
• File operations work (Read, Write, Edit)
• Tool calls execute correctly
• Existing conversations load properly
• Token usage is reduced (check console)

✅ Should Pass

• Response quality remains high
• Understands Obsidian concepts
• Relative paths work correctly
• No functionality lost

⚠️ Acceptable Trade-offs

• May forget context from messages 16+ back (windowed out)
• Less verbose explanations
• Slightly less hand-holding in responses

---

Troubleshooting

Issue: Plugin Won't Load

Symptoms: Error on enable, plugin stays disabled

Fix:

# Restore backup
cd /path/to/vault
cp .obsidian/plugins/claudian/main.js.backup .obsidian/plugins/claudian/main.js

Then reload Obsidian

Issue: Errors in Console

Symptoms: JavaScript errors in console after sending message

What to do:

1. Copy the full error message
2. Check which function is failing
3. Restore backup if critical

Issue: Poor Response Quality

Symptoms: Claudian doesn't understand Obsidian concepts, paths are wrong

Possible causes:

• System prompt too compressed
• Missing essential instructions

Fix: Can restore specific sections (e.g., uncomment image instructions if needed)

Issue: Still Using Too Many Tokens

Symptoms: Token usage not significantly reduced

Check:

1. Is session resumption working? (Should use persistent sessions normally)
2. Are you testing with session breaks? (History windowing only applies when session breaks)
3. Check if custom instructions in settings add lots of tokens

---

Rollback Procedure

If anything breaks:

1. Full Rollback:

cp .obsidian/plugins/claudian/main.js.backup .obsidian/plugins/claudian/main.js

2. Reload Plugin:

   • Settings → Community Plugins
   • Disable Claudian
   • Enable Claudian

3. Verify: Test that original version works

---

Measuring Token Reduction

Before Optimization (Typical Values)

System Prompt: ~3,200-6,500 tokens History (30 msg conversation with session break): ~15,000-18,000 tokens Total: ~18,000-24,000 tokens per request

After Optimization (Expected Values)

System Prompt: ~1,600-2,000 tokens History (30 msg conversation with session break): ~6,000-8,000 tokens Total: ~7,600-10,000 tokens per request

Overall Reduction: ~60-70% for medium/long conversations

---

Phase 2 Preview (Future Enhancements)

If Phase 1 is successful, Phase 2 will add:

1. User Settings:

   • Configurable history window size (15-30 messages)
   • Tool result max length setting
   • Toggle image/editor instructions

2. Smart Message Selection:

   • Preserve messages with tool calls
   • Preserve longer/important messages
   • Dynamic window sizing

3. Token Warnings:

   • Visual warning at 50% context usage
   • Auto-suggest starting new conversation at 80%

4. Token Usage UI:

   • Show breakdown: System | History | Current
   • Display in conversation header

---

Notes

• All changes are backwards compatible
• JSONL conversation files unchanged
• Session management logic unchanged
• Can selectively restore sections if needed
• Backup file preserved at: .obsidian/plugins/claudian/main.js.backup

---

Testing Checklist

Print this section and check off as you test:

Basic Tests:
[ ] Plugin loads without errors
[ ] New conversation works
[ ] Can read files
[ ] Can write files
[ ] Existing conversation loads
[ ] Token usage reduced (check console)

Quality Tests:
[ ] Understands Markdown
[ ] Understands Wiki-links
[ ] Uses relative paths correctly
[ ] Tool calls work
[ ] Response quality good

Edge Cases:
[ ] Long conversation (30+ msgs)
[ ] Session break handling
[ ] Current note context
[ ] File operations in subfolders

If all checked: ✅ Phase 1 Complete!
If issues found: Document below and consider rollback/adjustments

---

Feedback Section

Date Tested: _____________

Token Reduction Observed: _______% (compare console before/after)

Issues Found:

Quality Assessment:

• Response coherence: ___/10
• Obsidian understanding: ___/10
• Tool usage: ___/10
• Overall satisfaction: ___/10

Recommendation:

• Keep Phase 1 changes, proceed to Phase 2
• Keep changes, adjust window size to: ___
• Restore specific section: ____________
• Full rollback needed