my-vault/06_Metadata/claudian-token-optimization-testing.md

# Claudian Token Optimization - Testing Guide

## Implementation Summary

**Date**: 2026-01-08
**Phase**: Phase 1 - Quick Wins
**Status**: ✅ Code changes completed, awaiting user testing

---

## Changes Made

### 1. System Prompt Compression (~800 tokens saved)

| Section | Before | After | Savings |
|---------|--------|-------|---------|
| Identity & Role | 7 lines verbose | 2 lines concise | ~150 tokens |
| Path Rules | 9 lines with examples | 1 line | ~100 tokens |
| Message Format | 14 lines with explanations | 3 lines examples only | ~80 tokens |
| Obsidian Context | 7 lines detailed | 2 bullet points | ~70 tokens |
| Tool Guidelines | 95 lines verbose | 6 bullet points | ~400 tokens |

**Total System Prompt**: Reduced from ~2,400 tokens → ~1,200 tokens (50% reduction)

### 2. History Windowing (10,000-25,000 tokens saved)

**Location**: `buildContextFromHistory()` function (line 22465)

**Logic**:
- Limit to **15 most recent messages**
- Always preserve **first message** (context establishment)
- Add truncation notice when messages omitted

**Example**:
```
100 message conversation:
Before: All 100 messages sent (~40,000 tokens)
After: First + last 15 messages (16 total, ~6,500 tokens)
Savings: 33,500 tokens (84% reduction)
```

### 3. Tool Result Truncation (300-500 tokens saved)

**Location**: `truncateToolResult()` function (line 22453)

**Change**: Max length 800 → 300 characters

**Impact**: Tool results in history rebuilds are more aggressive truncated

### 4. Optional Sections Removed (600-800 tokens saved)

**Location**: `buildSystemPrompt()` function (line 23471)

**Commented out**:
- Image instructions (~400 tokens) - line 23474
- Editor context instructions (~150 tokens) - lines 23480-23482

---

## Testing Instructions

### Step 1: Reload Plugin in Obsidian

1. Open Obsidian
2. Navigate to **Settings → Community Plugins**
3. Find **Claudian** plugin
4. Click **Disable** (wait 2-3 seconds)
5. Click **Enable**
6. Close Settings

**Expected**: Plugin enables without errors

### Step 2: Basic Functionality Test

**Test 1: New Conversation**
1. Start a new Claudian conversation
2. Send a simple message: "List files in the vault root"
3. Verify Claudian responds normally

**Expected**: Response should be coherent and functional

**Test 2: File Operations**
1. Ask: "Read the file 00_Inbox/2026-01-07.md"
2. Verify file is read correctly
3. Ask: "What's in this note?"

**Expected**: Claudian can read files and understand content

**Test 3: Existing Conversation**
1. Open an existing conversation (preferably 20+ messages)
2. Send a new message
3. Verify response is coherent

**Expected**: Plugin handles existing conversations correctly

### Step 3: Token Usage Verification

**Check Console for Token Info**:
1. Press **Ctrl+Shift+I** (or Cmd+Option+I on Mac) to open DevTools
2. Click **Console** tab
3. Send a message in Claudian
4. Look for token usage info in console output

**What to Look For**:
- `inputTokens`: Should be significantly lower
- `contextTokens`: Total tokens used
- Compare with previous conversations (if you remember typical values)

**Expected Reductions**:
- Short conversations (5 msgs): ~1,800 tokens total
- Medium conversations (20-30 msgs): ~6,500 tokens total
- Long conversations (50+ msgs): ~8,000-10,000 tokens total

### Step 4: Quality Checks

**Test Understanding of Obsidian Context**:
1. Ask: "Create a note in 00_Inbox with today's date"
2. Ask: "Add a wiki-link to another note"
3. Verify Claudian still understands Obsidian concepts

**Expected**: Should still understand:
- Markdown formatting
- Wiki-links [[note]]
- Vault structure
- Relative paths

**Test Tool Usage**:
1. Ask: "Find all markdown files with the word 'TODO'"
2. Ask: "Read the first result"
3. Ask: "Update it to remove the TODO"

**Expected**: Should correctly use Read, Grep, Edit tools

### Step 5: Edge Cases

**Test 1: Long Conversation with Session Break**
1. Open a conversation with 30+ messages
2. Send a message that requires context from early messages
3. Observe if Claudian maintains enough context

**Expected**: May not remember details from messages 16-29 (that were windowed out), but should handle gracefully with truncation notice

**Test 2: Current Note Context**
1. Open a note in Obsidian
2. Open Claudian sidebar
3. Send a message referencing "this note"

**Expected**: Should still understand current note context

---

## Success Criteria

### ✅ Must Pass
- [ ] Plugin loads without errors
- [ ] Can start new conversations
- [ ] Can send/receive messages
- [ ] File operations work (Read, Write, Edit)
- [ ] Tool calls execute correctly
- [ ] Existing conversations load properly
- [ ] Token usage is reduced (check console)

### ✅ Should Pass
- [ ] Response quality remains high
- [ ] Understands Obsidian concepts
- [ ] Relative paths work correctly
- [ ] No functionality lost

### ⚠️ Acceptable Trade-offs
- [ ] May forget context from messages 16+ back (windowed out)
- [ ] Less verbose explanations
- [ ] Slightly less hand-holding in responses

---

## Troubleshooting

### Issue: Plugin Won't Load

**Symptoms**: Error on enable, plugin stays disabled

**Fix**:
```bash
# Restore backup
cd /path/to/vault
cp .obsidian/plugins/claudian/main.js.backup .obsidian/plugins/claudian/main.js
```
Then reload Obsidian

### Issue: Errors in Console

**Symptoms**: JavaScript errors in console after sending message

**What to do**:
1. Copy the full error message
2. Check which function is failing
3. Restore backup if critical

### Issue: Poor Response Quality

**Symptoms**: Claudian doesn't understand Obsidian concepts, paths are wrong

**Possible causes**:
- System prompt too compressed
- Missing essential instructions

**Fix**: Can restore specific sections (e.g., uncomment image instructions if needed)

### Issue: Still Using Too Many Tokens

**Symptoms**: Token usage not significantly reduced

**Check**:
1. Is session resumption working? (Should use persistent sessions normally)
2. Are you testing with session breaks? (History windowing only applies when session breaks)
3. Check if custom instructions in settings add lots of tokens

---

## Rollback Procedure

If anything breaks:

1. **Full Rollback**:
```bash
cp .obsidian/plugins/claudian/main.js.backup .obsidian/plugins/claudian/main.js
```

2. **Reload Plugin**:
   - Settings → Community Plugins
   - Disable Claudian
   - Enable Claudian

3. **Verify**: Test that original version works

---

## Measuring Token Reduction

### Before Optimization (Typical Values)

**System Prompt**: ~3,200-6,500 tokens
**History (30 msg conversation with session break)**: ~15,000-18,000 tokens
**Total**: ~18,000-24,000 tokens per request

### After Optimization (Expected Values)

**System Prompt**: ~1,600-2,000 tokens
**History (30 msg conversation with session break)**: ~6,000-8,000 tokens
**Total**: ~7,600-10,000 tokens per request

**Overall Reduction**: ~60-70% for medium/long conversations

---

## Phase 2 Preview (Future Enhancements)

If Phase 1 is successful, Phase 2 will add:

1. **User Settings**:
   - Configurable history window size (15-30 messages)
   - Tool result max length setting
   - Toggle image/editor instructions

2. **Smart Message Selection**:
   - Preserve messages with tool calls
   - Preserve longer/important messages
   - Dynamic window sizing

3. **Token Warnings**:
   - Visual warning at 50% context usage
   - Auto-suggest starting new conversation at 80%

4. **Token Usage UI**:
   - Show breakdown: System | History | Current
   - Display in conversation header

---

## Notes

- All changes are backwards compatible
- JSONL conversation files unchanged
- Session management logic unchanged
- Can selectively restore sections if needed
- Backup file preserved at: `.obsidian/plugins/claudian/main.js.backup`

---

## Testing Checklist

Print this section and check off as you test:

```
Basic Tests:
[ ] Plugin loads without errors
[ ] New conversation works
[ ] Can read files
[ ] Can write files
[ ] Existing conversation loads
[ ] Token usage reduced (check console)

Quality Tests:
[ ] Understands Markdown
[ ] Understands Wiki-links
[ ] Uses relative paths correctly
[ ] Tool calls work
[ ] Response quality good

Edge Cases:
[ ] Long conversation (30+ msgs)
[ ] Session break handling
[ ] Current note context
[ ] File operations in subfolders

If all checked: ✅ Phase 1 Complete!
If issues found: Document below and consider rollback/adjustments
```

---

## Feedback Section

**Date Tested**: _____________

**Token Reduction Observed**: _______% (compare console before/after)

**Issues Found**:
-
-
-

**Quality Assessment**:
- Response coherence: ___/10
- Obsidian understanding: ___/10
- Tool usage: ___/10
- Overall satisfaction: ___/10

**Recommendation**:
- [ ] Keep Phase 1 changes, proceed to Phase 2
- [ ] Keep changes, adjust window size to: ___
- [ ] Restore specific section: ____________
- [ ] Full rollback needed