Initial commit - Claudesidian v0.2.0
Claude Code + Obsidian starter kit for AI-powered knowledge management. Features: - PARA method folder structure - Bootstrap initialization system - Pre-configured Claude Code commands and agents - Gemini Vision MCP server with video support - Helper scripts for vault management - Automated release management See README.md for setup instructions.
This commit is contained in:
Noah Brier
@@ -0,0 +1,285 @@
|
||||
# Gemini Vision MCP Server - Quick Start Guide
|
||||
|
||||
**For getting Gemini Vision working on a new machine in under 5 minutes**
|
||||
|
||||
## Prerequisites Check
|
||||
|
||||
Run these commands to verify you have everything needed:
|
||||
```bash
|
||||
node --version # Should be v22+
|
||||
pnpm --version # Should be installed
|
||||
claude --version # Claude Code should be installed
|
||||
```
|
||||
|
||||
If any are missing:
|
||||
- Node.js: Install from [nodejs.org](https://nodejs.org/) (v22+)
|
||||
- pnpm: `npm install -g pnpm`
|
||||
- Claude Code: Download from [claude.ai/code](https://claude.ai/code)
|
||||
|
||||
## Step 1: Get Your Gemini API Key
|
||||
|
||||
1. Go to [https://aistudio.google.com/apikey](https://aistudio.google.com/apikey)
|
||||
2. Click "Create API Key"
|
||||
3. Copy the key (starts with `AIzaSy...`)
|
||||
|
||||
## Step 2: Set Up Environment Variable
|
||||
|
||||
### For Linux/macOS with Bash:
|
||||
```bash
|
||||
echo 'export GEMINI_API_KEY="your-actual-api-key-here"' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
echo $GEMINI_API_KEY # Verify it shows your key
|
||||
```
|
||||
|
||||
### For Linux/macOS with Zsh:
|
||||
```bash
|
||||
echo 'export GEMINI_API_KEY="your-actual-api-key-here"' >> ~/.zshrc
|
||||
source ~/.zshrc
|
||||
echo $GEMINI_API_KEY # Verify it shows your key
|
||||
```
|
||||
|
||||
### For Windows PowerShell:
|
||||
```powershell
|
||||
[System.Environment]::SetEnvironmentVariable('GEMINI_API_KEY', 'your-key-here', 'User')
|
||||
# Restart PowerShell
|
||||
$env:GEMINI_API_KEY # Verify it shows your key
|
||||
```
|
||||
|
||||
## Step 3: Install Dependencies
|
||||
|
||||
**⚠️ CRITICAL: This step MUST be done before adding the MCP server!**
|
||||
|
||||
Navigate to your Obsidian vault:
|
||||
```bash
|
||||
cd ~/dev/02_Areas/Obsidian # Or wherever your vault is
|
||||
```
|
||||
|
||||
Install the required dependencies:
|
||||
```bash
|
||||
# Install npm packages (REQUIRED - do this first!)
|
||||
pnpm install
|
||||
|
||||
# This installs:
|
||||
# - @google/generative-ai (Gemini API client)
|
||||
# - @modelcontextprotocol/sdk (MCP server framework)
|
||||
# - Other dependencies from package.json
|
||||
```
|
||||
|
||||
**Common Error Fix**: If you see `Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@modelcontextprotocol/sdk'`, you forgot to run `pnpm install`!
|
||||
|
||||
**To hide node_modules from Obsidian** (optional but recommended):
|
||||
1. Open Obsidian
|
||||
2. Go to Settings → Files & Links → Excluded files
|
||||
3. Click "Manage"
|
||||
4. Add `node_modules/` to the list
|
||||
5. Optionally also add: `pnpm-lock.yaml`, `.gitignore`
|
||||
|
||||
This keeps your vault clean while using standard Node.js module resolution.
|
||||
|
||||
## Step 4: Register the MCP Server
|
||||
|
||||
**For project-scoped installation (recommended for team use):**
|
||||
```bash
|
||||
# Add server to project (creates .mcp.json file)
|
||||
claude mcp add --scope project gemini-vision node .claude/mcp-servers/gemini-vision.mjs
|
||||
```
|
||||
|
||||
**For user-scoped installation (personal use across all projects):**
|
||||
```bash
|
||||
# Add server to your user config
|
||||
claude mcp add --scope user gemini-vision node .claude/mcp-servers/gemini-vision.mjs
|
||||
```
|
||||
|
||||
After adding, you'll need to edit the `.mcp.json` file to add your API key:
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"gemini-vision": {
|
||||
"type": "stdio",
|
||||
"command": "node",
|
||||
"args": [".claude/mcp-servers/gemini-vision.mjs"],
|
||||
"env": {
|
||||
"GEMINI_API_KEY": "your-api-key-here"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**IMPORTANT**:
|
||||
- The command must be run from the Obsidian vault root directory
|
||||
- You MUST have run `pnpm install` first
|
||||
- The `.mcp.json` file is gitignored for security
|
||||
|
||||
## Step 5: Verify It's Working
|
||||
|
||||
1. **Open a NEW Claude Code window** (critical - must be new):
|
||||
```bash
|
||||
cd ~/dev/Obsidian
|
||||
claude
|
||||
```
|
||||
|
||||
2. **Check the server is connected**:
|
||||
Type `/mcp` in Claude
|
||||
|
||||
You should see:
|
||||
```
|
||||
gemini-vision ✔ connected
|
||||
```
|
||||
|
||||
3. **Test with an actual command**:
|
||||
```
|
||||
Use gemini-vision to extract text from 05 Attachments/[any-image.png]
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "gemini-vision failed" or not showing in /mcp
|
||||
|
||||
1. **MOST COMMON ISSUE - Dependencies not installed**:
|
||||
```bash
|
||||
# If you see: Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@modelcontextprotocol/sdk'
|
||||
# Run this:
|
||||
pnpm install
|
||||
```
|
||||
Then reconnect the MCP server in Claude Code.
|
||||
|
||||
2. **Check API key is configured**:
|
||||
- For project-scoped: Check `.mcp.json` has your API key in the env section
|
||||
- For user-scoped: Check `~/.claude.json` has your API key
|
||||
- The key should be in the format: `"GEMINI_API_KEY": "AIzaSy..."`
|
||||
|
||||
3. **Test server can run directly**:
|
||||
```bash
|
||||
export GEMINI_API_KEY="your-api-key-here"
|
||||
node .claude/mcp-servers/gemini-vision.mjs
|
||||
```
|
||||
Should show: "🚀 Gemini Vision MCP Server running"
|
||||
Press Ctrl+C to exit.
|
||||
|
||||
4. **Re-add the server (for project scope)**:
|
||||
```bash
|
||||
claude mcp remove gemini-vision --scope project
|
||||
claude mcp add --scope project gemini-vision node .claude/mcp-servers/gemini-vision.mjs
|
||||
# Then edit .mcp.json to add your API key
|
||||
```
|
||||
|
||||
4. **Check logs**:
|
||||
```bash
|
||||
# Find log directory
|
||||
ls ~/Library/Caches/claude-cli-nodejs/*/mcp-logs-gemini-vision/
|
||||
# Or on Linux:
|
||||
ls ~/.cache/claude-cli-nodejs/*/mcp-logs-gemini-vision/
|
||||
|
||||
# View latest log
|
||||
tail -f [log-directory]/*.txt
|
||||
```
|
||||
|
||||
### "Cannot find module" errors
|
||||
|
||||
1. **Verify package.json exists**:
|
||||
```bash
|
||||
cat package.json
|
||||
```
|
||||
Should show @google/generative-ai and @modelcontextprotocol/sdk
|
||||
|
||||
2. **Reinstall dependencies**:
|
||||
```bash
|
||||
rm -rf node_modules pnpm-lock.yaml
|
||||
pnpm install
|
||||
```
|
||||
|
||||
3. **Check node_modules was created**:
|
||||
```bash
|
||||
ls node_modules/@google/generative-ai
|
||||
```
|
||||
|
||||
### Server runs but tools don't work
|
||||
|
||||
1. **Test API key directly**:
|
||||
```bash
|
||||
curl "https://generativelanguage.googleapis.com/v1beta/models?key=$GEMINI_API_KEY"
|
||||
```
|
||||
Should return a list of models, not an error.
|
||||
|
||||
2. **Check file paths**:
|
||||
- Use absolute paths from vault root
|
||||
- Example: `05 Attachments/image.png` not `./05 Attachments/image.png`
|
||||
|
||||
## Available Tools
|
||||
|
||||
Once working, you can use these in Claude:
|
||||
|
||||
### Image Analysis
|
||||
```
|
||||
# Analyze an image
|
||||
Use gemini-vision to analyze 05 Attachments/screenshot.png
|
||||
|
||||
# Extract text (OCR)
|
||||
Use gemini-vision to extract text from 05 Attachments/document.jpg
|
||||
|
||||
# Compare images
|
||||
Use gemini-vision to compare image1.png and image2.png
|
||||
|
||||
# Suggest a filename
|
||||
Use gemini-vision to suggest a filename for IMG_1234.jpg
|
||||
|
||||
# Analyze multiple images
|
||||
Use gemini-vision to analyze multiple: image1.png, image2.png, image3.png
|
||||
```
|
||||
|
||||
### Video Analysis (NEW!)
|
||||
```
|
||||
# Analyze a local video file
|
||||
Use gemini-vision to analyze video 05 Attachments/video.mp4
|
||||
|
||||
# Analyze a YouTube video
|
||||
Use gemini-vision to analyze YouTube video https://www.youtube.com/watch?v=VIDEO_ID
|
||||
|
||||
# Custom video analysis prompt
|
||||
Use gemini-vision to analyze video file.mp4 and extract all visible text
|
||||
```
|
||||
|
||||
**Note:** Video processing may take 30-60 seconds as files need to reach ACTIVE state before analysis. The server will automatically wait and show progress updates.
|
||||
|
||||
### Supported Formats
|
||||
|
||||
**Images:** JPG, JPEG, PNG, GIF, BMP, WebP
|
||||
**Videos:** MP4, AVI, MOV, WebM, MKV, WMV, FLV, 3GP, M4V
|
||||
**Documents:** PDF, TXT, DOC, DOCX, ODT, RTF
|
||||
**Special:** YouTube URLs (direct support without download)
|
||||
|
||||
## Quick Reinstall (If Already Set Up Once)
|
||||
|
||||
If you've already set up the API key in your shell profile:
|
||||
|
||||
```bash
|
||||
cd ~/dev/Obsidian
|
||||
git pull
|
||||
pnpm install
|
||||
claude mcp add gemini-vision \
|
||||
--scope local \
|
||||
--env GEMINI_API_KEY=$GEMINI_API_KEY \
|
||||
-- node .claude/mcp-servers/gemini-vision.mjs
|
||||
```
|
||||
|
||||
Then open a new Claude window and test.
|
||||
|
||||
## File Locations
|
||||
|
||||
- **Server code**: `.claude/mcp-servers/gemini-vision.mjs`
|
||||
- **Dependencies**: `package.json`
|
||||
- **This guide**: `07 Readme/GEMINI_VISION_QUICK_START.md`
|
||||
- **Detailed docs**: `07 Readme/GEMINI_VISION_INSTALLATION.md`
|
||||
- **Development guide**: `07 Readme/MCP_DEVELOPMENT_GUIDE.md`
|
||||
|
||||
## Need Help?
|
||||
|
||||
1. Check the troubleshooting section above
|
||||
2. Verify all prerequisites are installed
|
||||
3. Make sure you're in the Obsidian vault root directory
|
||||
4. Ensure the API key is properly set in your environment
|
||||
|
||||
---
|
||||
|
||||
*Last tested: September 2025*
|
||||
@@ -0,0 +1,104 @@
|
||||
# MCP Servers
|
||||
|
||||
Model Context Protocol servers extend Claude Code's capabilities.
|
||||
|
||||
## Gemini Vision MCP
|
||||
|
||||
Adds powerful image and document analysis capabilities using Google's Gemini model.
|
||||
|
||||
### Features
|
||||
|
||||
- **Image Analysis**: Describe, analyze, and extract text from images
|
||||
- **Document Processing**: Analyze PDFs and documents
|
||||
- **Multi-Image Comparison**: Compare multiple images at once
|
||||
- **OCR**: Extract text from images
|
||||
- **Smart Filename Suggestions**: Generate descriptive filenames for images
|
||||
|
||||
### Setup
|
||||
|
||||
1. **Get a Gemini API Key**
|
||||
- Visit: https://aistudio.google.com/apikey
|
||||
- Create a free API key
|
||||
|
||||
2. **Add to Environment**
|
||||
```bash
|
||||
# Add to ~/.zshrc or ~/.bashrc
|
||||
export GEMINI_API_KEY='your-key-here'
|
||||
|
||||
# Reload shell
|
||||
source ~/.zshrc
|
||||
```
|
||||
|
||||
3. **Install Dependencies**
|
||||
```bash
|
||||
pnpm install
|
||||
```
|
||||
|
||||
4. **Test Setup**
|
||||
```bash
|
||||
pnpm test-gemini
|
||||
```
|
||||
|
||||
### Available Commands
|
||||
|
||||
Once configured, these commands become available in Claude Code:
|
||||
|
||||
- `mcp__gemini-vision__analyze_image` - Analyze a single image
|
||||
- `mcp__gemini-vision__analyze_multiple` - Compare multiple images
|
||||
- `mcp__gemini-vision__extract_text` - OCR text extraction
|
||||
- `mcp__gemini-vision__compare_images` - Compare two images
|
||||
- `mcp__gemini-vision__suggest_image_filename` - Generate descriptive filename
|
||||
- `mcp__gemini-vision__analyze_document` - Analyze PDFs and documents
|
||||
|
||||
### Usage Examples
|
||||
|
||||
**Analyze Screenshot**
|
||||
```
|
||||
Analyze the image at 05_Attachments/screenshot.png
|
||||
and tell me what it contains.
|
||||
```
|
||||
|
||||
**Process Multiple Images**
|
||||
```
|
||||
Compare all images in 05_Attachments/Organized/
|
||||
and identify common themes.
|
||||
```
|
||||
|
||||
**Extract Text**
|
||||
```
|
||||
Extract all text from the PDF at
|
||||
05_Attachments/document.pdf
|
||||
```
|
||||
|
||||
**Rename Images**
|
||||
```
|
||||
Suggest better names for all images
|
||||
in 05_Attachments/ based on their content.
|
||||
```
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
**"GEMINI_API_KEY not found"**
|
||||
- Make sure you've added the key to your shell profile
|
||||
- Restart your terminal and Claude Code
|
||||
|
||||
**"File not found"**
|
||||
- Use absolute paths or paths relative to vault root
|
||||
- Check file permissions
|
||||
|
||||
**Rate Limits**
|
||||
- Free tier: 15 requests per minute
|
||||
- Consider upgrading for heavy usage
|
||||
|
||||
## Adding More MCPs
|
||||
|
||||
1. Place MCP server file in `.claude/mcp-servers/`
|
||||
2. Add configuration to Claude settings
|
||||
3. Document setup here
|
||||
4. Add usage examples
|
||||
|
||||
## Resources
|
||||
|
||||
- [MCP Documentation](https://modelcontextprotocol.io)
|
||||
- [Gemini API Docs](https://ai.google.dev)
|
||||
- [Claude Code MCP Guide](https://claude.ai/docs/mcp)
|
||||
@@ -0,0 +1,430 @@
|
||||
#!/usr/bin/env node
|
||||
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
|
||||
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
|
||||
import {
|
||||
ListToolsRequestSchema,
|
||||
CallToolRequestSchema
|
||||
} from "@modelcontextprotocol/sdk/types.js";
|
||||
import { GoogleGenerativeAI } from "@google/generative-ai";
|
||||
import { GoogleAIFileManager } from "@google/generative-ai/server";
|
||||
import fs from "fs/promises";
|
||||
import path from "path";
|
||||
import os from "os";
|
||||
|
||||
const apiKey = process.env.GEMINI_API_KEY;
|
||||
if (!apiKey) {
|
||||
console.error("❌ GEMINI_API_KEY environment variable is required");
|
||||
console.error("");
|
||||
console.error("To fix this:");
|
||||
console.error("");
|
||||
console.error("1. Get your API key from: https://aistudio.google.com/apikey");
|
||||
console.error("");
|
||||
console.error("2. Add to your shell profile:");
|
||||
console.error(" For macOS/Linux (add to ~/.zshrc or ~/.bashrc):");
|
||||
console.error(" export GEMINI_API_KEY='your-actual-api-key-here'");
|
||||
console.error("");
|
||||
console.error(" For Windows PowerShell:");
|
||||
console.error(" [System.Environment]::SetEnvironmentVariable('GEMINI_API_KEY', 'your-key', 'User')");
|
||||
console.error("");
|
||||
console.error("3. Reload your terminal:");
|
||||
console.error(" source ~/.zshrc (or source ~/.bashrc)");
|
||||
console.error("");
|
||||
console.error("4. Restart Claude Code");
|
||||
console.error("");
|
||||
console.error("For detailed instructions, see GEMINI_VISION_SETUP.md");
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const genAI = new GoogleGenerativeAI(apiKey);
|
||||
const fileManager = new GoogleAIFileManager(apiKey);
|
||||
const model = genAI.getGenerativeModel({ model: "gemini-2.5-flash" });
|
||||
|
||||
// Expand home directory in paths
|
||||
function expandPath(filepath) {
|
||||
if (filepath.startsWith("~/")) {
|
||||
return path.join(os.homedir(), filepath.slice(2));
|
||||
}
|
||||
return filepath;
|
||||
}
|
||||
|
||||
// Helper function to wait/sleep
|
||||
function sleep(ms) {
|
||||
return new Promise(resolve => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
// Upload file to Gemini
|
||||
async function uploadFile(filePath) {
|
||||
const expandedPath = expandPath(filePath);
|
||||
|
||||
try {
|
||||
await fs.access(expandedPath);
|
||||
} catch {
|
||||
throw new Error(`File not found: ${filePath}`);
|
||||
}
|
||||
|
||||
const ext = path.extname(expandedPath).toLowerCase();
|
||||
const mimeTypes = {
|
||||
'.jpg': 'image/jpeg',
|
||||
'.jpeg': 'image/jpeg',
|
||||
'.png': 'image/png',
|
||||
'.gif': 'image/gif',
|
||||
'.bmp': 'image/bmp',
|
||||
'.webp': 'image/webp',
|
||||
'.pdf': 'application/pdf',
|
||||
'.txt': 'text/plain',
|
||||
'.doc': 'application/msword',
|
||||
'.docx': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'.odt': 'application/vnd.oasis.opendocument.text',
|
||||
'.rtf': 'application/rtf',
|
||||
// Video formats
|
||||
'.mp4': 'video/mp4',
|
||||
'.avi': 'video/x-msvideo',
|
||||
'.mov': 'video/quicktime',
|
||||
'.webm': 'video/webm',
|
||||
'.mkv': 'video/x-matroska',
|
||||
'.wmv': 'video/x-ms-wmv',
|
||||
'.flv': 'video/x-flv',
|
||||
'.3gp': 'video/3gpp',
|
||||
'.m4v': 'video/x-m4v',
|
||||
};
|
||||
|
||||
const uploadResult = await fileManager.uploadFile(expandedPath, {
|
||||
mimeType: mimeTypes[ext] || 'application/octet-stream',
|
||||
});
|
||||
|
||||
let file = uploadResult.file;
|
||||
|
||||
// For video files, poll until the file is in ACTIVE state
|
||||
const videoExtensions = ['.mp4', '.avi', '.mov', '.webm', '.mkv', '.wmv', '.flv', '.3gp', '.m4v'];
|
||||
if (videoExtensions.includes(ext)) {
|
||||
console.error(`Waiting for video file to process: ${path.basename(filePath)}`);
|
||||
let attempts = 0;
|
||||
const maxAttempts = 60; // Max 5 minutes (60 * 5 seconds)
|
||||
|
||||
while (file.state !== 'ACTIVE' && attempts < maxAttempts) {
|
||||
await sleep(5000); // Wait 5 seconds
|
||||
attempts++;
|
||||
|
||||
// Get updated file status
|
||||
const fileStatus = await fileManager.getFile(file.name);
|
||||
file = fileStatus;
|
||||
|
||||
console.error(`Video processing status: ${file.state} (attempt ${attempts}/${maxAttempts})`);
|
||||
|
||||
if (file.state === 'FAILED') {
|
||||
throw new Error(`Video processing failed for: ${filePath}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (file.state !== 'ACTIVE') {
|
||||
throw new Error(`Video processing timeout for: ${filePath}. File state: ${file.state}`);
|
||||
}
|
||||
|
||||
console.error('Video file is ready for analysis');
|
||||
}
|
||||
|
||||
return file;
|
||||
}
|
||||
|
||||
// Tool handlers
|
||||
async function analyzeImage(args) {
|
||||
const imagePath = args.image_path;
|
||||
const prompt = args.prompt || "Describe this image in detail";
|
||||
|
||||
const file = await uploadFile(imagePath);
|
||||
const result = await model.generateContent([
|
||||
prompt,
|
||||
{ fileData: { fileUri: file.uri, mimeType: file.mimeType }}
|
||||
]);
|
||||
|
||||
return result.response.text();
|
||||
}
|
||||
|
||||
async function analyzeMultiple(args) {
|
||||
const imagePaths = args.image_paths;
|
||||
const prompt = args.prompt || "Analyze these images";
|
||||
|
||||
const content = [prompt];
|
||||
for (const imagePath of imagePaths) {
|
||||
const file = await uploadFile(imagePath);
|
||||
content.push({ fileData: { fileUri: file.uri, mimeType: file.mimeType }});
|
||||
}
|
||||
|
||||
const result = await model.generateContent(content);
|
||||
return result.response.text();
|
||||
}
|
||||
|
||||
async function extractText(args) {
|
||||
const imagePath = args.image_path;
|
||||
const format = args.format || "plain";
|
||||
|
||||
const prompts = {
|
||||
plain: "Extract and transcribe all text from this image. Return only the text, nothing else.",
|
||||
markdown: "Extract all text from this image and format it in markdown, preserving structure.",
|
||||
structured: "Extract all text from this image and organize it with clear sections and structure."
|
||||
};
|
||||
|
||||
const file = await uploadFile(imagePath);
|
||||
const result = await model.generateContent([
|
||||
prompts[format] || prompts.plain,
|
||||
{ fileData: { fileUri: file.uri, mimeType: file.mimeType }}
|
||||
]);
|
||||
|
||||
return result.response.text();
|
||||
}
|
||||
|
||||
async function compareImages(args) {
|
||||
const image1Path = args.image1_path;
|
||||
const image2Path = args.image2_path;
|
||||
const focus = args.focus || "differences";
|
||||
|
||||
const prompts = {
|
||||
differences: "Compare these two images and describe all the differences you can find.",
|
||||
similarities: "Compare these two images and describe what they have in common.",
|
||||
changes: "Describe what has changed between the first and second image."
|
||||
};
|
||||
|
||||
const [file1, file2] = await Promise.all([
|
||||
uploadFile(image1Path),
|
||||
uploadFile(image2Path)
|
||||
]);
|
||||
|
||||
const result = await model.generateContent([
|
||||
prompts[focus] || prompts.differences,
|
||||
{ fileData: { fileUri: file1.uri, mimeType: file1.mimeType }},
|
||||
{ fileData: { fileUri: file2.uri, mimeType: file2.mimeType }}
|
||||
]);
|
||||
|
||||
return result.response.text();
|
||||
}
|
||||
|
||||
async function suggestFilename(args) {
|
||||
const imagePath = args.image_path;
|
||||
const maxLength = args.max_length || 60;
|
||||
const includeDate = args.include_date || false;
|
||||
|
||||
const prompt = `Analyze this image and suggest a descriptive filename for it.
|
||||
Requirements:
|
||||
- Maximum ${maxLength} characters (not including extension)
|
||||
- Use title case with spaces (will be converted to hyphens)
|
||||
- Be specific and descriptive about the content
|
||||
- ${includeDate ? 'Include YYYY-MM-DD prefix if a date is visible in the image' : 'Do not include date prefix'}
|
||||
- Focus on the main subject or purpose of the image
|
||||
- For screenshots: include the application or website name
|
||||
- For diagrams: include the type and subject
|
||||
- For photos: include the subject and context
|
||||
- Return ONLY the filename suggestion, no explanation or extension`;
|
||||
|
||||
const file = await uploadFile(imagePath);
|
||||
const result = await model.generateContent([
|
||||
prompt,
|
||||
{ fileData: { fileUri: file.uri, mimeType: file.mimeType }}
|
||||
]);
|
||||
|
||||
// Clean up the suggestion and format it
|
||||
let suggestion = result.response.text().trim();
|
||||
// Remove any file extension if accidentally included
|
||||
suggestion = suggestion.replace(/\.(png|jpg|jpeg|gif|webp|pdf)$/i, '');
|
||||
// Replace spaces with hyphens
|
||||
suggestion = suggestion.replace(/\s+/g, ' ').replace(/ /g, ' - ');
|
||||
// Ensure it doesn't exceed max length
|
||||
if (suggestion.length > maxLength) {
|
||||
suggestion = suggestion.substring(0, maxLength).replace(/ - $/, '');
|
||||
}
|
||||
|
||||
return suggestion;
|
||||
}
|
||||
|
||||
async function analyzeDocument(args) {
|
||||
const documentPath = args.document_path;
|
||||
const prompt = args.prompt || "Analyze this document and provide a comprehensive summary";
|
||||
|
||||
const file = await uploadFile(documentPath);
|
||||
const result = await model.generateContent([
|
||||
prompt,
|
||||
{ fileData: { fileUri: file.uri, mimeType: file.mimeType }}
|
||||
]);
|
||||
|
||||
return result.response.text();
|
||||
}
|
||||
|
||||
// Analyze video files or YouTube URLs
|
||||
async function analyzeVideo(args) {
|
||||
const videoPath = args.video_path;
|
||||
const youtubeUrl = args.youtube_url;
|
||||
const prompt = args.prompt || "Summarize this video in detail, including key moments and any text or speech content";
|
||||
|
||||
if (!videoPath && !youtubeUrl) {
|
||||
throw new Error("Either video_path or youtube_url is required");
|
||||
}
|
||||
|
||||
if (videoPath && youtubeUrl) {
|
||||
throw new Error("Please provide either video_path or youtube_url, not both");
|
||||
}
|
||||
|
||||
let fileData;
|
||||
|
||||
if (youtubeUrl) {
|
||||
// YouTube URLs can be passed directly to the API
|
||||
fileData = { fileUri: youtubeUrl };
|
||||
} else {
|
||||
// Upload local video file
|
||||
const file = await uploadFile(videoPath);
|
||||
fileData = { fileUri: file.uri, mimeType: file.mimeType };
|
||||
}
|
||||
|
||||
const result = await model.generateContent([
|
||||
prompt,
|
||||
{ fileData }
|
||||
]);
|
||||
|
||||
return result.response.text();
|
||||
}
|
||||
|
||||
// Create MCP server
|
||||
const server = new Server(
|
||||
{ name: "gemini-vision", version: "1.0.0" },
|
||||
{ capabilities: { tools: {} }}
|
||||
);
|
||||
|
||||
// List available tools
|
||||
server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
||||
tools: [
|
||||
{
|
||||
name: "analyze_image",
|
||||
description: "Analyze an image - transcribe text, describe content, or answer questions",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
image_path: { type: "string", description: "Path to the image file" },
|
||||
prompt: { type: "string", description: "What to do with the image", default: "Describe this image" }
|
||||
},
|
||||
required: ["image_path"]
|
||||
}
|
||||
},
|
||||
{
|
||||
name: "analyze_multiple",
|
||||
description: "Analyze multiple images at once",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
image_paths: { type: "array", items: { type: "string" }, description: "List of image paths" },
|
||||
prompt: { type: "string", description: "What to do with the images", default: "Analyze these images" }
|
||||
},
|
||||
required: ["image_paths"]
|
||||
}
|
||||
},
|
||||
{
|
||||
name: "extract_text",
|
||||
description: "Extract and transcribe all text from an image (OCR)",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
image_path: { type: "string", description: "Path to the image file" },
|
||||
format: { type: "string", enum: ["plain", "markdown", "structured"], default: "plain" }
|
||||
},
|
||||
required: ["image_path"]
|
||||
}
|
||||
},
|
||||
{
|
||||
name: "compare_images",
|
||||
description: "Compare two images and describe differences or similarities",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
image1_path: { type: "string", description: "Path to first image" },
|
||||
image2_path: { type: "string", description: "Path to second image" },
|
||||
focus: { type: "string", enum: ["differences", "similarities", "changes"], default: "differences" }
|
||||
},
|
||||
required: ["image1_path", "image2_path"]
|
||||
}
|
||||
},
|
||||
{
|
||||
name: "suggest_image_filename",
|
||||
description: "Analyze an image and suggest a descriptive filename (without extension)",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
image_path: { type: "string", description: "Path to the image file" },
|
||||
max_length: { type: "number", description: "Maximum filename length", default: 60 },
|
||||
include_date: { type: "boolean", description: "Include date prefix in suggestion", default: false }
|
||||
},
|
||||
required: ["image_path"]
|
||||
}
|
||||
},
|
||||
{
|
||||
name: "analyze_video",
|
||||
description: "Analyze video files or YouTube URLs - extract content, summarize, transcribe speech, identify objects/text. Provide either video_path OR youtube_url",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
video_path: { type: "string", description: "Path to local video file (MP4, AVI, MOV, etc.)" },
|
||||
youtube_url: { type: "string", description: "YouTube video URL (e.g., https://www.youtube.com/watch?v=...)" },
|
||||
prompt: { type: "string", description: "What to analyze in the video", default: "Summarize this video in detail" }
|
||||
},
|
||||
required: []
|
||||
}
|
||||
},
|
||||
{
|
||||
name: "analyze_document",
|
||||
description: "Analyze a PDF or document with custom prompts - extract specific information, find mentions of topics, summarize sections, etc.",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
document_path: { type: "string", description: "Path to the document file (PDF, DOC, DOCX, ODT, RTF, TXT)" },
|
||||
prompt: { type: "string", description: "What to analyze or extract from the document", default: "Analyze this document and provide a comprehensive summary" }
|
||||
},
|
||||
required: ["document_path"]
|
||||
}
|
||||
}
|
||||
]
|
||||
}));
|
||||
|
||||
// Handle tool calls
|
||||
server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
||||
const { name, arguments: args } = request.params;
|
||||
|
||||
try {
|
||||
let result;
|
||||
switch (name) {
|
||||
case "analyze_image":
|
||||
result = await analyzeImage(args);
|
||||
break;
|
||||
case "analyze_multiple":
|
||||
result = await analyzeMultiple(args);
|
||||
break;
|
||||
case "extract_text":
|
||||
result = await extractText(args);
|
||||
break;
|
||||
case "compare_images":
|
||||
result = await compareImages(args);
|
||||
break;
|
||||
case "suggest_image_filename":
|
||||
result = await suggestFilename(args);
|
||||
break;
|
||||
case "analyze_document":
|
||||
result = await analyzeDocument(args);
|
||||
break;
|
||||
case "analyze_video":
|
||||
result = await analyzeVideo(args);
|
||||
break;
|
||||
default:
|
||||
throw new Error(`Unknown tool: ${name}`);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [{ type: "text", text: result }]
|
||||
};
|
||||
} catch (error) {
|
||||
throw new Error(`Tool execution failed: ${error.message}`);
|
||||
}
|
||||
});
|
||||
|
||||
// Start server
|
||||
async function main() {
|
||||
const transport = new StdioServerTransport();
|
||||
await server.connect(transport);
|
||||
console.error("🚀 Gemini Vision MCP Server running");
|
||||
}
|
||||
|
||||
main().catch(console.error);
|
||||
Reference in New Issue
Block a user