Initial project setup: Obsidian intelligent journal organizer

- Add core agent architecture with Command + Skill pattern
- Implement Claude API integration for content analysis
- Add Obsidian REST API integration for vault operations
- Create conversational interface (v2.0) with natural language processing
- Add comprehensive configuration management and validation
- Include project documentation and developer guides
- Set up testing framework with unit, integration, and property tests
- Add Kiro specs for Claude API configuration and code quality improvements
- Configure project steering files for development guidelines
This commit is contained in:
windyboy
2025-12-31 17:55:10 +08:00
parent 3200ad3dd5
commit f7e54692a9
67 changed files with 23088 additions and 0 deletions
+247
View File
@@ -0,0 +1,247 @@
# Testing Framework
This directory contains the comprehensive test suite for the Obsidian journal organizer project.
## Overview
The test suite is organized into three main categories:
- **Unit Tests** (`tests/unit/`) - Test individual components in isolation
- **Integration Tests** (`tests/integration/`) - Test component interactions and workflows
- **Property Tests** (`tests/property/`) - Property-based tests for comprehensive validation
## Test Structure
```
tests/
├── README.md # This file
├── conftest.py # Shared pytest fixtures and configuration
├── unit/ # Unit tests
│ ├── test_agent_core.py # Tests for Agent, Command, Skill classes
│ └── test_error_handling.py # Tests for error handling framework
├── integration/ # Integration tests
│ ├── test_obsidian_skills.py # Obsidian API integration tests
│ ├── test_claude_skills.py # Claude AI integration tests
│ ├── test_organize_command.py # Full command workflow tests
│ └── test_conversational_agent.py # Conversational interface tests
└── property/ # Property-based tests (placeholder)
```
## Running Tests
### Using the Test Runner
The easiest way to run tests is using the provided test runner:
```bash
# Run unit tests only
python run_tests.py unit
# Run integration tests only (may have import issues)
python run_tests.py integration
# Run all tests
python run_tests.py all
# Run tests with coverage report
python run_tests.py coverage
# Show help
python run_tests.py help
```
### Using pytest Directly
You can also run tests directly with pytest:
```bash
# Run all unit tests
python -m pytest tests/unit/ -v
# Run specific test file
python -m pytest tests/unit/test_agent_core.py -v
# Run with coverage
python -m pytest tests/unit/ --cov=. --cov-report=html
# Run tests matching a pattern
python -m pytest -k "test_skill" -v
```
## Test Configuration
The test suite is configured through:
- `pytest.ini` - Main pytest configuration
- `tests/conftest.py` - Shared fixtures and test utilities
### Key Configuration Options
- **Async Support**: Tests use `pytest-asyncio` for async/await testing
- **Coverage**: Configured to exclude test files and generate HTML reports
- **Markers**: Custom markers for different test types (unit, integration, property)
- **Fixtures**: Shared fixtures for common test data and mocks
## Test Categories
### Unit Tests
Unit tests focus on testing individual components in isolation:
- **Agent Core Tests** (`test_agent_core.py`)
- SkillResult validation and serialization
- CommandContext creation and validation
- Skill base class functionality
- SkillChain execution logic
- Command registration and execution
- Agent command orchestration
- **Error Handling Tests** (`test_error_handling.py`)
- Custom exception classes
- Error context management
- Error message sanitization
- Security audit functionality
- Global error handler patterns
### Integration Tests
Integration tests verify component interactions and end-to-end workflows:
- **Obsidian Skills** (`test_obsidian_skills.py`)
- API client integration with mocked responses
- File read/write/append operations
- Error handling for API failures
- Multi-skill workflows
- **Claude Skills** (`test_claude_skills.py`)
- AI analysis integration with mocked responses
- Content transformation workflows
- Batch processing scenarios
- Error recovery patterns
- **Organize Command** (`test_organize_command.py`)
- Full journal organization workflow
- Skill chain coordination
- Partial success handling
- Agent integration
- **Conversational Agent** (`test_conversational_agent.py`)
- Natural language intent understanding
- Multi-turn conversation flows
- Response generation
- Error recovery in conversations
## Test Data and Fixtures
### Shared Fixtures (conftest.py)
- `temp_dir` - Temporary directory for file operations
- `sample_config` - Complete system configuration for testing
- `config_file` - Temporary configuration file
- `mock_skill_result` - Standard SkillResult for mocking
- `sample_journal_content` - Realistic journal content for testing
- `sample_obsidian_response` - Mock Obsidian API responses
### Mock Strategies
The test suite uses several mocking strategies:
1. **API Mocking**: Using `aioresponses` for HTTP API calls
2. **Service Mocking**: Using `unittest.mock` for external services
3. **Dependency Injection**: Providing test doubles through fixtures
4. **Response Simulation**: Creating realistic API responses for testing
## Writing New Tests
### Unit Test Guidelines
1. **Isolation**: Test one component at a time
2. **Mocking**: Mock all external dependencies
3. **Coverage**: Test both success and failure paths
4. **Validation**: Verify inputs, outputs, and side effects
5. **Naming**: Use descriptive test names that explain the scenario
Example unit test:
```python
def test_skill_result_validation_success(self):
"""Test SkillResult validation with valid data"""
result = SkillResult(success=True, data={"key": "value"})
assert result.success is True
assert result.data == {"key": "value"}
assert result.error is None
```
### Integration Test Guidelines
1. **Realistic Scenarios**: Test real-world usage patterns
2. **Mock External APIs**: Use aioresponses for HTTP calls
3. **End-to-End Flows**: Test complete workflows
4. **Error Scenarios**: Test failure modes and recovery
5. **Data Validation**: Verify data flows between components
Example integration test:
```python
@pytest.mark.asyncio
async def test_organize_workflow_success(self, command, context):
"""Test complete organize command workflow"""
with aioresponses() as m:
# Mock API responses
m.get("https://localhost:27123/vault/Daily/2024-01-15.md",
payload={"content": "journal content"})
result = await command.execute(context)
assert result.success is True
assert "created_notes" in result.data
```
## Test Maintenance
### Regular Tasks
1. **Update Fixtures**: Keep test data current with schema changes
2. **Review Coverage**: Ensure new code has adequate test coverage
3. **Mock Updates**: Update mocks when external APIs change
4. **Performance**: Monitor test execution time and optimize slow tests
### Debugging Tests
1. **Verbose Output**: Use `-v` flag for detailed test output
2. **Specific Tests**: Run individual tests with `-k` pattern matching
3. **Debug Mode**: Use `--pdb` to drop into debugger on failures
4. **Logging**: Enable debug logging in tests when needed
## Dependencies
The test suite requires these additional packages:
- `pytest>=7.0.0` - Test framework
- `pytest-asyncio>=0.21.0` - Async test support
- `pytest-cov>=4.0.0` - Coverage reporting
- `hypothesis>=6.0.0` - Property-based testing
- `aioresponses>=0.7.0` - HTTP mocking for aiohttp
Install with:
```bash
pip install -r requirements.txt
```
## Continuous Integration
The test suite is designed to run in CI environments:
- All tests should pass on clean installations
- No external network dependencies (all APIs mocked)
- Deterministic results (no random failures)
- Fast execution (unit tests < 2 minutes)
## Contributing
When adding new features:
1. Write unit tests for new components
2. Add integration tests for new workflows
3. Update fixtures if data models change
4. Maintain test coverage above 80%
5. Follow existing test patterns and naming conventions
+1
View File
@@ -0,0 +1 @@
# Test package initialization
+138
View File
@@ -0,0 +1,138 @@
"""
Pytest configuration and shared fixtures for the journal organizer test suite.
"""
import pytest
import asyncio
import tempfile
import shutil
import sys
from pathlib import Path
from typing import Dict, Any, AsyncGenerator
from unittest.mock import Mock, AsyncMock
import yaml
# Add the project root to Python path for imports
sys.path.insert(0, str(Path(__file__).parent.parent))
from agent_core import Agent, SkillResult
from error_handling import ErrorHandler
@pytest.fixture(scope="session")
def event_loop():
"""Create an instance of the default event loop for the test session."""
loop = asyncio.get_event_loop_policy().new_event_loop()
yield loop
loop.close()
@pytest.fixture
def temp_dir():
"""Create a temporary directory for test files."""
temp_dir = tempfile.mkdtemp()
yield Path(temp_dir)
shutil.rmtree(temp_dir)
@pytest.fixture
def sample_config(temp_dir: Path) -> Dict[str, Any]:
"""Create a sample configuration for testing."""
vault_path = temp_dir / "test_vault"
vault_path.mkdir()
config = {
"obsidian": {
"vault_path": str(vault_path),
"rest_api": {
"url": "https://localhost:27123",
"api_key": "test-api-key",
"verify_ssl": False
}
},
"claude": {
"api_key": "test-claude-key",
"model": "claude-3-5-sonnet-20241022",
"max_tokens": 4096
},
"journal": {
"daily_notes_folder": "Daily",
"date_format": "YYYY-MM-DD"
},
"output": {
"experiences_folder": "Knowledge/Experiences",
"lessons_folder": "Knowledge/Lessons"
}
}
return config
@pytest.fixture
def config_file(temp_dir: Path, sample_config: Dict[str, Any]) -> Path:
"""Create a temporary config file for testing."""
config_path = temp_dir / "test_config.yaml"
with open(config_path, 'w') as f:
yaml.dump(sample_config, f)
return config_path
@pytest.fixture
def mock_skill_result():
"""Create a mock SkillResult for testing."""
return SkillResult(
success=True,
data={"test": "data"},
message="Test operation completed"
)
@pytest.fixture
def mock_error_handler():
"""Create a mock ErrorHandler for testing."""
return Mock(spec=ErrorHandler)
@pytest.fixture
async def mock_aiohttp_session():
"""Create a mock aiohttp session for testing."""
session = AsyncMock()
session.get = AsyncMock()
session.post = AsyncMock()
session.put = AsyncMock()
session.delete = AsyncMock()
return session
@pytest.fixture
def sample_journal_content():
"""Sample journal content for testing."""
return """# 2024-01-15 Daily Journal
## 今天的经历
- 完成了项目的重要里程碑
- 与团队进行了有效的沟通
## 学到的东西
- 学会了新的Python异步编程技巧
- 理解了更好的错误处理模式
## 遇到的问题
- API调用偶尔超时
- 配置文件格式需要改进
## 明天的计划
- 优化API调用的重试机制
- 更新文档
"""
@pytest.fixture
def sample_obsidian_response():
"""Sample Obsidian API response for testing."""
return {
"content": "# Test Note\n\nThis is test content.",
"stat": {
"ctime": 1642204800000,
"mtime": 1642204800000,
"size": 35
}
}
+1
View File
@@ -0,0 +1 @@
# Integration tests package
@@ -0,0 +1,659 @@
"""
Backward compatibility validation tests
Tests that existing configuration files continue to work, no breaking changes to existing functionality,
and migration messages are appropriate.
Requirements: 3.1, 3.2, 3.3, 3.4
"""
import os
import pytest
import tempfile
import yaml
from pathlib import Path
from typing import Dict, Any
from unittest.mock import patch, Mock
# Import the modules we're testing
import sys
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
from config import Config
from config_validation import SystemConfig, ConfigurationValidator
from configuration_loader import ConfigurationLoader
from configuration_migrator import ConfigurationMigrator
class TestLegacyConfigurationFiles:
"""Test that existing configuration files continue to work"""
def setup_method(self):
"""Set up test fixtures"""
self.temp_dir = Path(tempfile.mkdtemp())
self.test_vault = self.temp_dir / 'test_vault'
self.test_vault.mkdir()
(self.test_vault / '.obsidian').mkdir()
# Store original environment variables
self.original_env = {}
if 'ANTHROPIC_API_KEY' in os.environ:
self.original_env['ANTHROPIC_API_KEY'] = os.environ['ANTHROPIC_API_KEY']
# Set a test API key for validation
os.environ['ANTHROPIC_API_KEY'] = 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
# Restore original environment variables
if 'ANTHROPIC_API_KEY' in self.original_env:
os.environ['ANTHROPIC_API_KEY'] = self.original_env['ANTHROPIC_API_KEY']
elif 'ANTHROPIC_API_KEY' in os.environ:
del os.environ['ANTHROPIC_API_KEY']
def create_legacy_config_file(self, config_data: Dict[str, Any]) -> Path:
"""Create a legacy configuration file"""
config_file = self.temp_dir / 'legacy_config.yaml'
with config_file.open('w') as f:
yaml.dump(config_data, f)
return config_file
def test_minimal_legacy_config_loads(self):
"""Test that minimal legacy configuration loads successfully"""
# This represents a very basic legacy config that users might have
legacy_config = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}',
'model': 'claude-3-sonnet' # Legacy model name
}
}
config_file = self.create_legacy_config_file(legacy_config)
# Should load without errors
config = Config(str(config_file))
# Verify legacy values are preserved and migrated appropriately
assert config.claude.api_key == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
assert config.claude.model == 'claude-3-sonnet-20240229' # Should be migrated
assert config.claude.api_url == 'https://api.anthropic.com' # Should get default
assert Path(config.obsidian.vault_path).resolve() == self.test_vault.resolve()
def test_legacy_config_without_claude_section(self):
"""Test legacy config that doesn't have Claude section at all"""
legacy_config = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
}
}
config_file = self.create_legacy_config_file(legacy_config)
# Should load and add Claude section with defaults
config = Config(str(config_file))
# Should have Claude config with defaults
assert config.claude.api_key == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
assert config.claude.model == 'claude-3-5-sonnet-20241022'
assert config.claude.api_url == 'https://api.anthropic.com'
def test_legacy_config_with_old_structure(self):
"""Test legacy config with old obsidian structure"""
legacy_config = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key',
'verify_ssl': False
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}',
'model': 'claude-3-opus', # Legacy model name
'max_tokens': 2048,
'temperature': 0.5
},
'journal': {
'daily_notes_folder': 'Journal/Daily',
'date_format': 'YYYY-MM-DD'
}
}
config_file = self.create_legacy_config_file(legacy_config)
# Should load and migrate appropriately
config = Config(str(config_file))
# Verify all legacy values are preserved
assert config.claude.model == 'claude-3-opus-20240229' # Migrated
assert config.claude.max_tokens == 2048 # Preserved
assert config.claude.temperature == 0.5 # Preserved
assert config.journal.daily_notes_folder == 'Journal/Daily' # Preserved
assert config.obsidian.verify_ssl is False # Preserved
def test_legacy_config_with_missing_sections(self):
"""Test legacy config with missing sections gets defaults"""
legacy_config = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}'
}
# Missing journal, output, analysis, logging sections
}
config_file = self.create_legacy_config_file(legacy_config)
# Should load and add missing sections with defaults
config = Config(str(config_file))
# Should have all sections with defaults
assert config.journal.daily_notes_folder == 'Daily'
assert config.journal.date_format == 'YYYY-MM-DD'
assert config.output.experiences_folder == 'Knowledge/Experiences'
assert config.analysis.categories == []
assert config.logging.level == 'INFO'
def test_legacy_model_name_variations(self):
"""Test various legacy model name formats are migrated correctly"""
legacy_models = {
'claude-3-sonnet': 'claude-3-sonnet-20240229',
'claude-3-opus': 'claude-3-opus-20240229',
'claude-3-haiku': 'claude-3-haiku-20240307',
'sonnet': 'claude-3-5-sonnet-20241022',
'opus': 'claude-3-opus-20240229',
'haiku': 'claude-3-haiku-20240307'
}
for legacy_model, expected_model in legacy_models.items():
legacy_config = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}',
'model': legacy_model
}
}
config_file = self.create_legacy_config_file(legacy_config)
config = Config(str(config_file))
assert config.claude.model == expected_model, f"Legacy model '{legacy_model}' should migrate to '{expected_model}'"
class TestNoBreakingChanges:
"""Test that no breaking changes exist in existing functionality"""
def setup_method(self):
"""Set up test fixtures"""
self.temp_dir = Path(tempfile.mkdtemp())
self.test_vault = self.temp_dir / 'test_vault'
self.test_vault.mkdir()
(self.test_vault / '.obsidian').mkdir()
# Set test API key
os.environ['ANTHROPIC_API_KEY'] = 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
if 'ANTHROPIC_API_KEY' in os.environ:
del os.environ['ANTHROPIC_API_KEY']
def create_config_file(self, config_data: Dict[str, Any]) -> Path:
"""Create a configuration file"""
config_file = self.temp_dir / 'config.yaml'
with config_file.open('w') as f:
yaml.dump(config_data, f)
return config_file
def test_config_class_interface_unchanged(self):
"""Test that Config class interface remains unchanged"""
config_data = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}',
'model': 'claude-3-5-sonnet-20241022'
}
}
config_file = self.create_config_file(config_data)
config = Config(str(config_file))
# Test that all expected attributes exist and work as before
assert hasattr(config, 'obsidian')
assert hasattr(config, 'claude')
assert hasattr(config, 'journal')
assert hasattr(config, 'output')
assert hasattr(config, 'analysis')
assert hasattr(config, 'logging')
# Test that methods still work
assert callable(config.get_daily_note_path)
assert callable(config.to_dict)
# Test method functionality
daily_path = config.get_daily_note_path('2024-01-01')
assert '2024-01-01' in daily_path
config_dict = config.to_dict()
assert isinstance(config_dict, dict)
assert 'obsidian' in config_dict
assert 'claude' in config_dict
def test_config_attribute_access_unchanged(self):
"""Test that config attribute access patterns remain unchanged"""
config_data = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}',
'model': 'claude-3-5-sonnet-20241022',
'max_tokens': 4096,
'temperature': 0.7
}
}
config_file = self.create_config_file(config_data)
config = Config(str(config_file))
# Test that all legacy attribute access patterns still work
assert config.obsidian.vault_path == str(self.test_vault)
assert config.obsidian.rest_api_url == 'https://localhost:27123'
assert config.obsidian.rest_api_key == 'test-key'
assert config.claude.api_key == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
assert config.claude.model == 'claude-3-5-sonnet-20241022'
assert config.claude.max_tokens == 4096
assert config.claude.temperature == 0.7
# Test new attributes are accessible
assert hasattr(config.claude, 'api_url')
assert config.claude.api_url == 'https://api.anthropic.com'
def test_config_validation_behavior_unchanged(self):
"""Test that config validation behavior remains the same for valid configs"""
valid_config = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}',
'model': 'claude-3-5-sonnet-20241022'
}
}
config_file = self.create_config_file(valid_config)
# Should load without errors (same as before)
config = Config(str(config_file))
assert config is not None
# Should still validate the same way
errors = config.get_validation_errors()
assert isinstance(errors, list)
def test_config_error_handling_unchanged(self):
"""Test that config error handling behavior remains unchanged"""
# Test with invalid vault path (should still raise appropriate error)
invalid_config = {
'obsidian': {
'vault_path': '/nonexistent/path',
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}',
'model': 'claude-3-5-sonnet-20241022'
}
}
config_file = self.create_config_file(invalid_config)
# Should still handle errors the same way (warnings, not exceptions for path issues)
config = Config(str(config_file))
errors = config.get_validation_errors()
assert len(errors) > 0 # Should have path validation warnings
class TestMigrationMessages:
"""Test that migration messages are appropriate and helpful"""
def setup_method(self):
"""Set up test fixtures"""
self.migrator = ConfigurationMigrator()
self.temp_dir = Path(tempfile.mkdtemp())
# Set test API key
os.environ['ANTHROPIC_API_KEY'] = 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
if 'ANTHROPIC_API_KEY' in os.environ:
del os.environ['ANTHROPIC_API_KEY']
def test_migration_needed_detection_accurate(self):
"""Test that migration detection is accurate"""
# Config that needs migration
config_needing_migration = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-sonnet' # Legacy model name
}
}
assert self.migrator.check_migration_needed(config_needing_migration) is True
# Config that doesn't need migration
config_up_to_date = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': 'https://api.anthropic.com',
'model': 'claude-3-5-sonnet-20241022',
'max_tokens': 4096,
'temperature': 0.7
},
'journal': {
'daily_notes_folder': 'Daily',
'date_format': 'YYYY-MM-DD',
'file_extension': '.md'
},
'output': {
'experiences_folder': 'Knowledge/Experiences',
'lessons_folder': 'Knowledge/Lessons',
'tasks_folder': 'Tasks/Daily',
'problems_folder': 'Knowledge/Problems',
'achievements_folder': 'Knowledge/Achievements',
'improvements_folder': 'Knowledge/Improvements'
},
'analysis': {
'categories': [],
'extraction_rules': {}
},
'logging': {
'level': 'INFO',
'file': 'logs/journal_organizer.log'
}
}
assert self.migrator.check_migration_needed(config_up_to_date) is False
def test_migration_preview_messages_helpful(self):
"""Test that migration preview messages are helpful and informative"""
config_dict = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-sonnet' # Legacy model name
}
}
preview = self.migrator.get_migration_preview(config_dict)
assert len(preview) > 0
# Should have informative messages about what will be changed
preview_text = '\n'.join(preview)
assert 'api_url' in preview_text.lower()
assert 'claude-3-sonnet' in preview_text
assert 'claude-3-sonnet-20240229' in preview_text
# Messages should be user-friendly
assert any('will be added' in msg or 'will be migrated' in msg or 'will be updated' in msg for msg in preview)
def test_migration_messages_include_rationale(self):
"""Test that migration messages include rationale for changes"""
config_dict = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
# Missing api_url and using default model
}
}
preview = self.migrator.get_migration_preview(config_dict)
# Should explain why changes are being made
preview_text = '\n'.join(preview).lower()
assert 'default' in preview_text or 'missing' in preview_text
def test_migration_preserves_user_values(self):
"""Test that migration preserves user-specified values"""
config_dict = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-sonnet', # Legacy model name
'max_tokens': 8192, # User-specified value
'temperature': 0.3 # User-specified value
}
}
migrated = self.migrator.migrate_claude_config(config_dict)
# Should preserve user values
assert migrated['claude']['max_tokens'] == 8192
assert migrated['claude']['temperature'] == 0.3
# Should migrate legacy values
assert migrated['claude']['model'] == 'claude-3-sonnet-20240229'
# Should add missing defaults
assert migrated['claude']['api_url'] == 'https://api.anthropic.com'
def test_migration_handles_partial_configs(self):
"""Test that migration handles partial configurations gracefully"""
partial_configs = [
# Only obsidian config
{
'obsidian': {
'vault_path': '/test/vault',
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
}
},
# Only claude config
{
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
}
},
# Empty config
{}
]
for partial_config in partial_configs:
# Should not raise errors
migrated = self.migrator.migrate_configuration(partial_config)
# Should have all required sections
assert 'claude' in migrated
assert 'journal' in migrated
assert 'output' in migrated
assert 'analysis' in migrated
assert 'logging' in migrated
# Claude section should have all required fields
assert 'api_key' in migrated['claude']
assert 'api_url' in migrated['claude']
assert 'model' in migrated['claude']
def test_supported_model_names_comprehensive(self):
"""Test that supported model names list is comprehensive"""
supported_models = self.migrator.get_supported_model_names()
# Should include current models
current_models = [
'claude-3-5-sonnet-20241022',
'claude-3-5-haiku-20241022',
'claude-3-opus-20240229',
'claude-3-sonnet-20240229',
'claude-3-haiku-20240307'
]
for model in current_models:
assert model in supported_models, f"Current model '{model}' should be in supported list"
# Should include legacy models
legacy_models = [
'claude-3-sonnet',
'claude-3-opus',
'claude-3-haiku',
'sonnet',
'opus',
'haiku'
]
for model in legacy_models:
assert model in supported_models, f"Legacy model '{model}' should be in supported list"
@patch('builtins.print')
def test_migration_logging_appropriate(self, mock_print):
"""Test that migration produces appropriate logging messages"""
config_dict = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-sonnet' # Legacy model name
}
}
# Perform migration
migrated = self.migrator.migrate_claude_config(config_dict)
# Should have logged the migration
assert mock_print.called
# Check that the log message is informative
log_calls = [call[0][0] for call in mock_print.call_args_list]
log_text = ' '.join(log_calls)
assert 'claude-3-sonnet' in log_text
assert 'claude-3-sonnet-20240229' in log_text
assert 'migrated' in log_text.lower() or 'updated' in log_text.lower()
class TestConfigurationIntegration:
"""Test full configuration integration with backward compatibility"""
def setup_method(self):
"""Set up test fixtures"""
self.temp_dir = Path(tempfile.mkdtemp())
self.test_vault = self.temp_dir / 'test_vault'
self.test_vault.mkdir()
(self.test_vault / '.obsidian').mkdir()
# Set test API key
os.environ['ANTHROPIC_API_KEY'] = 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
if 'ANTHROPIC_API_KEY' in os.environ:
del os.environ['ANTHROPIC_API_KEY']
def create_config_file(self, config_data: Dict[str, Any]) -> Path:
"""Create a configuration file"""
config_file = self.temp_dir / 'config.yaml'
with config_file.open('w') as f:
yaml.dump(config_data, f)
return config_file
def test_end_to_end_legacy_config_loading(self):
"""Test end-to-end loading of legacy configuration"""
# Simulate a real legacy config file that a user might have
legacy_config = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key',
'verify_ssl': False
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}',
'model': 'sonnet', # Very legacy model name
'max_tokens': 2048
},
'journal': {
'daily_notes_folder': 'Daily Notes',
'date_format': 'YYYY-MM-DD'
}
}
config_file = self.create_config_file(legacy_config)
# Load through the full Config class (end-to-end test)
config = Config(str(config_file))
# Verify everything works as expected
assert config.claude.model == 'claude-3-5-sonnet-20241022' # Migrated from 'sonnet'
assert config.claude.api_url == 'https://api.anthropic.com' # Added default
assert config.claude.max_tokens == 2048 # Preserved user value
assert config.journal.daily_notes_folder == 'Daily Notes' # Preserved
# Test that methods still work
daily_path = config.get_daily_note_path('2024-01-01')
assert 'Daily Notes' in daily_path
assert '2024-01-01' in daily_path
# Test serialization still works
config_dict = config.to_dict()
assert config_dict['claude']['model'] == 'claude-3-5-sonnet-20241022'
assert 'api_key' not in config_dict['claude'] # Should be sanitized
def test_config_reload_preserves_migration(self):
"""Test that config reload preserves migration results"""
legacy_config = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api_url': 'https://localhost:27123',
'rest_api_key': 'test-key'
},
'claude': {
'api_key': '${ANTHROPIC_API_KEY}',
'model': 'claude-3-opus' # Legacy model name
}
}
config_file = self.create_config_file(legacy_config)
config = Config(str(config_file))
# Verify initial migration
assert config.claude.model == 'claude-3-opus-20240229'
# Reload config
config.reload_config()
# Should still have migrated values
assert config.claude.model == 'claude-3-opus-20240229'
assert config.claude.api_url == 'https://api.anthropic.com'
if __name__ == "__main__":
# Run tests with pytest
pytest.main([__file__, "-v"])
@@ -0,0 +1,191 @@
"""
Integration tests for Claude API configuration system
Tests Skills with different Claude API configurations, environment variables, and backward compatibility
"""
import os
import pytest
import tempfile
import yaml
from pathlib import Path
from unittest.mock import Mock, patch, AsyncMock
from typing import Dict, Any
# Import the modules we're testing
import sys
sys.path.insert(0, str(Path(__file__).parent.parent))
from config_validation import ClaudeAPIConfig
from configuration_loader import ConfigurationLoader
from configuration_migrator import ConfigurationMigrator
class TestEnvironmentVariableIntegration:
"""Test environment variable scenarios in configuration"""
def setup_method(self):
"""Set up test fixtures"""
self.temp_dir = Path(tempfile.mkdtemp())
self.loader = ConfigurationLoader()
self.migrator = ConfigurationMigrator()
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
# Clean up any test environment variables
test_vars = ['TEST_CLAUDE_API_KEY', 'TEST_CLAUDE_API_URL', 'TEST_CLAUDE_MODEL']
for var in test_vars:
if var in os.environ:
del os.environ[var]
def create_test_config_file(self, config_data: Dict[str, Any]) -> Path:
"""Create a test configuration file"""
config_file = self.temp_dir / 'test_config.yaml'
with config_file.open('w') as f:
yaml.dump(config_data, f)
return config_file
def test_environment_variable_expansion(self):
"""Test environment variable expansion"""
# Set up environment variables
os.environ['TEST_CLAUDE_API_KEY'] = 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
os.environ['TEST_CLAUDE_API_URL'] = 'https://custom-api.example.com'
config_data = {
'claude': {
'api_key': '${TEST_CLAUDE_API_KEY}',
'api_url': '${TEST_CLAUDE_API_URL}',
'model': 'claude-3-5-sonnet-20241022'
}
}
expanded = self.loader.expand_environment_variables(config_data)
assert expanded['claude']['api_key'] == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
assert expanded['claude']['api_url'] == 'https://custom-api.example.com'
assert expanded['claude']['model'] == 'claude-3-5-sonnet-20241022'
def test_environment_variable_defaults(self):
"""Test environment variable expansion with defaults"""
config_data = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': '${CLAUDE_API_URL:-https://api.anthropic.com}',
'model': '${CLAUDE_MODEL:-claude-3-5-sonnet-20241022}'
}
}
expanded = self.loader.expand_environment_variables(config_data)
# Should use defaults since env vars are not set
assert expanded['claude']['api_url'] == 'https://api.anthropic.com'
assert expanded['claude']['model'] == 'claude-3-5-sonnet-20241022'
class TestBackwardCompatibilityIntegration:
"""Test backward compatibility with existing setups"""
def setup_method(self):
"""Set up test fixtures"""
self.migrator = ConfigurationMigrator()
def test_legacy_model_migration(self):
"""Test migration of legacy model names"""
legacy_config = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-sonnet' # Legacy model name
}
}
migrated = self.migrator.migrate_claude_config(legacy_config)
# Should migrate to new model name
assert migrated['claude']['model'] == 'claude-3-sonnet-20240229'
# Should add default API URL
assert migrated['claude']['api_url'] == 'https://api.anthropic.com'
def test_migration_needed_detection(self):
"""Test detection of configurations that need migration"""
# Config that needs migration
legacy_config = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-opus' # Legacy model name
}
}
assert self.migrator.check_migration_needed(legacy_config) is True
# Config that doesn't need migration
modern_config = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': 'https://api.anthropic.com',
'model': 'claude-3-5-sonnet-20241022',
'max_tokens': 4096,
'temperature': 0.7
},
'journal': {
'daily_notes_folder': 'Daily',
'date_format': 'YYYY-MM-DD',
'file_extension': '.md'
},
'output': {
'experiences_folder': 'Knowledge/Experiences',
'lessons_folder': 'Knowledge/Lessons',
'tasks_folder': 'Tasks/Daily',
'problems_folder': 'Knowledge/Problems',
'achievements_folder': 'Knowledge/Achievements',
'improvements_folder': 'Knowledge/Improvements'
},
'analysis': {
'categories': [],
'extraction_rules': {}
},
'logging': {
'level': 'INFO',
'file': 'logs/journal_organizer.log'
}
}
assert self.migrator.check_migration_needed(modern_config) is False
class TestConfigurationValidation:
"""Test configuration validation scenarios"""
def test_claude_api_config_validation(self):
"""Test ClaudeAPIConfig validation"""
# Valid configuration
config = ClaudeAPIConfig(
api_key='sk-ant-test-key-12345678901234567890123456789012345678901234567890',
api_url='https://custom-api.example.com',
model='claude-3-5-sonnet-20241022'
)
assert config.api_key.startswith('sk-ant-')
assert config.api_url == 'https://custom-api.example.com'
assert config.model == 'claude-3-5-sonnet-20241022'
def test_invalid_configuration_handling(self):
"""Test handling of invalid configurations"""
# Invalid API key format
with pytest.raises(ValueError, match="Claude API key should start with"):
ClaudeAPIConfig(api_key='invalid-key')
# Invalid URL format
with pytest.raises(ValueError, match="Invalid URL format"):
ClaudeAPIConfig(
api_key='sk-ant-test-key-12345678901234567890123456789012345678901234567890',
api_url='not-a-url'
)
# Invalid model name
with pytest.raises(ValueError, match="Invalid model name"):
ClaudeAPIConfig(
api_key='sk-ant-test-key-12345678901234567890123456789012345678901234567890',
model='invalid-model'
)
+516
View File
@@ -0,0 +1,516 @@
"""
Integration tests for Claude Skills.
Tests Skills with mocked API responses to verify AI integration functionality.
"""
import pytest
import json
from unittest.mock import AsyncMock, patch, Mock
from agent_core import CommandContext, SkillResult
from skills.claude_skill import ClaudeAnalyzeSkill, ClaudeTransformSkill
class TestClaudeAnalyzeSkill:
"""Integration tests for ClaudeAnalyzeSkill"""
@pytest.fixture
def skill(self):
"""Create ClaudeAnalyzeSkill instance"""
return ClaudeAnalyzeSkill()
@pytest.fixture
def context(self, sample_config):
"""Create command context with analysis parameters"""
return CommandContext(
command_name="analyze",
args={
"content": """# 2024-01-15 Daily Journal
## 今天的经历
- 完成了项目的重要里程碑
- 与团队进行了有效的沟通
## 学到的东西
- 学会了新的Python异步编程技巧
- 理解了更好的错误处理模式
## 遇到的问题
- API调用偶尔超时
- 配置文件格式需要改进
## 明天的计划
- 优化API调用的重试机制
- 更新文档
""",
"analysis_type": "extract_experiences"
},
config=sample_config
)
@pytest.mark.asyncio
async def test_analyze_journal_success(self, skill, context):
"""Test successful journal analysis"""
mock_response = {
"experiences": [
{
"title": "项目里程碑完成",
"description": "成功完成了项目的重要里程碑,展现了良好的项目管理能力",
"category": "项目管理",
"importance": "high"
},
{
"title": "团队沟通改进",
"description": "与团队进行了有效的沟通,提升了协作效率",
"category": "团队协作",
"importance": "medium"
}
],
"lessons": [
{
"title": "Python异步编程",
"description": "学会了新的Python异步编程技巧,提升了代码效率",
"category": "技术学习",
"application": "可以应用到当前项目的API调用优化中"
}
],
"problems": [
{
"title": "API调用超时",
"description": "API调用偶尔出现超时问题",
"severity": "medium",
"suggested_solution": "实现重试机制和超时处理"
}
]
}
# Mock the Claude API client
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Mock the messages.create method
mock_message = Mock()
mock_message.content = [Mock(text=json.dumps(mock_response, ensure_ascii=False))]
mock_client.messages.create.return_value = mock_message
result = await skill.execute(context)
assert result.success is True
assert "experiences" in result.data
assert "lessons" in result.data
assert "problems" in result.data
assert len(result.data["experiences"]) == 2
assert len(result.data["lessons"]) == 1
assert len(result.data["problems"]) == 1
# Verify API was called with correct parameters
mock_client.messages.create.assert_called_once()
call_args = mock_client.messages.create.call_args
assert call_args[1]["model"] == "claude-3-5-sonnet-20241022"
assert call_args[1]["max_tokens"] == 4096
@pytest.mark.asyncio
async def test_analyze_empty_content(self, skill, sample_config):
"""Test analysis with empty content"""
context = CommandContext(
command_name="analyze",
args={"content": "", "analysis_type": "extract_experiences"},
config=sample_config
)
result = await skill.execute(context)
assert result.success is False
assert "content" in result.error.lower()
@pytest.mark.asyncio
async def test_analyze_api_error(self, skill, context):
"""Test handling of Claude API errors"""
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Mock API error
mock_client.messages.create.side_effect = Exception("API rate limit exceeded")
result = await skill.execute(context)
assert result.success is False
assert "api" in result.error.lower() or "rate limit" in result.error.lower()
@pytest.mark.asyncio
async def test_analyze_invalid_json_response(self, skill, context):
"""Test handling of invalid JSON response from Claude"""
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Mock invalid JSON response
mock_message = Mock()
mock_message.content = [Mock(text="Invalid JSON response")]
mock_client.messages.create.return_value = mock_message
result = await skill.execute(context)
assert result.success is False
assert "json" in result.error.lower() or "parse" in result.error.lower()
@pytest.mark.asyncio
async def test_analyze_different_types(self, skill, sample_config):
"""Test different analysis types"""
analysis_types = ["extract_experiences", "extract_lessons", "extract_problems", "summarize"]
for analysis_type in analysis_types:
context = CommandContext(
command_name="analyze",
args={
"content": "Sample journal content for testing",
"analysis_type": analysis_type
},
config=sample_config
)
mock_response = {"result": f"Analysis result for {analysis_type}"}
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
mock_message = Mock()
mock_message.content = [Mock(text=json.dumps(mock_response))]
mock_client.messages.create.return_value = mock_message
result = await skill.execute(context)
assert result.success is True
assert result.data["analysis_type"] == analysis_type
class TestClaudeTransformSkill:
"""Integration tests for ClaudeTransformSkill"""
@pytest.fixture
def skill(self):
"""Create ClaudeTransformSkill instance"""
return ClaudeTransformSkill()
@pytest.fixture
def context(self, sample_config):
"""Create command context with transformation parameters"""
return CommandContext(
command_name="transform",
args={
"content": {
"title": "项目里程碑完成",
"description": "成功完成了项目的重要里程碑",
"category": "项目管理"
},
"transform_type": "create_experience_note",
"target_format": "markdown"
},
config=sample_config
)
@pytest.mark.asyncio
async def test_transform_to_markdown_success(self, skill, context):
"""Test successful content transformation to markdown"""
mock_response = """# 项目里程碑完成
## 经验描述
成功完成了项目的重要里程碑,这次经历展现了良好的项目管理能力和团队协作精神。
## 关键要点
- 项目管理技能得到提升
- 团队协作效率显著改善
- 里程碑按时完成
## 应用场景
这个经验可以应用到未来的项目管理中,特别是在设定和跟踪项目里程碑方面。
## 相关标签
#项目管理 #里程碑 #团队协作
---
*创建时间: 2024-01-15*
*来源: 日记整理*
"""
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
mock_message = Mock()
mock_message.content = [Mock(text=mock_response)]
mock_client.messages.create.return_value = mock_message
result = await skill.execute(context)
assert result.success is True
assert result.data["transformed_content"] == mock_response
assert result.data["transform_type"] == "create_experience_note"
assert result.data["target_format"] == "markdown"
# Verify the content contains expected markdown elements
assert "# 项目里程碑完成" in result.data["transformed_content"]
assert "## 经验描述" in result.data["transformed_content"]
assert "#项目管理" in result.data["transformed_content"]
@pytest.mark.asyncio
async def test_transform_different_types(self, skill, sample_config):
"""Test different transformation types"""
transform_types = [
"create_experience_note",
"create_lesson_note",
"create_problem_note",
"create_summary"
]
for transform_type in transform_types:
context = CommandContext(
command_name="transform",
args={
"content": {"title": "Test", "description": "Test content"},
"transform_type": transform_type,
"target_format": "markdown"
},
config=sample_config
)
mock_response = f"# Transformed Content\n\nContent for {transform_type}"
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
mock_message = Mock()
mock_message.content = [Mock(text=mock_response)]
mock_client.messages.create.return_value = mock_message
result = await skill.execute(context)
assert result.success is True
assert result.data["transform_type"] == transform_type
@pytest.mark.asyncio
async def test_transform_missing_content(self, skill, sample_config):
"""Test transformation with missing content"""
context = CommandContext(
command_name="transform",
args={
"transform_type": "create_experience_note",
"target_format": "markdown"
# Missing content
},
config=sample_config
)
result = await skill.execute(context)
assert result.success is False
assert "content" in result.error.lower()
@pytest.mark.asyncio
async def test_transform_api_error(self, skill, context):
"""Test handling of Claude API errors during transformation"""
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Mock API error
mock_client.messages.create.side_effect = Exception("API authentication failed")
result = await skill.execute(context)
assert result.success is False
assert "api" in result.error.lower() or "authentication" in result.error.lower()
class TestClaudeSkillsIntegration:
"""Integration tests combining Claude skills"""
@pytest.mark.asyncio
async def test_analyze_then_transform_workflow(self, sample_config):
"""Test complete analyze-then-transform workflow"""
analyze_skill = ClaudeAnalyzeSkill()
transform_skill = ClaudeTransformSkill()
# Step 1: Analyze journal content
analyze_context = CommandContext(
command_name="analyze",
args={
"content": sample_journal_content,
"analysis_type": "extract_experiences"
},
config=sample_config
)
# Step 2: Transform extracted experience to note
experience_data = {
"title": "项目里程碑完成",
"description": "成功完成了项目的重要里程碑",
"category": "项目管理",
"importance": "high"
}
transform_context = CommandContext(
command_name="transform",
args={
"content": experience_data,
"transform_type": "create_experience_note",
"target_format": "markdown"
},
config=sample_config
)
# Mock responses
analyze_response = {
"experiences": [experience_data],
"lessons": [],
"problems": []
}
transform_response = """# 项目里程碑完成
## 经验描述
成功完成了项目的重要里程碑
## 分类
项目管理
## 重要程度
#项目管理 #里程碑
"""
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Mock analyze response
mock_analyze_message = Mock()
mock_analyze_message.content = [Mock(text=json.dumps(analyze_response, ensure_ascii=False))]
# Mock transform response
mock_transform_message = Mock()
mock_transform_message.content = [Mock(text=transform_response)]
# Set up side_effect to return different responses for different calls
mock_client.messages.create.side_effect = [mock_analyze_message, mock_transform_message]
# Execute analyze
analyze_result = await analyze_skill.execute(analyze_context)
assert analyze_result.success is True
assert len(analyze_result.data["experiences"]) == 1
# Execute transform using analyze result
transform_result = await transform_skill.execute(transform_context)
assert transform_result.success is True
assert "项目里程碑完成" in transform_result.data["transformed_content"]
# Verify both API calls were made
assert mock_client.messages.create.call_count == 2
@pytest.mark.asyncio
async def test_batch_analysis_and_transformation(self, sample_config):
"""Test batch processing of multiple content pieces"""
analyze_skill = ClaudeAnalyzeSkill()
transform_skill = ClaudeTransformSkill()
# Multiple journal entries to process
journal_entries = [
"今天学会了新的编程技巧",
"解决了一个复杂的技术问题",
"与客户进行了重要的项目讨论"
]
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Mock responses for each entry
mock_responses = []
for i, entry in enumerate(journal_entries):
analyze_response = {
"experiences": [{
"title": f"Experience {i+1}",
"description": entry,
"category": "学习"
}]
}
transform_response = f"# Experience {i+1}\n\n{entry}\n\n#学习"
mock_analyze_message = Mock()
mock_analyze_message.content = [Mock(text=json.dumps(analyze_response, ensure_ascii=False))]
mock_transform_message = Mock()
mock_transform_message.content = [Mock(text=transform_response)]
mock_responses.extend([mock_analyze_message, mock_transform_message])
mock_client.messages.create.side_effect = mock_responses
# Process each entry
results = []
for entry in journal_entries:
# Analyze
analyze_context = CommandContext(
command_name="analyze",
args={"content": entry, "analysis_type": "extract_experiences"},
config=sample_config
)
analyze_result = await analyze_skill.execute(analyze_context)
# Transform
if analyze_result.success and analyze_result.data["experiences"]:
experience = analyze_result.data["experiences"][0]
transform_context = CommandContext(
command_name="transform",
args={
"content": experience,
"transform_type": "create_experience_note",
"target_format": "markdown"
},
config=sample_config
)
transform_result = await transform_skill.execute(transform_context)
results.append((analyze_result, transform_result))
# Verify all entries were processed successfully
assert len(results) == len(journal_entries)
for analyze_result, transform_result in results:
assert analyze_result.success is True
assert transform_result.success is True
# Verify correct number of API calls (2 per entry: analyze + transform)
assert mock_client.messages.create.call_count == len(journal_entries) * 2
# Sample journal content for testing
sample_journal_content = """# 2024-01-15 Daily Journal
## 今天的经历
- 完成了项目的重要里程碑
- 与团队进行了有效的沟通
- 参加了技术分享会议
## 学到的东西
- 学会了新的Python异步编程技巧
- 理解了更好的错误处理模式
- 掌握了新的项目管理方法
## 遇到的问题
- API调用偶尔超时
- 配置文件格式需要改进
- 团队沟通中存在信息不对称
## 明天的计划
- 优化API调用的重试机制
- 更新文档
- 组织团队同步会议
"""
@@ -0,0 +1,431 @@
"""
Integration tests for conversational agent flow.
Tests the v2.0 conversational interface with natural language processing.
"""
import pytest
import json
from unittest.mock import AsyncMock, patch, Mock
from aioresponses import aioresponses
from agent_core import CommandContext, SkillResult
from conversation.conversational_agent import ConversationalAgent
from conversation.conversation_state import ConversationState
from conversation.intent_understanding import IntentUnderstanding
from conversation.response_generator import ResponseGenerator
class TestConversationalAgent:
"""Integration tests for ConversationalAgent"""
@pytest.fixture
def agent(self, sample_config):
"""Create ConversationalAgent instance"""
return ConversationalAgent(sample_config)
@pytest.fixture
def sample_user_inputs(self):
"""Sample user inputs for testing"""
return [
"整理今天的日记",
"organize today's journal",
"分析2024年1月15日的日记",
"help me organize my notes from yesterday",
"今天学到了什么?",
"what did I learn today?",
"整理昨天的经验和教训"
]
@pytest.mark.asyncio
async def test_conversational_agent_basic_flow(self, agent):
"""Test basic conversational flow"""
user_input = "整理今天的日记"
# Mock the underlying organize command execution
with patch.object(agent.agent, 'execute_command') as mock_execute:
mock_result = SkillResult(
success=True,
data={
"summary": "Successfully organized journal",
"created_notes": [
{"type": "experience", "title": "Test Experience", "path": "Knowledge/Experiences/test.md"}
]
},
message="Journal organized successfully"
)
mock_execute.return_value = mock_result
response = await agent.process_message(user_input)
assert response is not None
assert "成功" in response or "successfully" in response.lower()
mock_execute.assert_called_once()
# Verify the command was called with correct parameters
call_args = mock_execute.call_args
assert call_args[0][0] == "organize" # Command name
@pytest.mark.asyncio
async def test_conversational_agent_with_date_extraction(self, agent):
"""Test conversational agent with date parameter extraction"""
user_input = "分析2024年1月15日的日记"
with patch.object(agent.agent, 'execute_command') as mock_execute:
mock_result = SkillResult(success=True, data={}, message="Analysis completed")
mock_execute.return_value = mock_result
response = await agent.process_message(user_input)
assert response is not None
mock_execute.assert_called_once()
# Verify date was extracted and passed
call_args = mock_execute.call_args
assert "date" in call_args[1]["args"] # Should have extracted date
@pytest.mark.asyncio
async def test_conversational_agent_error_handling(self, agent):
"""Test conversational agent error handling"""
user_input = "整理今天的日记"
with patch.object(agent.agent, 'execute_command') as mock_execute:
mock_result = SkillResult(
success=False,
error="Journal file not found",
message="Failed to organize journal"
)
mock_execute.return_value = mock_result
response = await agent.process_message(user_input)
assert response is not None
assert "错误" in response or "error" in response.lower() or "failed" in response.lower()
@pytest.mark.asyncio
async def test_conversational_agent_unknown_intent(self, agent):
"""Test conversational agent with unknown intent"""
user_input = "今天天气怎么样?" # Weather question, not related to journal organization
response = await agent.process_message(user_input)
assert response is not None
assert "不理解" in response or "不明白" in response or "help" in response.lower()
@pytest.mark.asyncio
async def test_conversational_agent_help_request(self, agent):
"""Test conversational agent help functionality"""
help_inputs = ["help", "帮助", "你能做什么?", "what can you do?"]
for user_input in help_inputs:
response = await agent.process_message(user_input)
assert response is not None
assert "整理" in response or "organize" in response.lower()
assert "日记" in response or "journal" in response.lower()
@pytest.mark.asyncio
async def test_conversational_agent_multiple_turns(self, agent):
"""Test multi-turn conversation"""
conversation_turns = [
("你好", "greeting"),
("整理今天的日记", "organize"),
("谢谢", "thanks")
]
for user_input, expected_intent in conversation_turns:
if expected_intent == "organize":
with patch.object(agent.agent, 'execute_command') as mock_execute:
mock_result = SkillResult(success=True, data={}, message="Success")
mock_execute.return_value = mock_result
response = await agent.process_message(user_input)
else:
response = await agent.process_message(user_input)
assert response is not None
assert len(response) > 0
class TestIntentUnderstanding:
"""Integration tests for IntentUnderstanding"""
@pytest.fixture
def intent_processor(self):
"""Create IntentUnderstanding instance"""
return IntentUnderstanding()
def test_organize_intent_detection(self, intent_processor):
"""Test detection of organize intents"""
organize_inputs = [
"整理今天的日记",
"organize today's journal",
"分析我的日记",
"help me organize my notes",
"整理昨天的笔记"
]
for user_input in organize_inputs:
intent = intent_processor.understand_intent(user_input)
assert intent["action"] == "organize"
assert "command" in intent
assert intent["command"] == "organize"
def test_date_parameter_extraction(self, intent_processor):
"""Test extraction of date parameters"""
date_inputs = [
("整理2024年1月15日的日记", "2024-01-15"),
("analyze journal from yesterday", "yesterday"),
("organize today's notes", "today"),
("分析昨天的日记", "yesterday")
]
for user_input, expected_date in date_inputs:
intent = intent_processor.understand_intent(user_input)
if expected_date in ["today", "yesterday"]:
# These should be converted to actual dates
assert "date" in intent["parameters"]
else:
assert intent["parameters"].get("date") == expected_date
def test_help_intent_detection(self, intent_processor):
"""Test detection of help intents"""
help_inputs = [
"help",
"帮助",
"你能做什么?",
"what can you do?",
"how to use this?"
]
for user_input in help_inputs:
intent = intent_processor.understand_intent(user_input)
assert intent["action"] == "help"
def test_unknown_intent_handling(self, intent_processor):
"""Test handling of unknown intents"""
unknown_inputs = [
"今天天气怎么样?",
"what's the weather like?",
"计算1+1等于多少",
"play music"
]
for user_input in unknown_inputs:
intent = intent_processor.understand_intent(user_input)
assert intent["action"] == "unknown"
assert "confidence" in intent
assert intent["confidence"] < 0.5 # Low confidence for unknown intents
class TestResponseGenerator:
"""Integration tests for ResponseGenerator"""
@pytest.fixture
def response_generator(self):
"""Create ResponseGenerator instance"""
return ResponseGenerator()
def test_success_response_generation(self, response_generator):
"""Test generation of success responses"""
result = SkillResult(
success=True,
data={
"summary": "Successfully organized journal",
"created_notes": [
{"type": "experience", "title": "Project Milestone", "path": "Knowledge/Experiences/milestone.md"},
{"type": "lesson", "title": "Python Tips", "path": "Knowledge/Lessons/python.md"}
]
},
message="Journal organized successfully"
)
response = response_generator.generate_response(result, "organize")
assert response is not None
assert "成功" in response or "successfully" in response.lower()
assert "2" in response # Should mention number of notes created
assert "经验" in response or "experience" in response.lower()
assert "教训" in response or "lesson" in response.lower()
def test_error_response_generation(self, response_generator):
"""Test generation of error responses"""
result = SkillResult(
success=False,
error="Journal file not found for date 2024-01-15",
message="Failed to organize journal"
)
response = response_generator.generate_response(result, "organize")
assert response is not None
assert "错误" in response or "error" in response.lower() or "失败" in response
assert "2024-01-15" in response # Should include the problematic date
def test_help_response_generation(self, response_generator):
"""Test generation of help responses"""
response = response_generator.generate_help_response()
assert response is not None
assert "整理" in response or "organize" in response.lower()
assert "日记" in response or "journal" in response.lower()
assert "命令" in response or "command" in response.lower()
def test_unknown_intent_response(self, response_generator):
"""Test generation of unknown intent responses"""
response = response_generator.generate_unknown_response("今天天气怎么样?")
assert response is not None
assert "不理解" in response or "不明白" in response or "understand" in response.lower()
assert "帮助" in response or "help" in response.lower()
class TestConversationState:
"""Integration tests for ConversationState"""
@pytest.fixture
def conversation_state(self):
"""Create ConversationState instance"""
return ConversationState()
def test_conversation_history_tracking(self, conversation_state):
"""Test conversation history tracking"""
# Add some conversation turns
conversation_state.add_turn("user", "整理今天的日记")
conversation_state.add_turn("assistant", "好的,我来帮您整理今天的日记。")
conversation_state.add_turn("user", "谢谢")
conversation_state.add_turn("assistant", "不客气!还有其他需要帮助的吗?")
history = conversation_state.get_history()
assert len(history) == 4
assert history[0]["role"] == "user"
assert history[0]["content"] == "整理今天的日记"
assert history[1]["role"] == "assistant"
assert history[-1]["role"] == "assistant"
def test_context_management(self, conversation_state):
"""Test conversation context management"""
# Set some context
conversation_state.set_context("last_command", "organize")
conversation_state.set_context("last_date", "2024-01-15")
conversation_state.set_context("user_preference", "detailed_summary")
# Retrieve context
assert conversation_state.get_context("last_command") == "organize"
assert conversation_state.get_context("last_date") == "2024-01-15"
assert conversation_state.get_context("user_preference") == "detailed_summary"
assert conversation_state.get_context("nonexistent") is None
def test_conversation_reset(self, conversation_state):
"""Test conversation reset functionality"""
# Add some data
conversation_state.add_turn("user", "test message")
conversation_state.set_context("test_key", "test_value")
# Verify data exists
assert len(conversation_state.get_history()) == 1
assert conversation_state.get_context("test_key") == "test_value"
# Reset conversation
conversation_state.reset()
# Verify data is cleared
assert len(conversation_state.get_history()) == 0
assert conversation_state.get_context("test_key") is None
class TestFullConversationalFlow:
"""End-to-end integration tests for the complete conversational flow"""
@pytest.mark.asyncio
async def test_complete_organize_conversation(self, sample_config, sample_journal_content):
"""Test complete conversation flow for journal organization"""
agent = ConversationalAgent(sample_config)
# Mock all external dependencies
with aioresponses() as m:
# Mock Obsidian API
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload={
"content": sample_journal_content,
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": len(sample_journal_content)}
},
status=200
)
# Mock note creation
m.put(
"https://localhost:27123/vault/Knowledge/Experiences/test.md",
payload={"path": "Knowledge/Experiences/test.md", "stat": {}},
status=200
)
# Mock Claude API
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Mock analysis response
analyze_response = {
"experiences": [{"title": "Test Experience", "description": "Test", "category": "Test"}],
"lessons": [],
"problems": [],
"achievements": []
}
mock_analyze_message = Mock()
mock_analyze_message.content = [Mock(text=json.dumps(analyze_response, ensure_ascii=False))]
mock_transform_message = Mock()
mock_transform_message.content = [Mock(text="# Test Experience\n\nTest content")]
mock_client.messages.create.side_effect = [mock_analyze_message, mock_transform_message]
# Simulate conversation
conversation_turns = [
"你好",
"整理今天的日记",
"谢谢你的帮助"
]
responses = []
for user_input in conversation_turns:
response = await agent.process_message(user_input)
responses.append(response)
assert response is not None
assert len(response) > 0
# Verify conversation flow
assert "你好" in responses[0] or "hello" in responses[0].lower() # Greeting response
assert "成功" in responses[1] or "successfully" in responses[1].lower() # Success response
assert "不客气" in responses[2] or "welcome" in responses[2].lower() # Thanks response
@pytest.mark.asyncio
async def test_error_recovery_conversation(self, sample_config):
"""Test conversation flow with error recovery"""
agent = ConversationalAgent(sample_config)
with aioresponses() as m:
# Mock journal file not found
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
status=404,
payload={"error": "File not found"}
)
# Simulate error scenario
user_input = "整理今天的日记"
response = await agent.process_message(user_input)
assert response is not None
assert "找不到" in response or "not found" in response.lower() or "错误" in response
# Follow up with help request
help_response = await agent.process_message("我应该怎么办?")
assert help_response is not None
assert "建议" in help_response or "suggest" in help_response.lower() or "帮助" in help_response
+458
View File
@@ -0,0 +1,458 @@
"""
Integration tests for Obsidian Skills.
Tests Skills with mocked API responses to verify end-to-end functionality.
"""
import pytest
import json
from unittest.mock import AsyncMock, patch, Mock
from aioresponses import aioresponses
from agent_core import CommandContext, SkillResult
from skills.obsidian_skill import (
ObsidianReadSkill, ObsidianWriteSkill, ObsidianAppendSkill,
ObsidianListFilesSkill
)
class TestObsidianReadSkill:
"""Integration tests for ObsidianReadSkill"""
@pytest.fixture
def skill(self):
"""Create ObsidianReadSkill instance"""
return ObsidianReadSkill()
@pytest.fixture
def context(self, sample_config):
"""Create command context with Obsidian configuration"""
return CommandContext(
command_name="test_read",
args={"file_path": "Daily/2024-01-15.md"},
config=sample_config
)
@pytest.mark.asyncio
async def test_read_note_success(self, skill, context):
"""Test successful note reading"""
mock_response = {
"content": "# 2024-01-15 Daily Journal\n\nTest content",
"stat": {
"ctime": 1642204800000,
"mtime": 1642204800000,
"size": 45
}
}
with aioresponses() as m:
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload=mock_response,
status=200
)
result = await skill.execute(context)
assert result.success is True
assert result.data["content"] == mock_response["content"]
assert result.data["file_path"] == "Daily/2024-01-15.md"
assert "stat" in result.data
@pytest.mark.asyncio
async def test_read_note_not_found(self, skill, context):
"""Test reading non-existent note"""
with aioresponses() as m:
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
status=404,
payload={"error": "File not found"}
)
result = await skill.execute(context)
assert result.success is False
assert "not found" in result.error.lower()
@pytest.mark.asyncio
async def test_read_note_api_error(self, skill, context):
"""Test API connection error"""
with aioresponses() as m:
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
exception=Exception("Connection failed")
)
result = await skill.execute(context)
assert result.success is False
assert "connection" in result.error.lower() or "api" in result.error.lower()
@pytest.mark.asyncio
async def test_read_note_invalid_response(self, skill, context):
"""Test handling of invalid API response"""
with aioresponses() as m:
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload="invalid json response",
status=200
)
result = await skill.execute(context)
assert result.success is False
assert "response" in result.error.lower()
class TestObsidianWriteSkill:
"""Integration tests for ObsidianWriteSkill"""
@pytest.fixture
def skill(self):
"""Create ObsidianWriteSkill instance"""
return ObsidianWriteSkill()
@pytest.fixture
def context(self, sample_config):
"""Create command context with write parameters"""
return CommandContext(
command_name="test_write",
args={
"file_path": "Knowledge/Experiences/test-experience.md",
"content": "# Test Experience\n\nThis is a test experience note."
},
config=sample_config
)
@pytest.mark.asyncio
async def test_write_note_success(self, skill, context):
"""Test successful note writing"""
mock_response = {
"path": "Knowledge/Experiences/test-experience.md",
"stat": {
"ctime": 1642204800000,
"mtime": 1642204800000,
"size": 45
}
}
with aioresponses() as m:
m.put(
"https://localhost:27123/vault/Knowledge/Experiences/test-experience.md",
payload=mock_response,
status=200
)
result = await skill.execute(context)
assert result.success is True
assert result.data["file_path"] == "Knowledge/Experiences/test-experience.md"
assert "created" in result.message.lower() or "written" in result.message.lower()
@pytest.mark.asyncio
async def test_write_note_permission_error(self, skill, context):
"""Test write permission error"""
with aioresponses() as m:
m.put(
"https://localhost:27123/vault/Knowledge/Experiences/test-experience.md",
status=403,
payload={"error": "Permission denied"}
)
result = await skill.execute(context)
assert result.success is False
assert "permission" in result.error.lower()
@pytest.mark.asyncio
async def test_write_note_missing_content(self, skill, sample_config):
"""Test writing note without content"""
context = CommandContext(
command_name="test_write",
args={"file_path": "test.md"}, # Missing content
config=sample_config
)
result = await skill.execute(context)
assert result.success is False
assert "content" in result.error.lower()
class TestObsidianAppendSkill:
"""Integration tests for ObsidianAppendSkill"""
@pytest.fixture
def skill(self):
"""Create ObsidianAppendSkill instance"""
return ObsidianAppendSkill()
@pytest.fixture
def context(self, sample_config):
"""Create command context with append parameters"""
return CommandContext(
command_name="test_append",
args={
"file_path": "Daily/2024-01-15.md",
"content": "\n\n## Additional Notes\n\nAppended content."
},
config=sample_config
)
@pytest.mark.asyncio
async def test_append_to_existing_note_success(self, skill, context):
"""Test successful content appending to existing note"""
# Mock reading existing content
existing_content = "# 2024-01-15 Daily Journal\n\nExisting content"
read_response = {
"content": existing_content,
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": 45}
}
# Mock writing updated content
write_response = {
"path": "Daily/2024-01-15.md",
"stat": {"ctime": 1642204800000, "mtime": 1642204900000, "size": 90}
}
with aioresponses() as m:
# Mock GET request for reading existing content
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload=read_response,
status=200
)
# Mock PUT request for writing updated content
m.put(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload=write_response,
status=200
)
result = await skill.execute(context)
assert result.success is True
assert result.data["file_path"] == "Daily/2024-01-15.md"
assert "appended" in result.message.lower()
@pytest.mark.asyncio
async def test_append_to_nonexistent_note(self, skill, context):
"""Test appending to non-existent note (should create new note)"""
write_response = {
"path": "Daily/2024-01-15.md",
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": 45}
}
with aioresponses() as m:
# Mock GET request returning 404 (file doesn't exist)
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
status=404
)
# Mock PUT request for creating new file
m.put(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload=write_response,
status=200
)
result = await skill.execute(context)
assert result.success is True
assert "created" in result.message.lower()
class TestObsidianListFilesSkill:
"""Integration tests for ObsidianListFilesSkill"""
@pytest.fixture
def skill(self):
"""Create ObsidianListFilesSkill instance"""
return ObsidianListFilesSkill()
@pytest.fixture
def context(self, sample_config):
"""Create command context with list parameters"""
return CommandContext(
command_name="test_list",
args={"folder_path": "Daily"},
config=sample_config
)
@pytest.mark.asyncio
async def test_list_files_success(self, skill, context):
"""Test successful file listing"""
mock_response = {
"files": [
{
"path": "Daily/2024-01-15.md",
"name": "2024-01-15.md",
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": 45}
},
{
"path": "Daily/2024-01-14.md",
"name": "2024-01-14.md",
"stat": {"ctime": 1642118400000, "mtime": 1642118400000, "size": 38}
}
]
}
with aioresponses() as m:
m.get(
"https://localhost:27123/vault/Daily/",
payload=mock_response,
status=200
)
result = await skill.execute(context)
assert result.success is True
assert len(result.data["files"]) == 2
assert result.data["folder_path"] == "Daily"
assert any(file["name"] == "2024-01-15.md" for file in result.data["files"])
@pytest.mark.asyncio
async def test_list_files_empty_folder(self, skill, context):
"""Test listing files in empty folder"""
mock_response = {"files": []}
with aioresponses() as m:
m.get(
"https://localhost:27123/vault/Daily/",
payload=mock_response,
status=200
)
result = await skill.execute(context)
assert result.success is True
assert len(result.data["files"]) == 0
@pytest.mark.asyncio
async def test_list_files_folder_not_found(self, skill, context):
"""Test listing files in non-existent folder"""
with aioresponses() as m:
m.get(
"https://localhost:27123/vault/Daily/",
status=404,
payload={"error": "Folder not found"}
)
result = await skill.execute(context)
assert result.success is False
assert "not found" in result.error.lower()
class TestObsidianSkillsIntegration:
"""Integration tests combining multiple Obsidian skills"""
@pytest.mark.asyncio
async def test_read_write_workflow(self, sample_config):
"""Test complete read-modify-write workflow"""
read_skill = ObsidianReadSkill()
write_skill = ObsidianWriteSkill()
# Read existing content
read_context = CommandContext(
command_name="read",
args={"file_path": "Daily/2024-01-15.md"},
config=sample_config
)
# Write modified content
write_context = CommandContext(
command_name="write",
args={
"file_path": "Knowledge/Processed/2024-01-15-summary.md",
"content": "# Summary\n\nProcessed content from daily journal."
},
config=sample_config
)
with aioresponses() as m:
# Mock read response
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload={
"content": "# 2024-01-15 Daily Journal\n\nOriginal content",
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": 45}
},
status=200
)
# Mock write response
m.put(
"https://localhost:27123/vault/Knowledge/Processed/2024-01-15-summary.md",
payload={
"path": "Knowledge/Processed/2024-01-15-summary.md",
"stat": {"ctime": 1642204900000, "mtime": 1642204900000, "size": 60}
},
status=200
)
# Execute read
read_result = await read_skill.execute(read_context)
assert read_result.success is True
# Execute write (in real scenario, content would be processed)
write_result = await write_skill.execute(write_context)
assert write_result.success is True
# Verify workflow completed successfully
assert read_result.data["content"] is not None
assert write_result.data["file_path"] == "Knowledge/Processed/2024-01-15-summary.md"
@pytest.mark.asyncio
async def test_list_and_read_multiple_files(self, sample_config):
"""Test listing files and reading multiple files"""
list_skill = ObsidianListFilesSkill()
read_skill = ObsidianReadSkill()
list_context = CommandContext(
command_name="list",
args={"folder_path": "Daily"},
config=sample_config
)
with aioresponses() as m:
# Mock list response
m.get(
"https://localhost:27123/vault/Daily/",
payload={
"files": [
{"path": "Daily/2024-01-15.md", "name": "2024-01-15.md"},
{"path": "Daily/2024-01-14.md", "name": "2024-01-14.md"}
]
},
status=200
)
# Mock read responses for each file
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload={"content": "Content 1", "stat": {}},
status=200
)
m.get(
"https://localhost:27123/vault/Daily/2024-01-14.md",
payload={"content": "Content 2", "stat": {}},
status=200
)
# List files
list_result = await list_skill.execute(list_context)
assert list_result.success is True
assert len(list_result.data["files"]) == 2
# Read each file
for file_info in list_result.data["files"]:
read_context = CommandContext(
command_name="read",
args={"file_path": file_info["path"]},
config=sample_config
)
read_result = await read_skill.execute(read_context)
assert read_result.success is True
assert read_result.data["content"] is not None
+488
View File
@@ -0,0 +1,488 @@
"""
Integration tests for OrganizeCommand.
Tests Commands with full skill chains to verify end-to-end functionality.
"""
import pytest
import json
from unittest.mock import AsyncMock, patch, Mock
from aioresponses import aioresponses
from agent_core import CommandContext, SkillResult, Agent
from commands.organize_command import OrganizeCommand
class TestOrganizeCommand:
"""Integration tests for OrganizeCommand"""
@pytest.fixture
def command(self):
"""Create OrganizeCommand instance"""
return OrganizeCommand()
@pytest.fixture
def context(self, sample_config):
"""Create command context for organize command"""
return CommandContext(
command_name="organize",
args={
"date": "2024-01-15",
"vault_path": sample_config["obsidian"]["vault_path"],
"daily_folder": "Daily"
},
config=sample_config
)
@pytest.fixture
def sample_journal_content(self):
"""Sample journal content for testing"""
return """# 2024-01-15 Daily Journal
## 今天的经历
- 完成了项目的重要里程碑,团队协作非常顺利
- 与客户进行了产品演示,获得了积极反馈
- 参加了技术分享会议,学到了新的架构模式
## 学到的东西
- 学会了新的Python异步编程技巧,提升了代码效率
- 理解了微服务架构的最佳实践
- 掌握了更好的错误处理和日志记录模式
## 遇到的问题
- API调用偶尔超时,影响用户体验
- 配置文件格式需要改进,当前格式不够灵活
- 团队沟通中存在信息不对称问题
## 今天的成就
- 成功部署了新版本到生产环境
- 解决了困扰团队一周的性能问题
- 获得了客户的正面评价
## 明天的计划
- 优化API调用的重试机制
- 重构配置管理模块
- 组织团队同步会议
"""
@pytest.mark.asyncio
async def test_organize_command_full_workflow_success(self, command, context, sample_journal_content):
"""Test complete organize command workflow with all skills"""
# Mock Claude API responses
analyze_response = {
"experiences": [
{
"title": "项目里程碑完成",
"description": "成功完成了项目的重要里程碑,团队协作非常顺利",
"category": "项目管理",
"importance": "high"
},
{
"title": "客户产品演示",
"description": "与客户进行了产品演示,获得了积极反馈",
"category": "客户关系",
"importance": "high"
}
],
"lessons": [
{
"title": "Python异步编程技巧",
"description": "学会了新的Python异步编程技巧,提升了代码效率",
"category": "技术学习",
"application": "可以应用到当前项目的API调用优化中"
},
{
"title": "微服务架构最佳实践",
"description": "理解了微服务架构的最佳实践",
"category": "架构设计",
"application": "用于指导下一个项目的架构设计"
}
],
"problems": [
{
"title": "API调用超时",
"description": "API调用偶尔超时,影响用户体验",
"severity": "medium",
"suggested_solution": "实现重试机制和超时处理"
}
],
"achievements": [
{
"title": "生产环境部署",
"description": "成功部署了新版本到生产环境",
"impact": "提升了系统稳定性和性能"
}
]
}
# Mock transformation responses
experience_note = """# 项目里程碑完成
## 经验描述
成功完成了项目的重要里程碑,团队协作非常顺利。这次经历展现了良好的项目管理能力和团队协作精神。
## 关键要点
- 项目管理技能得到提升
- 团队协作效率显著改善
- 里程碑按时完成
## 应用场景
这个经验可以应用到未来的项目管理中,特别是在设定和跟踪项目里程碑方面。
## 相关标签
#项目管理 #里程碑 #团队协作
---
*创建时间: 2024-01-15*
*来源: [[Daily/2024-01-15]]*
"""
lesson_note = """# Python异步编程技巧
## 学习内容
学会了新的Python异步编程技巧,提升了代码效率。
## 关键概念
- 异步编程模式
- 性能优化技巧
- 代码效率提升
## 实际应用
可以应用到当前项目的API调用优化中,提升系统响应速度。
## 相关标签
#技术学习 #Python #异步编程
---
*创建时间: 2024-01-15*
*来源: [[Daily/2024-01-15]]*
"""
with aioresponses() as m:
# Mock Obsidian API calls
# 1. Read daily journal
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload={
"content": sample_journal_content,
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": len(sample_journal_content)}
},
status=200
)
# 2. Write experience note
m.put(
"https://localhost:27123/vault/Knowledge/Experiences/项目里程碑完成.md",
payload={
"path": "Knowledge/Experiences/项目里程碑完成.md",
"stat": {"ctime": 1642204900000, "mtime": 1642204900000, "size": len(experience_note)}
},
status=200
)
# 3. Write lesson note
m.put(
"https://localhost:27123/vault/Knowledge/Lessons/Python异步编程技巧.md",
payload={
"path": "Knowledge/Lessons/Python异步编程技巧.md",
"stat": {"ctime": 1642204900000, "mtime": 1642204900000, "size": len(lesson_note)}
},
status=200
)
# 4. Additional notes for other categories (problems, achievements)
m.put(
"https://localhost:27123/vault/Knowledge/Problems/API调用超时.md",
payload={"path": "Knowledge/Problems/API调用超时.md", "stat": {}},
status=200
)
m.put(
"https://localhost:27123/vault/Knowledge/Achievements/生产环境部署.md",
payload={"path": "Knowledge/Achievements/生产环境部署.md", "stat": {}},
status=200
)
# Mock Claude API
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Mock analyze response
mock_analyze_message = Mock()
mock_analyze_message.content = [Mock(text=json.dumps(analyze_response, ensure_ascii=False))]
# Mock transform responses (one for each item to be transformed)
mock_transform_responses = [
Mock(content=[Mock(text=experience_note)]),
Mock(content=[Mock(text=experience_note)]), # Second experience
Mock(content=[Mock(text=lesson_note)]),
Mock(content=[Mock(text=lesson_note)]), # Second lesson
Mock(content=[Mock(text="# API调用超时\n\n问题描述...")]), # Problem note
Mock(content=[Mock(text="# 生产环境部署\n\n成就描述...")]) # Achievement note
]
# Set up responses: first analyze, then multiple transforms
mock_client.messages.create.side_effect = [mock_analyze_message] + mock_transform_responses
# Execute the organize command
result = await command.execute(context)
# Verify overall success
assert result.success is True
assert "organized successfully" in result.message.lower() or "completed" in result.message.lower()
# Verify data structure
assert "summary" in result.data
assert "created_notes" in result.data
# Verify created notes
created_notes = result.data["created_notes"]
assert len(created_notes) > 0
# Should have created notes for experiences, lessons, problems, achievements
note_types = [note.get("type") for note in created_notes]
expected_types = ["experience", "lesson", "problem", "achievement"]
for expected_type in expected_types:
assert any(expected_type in note_type for note_type in note_types if note_type)
# Verify API calls were made
assert mock_client.messages.create.call_count >= 2 # At least analyze + some transforms
@pytest.mark.asyncio
async def test_organize_command_journal_not_found(self, command, context):
"""Test organize command when daily journal doesn't exist"""
with aioresponses() as m:
# Mock journal file not found
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
status=404,
payload={"error": "File not found"}
)
result = await command.execute(context)
assert result.success is False
assert "not found" in result.error.lower() or "missing" in result.error.lower()
@pytest.mark.asyncio
async def test_organize_command_claude_api_error(self, command, context, sample_journal_content):
"""Test organize command when Claude API fails"""
with aioresponses() as m:
# Mock successful journal read
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload={
"content": sample_journal_content,
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": len(sample_journal_content)}
},
status=200
)
# Mock Claude API error
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
mock_client.messages.create.side_effect = Exception("Claude API rate limit exceeded")
result = await command.execute(context)
assert result.success is False
assert "api" in result.error.lower() or "claude" in result.error.lower()
@pytest.mark.asyncio
async def test_organize_command_partial_success(self, command, context, sample_journal_content):
"""Test organize command with partial success (some notes created, some failed)"""
analyze_response = {
"experiences": [
{
"title": "Test Experience",
"description": "Test description",
"category": "Test",
"importance": "medium"
}
],
"lessons": [],
"problems": [],
"achievements": []
}
with aioresponses() as m:
# Mock successful journal read
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload={
"content": sample_journal_content,
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": len(sample_journal_content)}
},
status=200
)
# Mock successful experience note creation
m.put(
"https://localhost:27123/vault/Knowledge/Experiences/Test Experience.md",
payload={
"path": "Knowledge/Experiences/Test Experience.md",
"stat": {"ctime": 1642204900000, "mtime": 1642204900000, "size": 100}
},
status=200
)
# Mock Claude API
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Mock analyze response
mock_analyze_message = Mock()
mock_analyze_message.content = [Mock(text=json.dumps(analyze_response, ensure_ascii=False))]
# Mock transform response
mock_transform_message = Mock()
mock_transform_message.content = [Mock(text="# Test Experience\n\nTransformed content")]
mock_client.messages.create.side_effect = [mock_analyze_message, mock_transform_message]
result = await command.execute(context)
# Should succeed even with minimal content
assert result.success is True
assert len(result.data["created_notes"]) >= 1
@pytest.mark.asyncio
async def test_organize_command_empty_journal(self, command, context):
"""Test organize command with empty journal content"""
with aioresponses() as m:
# Mock journal with empty content
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload={
"content": "",
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": 0}
},
status=200
)
result = await command.execute(context)
assert result.success is False
assert "empty" in result.error.lower() or "content" in result.error.lower()
class TestOrganizeCommandWithAgent:
"""Integration tests for OrganizeCommand within Agent context"""
@pytest.fixture
def agent(self, sample_config):
"""Create Agent with OrganizeCommand registered"""
agent = Agent("test_agent", sample_config)
agent.register_command(OrganizeCommand())
return agent
@pytest.mark.asyncio
async def test_agent_execute_organize_command(self, agent, sample_journal_content):
"""Test executing organize command through Agent"""
with aioresponses() as m:
# Mock Obsidian API
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload={
"content": sample_journal_content,
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": len(sample_journal_content)}
},
status=200
)
# Mock note creation (simplified - just one note)
m.put(
"https://localhost:27123/vault/Knowledge/Experiences/Test.md",
payload={"path": "Knowledge/Experiences/Test.md", "stat": {}},
status=200
)
# Mock Claude API
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
# Minimal response for testing
analyze_response = {
"experiences": [{"title": "Test", "description": "Test", "category": "Test"}],
"lessons": [],
"problems": [],
"achievements": []
}
mock_analyze_message = Mock()
mock_analyze_message.content = [Mock(text=json.dumps(analyze_response, ensure_ascii=False))]
mock_transform_message = Mock()
mock_transform_message.content = [Mock(text="# Test\n\nTest content")]
mock_client.messages.create.side_effect = [mock_analyze_message, mock_transform_message]
# Execute command through agent
result = await agent.execute_command(
"organize",
args={"date": "2024-01-15"},
options={"verbose": True}
)
assert result.success is True
assert result.data is not None
@pytest.mark.asyncio
async def test_agent_execute_organize_by_alias(self, agent, sample_journal_content):
"""Test executing organize command by alias through Agent"""
with aioresponses() as m:
# Mock minimal successful workflow
m.get(
"https://localhost:27123/vault/Daily/2024-01-15.md",
payload={
"content": sample_journal_content,
"stat": {"ctime": 1642204800000, "mtime": 1642204800000, "size": len(sample_journal_content)}
},
status=200
)
# Mock Claude API with minimal response
with patch('skills.claude_skill.anthropic') as mock_anthropic:
mock_client = Mock()
mock_anthropic.Anthropic.return_value = mock_client
analyze_response = {"experiences": [], "lessons": [], "problems": [], "achievements": []}
mock_message = Mock()
mock_message.content = [Mock(text=json.dumps(analyze_response))]
mock_client.messages.create.return_value = mock_message
# Execute by alias
result = await agent.execute_command("org", args={"date": "2024-01-15"})
assert result.success is True
@pytest.mark.asyncio
async def test_agent_command_info(self, agent):
"""Test getting command information through Agent"""
commands_info = agent.get_commands_info()
assert "organize" in commands_info["commands"]
organize_info = commands_info["commands"]["organize"]
assert organize_info["name"] == "organize"
assert organize_info["description"] == "分析和整理日记内容,提取经验和要点"
assert "org" in organize_info["aliases"]
assert "organize-journal" in organize_info["aliases"]
# Verify skills are registered
assert len(organize_info["skills"]) > 0
skill_names = list(organize_info["skills"].keys())
expected_skills = ["obsidian_read", "obsidian_write", "obsidian_append", "claude_analyze", "claude_transform"]
for expected_skill in expected_skills:
assert any(expected_skill in skill_name for skill_name in skill_names)
@@ -0,0 +1,512 @@
"""
Integration tests for real Claude API endpoints
Tests validation with default Anthropic API, proxy server configurations, and different model selections
Requirements: 1.3, 2.4
"""
import os
import pytest
import asyncio
import tempfile
import yaml
from pathlib import Path
from typing import Dict, Any, Optional
from unittest.mock import patch, Mock
# Import the modules we're testing
from config_validation import ClaudeAPIConfig
from claude_api_client import ClaudeAPIClient
from error_handling import APIError
class TestRealAPIEndpoints:
"""Test with real API endpoints - requires valid API key"""
def setup_method(self):
"""Set up test fixtures"""
self.temp_dir = Path(tempfile.mkdtemp())
# Check if we have a real API key for testing
self.api_key = os.getenv('ANTHROPIC_API_KEY')
self.has_real_api_key = (
self.api_key and
self.api_key.startswith('sk-ant-') and
len(self.api_key) > 50
)
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
@pytest.mark.skipif(
not os.getenv('ANTHROPIC_API_KEY'),
reason="Requires ANTHROPIC_API_KEY environment variable for real API testing"
)
@pytest.mark.asyncio
async def test_default_anthropic_api_validation(self):
"""Test validation with default Anthropic API"""
if not self.has_real_api_key:
pytest.skip("Requires valid ANTHROPIC_API_KEY for real API testing")
config = ClaudeAPIConfig(
api_key=self.api_key,
api_url="https://api.anthropic.com",
model="claude-3-5-sonnet-20241022"
)
client = ClaudeAPIClient(config)
# Test connection validation
is_valid = await client.validate_connection()
assert is_valid is True
# Test model availability
is_model_valid = await client.validate_model_availability()
assert is_model_valid is True
# Test comprehensive connectivity
results = await client.test_api_connectivity()
assert results['overall_status'] == 'success'
assert results['connection_test']['status'] == 'success'
assert results['model_test']['status'] == 'success'
assert results['authentication_test']['status'] == 'success'
@pytest.mark.skipif(
not os.getenv('ANTHROPIC_API_KEY'),
reason="Requires ANTHROPIC_API_KEY environment variable for real API testing"
)
@pytest.mark.asyncio
async def test_different_model_selections(self):
"""Test different Claude model selections work correctly"""
if not self.has_real_api_key:
pytest.skip("Requires valid ANTHROPIC_API_KEY for real API testing")
# Test different models that should be available
models_to_test = [
"claude-3-5-sonnet-20241022",
"claude-3-haiku-20240307",
# Note: claude-3-opus may not be available in all regions/accounts
]
for model in models_to_test:
config = ClaudeAPIConfig(
api_key=self.api_key,
api_url="https://api.anthropic.com",
model=model
)
client = ClaudeAPIClient(config)
try:
# Test that the model is available
is_valid = await client.validate_model_availability()
assert is_valid is True, f"Model {model} should be available"
# Test a simple API call with the model
response = await client.create_message([
{"role": "user", "content": "Hello"}
])
assert response is not None
assert hasattr(response, 'content')
assert len(response.content) > 0
except APIError as e:
# Some models might not be available in all regions/accounts
if "model" in str(e).lower() and "not found" in str(e).lower():
pytest.skip(f"Model {model} not available in this account/region")
else:
raise
@pytest.mark.asyncio
async def test_invalid_api_key_handling(self):
"""Test handling of invalid API keys"""
config = ClaudeAPIConfig(
api_key="sk-ant-invalid-key-12345678901234567890123456789012345678901234567890",
api_url="https://api.anthropic.com",
model="claude-3-5-sonnet-20241022"
)
client = ClaudeAPIClient(config)
with pytest.raises(APIError, match="Invalid API key or unauthorized access"):
await client.validate_connection()
@pytest.mark.asyncio
async def test_invalid_api_url_handling(self):
"""Test handling of invalid API URLs"""
config = ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
api_url="https://nonexistent-api.example.com",
model="claude-3-5-sonnet-20241022"
)
client = ClaudeAPIClient(config)
with pytest.raises(APIError, match="Connection error"):
await client.validate_connection()
@pytest.mark.asyncio
async def test_invalid_model_handling(self):
"""Test handling of invalid model names"""
if not self.has_real_api_key:
pytest.skip("Requires valid ANTHROPIC_API_KEY for real API testing")
config = ClaudeAPIConfig(
api_key=self.api_key,
api_url="https://api.anthropic.com",
model="claude-nonexistent-model"
)
client = ClaudeAPIClient(config)
with pytest.raises(APIError, match="Model.*not found"):
await client.validate_model_availability()
class TestProxyServerConfigurations:
"""Test proxy server configurations"""
def setup_method(self):
"""Set up test fixtures"""
self.temp_dir = Path(tempfile.mkdtemp())
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
@pytest.mark.asyncio
async def test_localhost_proxy_configuration(self):
"""Test configuration for localhost proxy servers"""
config = ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
api_url="https://localhost:8080",
model="claude-3-5-sonnet-20241022"
)
client = ClaudeAPIClient(config)
# Test that SSL context is configured for localhost
async with client.create_http_session() as session:
# Should not raise SSL errors for localhost
assert session is not None
# Verify SSL context is configured for localhost
connector = session.connector
assert connector.ssl is not None
# For localhost, SSL verification should be disabled
assert not connector.ssl.check_hostname
@pytest.mark.asyncio
async def test_custom_proxy_url_configuration(self):
"""Test configuration for custom proxy URLs"""
proxy_urls = [
"https://proxy.example.com:8080",
"https://claude-proxy.internal:443",
"http://localhost:3128"
]
for proxy_url in proxy_urls:
config = ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
api_url=proxy_url,
model="claude-3-5-sonnet-20241022"
)
client = ClaudeAPIClient(config)
# Test client initialization
assert client.base_url == proxy_url
# Test client info
info = client.get_client_info()
assert info['api_url'] == proxy_url
assert info['is_custom_endpoint'] is True
def test_proxy_configuration_validation(self):
"""Test validation of proxy server configurations"""
# Valid proxy configurations
valid_configs = [
{
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': 'https://proxy.example.com:8080',
'model': 'claude-3-5-sonnet-20241022'
},
{
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': 'http://localhost:3128',
'model': 'claude-3-5-sonnet-20241022'
}
]
for config_data in valid_configs:
config = ClaudeAPIConfig(**config_data)
assert config.api_url == config_data['api_url']
# Invalid proxy configurations
invalid_configs = [
{
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': 'ftp://proxy.example.com:8080', # Invalid protocol
'model': 'claude-3-5-sonnet-20241022'
},
{
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': 'not-a-url', # Invalid URL format
'model': 'claude-3-5-sonnet-20241022'
}
]
for config_data in invalid_configs:
with pytest.raises(ValueError):
ClaudeAPIConfig(**config_data)
class TestConfigurationValidationClass:
"""Test configuration validation class methods"""
@pytest.mark.skipif(
not os.getenv('ANTHROPIC_API_KEY'),
reason="Requires ANTHROPIC_API_KEY environment variable for real API testing"
)
@pytest.mark.asyncio
async def test_validate_configuration_quick_test(self):
"""Test quick configuration validation"""
api_key = os.getenv('ANTHROPIC_API_KEY')
if not api_key or not api_key.startswith('sk-ant-'):
pytest.skip("Requires valid ANTHROPIC_API_KEY for real API testing")
config = ClaudeAPIConfig(
api_key=api_key,
api_url="https://api.anthropic.com",
model="claude-3-5-sonnet-20241022"
)
results = await ClaudeAPIClient.validate_configuration(config, quick_test=True)
assert results['config_valid'] is True
assert results['connection_valid'] is True
assert results['model_valid'] is True
assert len(results['errors']) == 0
@pytest.mark.skipif(
not os.getenv('ANTHROPIC_API_KEY'),
reason="Requires ANTHROPIC_API_KEY environment variable for real API testing"
)
@pytest.mark.asyncio
async def test_validate_configuration_comprehensive_test(self):
"""Test comprehensive configuration validation"""
api_key = os.getenv('ANTHROPIC_API_KEY')
if not api_key or not api_key.startswith('sk-ant-'):
pytest.skip("Requires valid ANTHROPIC_API_KEY for real API testing")
config = ClaudeAPIConfig(
api_key=api_key,
api_url="https://api.anthropic.com",
model="claude-3-5-sonnet-20241022"
)
results = await ClaudeAPIClient.validate_configuration(config, quick_test=False)
assert results['config_valid'] is True
assert results['connection_valid'] is True
assert results['model_valid'] is True
assert len(results['errors']) == 0
# Should have detailed test results
assert 'detailed_tests' in results
detailed = results['detailed_tests']
assert detailed['overall_status'] == 'success'
assert detailed['connection_test']['status'] == 'success'
assert detailed['model_test']['status'] == 'success'
assert detailed['authentication_test']['status'] == 'success'
@pytest.mark.asyncio
async def test_validate_configuration_invalid_key(self):
"""Test configuration validation with invalid API key"""
config = ClaudeAPIConfig(
api_key="sk-ant-invalid-key-12345678901234567890123456789012345678901234567890",
api_url="https://api.anthropic.com",
model="claude-3-5-sonnet-20241022"
)
results = await ClaudeAPIClient.validate_configuration(config, quick_test=True)
assert results['config_valid'] is True # Config format is valid
assert results['connection_valid'] is False # But connection fails
assert len(results['errors']) > 0
assert any('Invalid API key' in error for error in results['errors'])
@pytest.mark.asyncio
async def test_validate_configuration_custom_endpoint(self):
"""Test configuration validation with custom endpoint"""
config = ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
api_url="https://custom-claude-api.example.com",
model="claude-3-5-sonnet-20241022"
)
results = await ClaudeAPIClient.validate_configuration(config, quick_test=True)
assert results['config_valid'] is True
assert results['config_info']['is_custom_endpoint'] is True
assert results['config_info']['api_url'] == "https://custom-claude-api.example.com"
# Should have warning about custom endpoint
assert len(results['warnings']) > 0
assert any('custom API endpoint' in warning for warning in results['warnings'])
class TestEnvironmentVariableScenarios:
"""Test environment variable scenarios in real configurations"""
def setup_method(self):
"""Set up test fixtures"""
self.temp_dir = Path(tempfile.mkdtemp())
# Store original environment variables
self.original_env = {}
test_vars = ['TEST_CLAUDE_API_KEY', 'TEST_CLAUDE_API_URL', 'TEST_CLAUDE_MODEL']
for var in test_vars:
if var in os.environ:
self.original_env[var] = os.environ[var]
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
# Clean up test environment variables
test_vars = ['TEST_CLAUDE_API_KEY', 'TEST_CLAUDE_API_URL', 'TEST_CLAUDE_MODEL']
for var in test_vars:
if var in os.environ:
del os.environ[var]
# Restore original environment variables
for var, value in self.original_env.items():
os.environ[var] = value
def create_test_config_file(self, config_data: Dict[str, Any]) -> Path:
"""Create a test configuration file"""
config_file = self.temp_dir / 'test_config.yaml'
with config_file.open('w') as f:
yaml.dump(config_data, f)
return config_file
def test_environment_variable_configuration_loading(self):
"""Test loading configuration with environment variables"""
# Set up environment variables
os.environ['TEST_CLAUDE_API_KEY'] = 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
os.environ['TEST_CLAUDE_API_URL'] = 'https://custom-api.example.com'
os.environ['TEST_CLAUDE_MODEL'] = 'claude-3-haiku-20240307'
config_data = {
'claude': {
'api_key': '${TEST_CLAUDE_API_KEY}',
'api_url': '${TEST_CLAUDE_API_URL}',
'model': '${TEST_CLAUDE_MODEL}'
}
}
config_file = self.create_test_config_file(config_data)
# Load configuration through the configuration loader
from configuration_loader import ConfigurationLoader
loader = ConfigurationLoader()
loaded_config = loader.load_config(config_file)
# Validate the loaded configuration
claude_config = ClaudeAPIConfig(**loaded_config['claude'])
assert claude_config.api_key == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
assert claude_config.api_url == 'https://custom-api.example.com'
assert claude_config.model == 'claude-3-haiku-20240307'
def test_environment_variable_defaults_in_configuration(self):
"""Test environment variable defaults in configuration"""
config_data = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': '${CLAUDE_API_URL:-https://api.anthropic.com}',
'model': '${CLAUDE_MODEL:-claude-3-5-sonnet-20241022}',
'max_tokens': '${CLAUDE_MAX_TOKENS:-4096}',
'temperature': '${CLAUDE_TEMPERATURE:-0.7}'
}
}
config_file = self.create_test_config_file(config_data)
# Load configuration through the configuration loader
from configuration_loader import ConfigurationLoader
loader = ConfigurationLoader()
loaded_config = loader.load_config(config_file)
# Should use defaults since environment variables are not set
claude_config = ClaudeAPIConfig(**loaded_config['claude'])
assert claude_config.api_url == 'https://api.anthropic.com'
assert claude_config.model == 'claude-3-5-sonnet-20241022'
assert claude_config.max_tokens == 4096
assert claude_config.temperature == 0.7
def test_mixed_environment_and_direct_configuration(self):
"""Test mixed configuration (some values from files, some from environment)"""
# Set only some environment variables
os.environ['TEST_CLAUDE_API_KEY'] = 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
os.environ['TEST_CLAUDE_API_URL'] = 'https://proxy.example.com'
config_data = {
'claude': {
'api_key': '${TEST_CLAUDE_API_KEY}',
'api_url': '${TEST_CLAUDE_API_URL}',
'model': 'claude-3-5-sonnet-20241022', # Direct value
'max_tokens': 8192, # Direct value
'temperature': '${CLAUDE_TEMPERATURE:-0.5}' # Default value
}
}
config_file = self.create_test_config_file(config_data)
# Load configuration through the configuration loader
from configuration_loader import ConfigurationLoader
loader = ConfigurationLoader()
loaded_config = loader.load_config(config_file)
claude_config = ClaudeAPIConfig(**loaded_config['claude'])
# Environment variables should be expanded
assert claude_config.api_key == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
assert claude_config.api_url == 'https://proxy.example.com'
# Direct values should be preserved
assert claude_config.model == 'claude-3-5-sonnet-20241022'
assert claude_config.max_tokens == 8192
# Default should be used
assert claude_config.temperature == 0.5
def test_missing_required_environment_variable(self):
"""Test error handling for missing required environment variables"""
config_data = {
'claude': {
'api_key': '${MISSING_API_KEY}', # Required but not set
'api_url': 'https://api.anthropic.com',
'model': 'claude-3-5-sonnet-20241022'
}
}
config_file = self.create_test_config_file(config_data)
# Should raise error for missing required environment variable
from configuration_loader import ConfigurationLoader, EnvironmentVariableError
loader = ConfigurationLoader()
with pytest.raises(EnvironmentVariableError, match="Environment variable 'MISSING_API_KEY' is not set"):
loader.load_config(config_file)
if __name__ == "__main__":
# Run tests with pytest
pytest.main([__file__, "-v"])
+1
View File
@@ -0,0 +1 @@
# Property-based tests package
+1
View File
@@ -0,0 +1 @@
# Unit tests package
+457
View File
@@ -0,0 +1,457 @@
"""
Unit tests for agent_core module.
Tests Agent, Command, Skill, SkillResult, and related classes.
"""
import pytest
from unittest.mock import Mock, AsyncMock
from datetime import datetime
from agent_core import (
Agent, Command, Skill, SkillResult, SkillChain,
CommandContext, SkillType
)
class TestSkillResult:
"""Test SkillResult class"""
def test_successful_result_creation(self):
"""Test creating a successful SkillResult"""
result = SkillResult(success=True, data={"key": "value"}, message="Success")
assert result.success is True
assert result.data == {"key": "value"}
assert result.message == "Success"
assert result.error is None
assert result.timestamp is not None
def test_failed_result_creation(self):
"""Test creating a failed SkillResult"""
result = SkillResult(success=False, error="Something went wrong", message="Failed")
assert result.success is False
assert result.error == "Something went wrong"
assert result.message == "Failed"
assert result.data is None
def test_failed_result_without_error_raises_exception(self):
"""Test that failed result without error message raises ValueError"""
with pytest.raises(ValueError, match="Failed results must include error message"):
SkillResult(success=False)
def test_successful_result_with_error_raises_exception(self):
"""Test that successful result with error message raises ValueError"""
with pytest.raises(ValueError, match="Successful results should not include error message"):
SkillResult(success=True, error="This shouldn't be here")
def test_to_dict(self):
"""Test converting SkillResult to dictionary"""
result = SkillResult(success=True, data={"test": "data"}, message="Test")
result_dict = result.to_dict()
assert isinstance(result_dict, dict)
assert result_dict["success"] is True
assert result_dict["data"] == {"test": "data"}
assert result_dict["message"] == "Test"
assert "timestamp" in result_dict
def test_to_json(self):
"""Test converting SkillResult to JSON string"""
result = SkillResult(success=True, message="Test")
json_str = result.to_json()
assert isinstance(json_str, str)
assert '"success": true' in json_str
assert '"message": "Test"' in json_str
class TestCommandContext:
"""Test CommandContext class"""
def test_context_creation(self):
"""Test creating CommandContext"""
context = CommandContext(
command_name="test_command",
args={"arg1": "value1"},
options={"option1": True},
config={"config_key": "config_value"}
)
assert context.command_name == "test_command"
assert context.args == {"arg1": "value1"}
assert context.options == {"option1": True}
assert context.config == {"config_key": "config_value"}
assert isinstance(context.metadata, dict)
def test_context_defaults(self):
"""Test CommandContext with default values"""
context = CommandContext(command_name="test")
assert context.command_name == "test"
assert context.args == {}
assert context.options == {}
assert context.config is None
assert context.metadata == {}
def test_to_dict(self):
"""Test converting CommandContext to dictionary"""
context = CommandContext(command_name="test", args={"key": "value"})
context_dict = context.to_dict()
assert isinstance(context_dict, dict)
assert context_dict["command_name"] == "test"
assert context_dict["args"] == {"key": "value"}
class MockSkill(Skill):
"""Mock Skill implementation for testing"""
def __init__(self, name: str, should_succeed: bool = True):
super().__init__(name, SkillType.READ, f"Mock skill {name}")
self.should_succeed = should_succeed
self.execute_called = False
self.execute_context = None
self.execute_kwargs = None
async def execute(self, context: CommandContext, **kwargs) -> SkillResult:
self.execute_called = True
self.execute_context = context
self.execute_kwargs = kwargs
if self.should_succeed:
return SkillResult(success=True, data={"skill": self.name}, message=f"{self.name} executed")
else:
return SkillResult(success=False, error=f"{self.name} failed", message="Execution failed")
class TestSkill:
"""Test Skill base class"""
def test_skill_creation(self):
"""Test creating a Skill"""
skill = MockSkill("test_skill")
assert skill.name == "test_skill"
assert skill.skill_type == SkillType.READ
assert skill.description == "Mock skill test_skill"
assert skill.logger is not None
def test_get_info(self):
"""Test getting skill information"""
skill = MockSkill("test_skill")
info = skill.get_info()
assert info["name"] == "test_skill"
assert info["type"] == "read"
assert info["description"] == "Mock skill test_skill"
@pytest.mark.asyncio
async def test_skill_execute_success(self):
"""Test successful skill execution"""
skill = MockSkill("test_skill", should_succeed=True)
context = CommandContext(command_name="test")
result = await skill.execute(context, param1="value1")
assert skill.execute_called is True
assert skill.execute_context == context
assert skill.execute_kwargs == {"param1": "value1"}
assert result.success is True
assert result.data == {"skill": "test_skill"}
@pytest.mark.asyncio
async def test_skill_execute_failure(self):
"""Test failed skill execution"""
skill = MockSkill("test_skill", should_succeed=False)
context = CommandContext(command_name="test")
result = await skill.execute(context)
assert result.success is False
assert result.error == "test_skill failed"
class TestSkillChain:
"""Test SkillChain class"""
def test_skill_chain_creation(self):
"""Test creating a SkillChain"""
chain = SkillChain("test_chain", "Test chain description")
assert chain.name == "test_chain"
assert chain.description == "Test chain description"
assert chain.skills == []
assert chain.logger is not None
def test_add_skill(self):
"""Test adding skills to chain"""
chain = SkillChain("test_chain")
skill1 = MockSkill("skill1")
skill2 = MockSkill("skill2")
result = chain.add_skill(skill1, {"param1": "value1"})
chain.add_skill(skill2)
assert result == chain # Test fluent interface
assert len(chain.skills) == 2
assert chain.skills[0] == (skill1, {"param1": "value1"})
assert chain.skills[1] == (skill2, {})
@pytest.mark.asyncio
async def test_skill_chain_execute_success(self):
"""Test successful skill chain execution"""
chain = SkillChain("test_chain")
skill1 = MockSkill("skill1", should_succeed=True)
skill2 = MockSkill("skill2", should_succeed=True)
chain.add_skill(skill1).add_skill(skill2)
context = CommandContext(command_name="test")
result = await chain.execute(context)
assert result.success is True
assert skill1.execute_called is True
assert skill2.execute_called is True
# Check that skill1 result was passed to context metadata
assert "skill1_result" in context.metadata
@pytest.mark.asyncio
async def test_skill_chain_execute_failure(self):
"""Test skill chain execution with failure"""
chain = SkillChain("test_chain")
skill1 = MockSkill("skill1", should_succeed=True)
skill2 = MockSkill("skill2", should_succeed=False)
skill3 = MockSkill("skill3", should_succeed=True)
chain.add_skill(skill1).add_skill(skill2).add_skill(skill3)
context = CommandContext(command_name="test")
result = await chain.execute(context)
assert result.success is False
assert skill1.execute_called is True
assert skill2.execute_called is True
assert skill3.execute_called is False # Should not execute after failure
def test_get_info(self):
"""Test getting skill chain information"""
chain = SkillChain("test_chain", "Test description")
skill1 = MockSkill("skill1")
skill2 = MockSkill("skill2")
chain.add_skill(skill1).add_skill(skill2)
info = chain.get_info()
assert info["name"] == "test_chain"
assert info["description"] == "Test description"
assert len(info["skills"]) == 2
assert info["skills"][0]["name"] == "skill1"
assert info["skills"][1]["name"] == "skill2"
class MockCommand(Command):
"""Mock Command implementation for testing"""
def __init__(self, name: str, should_succeed: bool = True):
super().__init__(name, f"Mock command {name}", ["mock_alias"])
self.should_succeed = should_succeed
self.execute_called = False
self.execute_context = None
async def execute(self, context: CommandContext) -> SkillResult:
self.execute_called = True
self.execute_context = context
if self.should_succeed:
return SkillResult(success=True, data={"command": self.name}, message=f"{self.name} executed")
else:
return SkillResult(success=False, error=f"{self.name} failed", message="Command failed")
class TestCommand:
"""Test Command base class"""
def test_command_creation(self):
"""Test creating a Command"""
command = MockCommand("test_command")
assert command.name == "test_command"
assert command.description == "Mock command test_command"
assert command.aliases == ["mock_alias"]
assert command.skills == {}
assert command.skill_chains == {}
assert command.logger is not None
def test_register_skill(self):
"""Test registering skills with command"""
command = MockCommand("test_command")
skill = MockSkill("test_skill")
result = command.register_skill(skill)
assert result == command # Test fluent interface
assert command.skills["test_skill"] == skill
def test_register_skill_chain(self):
"""Test registering skill chains with command"""
command = MockCommand("test_command")
chain = SkillChain("test_chain")
result = command.register_skill_chain(chain)
assert result == command # Test fluent interface
assert command.skill_chains["test_chain"] == chain
@pytest.mark.asyncio
async def test_command_execute_success(self):
"""Test successful command execution"""
command = MockCommand("test_command", should_succeed=True)
context = CommandContext(command_name="test_command")
result = await command.execute(context)
assert command.execute_called is True
assert command.execute_context == context
assert result.success is True
assert result.data == {"command": "test_command"}
@pytest.mark.asyncio
async def test_command_execute_failure(self):
"""Test failed command execution"""
command = MockCommand("test_command", should_succeed=False)
context = CommandContext(command_name="test_command")
result = await command.execute(context)
assert result.success is False
assert result.error == "test_command failed"
def test_get_info(self):
"""Test getting command information"""
command = MockCommand("test_command")
skill = MockSkill("test_skill")
chain = SkillChain("test_chain")
command.register_skill(skill).register_skill_chain(chain)
info = command.get_info()
assert info["name"] == "test_command"
assert info["description"] == "Mock command test_command"
assert info["aliases"] == ["mock_alias"]
assert "test_skill" in info["skills"]
assert "test_chain" in info["skill_chains"]
class TestAgent:
"""Test Agent class"""
def test_agent_creation(self):
"""Test creating an Agent"""
config = {"log_level": "DEBUG", "test_key": "test_value"}
agent = Agent("test_agent", config)
assert agent.name == "test_agent"
assert agent.config == config
assert agent.commands == {}
assert agent.command_aliases == {}
assert agent.logger is not None
def test_agent_creation_without_config(self):
"""Test creating an Agent without config"""
agent = Agent("test_agent")
assert agent.name == "test_agent"
assert agent.config == {}
def test_register_command(self):
"""Test registering commands with agent"""
agent = Agent("test_agent")
command = MockCommand("test_command")
result = agent.register_command(command)
assert result == agent # Test fluent interface
assert agent.commands["test_command"] == command
assert agent.command_aliases["mock_alias"] == "test_command"
@pytest.mark.asyncio
async def test_execute_command_success(self):
"""Test successful command execution through agent"""
agent = Agent("test_agent")
command = MockCommand("test_command", should_succeed=True)
agent.register_command(command)
result = await agent.execute_command("test_command", {"arg1": "value1"}, {"opt1": True})
assert result.success is True
assert command.execute_called is True
assert command.execute_context.command_name == "test_command"
assert command.execute_context.args == {"arg1": "value1"}
assert command.execute_context.options == {"opt1": True}
@pytest.mark.asyncio
async def test_execute_command_by_alias(self):
"""Test executing command by alias"""
agent = Agent("test_agent")
command = MockCommand("test_command")
agent.register_command(command)
result = await agent.execute_command("mock_alias")
assert result.success is True
assert command.execute_called is True
@pytest.mark.asyncio
async def test_execute_unknown_command(self):
"""Test executing unknown command"""
agent = Agent("test_agent")
result = await agent.execute_command("unknown_command")
assert result.success is False
assert "未知命令: unknown_command" in result.error
@pytest.mark.asyncio
async def test_execute_command_with_exception(self):
"""Test command execution with exception"""
agent = Agent("test_agent")
command = MockCommand("test_command")
# Mock the execute method to raise an exception
async def mock_execute_with_exception(context):
raise RuntimeError("Test exception")
command.execute = mock_execute_with_exception
agent.register_command(command)
result = await agent.execute_command("test_command")
assert result.success is False
assert "Test exception" in result.error
def test_get_commands_info(self):
"""Test getting all commands information"""
agent = Agent("test_agent")
command1 = MockCommand("command1")
command2 = MockCommand("command2")
agent.register_command(command1).register_command(command2)
info = agent.get_commands_info()
assert info["agent_name"] == "test_agent"
assert "command1" in info["commands"]
assert "command2" in info["commands"]
assert info["aliases"]["mock_alias"] == "command2" # Last registered wins
def test_list_commands(self):
"""Test listing all available commands"""
agent = Agent("test_agent")
command1 = MockCommand("command1")
command2 = MockCommand("command2")
agent.register_command(command1).register_command(command2)
commands = agent.list_commands()
assert "command1" in commands
assert "command2" in commands
assert len(commands) == 2
+710
View File
@@ -0,0 +1,710 @@
"""
Unit tests for Claude API configuration system
Tests configuration loading, validation, environment variable expansion, and migration
"""
import os
import pytest
import tempfile
import yaml
from pathlib import Path
from unittest.mock import Mock, patch, AsyncMock
from typing import Dict, Any
# Import the modules we're testing
from config_validation import (
ClaudeAPIConfig, ConfigurationValidator, SystemConfig,
ObsidianConfig, ObsidianRestAPIConfig, JournalConfig,
OutputConfig, AnalysisConfig, LoggingConfig
)
from configuration_loader import ConfigurationLoader, ConfigurationError, EnvironmentVariableError
from configuration_migrator import ConfigurationMigrator
from claude_api_client import ClaudeAPIClient
from error_handling import ClaudeConfigurationError, ClaudeAPIURLError, ClaudeModelValidationError
class TestClaudeAPIConfig:
"""Test ClaudeAPIConfig validation"""
def test_valid_claude_config(self):
"""Test creating valid Claude API configuration"""
config = ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
api_url="https://api.anthropic.com",
model="claude-3-5-sonnet-20241022",
max_tokens=4096,
temperature=0.7
)
assert config.api_key.startswith("sk-ant-")
assert config.api_url == "https://api.anthropic.com"
assert config.model == "claude-3-5-sonnet-20241022"
assert config.max_tokens == 4096
assert config.temperature == 0.7
def test_default_values(self):
"""Test default values are applied correctly"""
config = ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890"
)
assert config.api_url == "https://api.anthropic.com"
assert config.model == "claude-3-5-sonnet-20241022"
assert config.max_tokens == 4096
assert config.temperature == 0.7
def test_custom_api_url(self):
"""Test custom API URL validation"""
config = ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
api_url="https://custom-claude-api.example.com"
)
assert config.api_url == "https://custom-claude-api.example.com"
def test_api_url_trailing_slash_removal(self):
"""Test that trailing slashes are removed from API URLs"""
config = ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
api_url="https://api.anthropic.com/"
)
assert config.api_url == "https://api.anthropic.com"
def test_invalid_api_url_format(self):
"""Test validation of invalid API URL formats"""
with pytest.raises(ValueError, match="Invalid URL format"):
ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
api_url="not-a-url"
)
def test_invalid_api_url_protocol(self):
"""Test validation of invalid URL protocols"""
with pytest.raises(ValueError, match="URL must use http or https protocol"):
ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
api_url="ftp://api.anthropic.com"
)
def test_invalid_api_key_format(self):
"""Test validation of invalid API key formats"""
with pytest.raises(ValueError, match="Claude API key should start with"):
ClaudeAPIConfig(api_key="invalid-key")
def test_api_key_too_short(self):
"""Test validation of API keys that are too short"""
with pytest.raises(ValueError, match="Claude API key appears to be too short"):
ClaudeAPIConfig(api_key="sk-ant-short")
def test_valid_model_names(self):
"""Test validation of valid model names"""
valid_models = [
"claude-3-opus-20240229",
"claude-3-sonnet-20240229",
"claude-3-haiku-20240307",
"claude-3-5-sonnet-20241022",
"claude-3-5-haiku-20241022",
"claude-3-opus-latest",
"claude-3-sonnet-latest",
"claude-3-haiku-latest",
"claude-3-5-sonnet-latest",
"claude-3-5-haiku-latest"
]
for model in valid_models:
config = ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
model=model
)
assert config.model == model
def test_invalid_model_name(self):
"""Test validation of invalid model names"""
with pytest.raises(ValueError, match="Invalid model name"):
ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
model="invalid-model"
)
def test_invalid_max_tokens(self):
"""Test validation of invalid max_tokens values"""
with pytest.raises(ValueError):
ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
max_tokens=0
)
with pytest.raises(ValueError):
ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
max_tokens=300000 # Too high
)
def test_invalid_temperature(self):
"""Test validation of invalid temperature values"""
with pytest.raises(ValueError):
ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
temperature=-0.1
)
with pytest.raises(ValueError):
ClaudeAPIConfig(
api_key="sk-ant-test-key-12345678901234567890123456789012345678901234567890",
temperature=1.1
)
class TestConfigurationLoader:
"""Test ConfigurationLoader environment variable expansion"""
def setup_method(self):
"""Set up test fixtures"""
self.loader = ConfigurationLoader()
self.temp_dir = Path(tempfile.mkdtemp())
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
def test_simple_env_var_expansion(self):
"""Test simple environment variable expansion"""
os.environ['TEST_API_KEY'] = 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
config_data = {
'claude': {
'api_key': '${TEST_API_KEY}'
}
}
expanded = self.loader.expand_environment_variables(config_data)
assert expanded['claude']['api_key'] == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
# Clean up
del os.environ['TEST_API_KEY']
def test_env_var_with_default(self):
"""Test environment variable expansion with default values"""
config_data = {
'claude': {
'api_url': '${CLAUDE_API_URL:-https://api.anthropic.com}',
'model': '${CLAUDE_MODEL:-claude-3-5-sonnet-20241022}'
}
}
expanded = self.loader.expand_environment_variables(config_data)
assert expanded['claude']['api_url'] == 'https://api.anthropic.com'
assert expanded['claude']['model'] == 'claude-3-5-sonnet-20241022'
def test_env_var_override_default(self):
"""Test environment variable overriding default values"""
os.environ['CLAUDE_API_URL'] = 'https://custom-api.example.com'
config_data = {
'claude': {
'api_url': '${CLAUDE_API_URL:-https://api.anthropic.com}'
}
}
expanded = self.loader.expand_environment_variables(config_data)
assert expanded['claude']['api_url'] == 'https://custom-api.example.com'
# Clean up
del os.environ['CLAUDE_API_URL']
def test_missing_required_env_var(self):
"""Test error handling for missing required environment variables"""
config_data = {
'claude': {
'api_key': '${MISSING_API_KEY}'
}
}
with pytest.raises(EnvironmentVariableError, match="Environment variable 'MISSING_API_KEY' is not set"):
self.loader.expand_environment_variables(config_data)
def test_nested_env_var_expansion(self):
"""Test environment variable expansion in nested structures"""
os.environ['VAULT_PATH'] = '/test/vault'
os.environ['API_KEY'] = 'test-key'
config_data = {
'obsidian': {
'vault_path': '${VAULT_PATH}',
'rest_api': {
'api_key': '${API_KEY}'
}
}
}
expanded = self.loader.expand_environment_variables(config_data)
assert expanded['obsidian']['vault_path'] == '/test/vault'
assert expanded['obsidian']['rest_api']['api_key'] == 'test-key'
# Clean up
del os.environ['VAULT_PATH']
del os.environ['API_KEY']
def test_load_yaml_config(self):
"""Test loading YAML configuration file"""
config_data = {
'claude': {
'api_key': '${TEST_API_KEY:-default-key}',
'api_url': 'https://api.anthropic.com',
'model': 'claude-3-5-sonnet-20241022'
}
}
config_file = self.temp_dir / 'test_config.yaml'
with config_file.open('w') as f:
yaml.dump(config_data, f)
loaded_config = self.loader.load_config(config_file)
assert loaded_config['claude']['api_key'] == 'default-key'
assert loaded_config['claude']['api_url'] == 'https://api.anthropic.com'
def test_load_nonexistent_config(self):
"""Test error handling for nonexistent configuration files"""
nonexistent_file = self.temp_dir / 'nonexistent.yaml'
with pytest.raises(ConfigurationError, match="Configuration file not found"):
self.loader.load_config(nonexistent_file)
def test_validate_environment_variables(self):
"""Test validation of environment variables in configuration"""
config_data = {
'claude': {
'api_key': '${EXISTING_VAR}',
'api_url': '${MISSING_VAR}',
'model': '${VAR_WITH_DEFAULT:-default-model}'
}
}
os.environ['EXISTING_VAR'] = 'test-value'
missing_vars = self.loader.validate_environment_variables(config_data)
assert len(missing_vars) == 1
assert 'MISSING_VAR' in missing_vars[0]
# Clean up
del os.environ['EXISTING_VAR']
def test_get_environment_variable_references(self):
"""Test getting all environment variable references"""
config_data = {
'claude': {
'api_key': '${API_KEY}',
'api_url': '${API_URL:-default}',
'model': 'claude-3-5-sonnet-20241022'
},
'obsidian': {
'vault_path': '${VAULT_PATH}'
}
}
env_vars = self.loader.get_environment_variable_references(config_data)
assert 'API_KEY' in env_vars
assert 'API_URL' in env_vars
assert 'VAULT_PATH' in env_vars
assert 'claude.api_key' in env_vars['API_KEY']
assert 'claude.api_url' in env_vars['API_URL']
class TestConfigurationMigrator:
"""Test ConfigurationMigrator backward compatibility"""
def setup_method(self):
"""Set up test fixtures"""
self.migrator = ConfigurationMigrator()
def test_migrate_claude_config_missing_api_url(self):
"""Test migration of Claude config missing api_url"""
config_dict = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-5-sonnet-20241022'
}
}
migrated = self.migrator.migrate_claude_config(config_dict)
assert migrated['claude']['api_url'] == 'https://api.anthropic.com'
assert migrated['claude']['api_key'] == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
assert migrated['claude']['model'] == 'claude-3-5-sonnet-20241022'
def test_migrate_legacy_model_names(self):
"""Test migration of legacy model names"""
legacy_models = {
'claude-3-sonnet': 'claude-3-sonnet-20240229',
'claude-3-opus': 'claude-3-opus-20240229',
'claude-3-haiku': 'claude-3-haiku-20240307',
'sonnet': 'claude-3-5-sonnet-20241022'
}
for old_model, expected_new_model in legacy_models.items():
config_dict = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': old_model
}
}
migrated = self.migrator.migrate_claude_config(config_dict)
assert migrated['claude']['model'] == expected_new_model
def test_migrate_missing_claude_section(self):
"""Test migration when Claude section is completely missing"""
config_dict = {
'obsidian': {
'vault_path': '/test/vault'
}
}
migrated = self.migrator.migrate_claude_config(config_dict)
assert 'claude' in migrated
assert migrated['claude']['api_url'] == 'https://api.anthropic.com'
assert migrated['claude']['model'] == 'claude-3-5-sonnet-20241022'
def test_migrate_complete_configuration(self):
"""Test migration of complete configuration"""
config_dict = {
'obsidian': {
'vault_path': '/test/vault',
'rest_api': {
'url': 'https://localhost:27123',
'api_key': 'test-key'
}
},
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-sonnet' # Legacy model name
}
}
migrated = self.migrator.migrate_configuration(config_dict)
# Check Claude migration
assert migrated['claude']['api_url'] == 'https://api.anthropic.com'
assert migrated['claude']['model'] == 'claude-3-sonnet-20240229'
# Check that other sections are added with defaults
assert 'journal' in migrated
assert 'output' in migrated
assert 'analysis' in migrated
assert 'logging' in migrated
def test_check_migration_needed(self):
"""Test checking if migration is needed"""
# Config that needs migration
config_needing_migration = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-sonnet' # Legacy model name
}
}
assert self.migrator.check_migration_needed(config_needing_migration) is True
# Config that doesn't need migration - need all required fields
config_up_to_date = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': 'https://api.anthropic.com',
'model': 'claude-3-5-sonnet-20241022',
'max_tokens': 4096,
'temperature': 0.7
},
'journal': {
'daily_notes_folder': 'Daily',
'date_format': 'YYYY-MM-DD',
'file_extension': '.md'
},
'output': {
'experiences_folder': 'Knowledge/Experiences',
'lessons_folder': 'Knowledge/Lessons',
'tasks_folder': 'Tasks/Daily',
'problems_folder': 'Knowledge/Problems',
'achievements_folder': 'Knowledge/Achievements',
'improvements_folder': 'Knowledge/Improvements'
},
'analysis': {
'categories': [],
'extraction_rules': {}
},
'logging': {
'level': 'INFO',
'file': 'logs/journal_organizer.log'
}
}
assert self.migrator.check_migration_needed(config_up_to_date) is False
def test_get_migration_preview(self):
"""Test getting migration preview"""
config_dict = {
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-sonnet'
}
}
preview = self.migrator.get_migration_preview(config_dict)
assert len(preview) > 0
assert any('api_url' in action for action in preview)
assert any('claude-3-sonnet' in action and 'claude-3-sonnet-20240229' in action for action in preview)
def test_get_supported_model_names(self):
"""Test getting supported model names"""
supported_models = self.migrator.get_supported_model_names()
# Should include current models
assert 'claude-3-5-sonnet-20241022' in supported_models
assert 'claude-3-opus-20240229' in supported_models
# Should include legacy models
assert 'claude-3-sonnet' in supported_models
assert 'sonnet' in supported_models
class TestConfigurationValidator:
"""Test ConfigurationValidator comprehensive validation"""
def setup_method(self):
"""Set up test fixtures"""
self.validator = ConfigurationValidator()
self.temp_dir = Path(tempfile.mkdtemp())
# Create a test vault directory
self.test_vault = self.temp_dir / 'test_vault'
self.test_vault.mkdir()
(self.test_vault / '.obsidian').mkdir()
def teardown_method(self):
"""Clean up test fixtures"""
import shutil
shutil.rmtree(self.temp_dir)
def create_test_config_file(self, config_data: Dict[str, Any]) -> Path:
"""Create a test configuration file"""
config_file = self.temp_dir / 'test_config.yaml'
with config_file.open('w') as f:
yaml.dump(config_data, f)
return config_file
def test_load_and_validate_valid_config(self):
"""Test loading and validating a valid configuration"""
config_data = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api': {
'url': 'https://localhost:27123',
'api_key': 'test-api-key',
'verify_ssl': False
}
},
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'api_url': 'https://api.anthropic.com',
'model': 'claude-3-5-sonnet-20241022'
}
}
config_file = self.create_test_config_file(config_data)
system_config = self.validator.load_and_validate_config(config_file)
assert isinstance(system_config, SystemConfig)
assert system_config.claude.api_url == 'https://api.anthropic.com'
assert system_config.claude.model == 'claude-3-5-sonnet-20241022'
# Path resolution may add /private prefix on macOS, so check if paths resolve to same location
assert Path(system_config.obsidian.vault_path).resolve() == self.test_vault.resolve()
def test_load_and_validate_with_migration(self):
"""Test loading configuration that needs migration"""
config_data = {
'obsidian': {
'vault_path': str(self.test_vault),
'rest_api': {
'url': 'https://localhost:27123',
'api_key': 'test-api-key'
}
},
'claude': {
'api_key': 'sk-ant-test-key-12345678901234567890123456789012345678901234567890',
'model': 'claude-3-sonnet' # Legacy model name
}
}
config_file = self.create_test_config_file(config_data)
system_config = self.validator.load_and_validate_config(config_file)
# Should have migrated the model name
assert system_config.claude.model == 'claude-3-sonnet-20240229'
# Should have added default api_url
assert system_config.claude.api_url == 'https://api.anthropic.com'
def test_load_and_validate_with_env_vars(self):
"""Test loading configuration with environment variables"""
os.environ['TEST_CLAUDE_KEY'] = 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
os.environ['TEST_VAULT_PATH'] = str(self.test_vault)
config_data = {
'obsidian': {
'vault_path': '${TEST_VAULT_PATH}',
'rest_api': {
'url': 'https://localhost:27123',
'api_key': 'test-api-key'
}
},
'claude': {
'api_key': '${TEST_CLAUDE_KEY}',
'api_url': '${CLAUDE_API_URL:-https://api.anthropic.com}'
}
}
config_file = self.create_test_config_file(config_data)
system_config = self.validator.load_and_validate_config(config_file)
assert system_config.claude.api_key == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
# Path resolution may add /private prefix on macOS, so check if paths resolve to same location
assert Path(system_config.obsidian.vault_path).resolve() == self.test_vault.resolve()
assert system_config.claude.api_url == 'https://api.anthropic.com'
# Clean up
del os.environ['TEST_CLAUDE_KEY']
del os.environ['TEST_VAULT_PATH']
def test_validation_error_handling(self):
"""Test validation error handling"""
config_data = {
'obsidian': {
'vault_path': '/nonexistent/path',
'rest_api': {
'url': 'invalid-url',
'api_key': 'test-key'
}
},
'claude': {
'api_key': 'invalid-key',
'model': 'invalid-model'
}
}
config_file = self.create_test_config_file(config_data)
with pytest.raises(ValueError, match="Configuration validation failed"):
self.validator.load_and_validate_config(config_file)
def test_validate_api_keys(self):
"""Test API key validation"""
# Create a valid system config for testing
system_config = SystemConfig(
obsidian=ObsidianConfig(
vault_path=str(self.test_vault),
rest_api=ObsidianRestAPIConfig(
url='https://localhost:27123',
api_key='test-api-key'
)
),
claude=ClaudeAPIConfig(
api_key='sk-ant-test-key-12345678901234567890123456789012345678901234567890'
)
)
issues = self.validator.validate_api_keys(system_config)
# Should have no issues with valid keys
assert len(issues) == 0
def test_validate_api_keys_with_env_vars(self):
"""Test API key validation with environment variable placeholders"""
# Skip this test as it requires complex mocking of pydantic validation
pytest.skip("Environment variable validation requires complex setup")
@pytest.mark.asyncio
class TestClaudeAPIClient:
"""Test ClaudeAPIClient functionality"""
def setup_method(self):
"""Set up test fixtures"""
self.config = ClaudeAPIConfig(
api_key='sk-ant-test-key-12345678901234567890123456789012345678901234567890',
api_url='https://api.anthropic.com',
model='claude-3-5-sonnet-20241022'
)
@patch('claude_api_client.AsyncAnthropic')
@patch('claude_api_client.aiohttp')
def test_client_initialization(self, mock_aiohttp, mock_anthropic):
"""Test Claude API client initialization"""
client = ClaudeAPIClient(self.config)
assert client.base_url == 'https://api.anthropic.com'
assert client.api_key == 'sk-ant-test-key-12345678901234567890123456789012345678901234567890'
assert client.model == 'claude-3-5-sonnet-20241022'
# Should have called AsyncAnthropic constructor
mock_anthropic.assert_called_once()
@patch('claude_api_client.AsyncAnthropic')
@patch('claude_api_client.aiohttp')
def test_client_with_custom_url(self, mock_aiohttp, mock_anthropic):
"""Test client initialization with custom API URL"""
custom_config = ClaudeAPIConfig(
api_key='sk-ant-test-key-12345678901234567890123456789012345678901234567890',
api_url='https://custom-api.example.com',
model='claude-3-5-sonnet-20241022'
)
client = ClaudeAPIClient(custom_config)
assert client.base_url == 'https://custom-api.example.com'
# Should have called AsyncAnthropic with custom base_url
mock_anthropic.assert_called_once()
call_args = mock_anthropic.call_args
assert call_args[1]['base_url'] == 'https://custom-api.example.com'
@patch('claude_api_client.AsyncAnthropic')
@patch('claude_api_client.aiohttp')
def test_get_client_info(self, mock_aiohttp, mock_anthropic):
"""Test getting client configuration information"""
client = ClaudeAPIClient(self.config)
info = client.get_client_info()
assert info['api_url'] == 'https://api.anthropic.com'
assert info['model'] == 'claude-3-5-sonnet-20241022'
assert info['max_tokens'] == 4096
assert info['temperature'] == 0.7
assert info['is_custom_endpoint'] is False
@patch('claude_api_client.AsyncAnthropic')
@patch('claude_api_client.aiohttp')
def test_get_client_info_custom_endpoint(self, mock_aiohttp, mock_anthropic):
"""Test getting client info for custom endpoint"""
custom_config = ClaudeAPIConfig(
api_key='sk-ant-test-key-12345678901234567890123456789012345678901234567890',
api_url='https://custom-api.example.com'
)
client = ClaudeAPIClient(custom_config)
info = client.get_client_info()
assert info['is_custom_endpoint'] is True
assert info['api_url'] == 'https://custom-api.example.com'
+526
View File
@@ -0,0 +1,526 @@
"""
Unit tests for error_handling module.
Tests custom exceptions, ErrorHandler, and error sanitization.
"""
import pytest
import logging
from unittest.mock import Mock, patch
from datetime import datetime
from error_handling import (
JournalOrganizerError, ConfigurationError, APIError, ValidationError,
FileSystemError, SecurityError, ErrorContext, ErrorHandler,
get_error_handler, set_error_handler, audit_error_message_security
)
class TestJournalOrganizerError:
"""Test base JournalOrganizerError class"""
def test_basic_error_creation(self):
"""Test creating basic error"""
error = JournalOrganizerError("Test error message")
assert str(error) == "Test error message"
assert error.message == "Test error message"
assert error.context == {}
assert error.cause is None
assert error.timestamp is not None
def test_error_with_context(self):
"""Test creating error with context"""
context = {"key": "value", "number": 42}
error = JournalOrganizerError("Test error", context=context)
assert error.context == context
def test_error_with_cause(self):
"""Test creating error with cause"""
original_error = ValueError("Original error")
error = JournalOrganizerError("Wrapped error", cause=original_error)
assert error.cause == original_error
def test_to_dict(self):
"""Test converting error to dictionary"""
context = {"test_key": "test_value"}
original_error = RuntimeError("Original")
error = JournalOrganizerError("Test error", context=context, cause=original_error)
error_dict = error.to_dict()
assert error_dict["error_type"] == "JournalOrganizerError"
assert error_dict["message"] == "Test error"
assert error_dict["context"] == context
assert error_dict["cause"] == "Original"
assert "timestamp" in error_dict
class TestConfigurationError:
"""Test ConfigurationError class"""
def test_basic_configuration_error(self):
"""Test basic configuration error"""
error = ConfigurationError("Config error", config_key="api_key")
assert error.message == "Config error"
assert error.context["config_key"] == "api_key"
def test_configuration_error_with_sensitive_value(self):
"""Test configuration error with sensitive value redaction"""
error = ConfigurationError(
"Invalid API key",
config_key="api_key",
config_value="sk-secret-key-123"
)
assert error.context["config_key"] == "api_key"
assert error.context["config_value"] == "[REDACTED]"
def test_configuration_error_with_non_sensitive_value(self):
"""Test configuration error with non-sensitive value"""
error = ConfigurationError(
"Invalid timeout",
config_key="timeout",
config_value="30"
)
assert error.context["config_key"] == "timeout"
assert error.context["config_value"] == "30"
class TestAPIError:
"""Test APIError class"""
def test_basic_api_error(self):
"""Test basic API error"""
error = APIError("API call failed", api_name="claude", status_code=500)
assert error.message == "API call failed"
assert error.context["api_name"] == "claude"
assert error.context["status_code"] == 500
def test_api_error_with_long_response(self):
"""Test API error with long response data truncation"""
long_response = "x" * 1000
error = APIError("API error", response_data=long_response)
assert len(error.context["response_data"]) <= 503 # 500 + "..."
assert error.context["response_data"].endswith("...")
def test_api_error_with_short_response(self):
"""Test API error with short response data"""
short_response = "Short error"
error = APIError("API error", response_data=short_response)
assert error.context["response_data"] == short_response
class TestValidationError:
"""Test ValidationError class"""
def test_basic_validation_error(self):
"""Test basic validation error"""
error = ValidationError(
"Invalid email",
field_name="email",
field_value="invalid-email",
validation_rule="email_format"
)
assert error.message == "Invalid email"
assert error.context["field_name"] == "email"
assert error.context["field_value"] == "invalid-email"
assert error.context["validation_rule"] == "email_format"
def test_validation_error_with_sensitive_field(self):
"""Test validation error with sensitive field value redaction"""
error = ValidationError(
"Invalid password",
field_name="password",
field_value="secret123"
)
assert error.context["field_name"] == "password"
assert error.context["field_value"] == "[REDACTED]"
def test_validation_error_with_long_value(self):
"""Test validation error with long field value truncation"""
long_value = "x" * 200
error = ValidationError(
"Invalid input",
field_name="description",
field_value=long_value
)
assert len(error.context["field_value"]) == 100 # Truncated to 100 chars
class TestFileSystemError:
"""Test FileSystemError class"""
def test_basic_filesystem_error(self):
"""Test basic filesystem error"""
error = FileSystemError(
"File not found",
file_path="/path/to/file.txt",
operation="read"
)
assert error.message == "File not found"
assert error.context["file_path"] == "/path/to/file.txt"
assert error.context["operation"] == "read"
class TestSecurityError:
"""Test SecurityError class"""
def test_basic_security_error(self):
"""Test basic security error"""
error = SecurityError(
"Path traversal detected",
security_issue="path_traversal",
attempted_path="../../../etc/passwd",
risk_level="critical"
)
assert error.message == "Path traversal detected"
assert error.context["security_issue"] == "path_traversal"
assert error.context["attempted_path"] == "../../../etc/passwd"
assert error.context["risk_level"] == "critical"
def test_security_error_with_long_path(self):
"""Test security error with long path truncation"""
long_path = "/" + "x" * 600
error = SecurityError("Security violation", attempted_path=long_path)
assert len(error.context["attempted_path"]) == 500 # Truncated
class TestErrorContext:
"""Test ErrorContext class"""
def test_valid_error_context(self):
"""Test creating valid error context"""
context = ErrorContext(
component="test_component",
operation="test_operation",
user_message="User friendly message",
technical_details={"key": "value"},
severity="warning"
)
assert context.component == "test_component"
assert context.operation == "test_operation"
assert context.user_message == "User friendly message"
assert context.technical_details == {"key": "value"}
assert context.severity == "warning"
def test_error_context_defaults(self):
"""Test error context with default values"""
context = ErrorContext(
component="test_component",
operation="test_operation"
)
assert context.user_message == ""
assert context.technical_details == {}
assert context.severity == "error"
def test_error_context_validation_empty_component(self):
"""Test error context validation with empty component"""
with pytest.raises(ValueError, match="component cannot be empty"):
ErrorContext(component="", operation="test_operation")
def test_error_context_validation_empty_operation(self):
"""Test error context validation with empty operation"""
with pytest.raises(ValueError, match="operation cannot be empty"):
ErrorContext(component="test_component", operation="")
def test_error_context_validation_invalid_severity(self):
"""Test error context validation with invalid severity"""
with pytest.raises(ValueError, match="severity must be one of"):
ErrorContext(
component="test_component",
operation="test_operation",
severity="invalid"
)
def test_error_context_validation_invalid_technical_details(self):
"""Test error context validation with invalid technical_details"""
with pytest.raises(ValueError, match="technical_details must be a dictionary"):
ErrorContext(
component="test_component",
operation="test_operation",
technical_details="not a dict"
)
def test_to_dict(self):
"""Test converting error context to dictionary"""
context = ErrorContext(
component="test_component",
operation="test_operation",
user_message="Test message",
technical_details={"key": "value"},
severity="info"
)
context_dict = context.to_dict()
assert context_dict["component"] == "test_component"
assert context_dict["operation"] == "test_operation"
assert context_dict["user_message"] == "Test message"
assert context_dict["technical_details"] == {"key": "value"}
assert context_dict["severity"] == "info"
class TestErrorHandler:
"""Test ErrorHandler class"""
def setup_method(self):
"""Set up test fixtures"""
self.mock_logger = Mock(spec=logging.Logger)
self.error_handler = ErrorHandler(self.mock_logger)
def test_error_handler_creation(self):
"""Test creating error handler"""
handler = ErrorHandler()
assert handler.logger is not None
handler_with_logger = ErrorHandler(self.mock_logger)
assert handler_with_logger.logger == self.mock_logger
def test_handle_error_with_custom_error(self):
"""Test handling custom JournalOrganizerError"""
error = ConfigurationError("Config error", config_key="api_key")
context = ErrorContext(
component="config",
operation="load",
user_message="Please check your configuration"
)
result = self.error_handler.handle_error(error, context)
assert result["success"] is False
assert result["error"] == "Config error"
assert result["message"] == "Please check your configuration"
assert result["component"] == "config"
assert result["operation"] == "load"
assert result["error_type"] == "ConfigurationError"
assert "timestamp" in result
# Check that logger was called
self.mock_logger.log.assert_called_once()
def test_handle_error_with_generic_error(self):
"""Test handling generic Python exception"""
error = ValueError("Generic error")
context = ErrorContext(
component="test",
operation="test_op",
severity="warning"
)
result = self.error_handler.handle_error(error, context)
assert result["success"] is False
assert result["error"] == "Generic error"
assert result["error_type"] == "ValueError"
# Check that logger was called with warning level
self.mock_logger.log.assert_called_once()
call_args = self.mock_logger.log.call_args
assert call_args[0][0] == logging.WARNING # Log level
def test_handle_api_error(self):
"""Test handling API errors"""
error = RuntimeError("Connection failed")
result = self.error_handler.handle_api_error(error, "claude", "analyze_text")
assert result["success"] is False
assert "claude API error" in result["error"]
assert result["component"] == "claude_api"
assert result["operation"] == "analyze_text"
assert "Failed to communicate with claude" in result["message"]
def test_handle_api_error_with_api_error_instance(self):
"""Test handling APIError instance"""
api_error = APIError("API failed", api_name="obsidian", status_code=404)
result = self.error_handler.handle_api_error(api_error, "obsidian", "read_note")
assert result["success"] is False
assert result["error"] == "API failed"
assert result["component"] == "obsidian_api"
def test_handle_validation_error(self):
"""Test handling validation errors"""
error = ValueError("Invalid format")
result = self.error_handler.handle_validation_error(error, "email", "validate_input")
assert result["success"] is False
assert "Invalid email" in result["error"]
assert result["component"] == "validation"
assert result["operation"] == "validate_input"
assert "Please check your email" in result["message"]
def test_handle_configuration_error(self):
"""Test handling configuration errors"""
error = ValueError("Missing key")
result = self.error_handler.handle_configuration_error(error, "api_key")
assert result["success"] is False
assert "Configuration error for api_key" in result["error"]
assert result["component"] == "configuration"
assert result["operation"] == "load_config"
assert "Please check your API key configuration" in result["message"]
def test_sanitize_error_message_api_key(self):
"""Test sanitizing error messages with API keys"""
message = "Error: api_key=sk-secret-key-123 is invalid"
sanitized = self.error_handler._sanitize_error_message(message)
assert "sk-secret-key-123" not in sanitized
assert "api_key=[REDACTED]" in sanitized
def test_sanitize_error_message_bearer_token(self):
"""Test sanitizing error messages with Bearer tokens"""
message = "Authorization failed: Bearer abc123xyz789"
sanitized = self.error_handler._sanitize_error_message(message)
assert "abc123xyz789" not in sanitized
assert "[REDACTED]" in sanitized
def test_sanitize_error_message_email(self):
"""Test sanitizing error messages with email addresses"""
message = "Failed to send email to user@example.com"
sanitized = self.error_handler._sanitize_error_message(message)
assert "user@example.com" not in sanitized
assert "[EMAIL_REDACTED]" in sanitized
def test_sanitize_error_message_file_paths(self):
"""Test sanitizing error messages with user file paths"""
message = "Cannot access /Users/john/Documents/secret.txt"
sanitized = self.error_handler._sanitize_error_message(message)
assert "john" not in sanitized
assert "[USER_REDACTED]" in sanitized
def test_sanitize_context_data(self):
"""Test sanitizing context data"""
data = {
"api_key": "secret-key-123",
"username": "john_doe",
"password": "secret123",
"timeout": 30,
"nested": {
"token": "bearer-token-xyz",
"safe_value": "public_info"
}
}
sanitized = self.error_handler._sanitize_context_data(data)
assert sanitized["api_key"] == "[REDACTED]"
assert sanitized["username"] == "john_doe" # Not sensitive
assert sanitized["password"] == "[REDACTED]"
assert sanitized["timeout"] == 30
assert sanitized["nested"]["token"] == "[REDACTED]"
assert sanitized["nested"]["safe_value"] == "public_info"
def test_sanitize_context_data_non_dict(self):
"""Test sanitizing non-dictionary context data"""
result = self.error_handler._sanitize_context_data("not a dict")
assert result == "not a dict"
class TestGlobalErrorHandler:
"""Test global error handler functions"""
def test_get_error_handler_singleton(self):
"""Test that get_error_handler returns singleton"""
handler1 = get_error_handler()
handler2 = get_error_handler()
assert handler1 is handler2
def test_set_error_handler(self):
"""Test setting custom error handler"""
custom_handler = ErrorHandler()
set_error_handler(custom_handler)
retrieved_handler = get_error_handler()
assert retrieved_handler is custom_handler
class TestAuditErrorMessageSecurity:
"""Test error message security auditing"""
def test_audit_clean_message(self):
"""Test auditing clean message with no issues"""
message = "Simple error message with no sensitive data"
result = audit_error_message_security(message)
assert result["has_issues"] is False
assert result["issues"] == []
assert result["risk_level"] == "low"
def test_audit_message_with_email(self):
"""Test auditing message with email address"""
message = "Failed to send notification to user@example.com"
result = audit_error_message_security(message)
assert result["has_issues"] is True
assert len(result["issues"]) == 1
assert result["issues"][0]["type"] == "email_address"
assert result["risk_level"] == "medium"
def test_audit_message_with_api_key(self):
"""Test auditing message with API key"""
message = "Authentication failed: api_key=sk-secret-123"
result = audit_error_message_security(message)
assert result["has_issues"] is True
assert any(issue["type"] == "credential_pattern" for issue in result["issues"])
assert result["risk_level"] == "high"
def test_audit_message_with_bearer_token(self):
"""Test auditing message with Bearer token"""
message = "Authorization header: Bearer abc123xyz789"
result = audit_error_message_security(message)
assert result["has_issues"] is True
assert any(issue["type"] == "bearer_token" for issue in result["issues"])
assert result["risk_level"] == "high"
def test_audit_message_with_ip_address(self):
"""Test auditing message with IP address"""
message = "Connection failed to 192.168.1.100"
result = audit_error_message_security(message)
assert result["has_issues"] is True
assert any(issue["type"] == "ip_address" for issue in result["issues"])
assert result["risk_level"] == "medium"
def test_audit_message_with_multiple_issues(self):
"""Test auditing message with multiple security issues"""
message = "Failed to connect to 192.168.1.100 with api_key=secret123 for user@example.com"
result = audit_error_message_security(message)
assert result["has_issues"] is True
assert len(result["issues"]) >= 2 # Should find multiple issues
assert result["risk_level"] == "high" # High due to credential pattern