windy/dl-organizer
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Initial commit: Video Library Manager

Browse Source 
- Add core VLM modules (scanner, parser, planner, executor, analysis)
- Add CLI with quarantine, reports, rollback, and state management
- Add comprehensive test suite
- Add project configuration and documentation
- Add .gitignore for Python project
This commit is contained in:
windyboy
2026-02-09 17:43:35 +08:00
commit 1705275e99
38 changed files with 16694 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+49
View File
@@ -0,0 +1,49 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# Virtual environments
venv/
env/
ENV/
.venv
# Testing
.pytest_cache/
.hypothesis/
.coverage
htmlcov/
.tox/
.benchmarks/
# IDEs
.vscode/
.idea/
*.swp
*.swo
*~
# OS
.DS_Store
Thumbs.db
# Logs
*.log
.kiro/settings/mcp.json
+4
View File
@@ -0,0 +1,4 @@
{
"mcpServers": {
}
}
.kiro/specs/video-library-manager/.config.kiro
+1
View File
@@ -0,0 +1 @@
{"generationMode": "requirements-first"}
.kiro/specs/video-library-manager/design.md
+1258
View File
File diff suppressed because it is too large Load Diff
.kiro/specs/video-library-manager/requirements.md
+236
View File
@@ -0,0 +1,236 @@
# Requirements Document
## Introduction
The Video Library Manager is a personal, semi-automated video library management tool written in Python for long-term use. It operates on video files stored on a FreeNAS system mounted via SMB on Windows 11. This tool provides a safe, transparent workflow to inspect, organize, and gradually clean an existing video library without acting as a downloader or media server.
The system follows a read-first, human-in-the-loop approach where all irreversible actions require explicit user confirmation. All file operations are expressed as reviewable execution plans, with no permanent deletion in v1 - unwanted files are moved to quarantine instead.
## Glossary
- **Video_Library_Manager**: The Python-based tool that manages video file organization
- **Inventory_Scanner**: Component that discovers and catalogs video files
- **Identity_Parser**: Component that extracts logical identity from filenames
- **Analysis_Engine**: Component that detects completeness and duplicates, and provides comparison data
- **Plan_Generator**: Component that creates structured, reviewable execution plans
- **Execution_Engine**: Component that safely executes file operations
- **Quarantine_Directory**: A `.quarantine/` subdirectory within the library root where unwanted files are moved
- **Dry_Run_Mode**: A simulation mode that shows what would happen without making changes
- **Execution_Plan**: A structured, reviewable document describing all file operations to be performed
- **Video_File**: A media file with extensions: .mp4, .mkv, .avi, .mov, .wmv, .flv, .webm, .m4v
- **Movie**: A standalone video file organized by title and year
- **Series**: An episodic TV series with continuous drama format
- **Season**: A collection of episodes within a series
- **Episode**: A single video file within a season
- **Anime**: Japanese animation content (deferred for v1 except inventory scanning)
- **Rollback_Log**: A record of all file operations that enables restoration to previous state
- **Configuration_File**: A user-editable file containing parsing rules, directory templates, and policies
## Requirements
### 1. Inventory Scanning
**User Story:** As a user, I want to scan my video library to discover all video files and record their metadata, so that I can understand the current state of my collection without making any modifications.
**Acceptance Criteria:**
1. WHEN the user initiates an inventory scan, THE Inventory_Scanner SHALL recursively discover all Video_Files within the specified root directory
2. WHEN a Video_File is discovered, THE Inventory_Scanner SHALL record its full path, filename, file size, and modification timestamp
3. WHERE ffprobe is available, THE Inventory_Scanner SHALL extract video metadata including resolution, codec, duration, and bitrate
4. WHEN scanning completes, THE Inventory_Scanner SHALL generate a structured inventory report in CSV format
5. THE Inventory_Scanner SHALL scan all top-level directories including movie, series, anime, and other
6. THE Inventory_Scanner SHALL perform read-only operations without modifying any files or directories
7. WHEN scanning encounters an inaccessible file or directory, THE Inventory_Scanner SHALL log the error and continue scanning remaining files
8. THE Inventory_Scanner SHALL save the inventory report to disk for later use
### 2. Identity Parsing for Movies
**User Story:** As a user, I want the system to extract movie titles and years from filenames, so that I can organize my movie collection into a predictable structure.
**Acceptance Criteria:**
1. WHEN a Video_File is located in the movie directory, THE Identity_Parser SHALL extract the movie title and release year from the filename
2. THE Identity_Parser SHALL handle common filename patterns including "Title (Year)", "Title.Year", "Title - Year", and variations with quality tags
3. WHEN a filename contains quality indicators (1080p, BluRay, WEB-DL), THE Identity_Parser SHALL exclude them from the title
4. WHEN a filename contains release group tags in brackets or parentheses, THE Identity_Parser SHALL exclude them from the title
5. WHEN the Identity_Parser cannot confidently extract a year, THE Identity_Parser SHALL mark the movie as requiring manual review
6. THE Identity_Parser SHALL normalize titles by removing extra whitespace and standardizing capitalization
7. WHEN multiple Video_Files share the same title and year, THE Identity_Parser SHALL flag them as potential duplicates
### 3. Identity Parsing for Series
**User Story:** As a user, I want the system to extract series titles, season numbers, and episode numbers from filenames, so that I can organize my TV series into a consistent structure.
**Acceptance Criteria:**
1. WHEN a Video_File is located in the series directory, THE Identity_Parser SHALL extract the series title, season number, and episode number from the filename
2. THE Identity_Parser SHALL recognize common episode patterns including "SXXEYY", "SXXeYY", "SeasonXEpisodeY", and "XXxYY"
3. THE Identity_Parser SHALL handle multi-episode files by extracting all episode numbers (e.g., "S01E01-E02")
4. WHEN a filename contains quality indicators or release group tags, THE Identity_Parser SHALL exclude them from the series title
5. WHEN the Identity_Parser cannot confidently extract season or episode numbers, THE Identity_Parser SHALL mark the file as requiring manual review
6. THE Identity_Parser SHALL normalize series titles by removing extra whitespace and standardizing capitalization
7. WHEN parsing completes, THE Identity_Parser SHALL group episodes by series title and season number
### 4. Series Completeness Analysis
**User Story:** As a user, I want to identify series with episode gaps in my collection, so that I can understand which series have missing episodes.
**Acceptance Criteria:**
1. WHEN analyzing a series, THE Analysis_Engine SHALL detect gaps in episode sequences within each season using heuristic detection
2. WHEN a season has episodes numbered 1, 2, 4, 5, THE Analysis_Engine SHALL report episode 3 as missing (gap in range [1, 5])
3. THE Analysis_Engine SHALL NOT calculate completeness percentages or determine if seasons are "complete" (v1 limitation: no external metadata)
4. WHEN a series has multiple seasons, THE Analysis_Engine SHALL analyze each season independently
5. THE Analysis_Engine SHALL generate a completeness report listing all series with detected episode gaps
6. THE completeness report SHALL show episodes_found and episodes_missing (gaps in [min, max]) for each season
### 5. Duplicate Detection
**User Story:** As a user, I want to identify duplicate or redundant video files, so that I can remove unnecessary copies and save storage space.
**Acceptance Criteria:**
1. WHEN multiple Video_Files have identical titles and years (for movies), THE Analysis_Engine SHALL flag them as potential duplicates
2. WHEN multiple Video_Files have identical series titles, season numbers, and episode numbers, THE Analysis_Engine SHALL flag them as potential duplicates
3. WHERE file size metadata is available, THE Analysis_Engine SHALL compare file sizes to help identify quality differences
4. WHERE video metadata is available, THE Analysis_Engine SHALL compare resolution and codec to help identify quality differences
5. THE Analysis_Engine SHALL generate a duplicate report grouping all potential duplicates with their metadata
6. THE Analysis_Engine SHALL provide comparison data without automatically recommending which file to keep
### 6. Execution Plan Generation
**User Story:** As a user, I want to review a detailed plan of all file operations before they are executed, so that I can verify the changes are correct and safe.
**Acceptance Criteria:**
1. WHEN the user requests organization, THE Plan_Generator SHALL create an Execution_Plan containing all proposed file operations
2. THE Execution_Plan SHALL specify operation type (rename, move, quarantine, no-op) for each Video_File
3. WHEN a movie should be organized, THE Execution_Plan SHALL specify moving it to "movie/Title (Year)/" directory structure
4. WHEN a series episode should be organized, THE Execution_Plan SHALL specify moving it to "series/Title/Season XX/" directory structure with "SXXEYY.ext" filename
5. THE Execution_Plan SHALL preserve the original top-level directory structure (movie, series, anime, other)
6. THE Execution_Plan SHALL include source path, destination path, and operation type for each file
7. THE Plan_Generator SHALL output the Execution_Plan in JSON format for both human readability and machine processing
8. WHEN generating plans, THE Plan_Generator SHALL detect potential conflicts (destination file already exists) and mark them for user review
9. THE Execution_Plan SHALL be editable by the user before execution
### 7. Safe File Operations
**User Story:** As a user, I want all file operations to be executed safely with dry-run support and explicit confirmation, so that I can prevent accidental data loss.
**Acceptance Criteria:**
1. THE Execution_Engine SHALL support Dry_Run_Mode that simulates operations without making changes
2. WHEN executing in Dry_Run_Mode, THE Execution_Engine SHALL log all operations that would be performed
3. WHEN executing file operations, THE Execution_Engine SHALL require explicit user confirmation before proceeding
4. WHEN moving or renaming files, THE Execution_Engine SHALL create destination directories if they do not exist
5. The Execution_Engine SHALL report the conflict and skip the affected operation without halting other planned operations.
6. THE Execution_Engine SHALL log all file operations with timestamps, source paths, destination paths, and operation results
7. WHEN any operation fails, THE Execution_Engine SHALL log the error and continue with remaining operations
8. THE Execution_Engine SHALL generate an execution summary showing successful operations, failed operations, and skipped operations
### 8. Quarantine Operations
**User Story:** As a user, I want to safely isolate unwanted files in a quarantine directory, so that I can review them before permanent deletion.
**Acceptance Criteria:**
1. WHEN the user marks files for quarantine, THE Execution_Engine SHALL move them to a Quarantine_Directory within the library root
2. THE Quarantine_Directory SHALL be located at `<library_root>/.quarantine/`
3. THE Execution_Engine SHALL preserve the relative directory structure within the Quarantine_Directory
4. WHEN moving files to quarantine, THE Execution_Engine SHALL record the original location in a quarantine manifest file
5. THE quarantine manifest SHALL be stored as JSON and include original path, quarantine path, timestamp, and optional reason
6. WHEN a quarantined file already exists at the destination, THE Execution_Engine SHALL append a numeric suffix to avoid overwriting
7. THE Execution_Engine SHALL support listing all quarantined files with their original locations
8. THE Execution_Engine SHALL prevent permanent deletion of files in v1
### 9. Rollback Operations
**User Story:** As a user, I want to undo file operations and restore files to their original locations, so that I can recover from mistakes.
**Acceptance Criteria:**
1. WHEN file operations are executed, THE Execution_Engine SHALL create a Rollback_Log containing all operations performed
2. THE Rollback_Log SHALL include operation type, source path, destination path, and timestamp for each operation
3. WHEN the user initiates a rollback, THE Execution_Engine SHALL restore files to their original locations based on the Rollback_Log
4. WHEN rolling back quarantine operations, THE Execution_Engine SHALL move files from Quarantine_Directory back to their original locations
5. WHEN rolling back move operations, THE Execution_Engine SHALL move files from destination back to source
6. WHEN rolling back rename operations, THE Execution_Engine SHALL rename files back to their original names
7. IF a rollback operation fails, THE Execution_Engine SHALL log the error and continue with remaining rollback operations
8. THE Execution_Engine SHALL generate a rollback summary showing successful rollbacks, failed rollbacks, and skipped operations
### 10. Configuration Management
**User Story:** As a user, I want to configure parsing rules, directory templates, and policies, so that I can customize the tool to my preferences.
**Acceptance Criteria:**
1. THE Video_Library_Manager SHALL load configuration from a Configuration_File
2. THE Configuration_File SHALL specify video file extensions to recognize
3. THE Configuration_File SHALL specify directory templates for movies and series
4. THE Configuration_File SHALL specify the library root path
5. Parsing patterns for movie titles and series episodes are hardcoded in v1 (not user-configurable)
6. WHEN the Configuration_File is missing, THE Video_Library_Manager SHALL create a default Configuration_File
7. WHEN the Configuration_File contains invalid syntax, THE Video_Library_Manager SHALL report the error and use default values
8. THE Video_Library_Manager SHALL validate configuration values and report errors for invalid settings
### 11. Reporting and Visualization
**User Story:** As a user, I want to view comprehensive reports about my video library, so that I can understand its current state and make informed decisions.
**Acceptance Criteria:**
1. THE Video_Library_Manager SHALL generate an inventory report listing all discovered Video_Files with metadata
2. THE Video_Library_Manager SHALL generate a completeness report showing incomplete series with missing episodes
3. THE Video_Library_Manager SHALL generate a duplicate report grouping potential duplicates with quality comparisons
4. THE Video_Library_Manager SHALL generate a summary report with total file count, total size, and category breakdown
5. THE Video_Library_Manager SHALL support exporting reports in CSV format for programmatic access
6. THE Video_Library_Manager SHALL support exporting reports in JSON format for programmatic access
7. THE Video_Library_Manager SHALL support exporting reports in human-readable text format
8. WHEN generating reports, THE Video_Library_Manager SHALL include generation timestamp and library root path
### 12. Anime Inventory Support
**User Story:** As a user, I want anime files to be included in inventory scans, so that I have a complete view of my library even though anime organization is deferred.
**Acceptance Criteria:**
1. WHEN scanning the anime directory, THE Inventory_Scanner SHALL discover and catalog all Video_Files
2. THE Inventory_Scanner SHALL record the same metadata for anime files as for other video files
3. THE Identity_Parser SHALL skip parsing logic for anime files in v1
4. THE Analysis_Engine SHALL skip completeness and duplicate analysis for anime files in v1
5. THE Plan_Generator SHALL not generate organization plans for anime files in v1
6. THE Video_Library_Manager SHALL include anime files in inventory reports with a flag indicating deferred processing
7. THE Video_Library_Manager SHALL preserve the anime directory structure without modifications
### 13. Error Handling and Logging
**User Story:** As a user, I want comprehensive error handling and logging, so that I can troubleshoot issues and understand what the tool is doing.
**Acceptance Criteria:**
1. WHEN any operation encounters an error, THE Video_Library_Manager SHALL log the error with timestamp, operation type, and error details
2. THE Video_Library_Manager SHALL continue processing remaining operations after encountering non-fatal errors
3. WHEN encountering file system errors (permission denied, file not found), THE Video_Library_Manager SHALL log the error and skip the affected file
4. WHEN encountering parsing errors, THE Video_Library_Manager SHALL log the error and mark the file for manual review
5. THE Video_Library_Manager SHALL support configurable log levels (DEBUG, INFO, WARNING, ERROR)
6. THE Video_Library_Manager SHALL write logs to both console output and a log file
7. WHEN the log file exceeds 10MB, THE Video_Library_Manager SHALL rotate the log file
### 14. Command-Line Interface
**User Story:** As a user, I want a clear command-line interface to interact with the tool, so that I can easily perform all operations. In v1, the CLI SHALL expose only the minimal commands required to complete the end-to-end workflow. Other commands may exist internally but are not required to be user-facing.
**Acceptance Criteria:**
1. THE Video_Library_Manager SHALL provide a command-line interface with subcommands for each major operation
2. THE Video_Library_Manager SHALL support a "scan" command to perform inventory scanning
3. THE Video_Library_Manager SHALL support a "parse" command to perform identity parsing
4. THE Video_Library_Manager SHALL support an "analyze" command to perform completeness and duplicate analysis
5. THE Video_Library_Manager SHALL support a "plan" command to generate execution plans
6. THE Video_Library_Manager SHALL support an "execute" command to execute plans with dry-run and confirmation options
7. THE Video_Library_Manager SHALL support a "quarantine" command to manage quarantined files
8. THE Video_Library_Manager SHALL support a "rollback" command to undo operations
9. THE Video_Library_Manager SHALL support a "report" command to generate and export reports
10. THE Video_Library_Manager SHALL display help text for each command with usage examples
11. WHEN the user provides invalid arguments, THE Video_Library_Manager SHALL display an error message and usage help
.kiro/specs/video-library-manager/tasks.md
+546
View File
@@ -0,0 +1,546 @@
# Implementation Plan: Video Library Manager
## Overview
The Video Library Manager is a Python-based CLI tool for managing personal video collections with a safety-first, human-in-the-loop approach. This implementation plan breaks down the design into discrete coding tasks that build incrementally toward a complete, tested system.
**Implementation Approach**: Bottom-up development starting with core data structures, then components, CLI interface, and finally integration testing.
**Testing Strategy**: Dual approach using both unit tests (specific examples, edge cases) and property-based tests (universal properties across inputs). Property tests validate the 57 correctness properties defined in the design document.
## Tasks
### 1. Project Foundation
- [x] 1.1 Set up Python project structure
- Create package structure: `src/vlm/` with `__init__.py`
- Create `pyproject.toml` with project metadata and dependencies
- Use `uv` for Python package management
- Add dependencies: Click, PyYAML, ffmpeg-python, pytest, hypothesis
- Create application data directory structure: `~/.vlm/` with subdirectories for logs, plans, rollback
- _Requirements: 10.1_
- [x] 1.2 Implement Configuration Manager
- Define `Config` dataclass with all configuration fields
- Implement `load_config(path: Path) -> Config` with YAML parsing
- Implement `create_default_config(path: Path) -> Config` to generate default config
- Implement `validate_config(config: Config) -> list[str]` for validation
- Handle missing config files by creating defaults
- Handle invalid YAML syntax with error reporting and fallback to defaults
- _Requirements: 10.1-10.8_
- [ ]* 1.3 Write property tests for Configuration Manager
- **Property 38: Configuration validation** - For any configuration with invalid values, validation SHALL detect and report errors
- **Property 39: Default configuration validity** - For any missing configuration file, creating default SHALL produce valid configuration
- **Property 40: Configuration error handling** - For any configuration with invalid YAML syntax, system SHALL report error and use defaults
- _Requirements: 10.6, 10.7, 10.8_
- [x] 1.4 Implement logging infrastructure
- Configure Python logging module with configurable log levels (DEBUG, INFO, WARNING, ERROR)
- Set up dual output: console (INFO+) and file (DEBUG+)
- Include timestamps, operation type, file paths in all log entries
- Implement log rotation at 10MB threshold
- _Requirements: 13.1-13.7_
- [ ]* 1.5 Write property tests for logging
- **Property 49: Error logging** - For any operation with error, log SHALL contain timestamp, operation type, and error details
- **Property 52: Log output** - For any logged message, message SHALL appear in console (if level permits) and log file
- **Property 53: Log rotation** - For any log file exceeding 10MB, system SHALL rotate the log file
- _Requirements: 13.1, 13.6, 13.7_
### 2. Core Data Structures
- [x] 2.1 Define video file and identity data structures
- Create `VideoFile` dataclass: path, filename, size_bytes, modified_timestamp, category, optional metadata (resolution, codec, duration, bitrate)
- Create `MovieIdentity` dataclass: title, year, confidence, needs_review, original_filename
- Create `SeriesIdentity` dataclass: title, season, episodes, confidence, needs_review, original_filename
- _Requirements: 1.2, 2.1, 3.1_
- [x] 2.2 Define execution and operation data structures
- Create `FileOperation` dataclass: operation_type, source_path, destination_path, reason, has_conflict, conflict_reason
- Create `ExecutionPlan` dataclass: plan_id, created_at, operations, summary
- Create `OperationResult` dataclass: operation, success, error_message, executed_at
- Create `RollbackLog` dataclass: log_id, execution_plan_id, executed_at, operations
- _Requirements: 6.1, 7.6, 9.1_
- [x] 2.3 Define quarantine and state data structures
- Create `QuarantineEntry` dataclass: original_path, quarantine_path, quarantined_at, reason, size_bytes, category
- Create `QuarantineManifest` dataclass: entries
- Create `FileState` dataclass: file_path, status, reason, updated_at
- Create `StateStore` dataclass: states (dict), version, last_updated
- _Requirements: 8.4, State Store design_
### 3. Inventory Scanner
- [x] 3.1 Implement core scanning functionality
- Implement recursive directory scanning using `os.scandir()`
- Filter files by configured video extensions
- Record file metadata: path, filename, size, modification timestamp
- Categorize files based on directory structure (movie/series/anime/other)
- Handle inaccessible files gracefully: log error and continue scanning
- _Requirements: 1.1, 1.2, 1.7_
- [x] 3.2 Add video metadata extraction
- Implement optional ffprobe integration for video metadata
- Extract resolution, codec, duration, bitrate when ffprobe available
- Handle ffprobe failures gracefully with non-blocking fallback
- _Requirements: 1.3_
- [x] 3.3 Implement inventory report generation
- Generate CSV inventory report with all file metadata (primary format)
- Implement CSV schema: path, filename, size_bytes, modified_timestamp, category, resolution, codec, duration_seconds, bitrate_kbps
- Support JSON export as optional format
- Include generation timestamp and library root in reports
- _Requirements: 1.4, 1.8, 11.1, 11.5, 11.6, 11.8_
- [ ]* 3.4 Write property tests for Inventory Scanner
- **Property 1: Scan completeness** - For any directory tree, scanning SHALL discover all and only files matching configured extensions
- **Property 2: Scan idempotence** - For any directory tree, multiple scans SHALL produce identical results
- **Property 3: Scan safety** - For any directory tree, scanning SHALL never modify files or directories
- **Property 4: Scan resilience** - For any directory tree with inaccessible files, scanning SHALL discover all accessible files and continue
- **Property 47: Anime inventory inclusion** - For any video files in anime directory, scanning SHALL discover and catalog them
- _Requirements: 1.1, 1.2, 1.6, 1.7, 12.1, 12.2_
- [ ]* 3.5 Write unit tests for Inventory Scanner
- Test scanning with various directory structures
- Test file extension filtering
- Test metadata extraction with and without ffprobe
- Test error handling for inaccessible files
- Test CSV and JSON report generation
- _Requirements: 1.1-1.8_
### 4. Identity Parser
- [x] 4.1 Implement movie identity parsing
- Parse patterns: "Title (Year)", "Title.Year", "Title - Year", "Title Year"
- Extract title and year with confidence scoring
- Implement quality tag removal: 1080p, 720p, 4K, BluRay, WEB-DL, HDTV, etc.
- Implement release group tag removal: [RARBG], (YTS), etc.
- Implement title normalization: remove extra whitespace, standardize capitalization
- Flag files without extractable year as needs_review
- _Requirements: 2.1-2.6_
- [x] 4.2 Implement series identity parsing
- Parse patterns: "SXXEYY", "SXXeYY", "SeasonXEpisodeY", "XXxYY"
- Extract series title, season number, episode numbers
- Handle multi-episode files: extract all episode numbers (e.g., S01E01-E02)
- Remove quality tags and release groups from series titles
- Normalize series titles
- Flag files with season=None or episodes=[] as needs_review (v1 constraint: these files excluded from organization plans)
- _Requirements: 3.1-3.6_
- [x] 4.3 Implement episode grouping
- Group parsed episodes by normalized series title and season number
- _Requirements: 3.7_
- [x] 4.4 Write property tests for Identity Parser
- **Property 5: Movie parsing** - For any filename matching common movie patterns, parser SHALL extract title and year
- **Property 6: Series parsing** - For any filename matching common series patterns, parser SHALL extract series title, season, and episodes
- **Property 7: Title normalization idempotence** - For any title string, normalizing multiple times SHALL produce same result
- **Property 8: Ambiguous filename flagging** - For any filename where season/episode cannot be extracted, parser SHALL mark as needs_review
- **Property 9: Episode grouping** - For any set of parsed episodes, grouping SHALL place episodes with identical normalized titles and seasons in same group
- **Property 51: Parsing error handling** - For any filename that cannot be parsed, system SHALL log error and mark file for review
- _Requirements: 2.1, 2.2, 2.5, 2.6, 3.1, 3.2, 3.5, 3.6, 3.7, 13.4_
- [x] 4.5 Write unit tests for Identity Parser
- Test various movie filename patterns with different quality tags
- Test various series filename patterns including multi-episode files
- Test title normalization edge cases
- Test ambiguous filename detection
- Test episode grouping logic
- _Requirements: 2.1-2.7, 3.1-3.7_
### 5. Analysis Engine
- [x] 5.1 Implement series completeness analysis
- Group episodes by series title and season
- For each season, find min and max episode numbers
- Detect gaps in episode sequence [min, max] using heuristic detection
- Generate completeness report showing episodes_found and episodes_missing (gaps only)
- Do NOT calculate percentages or "complete" status (v1 constraint: no external metadata, heuristic gap detection only)
- _Requirements: 4.1-4.6_
- [x] 5.2 Implement duplicate detection
- Group files by normalized identity (title+year for movies, title+season+episode for series)
- For groups with multiple files, flag as potential duplicates
- Extract quality comparison data: file size, resolution, codec
- Present comparison data without automatic recommendations
- _Requirements: 5.1-5.6_
- [x] 5.3 Implement report generation
- Generate completeness report (text/JSON) with series and detected gaps
- Generate duplicate report (text/JSON) with quality comparisons
- Generate summary report (text) with total files, size, category breakdown, files needing review
- Include generation timestamp and library root in all reports
- _Requirements: 11.2-11.4, 11.7, 11.8_
- [x] 5.4 Write property tests for Analysis Engine
- **Property 10: Gap detection** - For any set of episodes within season, analysis SHALL detect missing episode numbers in range [min, max]
- **Property 11: Multi-season independence** - For any series with multiple seasons, gap detection of one season SHALL not affect others
- **Property 12: Duplicate detection for movies** - For any set of movies with identical normalized titles and years, all SHALL be grouped as duplicates
- **Property 13: Duplicate detection for series** - For any set of series files with identical normalized titles, seasons, and episodes, all SHALL be grouped as duplicates
- **Property 14: Duplicate quality comparison** - For any duplicate group, comparison data SHALL include available metadata for each file
- **Property 42: Completeness report** - For any set of analyzed series, completeness report SHALL include all series with detected gaps
- **Property 43: Duplicate report grouping** - For any set of detected duplicates, duplicate report SHALL group files by identity with comparison data
- **Property 44: Summary report accuracy** - For any scanned library, summary report SHALL contain accurate counts and sizes
- _Requirements: 4.1, 4.2, 4.4, 5.1, 5.2, 5.3, 11.2, 11.3, 11.4_
- [x] 5.5 Write unit tests for Analysis Engine
- Test gap detection with various episode sequences
- Test multi-season analysis
- Test duplicate detection for movies and series
- Test quality comparison data extraction
- Test report generation in different formats
- _Requirements: 4.1-4.6, 5.1-5.6, 11.2-11.4_
### 6. Plan Generator
- [x] 6.1 Implement plan generation logic
- For each parsed identity, determine target directory using config templates
- Apply movie template: "movie/{title} ({year})/"
- Apply series template: "series/{title}/Season {season:02d}/"
- Apply filename templates for movies and series
- Create move/rename operations for movies and series with valid identities
- Generate no-op operations for anime, other, and series files with season=None or episodes=[] (v1 constraint: flagged for manual review)
- Preserve category boundaries (no cross-category moves)
- _Requirements: 6.1-6.5, 12.5_
- [x] 6.2 Implement conflict detection
- Check if destination file already exists for each operation
- Mark operations with conflicts: set has_conflict=True and conflict_reason
- _Requirements: 6.8_
- [x] 6.3 Implement plan serialization
- Save execution plan to JSON format (human-readable and editable)
- Include plan_id (UUID), created_at timestamp, operations list, summary
- _Requirements: 6.6, 6.7_
- [ ]* 6.4 Write property tests for Plan Generator
- **Property 15: Plan operation completeness** - For any set of video files with parsed identities, execution plan SHALL contain exactly one operation per file
- **Property 16: Movie template consistency** - For any movie identity, applying templates SHALL always produce same destination path
- **Property 17: Series template consistency** - For any series episode identity, applying templates SHALL always produce same destination path
- **Property 18: Category boundary preservation** - For any file in category directory, execution plan SHALL specify destination within same category
- **Property 19: Destination path uniqueness** - For any execution plan, no two operations SHALL specify same destination unless marked as conflict
- **Property 20: Conflict detection** - For any execution plan where destination file exists, operation SHALL be marked as conflict
- **Property 21: Plan JSON validity** - For any generated execution plan, output SHALL be valid JSON
- **Property 48: Anime processing deferral** - For any anime files, execution plans SHALL contain only no-op operations
- _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5, 6.7, 6.8, 12.3, 12.4, 12.5_
- [ ]* 6.5 Write unit tests for Plan Generator
- Test plan generation for various movie and series identities
- Test template application with different configurations
- Test no-op generation for anime and ambiguous files
- Test conflict detection scenarios
- Test JSON serialization and deserialization
- _Requirements: 6.1-6.9_
### 7. Execution Engine
- [x] 7.1 Implement execution mode handling
- Support mode parameter: "dry-run" (default) or "execute"
- In dry-run mode: simulate operations and log what would happen without making changes
- In execute mode: perform actual file operations
- Require --confirm flag for execute mode (v1 constraint: safety-first approach)
- _Requirements: 7.1, 7.2_
- [x] 7.2 Implement file operation execution
- Create destination directories if they don't exist
- Handle conflicts: skip operation and report (don't halt other operations, v1 constraint)
- Execute move/rename operations for each FileOperation
- Log all operations with timestamps, paths, and results
- Continue processing after non-fatal errors
- _Requirements: 7.4, 7.5, 7.6, 7.7_
- [x] 7.3 Implement execution summary and rollback log
- Generate execution summary with counts: successful, failed, skipped operations
- Save rollback log in execute mode only (not in dry-run)
- Include operation type, source, destination, timestamp for each operation
- _Requirements: 7.8, 9.1, 9.2_
- [ ]* 7.4 Write property tests for Execution Engine
- **Property 22: Execution mode safety** - For any execution plan, mode="dry-run" SHALL never modify files
- **Property 23: Directory creation** - For any file operation with destination in non-existent directory, executing SHALL create necessary parent directories
- **Property 24: Conflict handling** - For any execution plan with conflicts, executing SHALL skip conflicted operations and continue processing
- **Property 25: Operation logging** - For any executed operation, log SHALL contain timestamp, operation type, paths, and result
- **Property 26: Execution summary** - For any execution, summary SHALL contain accurate counts of successful, failed, and skipped operations
- **Property 50: Error resilience** - For any operation sequence with non-fatal errors, system SHALL log errors and continue processing
- _Requirements: 7.1, 7.2, 7.4, 7.5, 7.6, 7.7, 7.8, 13.2, 13.3_
- [ ]* 7.5 Write unit tests for Execution Engine
- Test dry-run mode (no file modifications)
- Test execute mode with actual file operations
- Test directory creation
- Test conflict handling
- Test error handling and resilience
- Test execution summary generation
- Test rollback log creation
- _Requirements: 7.1-7.8_
### 8. Rollback Functionality
- [x] 8.1 Implement rollback execution
- Load rollback log from JSON
- Reverse operation list (LIFO order for best-effort restoration)
- For each operation: attempt to move destination back to source
- Handle rollback failures gracefully: log error and continue
- Generate rollback summary with counts: successful, failed, skipped
- _Requirements: 9.1-9.8_
- [ ]* 8.2 Write property tests for rollback
- **Property 27: Rollback best-effort** - For any successful execution, rolling back SHALL attempt to restore all files to original locations
- **Property 28: Rollback idempotence** - For any rollback log, executing rollback multiple times SHALL produce same final state
- **Property 29: Rollback resilience** - For any rollback with failures, process SHALL log errors and continue processing
- **Property 30: Rollback summary** - For any rollback execution, summary SHALL contain accurate counts of successful, failed, and skipped operations
- _Requirements: 9.1, 9.2, 9.7, 9.8_
- [ ]* 8.3 Write unit tests for rollback
- Test rollback of move operations
- Test rollback of rename operations
- Test rollback failure handling
- Test rollback summary generation
- Test rollback idempotence
- _Requirements: 9.1-9.8_
### 9. Quarantine Manager
- [x] 9.1 Implement quarantine operations
- Verify file is in movie or series category (reject anime/other with error)
- Determine relative path from category root
- Construct quarantine path: `<category_root>/.quarantine/<relative_path>`
- Handle destination conflicts: append numeric suffix (_1, _2, etc.)
- Move file to category-specific quarantine directory
- _Requirements: 8.1, 8.2, 8.3, 8.6_
- [x] 9.2 Implement quarantine manifest management
- Update category-specific manifest.json with quarantine entry
- Include original_path, quarantine_path, timestamp, reason, size_bytes, category
- Save manifest to `<category_root>/.quarantine/manifest.json`
- _Requirements: 8.4, 8.5_
- [x] 9.3 Implement quarantine listing and restoration
- List quarantined files from category manifests, optionally filtered by category
- Restore files from quarantine to original locations (best-effort)
- _Requirements: 8.7_
- [ ]* 9.4 Write property tests for Quarantine Manager
- **Property 31: Quarantine location** - For any movie or series file marked for quarantine, file SHALL be moved to `<category_root>/.quarantine/`
- **Property 32: Quarantine category restriction** - For any file in anime or other categories, quarantine operations SHALL be rejected with error
- **Property 33: Quarantine manifest** - For any quarantined file, category-specific manifest SHALL contain original path, quarantine path, timestamp, and category
- **Property 34: Quarantine conflict handling** - For any file being quarantined where destination exists, system SHALL append numeric suffix
- **Property 35: Quarantine listing** - For any set of quarantined files, listing SHALL return all entries from category manifests, optionally filtered by category
- **Property 36: Quarantine restore** - For any quarantined file, restoring SHALL attempt to move file back to original location
- **Property 37: No permanent deletion in v1** - For any file operation in v1, system SHALL never permanently delete files
- _Requirements: 8.1, 8.2, 8.4, 8.5, 8.6, 8.7, 8.8_
- [ ]* 9.5 Write unit tests for Quarantine Manager
- Test quarantine operations for movie and series files
- Test category restriction enforcement
- Test manifest creation and updates
- Test conflict handling with numeric suffixes
- Test listing and restoration
- _Requirements: 8.1-8.8_
### 10. State Store
- [x] 10.1 Implement State Store operations
- Implement `load_state(path: Path) -> StateStore` to load from JSON
- Implement `save_state(store: StateStore, path: Path)` to save to JSON
- Implement `get_file_state(file_path: Path) -> Optional[FileState]` for path-based lookup
- Implement `set_file_state(file_path: Path, status: str, reason: Optional[str])` with idempotent updates and timestamp
- Implement `query_by_status(status: str) -> list[FileState]` to filter by status
- Implement `clear_state(file_path: Path)` to remove file state
- Validate status values: "reviewed", "ignored", "planned", "executed", "quarantined"
- _State Store design_
- [ ]* 10.2 Write property tests for State Store
- **Property 54: State persistence** - For any state store saved to disk, loading SHALL produce equivalent StateStore with all file states preserved
- **Property 55: State query** - For any status value, querying SHALL return all files with that status
- **Property 56: State update** - For any file path, setting state SHALL update store and subsequent queries return new status
- **Property 57: State JSON validity** - For any state store, saving SHALL produce valid JSON
- _State Store design_
- [ ]* 10.3 Write unit tests for State Store
- Test state loading and saving
- Test state queries by status
- Test state updates and idempotence
- Test state clearing
- Test status validation
- _State Store design_
### 11. Report Generator
- [x] 11.1 Implement report generation
- Generate inventory report in CSV/JSON formats with all file metadata
- Generate completeness report in text/JSON formats with series gaps
- Generate duplicate report in text/JSON formats with quality comparisons
- Generate summary report in text format with library statistics
- Include generation timestamp and library root in all reports
- _Requirements: 11.1-11.8_
- [ ]* 11.2 Write property tests for Report Generator
- **Property 41: Inventory report completeness** - For any set of scanned video files, inventory report SHALL include entry for each file
- **Property 45: Report format validity** - For any report, output SHALL be valid in specified format (CSV, JSON, or text)
- **Property 46: Report metadata** - For any generated report, report SHALL include generation timestamp and library root
- _Requirements: 11.1, 11.5, 11.6, 11.7, 11.8_
- [ ]* 11.3 Write unit tests for Report Generator
- Test inventory report generation in CSV and JSON
- Test completeness report generation
- Test duplicate report generation
- Test summary report generation
- Test report metadata inclusion
- _Requirements: 11.1-11.8_
### 12. CLI Interface
- [x] 12.1 Set up CLI framework
- Set up Click CLI with main entry point
- Implement global options: --config, --log-level
- Add help text for main command
- Implement error handling for invalid arguments
- _Requirements: 14.1, 14.10, 14.11_
- [x] 12.2 Implement scan command
- `vlm scan [--output inventory.csv]`
- Wire to Inventory Scanner
- Display progress and summary
- _Requirements: 14.2_
- [x] 12.3 Implement parse command
- `vlm parse [--input inventory.csv] [--output identities.json]`
- Wire to Identity Parser
- Display parsing statistics
- _Requirements: 14.3_
- [x] 12.4 Implement analyze command
- `vlm analyze [--input identities.json] [--output analysis.json]`
- Wire to Analysis Engine
- Display analysis summary
- _Requirements: 14.4_
- [x] 12.5 Implement plan command
- `vlm plan [--input identities.json] [--output plan.json]`
- Wire to Plan Generator
- Display plan summary with operation counts
- _Requirements: 14.5_
- [x] 12.6 Implement execute command
- `vlm execute [--plan plan.json] [--confirm]`
- Wire to Execution Engine
- Default to dry-run mode, require --confirm for actual execution
- Display execution progress and summary
- _Requirements: 14.6_
- [x] 12.7 Implement quarantine commands
- `vlm quarantine list [--category movie|series]`
- `vlm quarantine add <file> [--reason "duplicate"]`
- `vlm quarantine restore <file>`
- Wire to Quarantine Manager
- _Requirements: 14.7_
- [x] 12.8 Implement rollback command
- `vlm rollback [--log rollback.json]`
- Wire to rollback functionality
- Display rollback progress and summary
- _Requirements: 14.8_
- [x] 12.9 Implement report commands
- `vlm report inventory [--format csv|json|text]`
- `vlm report completeness [--format text|json]`
- `vlm report duplicates [--format text|json]`
- `vlm report summary`
- Wire to Report Generator
- _Requirements: 14.9_
- [x] 12.10 Implement state commands
- `vlm state show <file>`
- `vlm state set <file> --status reviewed [--reason "checked manually"]`
- `vlm state query --status ignored`
- `vlm state clear <file>`
- Wire to State Store
- _Requirements: 14.9_
- [x] 12.11 Implement config commands
- `vlm config init`
- `vlm config show`
- `vlm config validate`
- Wire to Configuration Manager
- _Requirements: 14.9_
- [ ]* 12.12 Write integration tests for CLI
- Test each command with various options
- Test error handling for invalid arguments
- Test help text display
- Test end-to-end workflow through CLI
- _Requirements: 14.1-14.11_
### 13. Integration and Polish
- [ ] 13.1 End-to-end integration testing
- Test complete workflow: scan → parse → analyze → plan → execute (dry-run) → execute (confirm) → rollback
- Test with real-world filename patterns and directory structures
- Test error scenarios: inaccessible files, invalid configs, conflicts
- Test quarantine operations end-to-end
- Test state management throughout workflow
- Verify all reports generate correctly with accurate data
- _All requirements_
- [ ] 13.2 Error handling validation
- Test parsing errors with ambiguous filenames
- Test file system errors (permissions, disk full, network issues)
- Test configuration errors
- Verify error logging and resilience
- _Requirements: 13.1-13.7_
- [ ] 13.3 Documentation
- Write README with installation instructions
- Document typical workflow with examples
- Add usage examples for all CLI commands
- Document configuration options
- Document v1 constraints and limitations
- _Requirements: 14.10_
- [ ] 13.4 Final checkpoint
- Ensure all tests pass (unit, property-based, integration)
- Verify all 57 correctness properties are validated
- Review code for security considerations (path validation, filename sanitization)
- Verify logging is comprehensive and useful
- Test with large library (performance validation)
- Ask user if any questions or issues arise
## Notes
### Testing Approach
- **Property-based tests** (marked with `*`): Validate universal correctness properties using Hypothesis with minimum 100 iterations per test. Each test references a specific property from the design document.
- **Unit tests** (marked with `*`): Test specific examples, edge cases, and error conditions for each component.
- **Integration tests**: Validate end-to-end workflows and component interactions.
### Task Dependencies
Tasks are ordered to build incrementally:
1. Foundation (config, logging, data structures)
2. Core components (scanner, parser, analysis)
3. Execution components (plan, execute, rollback)
4. Supporting features (quarantine, state, reports)
5. CLI interface
6. Integration and polish
### v1 Constraints
- **Quarantine**: Category-level only (movie/.quarantine/, series/.quarantine/), anime/other rejected
- **Completeness**: Heuristic gap detection only, no percentages or "complete" status
- **Parsing**: Hardcoded patterns, not user-configurable
- **Anime**: Inventory only, no organization
- **Deletion**: Quarantine only, no permanent deletion
- **State Store**: Simple path-based indexing, no fingerprint or move tracking
- **Execution**: Defaults to dry-run, requires --confirm for actual changes
- **Rollback**: Best-effort, not guaranteed restoration
### Property-Based Testing Configuration
- **Framework**: Hypothesis (Python PBT library)
- **Iterations**: Minimum 100 per test
- **Tagging**: Each test includes comment: `# Feature: video-library-manager, Property N: [description]`
- **Coverage**: All 57 correctness properties from design document
.markdownlint.json
+8
View File
@@ -0,0 +1,8 @@
{
"MD013": false,
"MD031": false,
"MD032": false,
"MD036": false,
"MD037": false,
"MD040": false
}
README.md
+35
View File
@@ -0,0 +1,35 @@
# Video Library Manager
A Python-based CLI tool for managing personal video collections with a safety-first, human-in-the-loop approach.
## Installation
This project uses `uv` for Python package management. To install:
```bash
# Install dependencies
uv pip install -e .
# Install with development dependencies
uv pip install -e ".[dev]"
```
## Usage
```bash
vlm --help
```
## Development
Run tests:
```bash
pytest
```
## Requirements
- Python >= 3.10
- uv (Python package manager)
- ffmpeg (optional, for video metadata extraction)
pyproject.toml
+36
View File
@@ -0,0 +1,36 @@
[project]
name = "video-library-manager"
version = "0.1.0"
description = "A Python-based CLI tool for managing personal video collections"
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
"click>=8.1.0",
"pyyaml>=6.0",
"ffmpeg-python>=0.2.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.4.0",
"hypothesis>=6.82.0",
]
[project.scripts]
vlm = "vlm.cli:main"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/vlm"]
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
[tool.hypothesis]
max_examples = 100
src/vlm/__init__.py
+3
View File
@@ -0,0 +1,3 @@
"""Video Library Manager - A Python-based CLI tool for managing personal video collections."""
__version__ = "0.1.0"
src/vlm/analysis.py
+188
View File
@@ -0,0 +1,188 @@
"""Analysis engine for detecting completeness issues and duplicates.
This module provides functionality to analyze video collections for:
- Series completeness (detecting episode gaps)
- Duplicate detection (finding duplicate content)
- Quality comparison (comparing video quality metrics)
"""
from vlm.models import SeriesIdentity, SeasonCompleteness, DuplicateGroup, VideoFile, MovieIdentity
def analyze_series_completeness(episodes: list[SeriesIdentity]) -> list[SeasonCompleteness]:
"""Analyze series completeness and detect episode gaps using heuristic detection.
This function groups episodes by series title and season, then detects gaps
in the episode sequence using heuristic detection. For each season, it finds
the minimum and maximum episode numbers and identifies missing episodes in
that range [min, max].
Note: This is heuristic gap detection only. It does NOT calculate percentages
or determine if seasons are "complete" (v1 constraint: no external metadata).
Args:
episodes: List of parsed series identities
Returns:
List of SeasonCompleteness objects for seasons with detected gaps
"""
from vlm.parser import group_episodes
# Group episodes by (title, season)
grouped = group_episodes(episodes)
completeness_results = []
# Analyze each season
for (series_title, season), episode_list in grouped.items():
# Collect all episode numbers from this season
all_episode_numbers = set()
for episode in episode_list:
all_episode_numbers.update(episode.episodes)
# Convert to sorted list
episodes_found = sorted(all_episode_numbers)
# Find min and max episode numbers
if not episodes_found:
continue
min_episode = min(episodes_found)
max_episode = max(episodes_found)
# Detect gaps in the range [min, max]
expected_episodes = set(range(min_episode, max_episode + 1))
found_episodes_set = set(episodes_found)
missing_episodes = sorted(expected_episodes - found_episodes_set)
# Only include seasons with gaps
if missing_episodes:
completeness_results.append(SeasonCompleteness(
series_title=series_title,
season=season,
episodes_found=episodes_found,
episodes_missing=missing_episodes
))
return completeness_results
def detect_duplicates(
identities: list[MovieIdentity | SeriesIdentity],
files: list[VideoFile]
) -> list[DuplicateGroup]:
"""Detect duplicate video files and provide quality comparison data.
Groups files by normalized identity (title+year for movies, title+season+episode
for series) and identifies groups with multiple files as potential duplicates.
Args:
identities: List of parsed identities (movies or series)
files: List of video files corresponding to the identities
Returns:
List of DuplicateGroup objects for files with duplicates
"""
# Create a mapping from original filename to VideoFile for quick lookup
file_map = {file.filename: file for file in files}
# Group identities by normalized identity
groups: dict[tuple, list[tuple[MovieIdentity | SeriesIdentity, VideoFile]]] = {}
for identity in identities:
# Create grouping key based on identity type
if isinstance(identity, MovieIdentity):
# For movies: group by (title, year)
# Skip if year is None (needs review)
if identity.year is None:
continue
key = ('movie', identity.title, identity.year)
else: # SeriesIdentity
# For series: group by (title, season, episode)
# Skip if season is None or episodes is empty (needs review)
if identity.season is None or not identity.episodes:
continue
# For multi-episode files, use the first episode for grouping
# Each episode in the list should be treated separately
for episode in identity.episodes:
key = ('series', identity.title, identity.season, episode)
# Get the corresponding VideoFile
video_file = file_map.get(identity.original_filename)
if video_file is None:
continue
# Add to group
if key not in groups:
groups[key] = []
groups[key].append((identity, video_file))
continue
# Get the corresponding VideoFile for movies
video_file = file_map.get(identity.original_filename)
if video_file is None:
continue
# Add to group
if key not in groups:
groups[key] = []
groups[key].append((identity, video_file))
# Filter groups to only those with multiple files (duplicates)
duplicate_groups = []
for key, items in groups.items():
if len(items) > 1:
# Extract identities and files
# Use the first identity as the representative
representative_identity = items[0][0]
duplicate_files = [item[1] for item in items]
# Generate quality comparison data
quality_comparison = compare_quality(duplicate_files)
duplicate_groups.append(DuplicateGroup(
identity=representative_identity,
files=duplicate_files,
quality_comparison=quality_comparison
))
return duplicate_groups
def compare_quality(files: list[VideoFile]) -> list[dict]:
"""Compare video quality metrics for a set of files.
Extracts and compares resolution, codec, file size, and other quality
indicators to help users decide which files to keep.
Args:
files: List of video files to compare
Returns:
List of dictionaries with quality comparison data for each file
"""
comparison_data = []
for file in files:
quality_info = {
'filename': file.filename,
'path': str(file.path),
'size_bytes': file.size_bytes,
}
# Add optional metadata if available
if file.resolution is not None:
quality_info['resolution'] = file.resolution
if file.codec is not None:
quality_info['codec'] = file.codec
if file.duration_seconds is not None:
quality_info['duration_seconds'] = file.duration_seconds
if file.bitrate_kbps is not None:
quality_info['bitrate_kbps'] = file.bitrate_kbps
comparison_data.append(quality_info)
return comparison_data
src/vlm/cli.py
+2019
View File
File diff suppressed because it is too large Load Diff
src/vlm/config.py
+208
View File
@@ -0,0 +1,208 @@
"""Configuration management for Video Library Manager."""
from dataclasses import dataclass, field
from pathlib import Path
from typing import Optional
import yaml
@dataclass
class Config:
"""Configuration for Video Library Manager.
Attributes:
library_root: Root directory of the video library
video_extensions: List of video file extensions to recognize
movie_template: Directory template for movies (e.g., "movie/{title} ({year})/")
series_template: Directory template for series (e.g., "series/{title}/Season {season:02d}/")
movie_filename_template: Filename template for movies (e.g., "{title} ({year}){ext}")
series_filename_template: Filename template for series (e.g., "S{season:02d}E{episode:02d}{ext}")
log_level: Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
quarantine_dir: Quarantine directory name relative to category root (e.g., ".quarantine")
"""
library_root: Path
video_extensions: list[str] = field(default_factory=lambda: [
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
])
movie_template: str = "movie/{title} ({year})/"
series_template: str = "series/{title}/Season {season:02d}/"
movie_filename_template: str = "{title} ({year}){ext}"
series_filename_template: str = "S{season:02d}E{episode:02d}{ext}"
log_level: str = "INFO"
quarantine_dir: str = ".quarantine"
def load_config(path: Path) -> Config:
"""Load configuration from YAML file.
Args:
path: Path to configuration file
Returns:
Config object with loaded settings
Raises:
FileNotFoundError: If config file doesn't exist (caller should handle by creating default)
yaml.YAMLError: If YAML syntax is invalid (caller should handle by using defaults)
"""
if not path.exists():
raise FileNotFoundError(f"Configuration file not found: {path}")
try:
with open(path, 'r', encoding='utf-8') as f:
data = yaml.safe_load(f)
except yaml.YAMLError as e:
raise yaml.YAMLError(f"Invalid YAML syntax in configuration file: {e}")
if data is None:
data = {}
# Extract library_root (required field)
library_root_str = data.get('library_root')
if not library_root_str:
raise ValueError("Configuration must specify 'library_root'")
library_root = Path(library_root_str).expanduser()
# Extract optional fields with defaults
video_extensions = data.get('video_extensions', [
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
])
# Extract templates
templates = data.get('templates', {})
movie_template = templates.get('movie_dir', "movie/{title} ({year})/")
series_template = templates.get('series_dir', "series/{title}/Season {season:02d}/")
movie_filename_template = templates.get('movie_filename', "{title} ({year}){ext}")
series_filename_template = templates.get('series_filename', "S{season:02d}E{episode:02d}{ext}")
# Extract other settings
quarantine_dir = data.get('quarantine_dir', '.quarantine')
log_level = data.get('log_level', 'INFO')
return Config(
library_root=library_root,
video_extensions=video_extensions,
movie_template=movie_template,
series_template=series_template,
movie_filename_template=movie_filename_template,
series_filename_template=series_filename_template,
log_level=log_level,
quarantine_dir=quarantine_dir
)
def create_default_config(path: Path) -> Config:
"""Create a default configuration file and return the Config object.
Args:
path: Path where configuration file should be created
Returns:
Config object with default settings
"""
# Create default config object
default_config = Config(
library_root=Path.home() / "Videos",
video_extensions=[".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"],
movie_template="movie/{title} ({year})/",
series_template="series/{title}/Season {season:02d}/",
movie_filename_template="{title} ({year}){ext}",
series_filename_template="S{season:02d}E{episode:02d}{ext}",
log_level="INFO",
quarantine_dir=".quarantine"
)
# Create YAML content
yaml_content = {
'library_root': str(default_config.library_root),
'video_extensions': default_config.video_extensions,
'templates': {
'movie_dir': default_config.movie_template,
'series_dir': default_config.series_template,
'movie_filename': default_config.movie_filename_template,
'series_filename': default_config.series_filename_template
},
'quarantine_dir': default_config.quarantine_dir,
'log_level': default_config.log_level
}
# Ensure parent directory exists
path.parent.mkdir(parents=True, exist_ok=True)
# Write configuration file
with open(path, 'w', encoding='utf-8') as f:
yaml.dump(yaml_content, f, default_flow_style=False, sort_keys=False)
return default_config
def validate_config(config: Config) -> list[str]:
"""Validate configuration and return list of error messages.
Args:
config: Configuration object to validate
Returns:
List of error messages (empty if valid)
"""
errors = []
# Validate library_root
if not isinstance(config.library_root, Path):
errors.append("library_root must be a Path object")
elif not str(config.library_root) or str(config.library_root) == ".":
errors.append("library_root cannot be empty")
# Validate video_extensions
if not config.video_extensions:
errors.append("video_extensions cannot be empty")
elif not isinstance(config.video_extensions, list):
errors.append("video_extensions must be a list")
else:
for ext in config.video_extensions:
if not isinstance(ext, str):
errors.append(f"video_extensions must contain strings, found: {type(ext)}")
break
if not ext.startswith('.'):
errors.append(f"video extension must start with '.': {ext}")
# Validate templates
if not config.movie_template:
errors.append("movie_template cannot be empty")
elif not isinstance(config.movie_template, str):
errors.append("movie_template must be a string")
if not config.series_template:
errors.append("series_template cannot be empty")
elif not isinstance(config.series_template, str):
errors.append("series_template must be a string")
if not config.movie_filename_template:
errors.append("movie_filename_template cannot be empty")
elif not isinstance(config.movie_filename_template, str):
errors.append("movie_filename_template must be a string")
if not config.series_filename_template:
errors.append("series_filename_template cannot be empty")
elif not isinstance(config.series_filename_template, str):
errors.append("series_filename_template must be a string")
# Validate log_level
valid_log_levels = ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
if not config.log_level:
errors.append("log_level cannot be empty")
elif not isinstance(config.log_level, str):
errors.append("log_level must be a string")
elif config.log_level.upper() not in valid_log_levels:
errors.append(f"log_level must be one of {valid_log_levels}, got: {config.log_level}")
# Validate quarantine_dir
if not config.quarantine_dir:
errors.append("quarantine_dir cannot be empty")
elif not isinstance(config.quarantine_dir, str):
errors.append("quarantine_dir must be a string")
elif config.quarantine_dir.startswith('/') or config.quarantine_dir.startswith('\\'):
errors.append("quarantine_dir must be relative to category root, not absolute")
return errors
src/vlm/executor.py
+591
View File
@@ -0,0 +1,591 @@
"""Execution engine for Video Library Manager.
This module provides safe execution of file operations with:
- Dry-run mode (default): simulates operations without making changes
- Execute mode: performs actual file operations (requires explicit confirmation)
- Comprehensive logging of all operations
- Error handling and resilience
"""
import json
import logging
from datetime import datetime
from pathlib import Path
from typing import Optional
from uuid import uuid4
from .logging_config import get_logger, log_operation
from .models import ExecutionPlan, FileOperation, OperationResult, RollbackLog
class ExecutionEngine:
"""Engine for executing file operations safely with dry-run support."""
def __init__(self, logger: Optional[logging.Logger] = None):
"""Initialize the execution engine.
Args:
logger: Optional logger instance (uses default if not provided)
"""
self.logger = logger or get_logger()
def execute_plan(
self,
plan: ExecutionPlan,
mode: str = "dry-run",
confirmed: bool = False
) -> tuple[list[OperationResult], dict, Optional[RollbackLog]]:
"""Execute an execution plan with the specified mode.
Args:
plan: The execution plan to execute
mode: Execution mode - "dry-run" (default) or "execute"
confirmed: Whether execution has been explicitly confirmed (required for execute mode)
Returns:
Tuple of (operation results, execution summary, rollback log)
- operation results: List of results for each operation
- execution summary: Dict with counts of successful, failed, and skipped operations
- rollback log: RollbackLog if mode is "execute", None for dry-run
Raises:
ValueError: If mode is invalid or execute mode used without confirmation
"""
# Validate mode
if mode not in ("dry-run", "execute"):
raise ValueError(f"Invalid mode: {mode}. Must be 'dry-run' or 'execute'")
# Require confirmation for execute mode
if mode == "execute" and not confirmed:
raise ValueError(
"Execute mode requires explicit confirmation. "
"Set confirmed=True or use --confirm flag in CLI"
)
log_operation(
self.logger,
logging.INFO,
f"Starting execution in {mode} mode with {len(plan.operations)} operations",
operation_type="execute"
)
# Execute all operations
results = []
for operation in plan.operations:
result = self.execute_operation(operation, mode)
results.append(result)
# Generate execution summary
summary = self._generate_execution_summary(results)
# Create rollback log only in execute mode
rollback_log = None
if mode == "execute":
# Only include successful operations in rollback log
successful_operations = [r for r in results if r.success]
rollback_log = RollbackLog(
log_id=str(uuid4()),
execution_plan_id=plan.plan_id,
executed_at=datetime.now(),
operations=successful_operations
)
log_operation(
self.logger,
logging.INFO,
f"Created rollback log with {len(successful_operations)} successful operations",
operation_type="execute"
)
# Log execution summary
self._log_execution_summary(summary, mode)
return results, summary, rollback_log
def execute_operation(
self,
operation: FileOperation,
mode: str
) -> OperationResult:
"""Execute a single file operation.
Args:
operation: The file operation to execute
mode: Execution mode - "dry-run" or "execute"
Returns:
OperationResult with success status and any error message
"""
executed_at = datetime.now()
# Handle no-op operations
if operation.operation_type == "no-op":
log_operation(
self.logger,
logging.DEBUG,
f"Skipping no-op operation: {operation.reason}",
operation_type="execute",
file_path=operation.source_path
)
return OperationResult(
operation=operation,
success=True,
error_message=None,
executed_at=executed_at
)
# Handle conflicted operations
if operation.has_conflict:
log_operation(
self.logger,
logging.WARNING,
f"Skipping conflicted operation: {operation.conflict_reason}",
operation_type="execute",
file_path=operation.source_path
)
return OperationResult(
operation=operation,
success=False,
error_message=f"Conflict: {operation.conflict_reason}",
executed_at=executed_at
)
# Execute based on mode
if mode == "dry-run":
return self._simulate_operation(operation, executed_at)
else:
return self._perform_operation(operation, executed_at)
def _simulate_operation(
self,
operation: FileOperation,
executed_at: datetime
) -> OperationResult:
"""Simulate an operation in dry-run mode without making changes.
Args:
operation: The file operation to simulate
executed_at: Timestamp of execution
Returns:
OperationResult indicating what would happen
"""
log_operation(
self.logger,
logging.INFO,
f"[DRY-RUN] Would {operation.operation_type}: "
f"{operation.source_path} -> {operation.destination_path}",
operation_type="execute",
file_path=operation.source_path
)
return OperationResult(
operation=operation,
success=True,
error_message=None,
executed_at=executed_at
)
def _perform_operation(
self,
operation: FileOperation,
executed_at: datetime
) -> OperationResult:
"""Perform an actual file operation.
Args:
operation: The file operation to perform
executed_at: Timestamp of execution
Returns:
OperationResult with success status and any error message
"""
try:
# Validate source file exists
if not operation.source_path.exists():
error_msg = f"Source file does not exist: {operation.source_path}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="execute",
file_path=operation.source_path
)
return OperationResult(
operation=operation,
success=False,
error_message=error_msg,
executed_at=executed_at
)
# Create destination directory if needed
if operation.destination_path:
operation.destination_path.parent.mkdir(parents=True, exist_ok=True)
# Perform the move/rename operation
operation.source_path.rename(operation.destination_path)
log_operation(
self.logger,
logging.INFO,
f"Successfully {operation.operation_type}: "
f"{operation.source_path} -> {operation.destination_path}",
operation_type="execute",
file_path=operation.source_path
)
return OperationResult(
operation=operation,
success=True,
error_message=None,
executed_at=executed_at
)
except Exception as e:
error_msg = f"Failed to {operation.operation_type}: {str(e)}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="execute",
file_path=operation.source_path
)
return OperationResult(
operation=operation,
success=False,
error_message=error_msg,
executed_at=executed_at
)
def _generate_execution_summary(
self,
results: list[OperationResult]
) -> dict:
"""Generate a structured execution summary.
Args:
results: List of operation results
Returns:
Dictionary with counts of successful, failed, and skipped operations
"""
successful = sum(1 for r in results if r.success)
failed = sum(1 for r in results if not r.success)
skipped = sum(
1 for r in results
if r.operation.operation_type == "no-op" or r.operation.has_conflict
)
return {
"successful": successful,
"failed": failed,
"skipped": skipped,
"total": len(results)
}
def _log_execution_summary(
self,
summary: dict,
mode: str
) -> None:
"""Log a summary of execution results.
Args:
summary: Execution summary dictionary
mode: Execution mode that was used
"""
summary_msg = (
f"Execution summary ({mode} mode): "
f"{summary['successful']} successful, {summary['failed']} failed, {summary['skipped']} skipped"
)
log_operation(
self.logger,
logging.INFO,
summary_msg,
operation_type="execute"
)
def save_rollback_log(
self,
rollback_log: RollbackLog,
output_path: Path
) -> None:
"""Save rollback log to disk in JSON format.
Args:
rollback_log: The rollback log to save
output_path: Path where the rollback log should be saved
"""
# Convert rollback log to JSON-serializable format
log_data = {
"log_id": rollback_log.log_id,
"execution_plan_id": rollback_log.execution_plan_id,
"executed_at": rollback_log.executed_at.isoformat(),
"operations": [
{
"operation_type": op.operation.operation_type,
"source_path": str(op.operation.source_path),
"destination_path": str(op.operation.destination_path) if op.operation.destination_path else None,
"reason": op.operation.reason,
"success": op.success,
"error_message": op.error_message,
"executed_at": op.executed_at.isoformat()
}
for op in rollback_log.operations
]
}
# Ensure output directory exists
output_path.parent.mkdir(parents=True, exist_ok=True)
# Write to file
with open(output_path, 'w', encoding='utf-8') as f:
json.dump(log_data, f, indent=2)
log_operation(
self.logger,
logging.INFO,
f"Saved rollback log to {output_path}",
operation_type="execute"
)
def load_rollback_log(
self,
log_path: Path
) -> RollbackLog:
"""Load rollback log from disk.
Args:
log_path: Path to the rollback log JSON file
Returns:
RollbackLog object loaded from the file
Raises:
FileNotFoundError: If the log file does not exist
ValueError: If the log file is invalid JSON or missing required fields
"""
if not log_path.exists():
raise FileNotFoundError(f"Rollback log not found: {log_path}")
try:
with open(log_path, 'r', encoding='utf-8') as f:
log_data = json.load(f)
# Reconstruct RollbackLog from JSON data
operations = []
for op_data in log_data["operations"]:
# Reconstruct FileOperation
file_op = FileOperation(
operation_type=op_data["operation_type"],
source_path=Path(op_data["source_path"]),
destination_path=Path(op_data["destination_path"]) if op_data["destination_path"] else None,
reason=op_data["reason"],
has_conflict=False, # Conflicts don't matter for rollback
conflict_reason=None
)
# Reconstruct OperationResult
op_result = OperationResult(
operation=file_op,
success=op_data["success"],
error_message=op_data["error_message"],
executed_at=datetime.fromisoformat(op_data["executed_at"])
)
operations.append(op_result)
rollback_log = RollbackLog(
log_id=log_data["log_id"],
execution_plan_id=log_data["execution_plan_id"],
executed_at=datetime.fromisoformat(log_data["executed_at"]),
operations=operations
)
log_operation(
self.logger,
logging.INFO,
f"Loaded rollback log from {log_path} with {len(operations)} operations",
operation_type="rollback"
)
return rollback_log
except (json.JSONDecodeError, KeyError) as e:
raise ValueError(f"Invalid rollback log format: {str(e)}")
def rollback(
self,
rollback_log: RollbackLog
) -> tuple[list[OperationResult], dict]:
"""Rollback operations from a rollback log (best-effort).
This method attempts to reverse all operations in the rollback log by moving
files from their destination back to their source. Operations are processed
in LIFO (Last In, First Out) order for best-effort restoration.
Args:
rollback_log: The rollback log containing operations to reverse
Returns:
Tuple of (rollback results, rollback summary)
- rollback results: List of OperationResult for each rollback attempt
- rollback summary: Dict with counts of successful, failed, and skipped operations
"""
log_operation(
self.logger,
logging.INFO,
f"Starting rollback of {len(rollback_log.operations)} operations (LIFO order)",
operation_type="rollback"
)
# Reverse the operation list (LIFO order)
reversed_operations = list(reversed(rollback_log.operations))
# Attempt to rollback each operation
results = []
for original_result in reversed_operations:
result = self._rollback_operation(original_result)
results.append(result)
# Generate rollback summary
summary = self._generate_rollback_summary(results)
# Log rollback summary
self._log_rollback_summary(summary)
return results, summary
def _rollback_operation(
self,
original_result: OperationResult
) -> OperationResult:
"""Rollback a single operation (best-effort).
Args:
original_result: The original operation result to rollback
Returns:
OperationResult indicating success or failure of the rollback
"""
operation = original_result.operation
executed_at = datetime.now()
# Skip no-op operations
if operation.operation_type == "no-op":
log_operation(
self.logger,
logging.DEBUG,
f"Skipping rollback of no-op operation",
operation_type="rollback",
file_path=operation.source_path
)
return OperationResult(
operation=operation,
success=True,
error_message=None,
executed_at=executed_at
)
try:
# For move/rename operations, reverse the direction
# Original: source -> destination
# Rollback: destination -> source
if operation.destination_path and operation.destination_path.exists():
# Move file back from destination to source
operation.destination_path.rename(operation.source_path)
log_operation(
self.logger,
logging.INFO,
f"Successfully rolled back: {operation.destination_path} -> {operation.source_path}",
operation_type="rollback",
file_path=operation.destination_path
)
return OperationResult(
operation=operation,
success=True,
error_message=None,
executed_at=executed_at
)
else:
# Destination file doesn't exist - cannot rollback
error_msg = f"Cannot rollback: destination file not found at {operation.destination_path}"
log_operation(
self.logger,
logging.WARNING,
error_msg,
operation_type="rollback",
file_path=operation.destination_path
)
return OperationResult(
operation=operation,
success=False,
error_message=error_msg,
executed_at=executed_at
)
except Exception as e:
# Handle rollback failures gracefully - log and continue
error_msg = f"Failed to rollback operation: {str(e)}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="rollback",
file_path=operation.destination_path
)
return OperationResult(
operation=operation,
success=False,
error_message=error_msg,
executed_at=executed_at
)
def _generate_rollback_summary(
self,
results: list[OperationResult]
) -> dict:
"""Generate a structured rollback summary.
Args:
results: List of rollback operation results
Returns:
Dictionary with counts of successful, failed, and skipped operations
"""
successful = sum(1 for r in results if r.success)
failed = sum(1 for r in results if not r.success)
skipped = sum(
1 for r in results
if r.operation.operation_type == "no-op"
)
return {
"successful": successful,
"failed": failed,
"skipped": skipped,
"total": len(results)
}
def _log_rollback_summary(
self,
summary: dict
) -> None:
"""Log a summary of rollback results.
Args:
summary: Rollback summary dictionary
"""
summary_msg = (
f"Rollback summary: "
f"{summary['successful']} successful, {summary['failed']} failed, {summary['skipped']} skipped"
)
log_operation(
self.logger,
logging.INFO,
summary_msg,
operation_type="rollback"
)
src/vlm/logging_config.py
+139
View File
@@ -0,0 +1,139 @@
"""Logging configuration for Video Library Manager.
This module provides centralized logging configuration with:
- Configurable log levels (DEBUG, INFO, WARNING, ERROR)
- Dual output: console (INFO+) and file (DEBUG+)
- Timestamps, operation type, and file paths in log entries
- Log rotation at 10MB threshold
"""
import logging
import logging.handlers
from pathlib import Path
from typing import Optional
# Default log directory
DEFAULT_LOG_DIR = Path.home() / ".vlm" / "logs"
DEFAULT_LOG_FILE = "vlm.log"
MAX_LOG_SIZE = 10 * 1024 * 1024 # 10MB in bytes
BACKUP_COUNT = 5 # Keep 5 backup log files
class OperationContextFilter(logging.Filter):
"""Filter to add operation context to log records."""
def filter(self, record: logging.LogRecord) -> bool:
"""Add operation_type and file_path attributes if not present."""
if not hasattr(record, 'operation_type'):
record.operation_type = 'general'
if not hasattr(record, 'file_path'):
record.file_path = ''
return True
def setup_logging(
log_level: str = "INFO",
log_dir: Optional[Path] = None,
log_file: str = DEFAULT_LOG_FILE
) -> logging.Logger:
"""Configure logging with dual output (console and file) and rotation.
Args:
log_level: Minimum log level for console output (DEBUG, INFO, WARNING, ERROR)
log_dir: Directory for log files (defaults to ~/.vlm/logs)
log_file: Name of the log file (defaults to vlm.log)
Returns:
Configured logger instance
Raises:
ValueError: If log_level is invalid
"""
# Validate log level
numeric_level = getattr(logging, log_level.upper(), None)
if not isinstance(numeric_level, int):
raise ValueError(f"Invalid log level: {log_level}")
# Use default log directory if not specified
if log_dir is None:
log_dir = DEFAULT_LOG_DIR
# Create log directory if it doesn't exist
log_dir.mkdir(parents=True, exist_ok=True)
# Get root logger
logger = logging.getLogger("vlm")
logger.setLevel(logging.DEBUG) # Capture all levels, handlers will filter
# Remove existing handlers to avoid duplicates
logger.handlers.clear()
# Create formatter with timestamps, operation type, and file paths
formatter = logging.Formatter(
fmt='%(asctime)s - %(levelname)s - [%(operation_type)s] - %(message)s%(file_path)s',
datefmt='%Y-%m-%d %H:%M:%S'
)
# Console handler (INFO+)
console_handler = logging.StreamHandler()
console_handler.setLevel(numeric_level)
console_handler.setFormatter(formatter)
console_handler.addFilter(OperationContextFilter())
logger.addHandler(console_handler)
# File handler with rotation (DEBUG+)
log_file_path = log_dir / log_file
file_handler = logging.handlers.RotatingFileHandler(
filename=log_file_path,
maxBytes=MAX_LOG_SIZE,
backupCount=BACKUP_COUNT,
encoding='utf-8'
)
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(formatter)
file_handler.addFilter(OperationContextFilter())
logger.addHandler(file_handler)
# Prevent propagation to root logger
logger.propagate = False
return logger
def get_logger() -> logging.Logger:
"""Get the configured VLM logger instance.
Returns:
Logger instance (creates default configuration if not already set up)
"""
logger = logging.getLogger("vlm")
# If logger has no handlers, set up default configuration
if not logger.handlers:
setup_logging()
return logger
def log_operation(
logger: logging.Logger,
level: int,
message: str,
operation_type: str = "general",
file_path: Optional[Path] = None
) -> None:
"""Log a message with operation context.
Args:
logger: Logger instance
level: Log level (logging.DEBUG, logging.INFO, etc.)
message: Log message
operation_type: Type of operation (scan, parse, execute, etc.)
file_path: Optional file path related to the operation
"""
extra = {
'operation_type': operation_type,
'file_path': f' - {file_path}' if file_path else ''
}
logger.log(level, message, extra=extra)
src/vlm/models.py
+234
View File
@@ -0,0 +1,234 @@
"""Data structures for Video Library Manager.
This module defines the core data structures used throughout the application
for representing video files and their parsed identities.
"""
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Optional
@dataclass
class VideoFile:
"""Represents a video file discovered during inventory scanning.
Attributes:
path: Full path to the video file
filename: Name of the file (without directory path)
size_bytes: File size in bytes
modified_timestamp: Last modification timestamp
category: Category of the video ("movie", "series", "anime", "other")
resolution: Optional video resolution (e.g., "1920x1080")
codec: Optional video codec (e.g., "h264")
duration_seconds: Optional video duration in seconds
bitrate_kbps: Optional video bitrate in kilobits per second
"""
path: Path
filename: str
size_bytes: int
modified_timestamp: datetime
category: str
# Optional metadata (if ffprobe available)
resolution: Optional[str] = None
codec: Optional[str] = None
duration_seconds: Optional[float] = None
bitrate_kbps: Optional[int] = None
@dataclass
class MovieIdentity:
"""Represents the parsed identity of a movie file.
Attributes:
title: Extracted movie title (normalized)
year: Extracted release year (None if not found)
confidence: Confidence score of the parsing (0.0 to 1.0)
needs_review: Flag indicating if manual review is needed
original_filename: Original filename before parsing
"""
title: str
year: Optional[int]
confidence: float
needs_review: bool
original_filename: str
@dataclass
class SeriesIdentity:
"""Represents the parsed identity of a TV series episode file.
Attributes:
title: Extracted series title (normalized)
season: Extracted season number (None if not found)
episodes: List of episode numbers (supports multi-episode files)
confidence: Confidence score of the parsing (0.0 to 1.0)
needs_review: Flag indicating if manual review is needed
original_filename: Original filename before parsing
"""
title: str
season: Optional[int]
episodes: list[int]
confidence: float
needs_review: bool
original_filename: str
@dataclass
class FileOperation:
"""Represents a single file operation in an execution plan.
Attributes:
operation_type: Type of operation ("move", "rename", "quarantine", "no-op")
source_path: Source file path
destination_path: Destination file path (None for no-op operations)
reason: Human-readable reason for the operation
has_conflict: Flag indicating if destination already exists
conflict_reason: Description of the conflict (None if no conflict)
"""
operation_type: str
source_path: Path
destination_path: Optional[Path]
reason: str
has_conflict: bool
conflict_reason: Optional[str] = None
@dataclass
class ExecutionPlan:
"""Represents a complete execution plan with all file operations.
Attributes:
plan_id: Unique identifier for the plan (UUID)
created_at: Timestamp when the plan was created
operations: List of file operations to execute
summary: Dictionary with operation counts by type
"""
plan_id: str
created_at: datetime
operations: list[FileOperation]
summary: dict
@dataclass
class OperationResult:
"""Represents the result of executing a single file operation.
Attributes:
operation: The file operation that was executed
success: Flag indicating if the operation succeeded
error_message: Error message if operation failed (None if successful)
executed_at: Timestamp when the operation was executed
"""
operation: FileOperation
success: bool
error_message: Optional[str]
executed_at: datetime
@dataclass
class RollbackLog:
"""Represents a log of executed operations for rollback purposes.
Attributes:
log_id: Unique identifier for the rollback log (UUID)
execution_plan_id: ID of the execution plan that was executed
executed_at: Timestamp when the operations were executed
operations: List of operation results that were executed
"""
log_id: str
execution_plan_id: str
executed_at: datetime
operations: list[OperationResult]
@dataclass
class QuarantineEntry:
"""Represents a single file in quarantine.
Attributes:
original_path: Original path of the file before quarantine
quarantine_path: Path to the file in quarantine directory
quarantined_at: Timestamp when the file was quarantined
reason: Optional reason for quarantining the file
size_bytes: File size in bytes
category: Category of the video ("movie" or "series")
"""
original_path: Path
quarantine_path: Path
quarantined_at: datetime
reason: Optional[str]
size_bytes: int
category: str
@dataclass
class QuarantineManifest:
"""Represents a manifest of all quarantined files in a category.
Attributes:
entries: List of quarantine entries
"""
entries: list[QuarantineEntry]
@dataclass
class FileState:
"""Represents the state of a file in the workflow.
Attributes:
file_path: Path to the file
status: Current status ("reviewed", "ignored", "planned", "executed", "quarantined")
reason: Optional reason for the status
updated_at: Timestamp when the state was last updated
"""
file_path: Path
status: str
reason: Optional[str]
updated_at: datetime
@dataclass
class StateStore:
"""Represents the persistent state store for all files.
Attributes:
states: Dictionary mapping file path strings to FileState objects
version: Version of the state store format
last_updated: Timestamp when the state store was last updated
"""
states: dict[str, FileState]
version: str
last_updated: datetime
@dataclass
class SeasonCompleteness:
"""Represents completeness analysis for a single season of a series.
Attributes:
series_title: Normalized series title
season: Season number
episodes_found: List of episode numbers that were found
episodes_missing: List of episode numbers missing in the range [min, max]
"""
series_title: str
season: int
episodes_found: list[int]
episodes_missing: list[int]
@dataclass
class DuplicateGroup:
"""Represents a group of duplicate video files.
Attributes:
identity: The shared identity (MovieIdentity or SeriesIdentity)
files: List of VideoFile objects that are duplicates
quality_comparison: List of dictionaries with quality metrics for each file
"""
identity: MovieIdentity | SeriesIdentity
files: list[VideoFile]
quality_comparison: list[dict]
src/vlm/parser.py
+267
View File
@@ -0,0 +1,267 @@
"""Identity parser for extracting movie and series information from filenames.
This module provides functionality to parse video filenames and extract
logical identities such as movie titles/years and series titles/seasons/episodes.
"""
import re
from typing import Optional
from vlm.models import MovieIdentity, SeriesIdentity
# Quality tags to remove from titles
QUALITY_TAGS = [
r'\b1080p\b', r'\b720p\b', r'\b480p\b', r'\b2160p\b',
r'\b4K\b', r'\bUHD\b', r'\bHD\b',
r'\bBluRay\b', r'\bBlu-Ray\b', r'\bBRRip\b', r'\bBDRip\b',
r'\bWEB-DL\b', r'\bWEBRip\b', r'\bWEB\b',
r'\bHDTV\b', r'\bHDRip\b',
r'\bDVDRip\b', r'\bDVD\b',
r'\bx264\b', r'\bx265\b', r'\bh264\b', r'\bh265\b', r'\bHEVC\b',
r'\bAAC\b', r'\bAC3\b', r'\bDTS\b',
r'\b10bit\b', r'\b8bit\b',
]
# Release group patterns (in brackets, but NOT years in parentheses)
RELEASE_GROUP_PATTERNS = [
r'\[[\w\s\-\.]+\]', # [RARBG], [YTS], etc.
]
def remove_quality_tags(text: str) -> str:
"""Remove quality indicators from text.
Args:
text: Input text containing potential quality tags
Returns:
Text with quality tags removed
"""
result = text
for pattern in QUALITY_TAGS:
result = re.sub(pattern, '', result, flags=re.IGNORECASE)
return result
def remove_release_groups(text: str) -> str:
"""Remove release group tags from text.
Args:
text: Input text containing potential release group tags
Returns:
Text with release group tags removed
"""
result = text
for pattern in RELEASE_GROUP_PATTERNS:
result = re.sub(pattern, '', result)
return result
def normalize_title(title: str) -> str:
"""Normalize a title by cleaning whitespace and standardizing capitalization.
Args:
title: Raw title string
Returns:
Normalized title with proper capitalization and spacing
"""
# Replace dots and underscores with spaces
title = title.replace('.', ' ').replace('_', ' ')
# Remove extra whitespace
title = ' '.join(title.split())
# Apply title case
title = title.title()
return title.strip()
def parse_movie(filename: str) -> MovieIdentity:
"""Parse a movie filename to extract title and year.
Supports patterns:
- Title (Year)
- Title.Year
- Title - Year
- Title Year
Args:
filename: Movie filename to parse
Returns:
MovieIdentity with extracted information
"""
# Remove file extension
name_without_ext = filename
for ext in ['.mp4', '.mkv', '.avi', '.mov', '.wmv', '.flv', '.webm', '.m4v']:
if name_without_ext.lower().endswith(ext):
name_without_ext = name_without_ext[:-len(ext)]
break
# Try different patterns in order of confidence BEFORE cleaning
# This preserves the year in parentheses
patterns = [
# Pattern: Title (Year) - High confidence
(r'^(.+?)\s*\((\d{4})\)', 0.9),
# Pattern: Title.Year or Title-Year - High confidence
(r'^(.+?)[\.\-](\d{4})', 0.9),
# Pattern: Title - Year - Medium confidence
(r'^(.+?)\s+-\s+(\d{4})', 0.7),
# Pattern: Title Year (4 digits at end) - Medium confidence
(r'^(.+?)\s+(\d{4})(?:\s|$)', 0.7),
]
for pattern, confidence in patterns:
match = re.search(pattern, name_without_ext)
if match:
title = match.group(1)
year = int(match.group(2))
# Now clean the title
title = remove_quality_tags(title)
title = remove_release_groups(title)
title = normalize_title(title)
return MovieIdentity(
title=title,
year=year,
confidence=confidence,
needs_review=False,
original_filename=filename
)
# No year found - clean and extract title, flag for review
cleaned = remove_quality_tags(name_without_ext)
cleaned = remove_release_groups(cleaned)
title = normalize_title(cleaned)
return MovieIdentity(
title=title,
year=None,
confidence=0.3,
needs_review=True,
original_filename=filename
)
def parse_series(filename: str) -> SeriesIdentity:
"""Parse a series filename to extract title, season, and episode numbers.
Supports patterns:
- SXXEYY (e.g., S01E01)
- SXXeYY (e.g., S01e01)
- SeasonXEpisodeY (e.g., Season1Episode1)
- XXxYY (e.g., 1x01)
- Multi-episode: S01E01-E02, S01E01E02, etc.
Args:
filename: Series filename to parse
Returns:
SeriesIdentity with extracted information
"""
# Remove file extension
name_without_ext = filename
for ext in ['.mp4', '.mkv', '.avi', '.mov', '.wmv', '.flv', '.webm', '.m4v']:
if name_without_ext.lower().endswith(ext):
name_without_ext = name_without_ext[:-len(ext)]
break
# Try different patterns in order of confidence
patterns = [
# Pattern: SXXEYY or SXXeYY - High confidence
# Also handles multi-episode: S01E01-E02, S01E01E02E03, etc.
(r'[Ss](\d{1,2})[Ee](\d{1,2})', 0.9),
# Pattern: XXxYY - High confidence
(r'(\d{1,2})x(\d{1,2})', 0.9),
# Pattern: Season X Episode Y - Medium confidence
(r'[Ss]eason\s*(\d{1,2})\s*[Ee]pisode\s*(\d{1,2})', 0.7),
]
season = None
episodes = []
confidence = 0.0
title_part = name_without_ext
for pattern, conf in patterns:
match = re.search(pattern, name_without_ext, re.IGNORECASE)
if match:
season = int(match.group(1))
episodes = [int(match.group(2))]
confidence = conf
# Extract title (everything before the match)
title_part = name_without_ext[:match.start()]
# Handle multi-episode files for SXXEYY pattern
if pattern.startswith(r'[Ss]'):
# Find the full episode section (from S01E01 onwards)
remaining = name_without_ext[match.start():]
# Look for all episode numbers: E01, -E02, E03, etc.
all_episode_matches = re.findall(r'[Ee](\d{1,2})', remaining)
if all_episode_matches:
episodes = [int(ep) for ep in all_episode_matches]
break
# Clean the title
if title_part:
title_part = remove_quality_tags(title_part)
title_part = remove_release_groups(title_part)
title_part = normalize_title(title_part)
else:
# If no title part found, use the whole filename cleaned
title_part = remove_quality_tags(name_without_ext)
title_part = remove_release_groups(title_part)
title_part = normalize_title(title_part)
# Determine if review is needed
needs_review = season is None or len(episodes) == 0
# If no pattern matched, set low confidence
if season is None:
confidence = 0.3
return SeriesIdentity(
title=title_part,
season=season,
episodes=episodes,
confidence=confidence,
needs_review=needs_review,
original_filename=filename
)
def group_episodes(episodes: list[SeriesIdentity]) -> dict[tuple[str, int], list[SeriesIdentity]]:
"""Group parsed episodes by normalized series title and season number.
Episodes are grouped by (normalized_title, season) tuple. Episodes with
season=None are excluded from grouping as they need manual review.
Args:
episodes: List of parsed series identities
Returns:
Dictionary mapping (title, season) tuples to lists of SeriesIdentity objects
"""
groups: dict[tuple[str, int], list[SeriesIdentity]] = {}
for episode in episodes:
# Skip episodes without season (they need manual review)
if episode.season is None:
continue
# Create grouping key from normalized title and season
key = (episode.title, episode.season)
# Add episode to the appropriate group
if key not in groups:
groups[key] = []
groups[key].append(episode)
return groups
src/vlm/planner.py
+378
View File
@@ -0,0 +1,378 @@
"""Plan generator for creating execution plans from parsed identities.
This module generates structured execution plans that specify how video files
should be organized based on their parsed identities and configuration templates.
"""
import json
import uuid
from datetime import datetime
from pathlib import Path
from typing import Union
from vlm.config import Config
from vlm.models import (
ExecutionPlan,
FileOperation,
MovieIdentity,
SeriesIdentity,
VideoFile,
)
def generate_plan(
identities: list[tuple[VideoFile, Union[MovieIdentity, SeriesIdentity, None]]],
config: Config
) -> ExecutionPlan:
"""Generate an execution plan from parsed identities.
Creates file operations for organizing video files based on their parsed
identities and configuration templates. Handles movies, series, anime,
and other categories according to v1 constraints.
Args:
identities: List of tuples containing (VideoFile, parsed_identity)
config: Configuration with templates and settings
Returns:
ExecutionPlan with all file operations and summary
"""
operations = []
for video_file, identity in identities:
operation = _create_operation(video_file, identity, config)
operations.append(operation)
# Generate summary counts
summary = _generate_summary(operations)
return ExecutionPlan(
plan_id=str(uuid.uuid4()),
created_at=datetime.now(),
operations=operations,
summary=summary
)
def _create_operation(
video_file: VideoFile,
identity: Union[MovieIdentity, SeriesIdentity, None],
config: Config
) -> FileOperation:
"""Create a file operation for a single video file.
Args:
video_file: The video file to create an operation for
identity: Parsed identity (MovieIdentity, SeriesIdentity, or None)
config: Configuration with templates
Returns:
FileOperation specifying what to do with the file
"""
# Handle anime category - generate no-op (v1 constraint)
if video_file.category == "anime":
return FileOperation(
operation_type="no-op",
source_path=video_file.path,
destination_path=None,
reason="Anime files not organized in v1",
has_conflict=False,
conflict_reason=None
)
# Handle other category - generate no-op (v1 constraint)
if video_file.category == "other":
return FileOperation(
operation_type="no-op",
source_path=video_file.path,
destination_path=None,
reason="Other files not organized in v1",
has_conflict=False,
conflict_reason=None
)
# Handle files without identity - generate no-op
if identity is None:
return FileOperation(
operation_type="no-op",
source_path=video_file.path,
destination_path=None,
reason="No identity parsed",
has_conflict=False,
conflict_reason=None
)
# Handle movie identity
if isinstance(identity, MovieIdentity):
return _create_movie_operation(video_file, identity, config)
# Handle series identity
if isinstance(identity, SeriesIdentity):
return _create_series_operation(video_file, identity, config)
# Fallback - should not reach here
return FileOperation(
operation_type="no-op",
source_path=video_file.path,
destination_path=None,
reason="Unknown identity type",
has_conflict=False,
conflict_reason=None
)
def _create_movie_operation(
video_file: VideoFile,
identity: MovieIdentity,
config: Config
) -> FileOperation:
"""Create operation for a movie file.
Args:
video_file: The movie file
identity: Parsed movie identity
config: Configuration with templates
Returns:
FileOperation for organizing the movie
"""
# If movie needs review (no year), generate no-op
if identity.needs_review or identity.year is None:
return FileOperation(
operation_type="no-op",
source_path=video_file.path,
destination_path=None,
reason="Movie needs manual review (no year found)",
has_conflict=False,
conflict_reason=None
)
# Apply movie directory template
target_dir = config.movie_template.format(
title=identity.title,
year=identity.year
)
# Get file extension
ext = video_file.path.suffix
# Apply movie filename template
target_filename = config.movie_filename_template.format(
title=identity.title,
year=identity.year,
ext=ext
)
# Construct full destination path
destination = config.library_root / target_dir / target_filename
# Check if source and destination are the same
if video_file.path.resolve() == destination.resolve():
return FileOperation(
operation_type="no-op",
source_path=video_file.path,
destination_path=None,
reason="File already at target location",
has_conflict=False,
conflict_reason=None
)
# Determine operation type (move or rename)
if video_file.path.parent == destination.parent:
operation_type = "rename"
else:
operation_type = "move"
# Check for conflicts - destination file already exists
has_conflict = destination.exists()
conflict_reason = None
if has_conflict:
conflict_reason = f"Destination file already exists: {destination}"
return FileOperation(
operation_type=operation_type,
source_path=video_file.path,
destination_path=destination,
reason=f"Organize movie: {identity.title} ({identity.year})",
has_conflict=has_conflict,
conflict_reason=conflict_reason
)
def _create_series_operation(
video_file: VideoFile,
identity: SeriesIdentity,
config: Config
) -> FileOperation:
"""Create operation for a series file.
Args:
video_file: The series file
identity: Parsed series identity
config: Configuration with templates
Returns:
FileOperation for organizing the series episode
"""
# If series needs review (no season or no episodes), generate no-op (v1 constraint)
if identity.needs_review or identity.season is None or len(identity.episodes) == 0:
return FileOperation(
operation_type="no-op",
source_path=video_file.path,
destination_path=None,
reason="Series needs manual review (no season/episode found)",
has_conflict=False,
conflict_reason=None
)
# Apply series directory template
target_dir = config.series_template.format(
title=identity.title,
season=identity.season
)
# Get file extension
ext = video_file.path.suffix
# Apply series filename template
# For multi-episode files, use the first episode number
target_filename = config.series_filename_template.format(
season=identity.season,
episode=identity.episodes[0],
ext=ext
)
# Construct full destination path
destination = config.library_root / target_dir / target_filename
# Check if source and destination are the same
if video_file.path.resolve() == destination.resolve():
return FileOperation(
operation_type="no-op",
source_path=video_file.path,
destination_path=None,
reason="File already at target location",
has_conflict=False,
conflict_reason=None
)
# Determine operation type (move or rename)
if video_file.path.parent == destination.parent:
operation_type = "rename"
else:
operation_type = "move"
# Check for conflicts - destination file already exists
has_conflict = destination.exists()
conflict_reason = None
if has_conflict:
conflict_reason = f"Destination file already exists: {destination}"
return FileOperation(
operation_type=operation_type,
source_path=video_file.path,
destination_path=destination,
reason=f"Organize series: {identity.title} S{identity.season:02d}E{identity.episodes[0]:02d}",
has_conflict=has_conflict,
conflict_reason=conflict_reason
)
def _generate_summary(operations: list[FileOperation]) -> dict:
"""Generate summary statistics for operations.
Args:
operations: List of file operations
Returns:
Dictionary with operation counts by type
"""
summary = {
"total": len(operations),
"move": 0,
"rename": 0,
"quarantine": 0,
"no-op": 0
}
for operation in operations:
op_type = operation.operation_type
if op_type in summary:
summary[op_type] += 1
return summary
def save_plan(plan: ExecutionPlan, output_path: Path) -> None:
"""Save execution plan to JSON file.
Serializes the execution plan to a human-readable and editable JSON format.
Includes plan_id, created_at timestamp, operations list, and summary.
Args:
plan: ExecutionPlan to save
output_path: Path where the JSON file should be saved
"""
# Convert ExecutionPlan to dictionary
plan_dict = {
"plan_id": plan.plan_id,
"created_at": plan.created_at.isoformat(),
"operations": [
{
"operation_type": op.operation_type,
"source_path": str(op.source_path),
"destination_path": str(op.destination_path) if op.destination_path else None,
"reason": op.reason,
"has_conflict": op.has_conflict,
"conflict_reason": op.conflict_reason
}
for op in plan.operations
],
"summary": plan.summary
}
# Write to JSON file with indentation for human readability
with open(output_path, 'w', encoding='utf-8') as f:
json.dump(plan_dict, f, indent=2, ensure_ascii=False)
def load_plan(input_path: Path) -> ExecutionPlan:
"""Load execution plan from JSON file.
Deserializes an execution plan from JSON format, reconstructing all
data structures including Path and datetime objects.
Args:
input_path: Path to the JSON file to load
Returns:
ExecutionPlan reconstructed from JSON
Raises:
FileNotFoundError: If the input file does not exist
json.JSONDecodeError: If the file contains invalid JSON
KeyError: If required fields are missing from the JSON
"""
with open(input_path, 'r', encoding='utf-8') as f:
plan_dict = json.load(f)
# Reconstruct FileOperation objects
operations = [
FileOperation(
operation_type=op["operation_type"],
source_path=Path(op["source_path"]),
destination_path=Path(op["destination_path"]) if op["destination_path"] else None,
reason=op["reason"],
has_conflict=op["has_conflict"],
conflict_reason=op.get("conflict_reason")
)
for op in plan_dict["operations"]
]
# Reconstruct ExecutionPlan
return ExecutionPlan(
plan_id=plan_dict["plan_id"],
created_at=datetime.fromisoformat(plan_dict["created_at"]),
operations=operations,
summary=plan_dict["summary"]
)
src/vlm/quarantine.py
+759
View File
@@ -0,0 +1,759 @@
"""Quarantine manager for Video Library Manager.
This module provides functionality to safely isolate unwanted files in
category-specific quarantine directories for review before deletion.
v1 Constraints:
- Only movie and series categories support quarantine
- Anime and other categories: quarantine operations rejected with error
- Each category has its own .quarantine/ subdirectory and manifest.json
"""
import json
import logging
from datetime import datetime
from pathlib import Path
from typing import Optional
from .config import Config
from .logging_config import get_logger, log_operation
from .models import QuarantineEntry, QuarantineManifest, OperationResult, FileOperation
class QuarantineManager:
"""Manager for quarantine operations on video files."""
def __init__(self, config: Config, logger: Optional[logging.Logger] = None):
"""Initialize the quarantine manager.
Args:
config: Configuration object with library settings
logger: Optional logger instance (uses default if not provided)
"""
self.config = config
self.logger = logger or get_logger()
def quarantine_file(
self,
file_path: Path,
reason: Optional[str] = None
) -> OperationResult:
"""Move a file to category-specific quarantine directory.
This method:
1. Verifies file is in movie or series category (rejects anime/other)
2. Determines relative path from category root
3. Constructs quarantine path: <category_root>/.quarantine/<relative_path>
4. Handles destination conflicts by appending numeric suffix
5. Moves file to quarantine directory
6. Updates category-specific manifest.json
Args:
file_path: Path to the file to quarantine
reason: Optional reason for quarantining the file
Returns:
OperationResult indicating success or failure
Raises:
ValueError: If file is in anime or other category (not supported in v1)
"""
executed_at = datetime.now()
# Verify file exists
if not file_path.exists():
error_msg = f"File does not exist: {file_path}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="quarantine",
file_path=file_path
)
return OperationResult(
operation=FileOperation(
operation_type="quarantine",
source_path=file_path,
destination_path=None,
reason=reason or "File not found",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
# Get file size before moving
try:
file_size = file_path.stat().st_size
except Exception as e:
error_msg = f"Failed to get file size: {str(e)}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="quarantine",
file_path=file_path
)
return OperationResult(
operation=FileOperation(
operation_type="quarantine",
source_path=file_path,
destination_path=None,
reason=reason or "Error getting file size",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
# Determine category from file path
category = self._determine_category(file_path)
# Reject anime and other categories (v1 constraint)
if category not in ("movie", "series"):
error_msg = (
f"Quarantine not supported for category '{category}'. "
f"Only 'movie' and 'series' categories are supported in v1."
)
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="quarantine",
file_path=file_path
)
raise ValueError(error_msg)
# Get category root directory
category_root = self.config.library_root / category
# Determine relative path from category root
try:
relative_path = file_path.relative_to(category_root)
except ValueError:
error_msg = f"File is not within category root {category_root}: {file_path}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="quarantine",
file_path=file_path
)
return OperationResult(
operation=FileOperation(
operation_type="quarantine",
source_path=file_path,
destination_path=None,
reason=reason or "Invalid path",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
# Construct quarantine path
quarantine_root = category_root / self.config.quarantine_dir
quarantine_path = quarantine_root / relative_path
# Handle destination conflicts by appending numeric suffix
quarantine_path = self._resolve_conflict(quarantine_path)
# Create quarantine directory structure
try:
quarantine_path.parent.mkdir(parents=True, exist_ok=True)
except Exception as e:
error_msg = f"Failed to create quarantine directory: {str(e)}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="quarantine",
file_path=file_path
)
return OperationResult(
operation=FileOperation(
operation_type="quarantine",
source_path=file_path,
destination_path=quarantine_path,
reason=reason or "Quarantine",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
# Move file to quarantine
try:
file_path.rename(quarantine_path)
log_operation(
self.logger,
logging.INFO,
f"Successfully quarantined file: {file_path} -> {quarantine_path}",
operation_type="quarantine",
file_path=file_path
)
# Update manifest
try:
self._update_manifest(
category=category,
original_path=file_path,
quarantine_path=quarantine_path,
quarantined_at=executed_at,
reason=reason,
size_bytes=file_size
)
except Exception as e:
# Log manifest update failure but don't fail the operation
# since the file was already moved successfully
log_operation(
self.logger,
logging.WARNING,
f"Failed to update manifest: {str(e)}",
operation_type="quarantine",
file_path=file_path
)
return OperationResult(
operation=FileOperation(
operation_type="quarantine",
source_path=file_path,
destination_path=quarantine_path,
reason=reason or "Quarantine",
has_conflict=False
),
success=True,
error_message=None,
executed_at=executed_at
)
except Exception as e:
error_msg = f"Failed to move file to quarantine: {str(e)}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="quarantine",
file_path=file_path
)
return OperationResult(
operation=FileOperation(
operation_type="quarantine",
source_path=file_path,
destination_path=quarantine_path,
reason=reason or "Quarantine",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
def _determine_category(self, file_path: Path) -> str:
"""Determine the category of a file based on its path.
Args:
file_path: Path to the file
Returns:
Category name ("movie", "series", "anime", "other")
"""
# Get path relative to library root
try:
relative_path = file_path.relative_to(self.config.library_root)
except ValueError:
return "other"
# First component of relative path is the category
parts = relative_path.parts
if not parts:
return "other"
category = parts[0].lower()
# Validate category
if category in ("movie", "series", "anime", "other"):
return category
else:
return "other"
def _resolve_conflict(self, quarantine_path: Path) -> Path:
"""Resolve destination conflicts by appending numeric suffix.
If the quarantine destination already exists, append _1, _2, etc.
until a non-existent path is found.
Args:
quarantine_path: Proposed quarantine path
Returns:
Resolved quarantine path that doesn't exist
"""
if not quarantine_path.exists():
return quarantine_path
# Extract stem and suffix
stem = quarantine_path.stem
suffix = quarantine_path.suffix
parent = quarantine_path.parent
# Try appending numeric suffixes
counter = 1
while True:
new_path = parent / f"{stem}_{counter}{suffix}"
if not new_path.exists():
log_operation(
self.logger,
logging.INFO,
f"Resolved quarantine conflict: {quarantine_path} -> {new_path}",
operation_type="quarantine"
)
return new_path
counter += 1
# Safety check to prevent infinite loop
if counter > 1000:
raise RuntimeError(
f"Could not resolve quarantine conflict after 1000 attempts: {quarantine_path}"
)
def _get_manifest_path(self, category: str) -> Path:
"""Get the path to the manifest file for a category.
Args:
category: Category name ("movie" or "series")
Returns:
Path to the manifest.json file
"""
category_root = self.config.library_root / category
quarantine_root = category_root / self.config.quarantine_dir
return quarantine_root / "manifest.json"
def _load_manifest(self, category: str) -> QuarantineManifest:
"""Load the quarantine manifest for a category.
Args:
category: Category name ("movie" or "series")
Returns:
QuarantineManifest object (empty if manifest doesn't exist)
"""
manifest_path = self._get_manifest_path(category)
if not manifest_path.exists():
return QuarantineManifest(entries=[])
try:
with open(manifest_path, 'r', encoding='utf-8') as f:
data = json.load(f)
# Parse entries
entries = []
for entry_data in data.get('entries', []):
entry = QuarantineEntry(
original_path=Path(entry_data['original_path']),
quarantine_path=Path(entry_data['quarantine_path']),
quarantined_at=datetime.fromisoformat(entry_data['quarantined_at']),
reason=entry_data.get('reason'),
size_bytes=entry_data['size_bytes'],
category=entry_data['category']
)
entries.append(entry)
return QuarantineManifest(entries=entries)
except Exception as e:
log_operation(
self.logger,
logging.WARNING,
f"Failed to load manifest for category '{category}': {str(e)}. Using empty manifest.",
operation_type="quarantine"
)
return QuarantineManifest(entries=[])
def _save_manifest(self, category: str, manifest: QuarantineManifest) -> None:
"""Save the quarantine manifest for a category.
Args:
category: Category name ("movie" or "series")
manifest: QuarantineManifest object to save
Raises:
Exception: If manifest cannot be saved
"""
manifest_path = self._get_manifest_path(category)
# Ensure quarantine directory exists
manifest_path.parent.mkdir(parents=True, exist_ok=True)
# Convert manifest to JSON-serializable format
data = {
'entries': [
{
'original_path': str(entry.original_path),
'quarantine_path': str(entry.quarantine_path),
'quarantined_at': entry.quarantined_at.isoformat(),
'reason': entry.reason,
'size_bytes': entry.size_bytes,
'category': entry.category
}
for entry in manifest.entries
]
}
# Save to file
with open(manifest_path, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2, ensure_ascii=False)
log_operation(
self.logger,
logging.DEBUG,
f"Saved manifest for category '{category}' with {len(manifest.entries)} entries",
operation_type="quarantine"
)
def _update_manifest(
self,
category: str,
original_path: Path,
quarantine_path: Path,
quarantined_at: datetime,
reason: Optional[str],
size_bytes: int
) -> None:
"""Update the quarantine manifest with a new entry.
Args:
category: Category name ("movie" or "series")
original_path: Original path of the file before quarantine
quarantine_path: Path to the file in quarantine
quarantined_at: Timestamp when the file was quarantined
reason: Optional reason for quarantining
size_bytes: File size in bytes
Raises:
Exception: If manifest cannot be updated
"""
# Load existing manifest
manifest = self._load_manifest(category)
# Create new entry
entry = QuarantineEntry(
original_path=original_path,
quarantine_path=quarantine_path,
quarantined_at=quarantined_at,
reason=reason,
size_bytes=size_bytes,
category=category
)
# Add entry to manifest
manifest.entries.append(entry)
# Save updated manifest
self._save_manifest(category, manifest)
log_operation(
self.logger,
logging.INFO,
f"Updated manifest for category '{category}': added entry for {original_path}",
operation_type="quarantine"
)
def list_quarantined(self, category: Optional[str] = None) -> list[QuarantineEntry]:
"""List quarantined files from category manifests.
Args:
category: Optional category filter ("movie" or "series").
If None, lists from all categories.
Returns:
List of QuarantineEntry objects
"""
entries = []
# Determine which categories to query
if category is not None:
# Validate category
if category not in ("movie", "series"):
log_operation(
self.logger,
logging.WARNING,
f"Invalid category '{category}' for listing. Only 'movie' and 'series' are supported.",
operation_type="quarantine"
)
return []
categories = [category]
else:
# List from all supported categories
categories = ["movie", "series"]
# Load manifests from each category
for cat in categories:
manifest = self._load_manifest(cat)
entries.extend(manifest.entries)
log_operation(
self.logger,
logging.INFO,
f"Listed {len(entries)} quarantined files" +
(f" from category '{category}'" if category else " from all categories"),
operation_type="quarantine"
)
return entries
def restore_from_quarantine(
self,
quarantine_path: Path
) -> OperationResult:
"""Restore a file from quarantine to its original location.
This is a best-effort operation. It attempts to:
1. Find the entry in the appropriate category manifest
2. Move the file from quarantine back to original location
3. Remove the entry from the manifest
Args:
quarantine_path: Path to the file in quarantine
Returns:
OperationResult indicating success or failure
"""
executed_at = datetime.now()
# Verify quarantine file exists
if not quarantine_path.exists():
error_msg = f"Quarantine file does not exist: {quarantine_path}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="restore",
file_path=quarantine_path
)
return OperationResult(
operation=FileOperation(
operation_type="restore",
source_path=quarantine_path,
destination_path=None,
reason="File not found",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
# Determine category from quarantine path
category = self._determine_category_from_quarantine(quarantine_path)
if category is None:
error_msg = f"Could not determine category for quarantine file: {quarantine_path}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="restore",
file_path=quarantine_path
)
return OperationResult(
operation=FileOperation(
operation_type="restore",
source_path=quarantine_path,
destination_path=None,
reason="Invalid quarantine path",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
# Load manifest to find original path
manifest = self._load_manifest(category)
# Find entry matching quarantine path
entry = None
for e in manifest.entries:
if e.quarantine_path == quarantine_path:
entry = e
break
if entry is None:
error_msg = f"No manifest entry found for quarantine file: {quarantine_path}"
log_operation(
self.logger,
logging.WARNING,
error_msg,
operation_type="restore",
file_path=quarantine_path
)
return OperationResult(
operation=FileOperation(
operation_type="restore",
source_path=quarantine_path,
destination_path=None,
reason="No manifest entry",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
original_path = entry.original_path
# Check if original location already has a file (conflict)
if original_path.exists():
error_msg = f"Cannot restore: original location already exists: {original_path}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="restore",
file_path=quarantine_path
)
return OperationResult(
operation=FileOperation(
operation_type="restore",
source_path=quarantine_path,
destination_path=original_path,
reason="Restore",
has_conflict=True,
conflict_reason="Destination already exists"
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
# Create parent directory if needed
try:
original_path.parent.mkdir(parents=True, exist_ok=True)
except Exception as e:
error_msg = f"Failed to create parent directory: {str(e)}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="restore",
file_path=quarantine_path
)
return OperationResult(
operation=FileOperation(
operation_type="restore",
source_path=quarantine_path,
destination_path=original_path,
reason="Restore",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
# Move file back to original location
try:
quarantine_path.rename(original_path)
log_operation(
self.logger,
logging.INFO,
f"Successfully restored file: {quarantine_path} -> {original_path}",
operation_type="restore",
file_path=quarantine_path
)
# Remove entry from manifest
try:
manifest.entries.remove(entry)
self._save_manifest(category, manifest)
log_operation(
self.logger,
logging.INFO,
f"Removed entry from manifest for category '{category}'",
operation_type="restore"
)
except Exception as e:
# Log manifest update failure but don't fail the operation
# since the file was already moved successfully
log_operation(
self.logger,
logging.WARNING,
f"Failed to update manifest after restore: {str(e)}",
operation_type="restore",
file_path=quarantine_path
)
return OperationResult(
operation=FileOperation(
operation_type="restore",
source_path=quarantine_path,
destination_path=original_path,
reason="Restore",
has_conflict=False
),
success=True,
error_message=None,
executed_at=executed_at
)
except Exception as e:
error_msg = f"Failed to restore file: {str(e)}"
log_operation(
self.logger,
logging.ERROR,
error_msg,
operation_type="restore",
file_path=quarantine_path
)
return OperationResult(
operation=FileOperation(
operation_type="restore",
source_path=quarantine_path,
destination_path=original_path,
reason="Restore",
has_conflict=False
),
success=False,
error_message=error_msg,
executed_at=executed_at
)
def _determine_category_from_quarantine(self, quarantine_path: Path) -> Optional[str]:
"""Determine the category from a quarantine path.
Args:
quarantine_path: Path to a file in quarantine
Returns:
Category name ("movie" or "series") or None if cannot be determined
"""
try:
relative_path = quarantine_path.relative_to(self.config.library_root)
except ValueError:
return None
parts = relative_path.parts
if len(parts) < 2:
return None
# First part should be category, second should be .quarantine
category = parts[0].lower()
quarantine_dir = parts[1]
if quarantine_dir != self.config.quarantine_dir:
return None
if category in ("movie", "series"):
return category
return None
src/vlm/reports.py
+532
View File
@@ -0,0 +1,532 @@
"""Report generation for Video Library Manager.
This module provides functionality to generate various reports about the video library:
- Inventory reports (all discovered files with metadata)
- Completeness reports (series with episode gaps)
- Duplicate reports (duplicate files with quality comparisons)
- Summary reports (library statistics)
"""
import csv
import json
import logging
from datetime import datetime, timezone
from io import StringIO
from pathlib import Path
from vlm.models import SeasonCompleteness, DuplicateGroup, VideoFile, MovieIdentity, SeriesIdentity
logger = logging.getLogger(__name__)
def generate_inventory_report(
files: list[VideoFile],
format: str,
library_root: Path
) -> str:
"""Generate inventory report listing all discovered video files with metadata.
Args:
files: List of VideoFile objects to include in the report
format: Output format ("csv" or "json")
library_root: Root of the library (included in report metadata)
Returns:
Formatted report as string
Raises:
ValueError: If format is not "csv" or "json"
"""
if format not in ["csv", "json"]:
raise ValueError(f"Invalid format: {format}. Must be 'csv' or 'json'")
generation_timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
if format == "json":
return _generate_inventory_json(files, generation_timestamp, library_root)
else: # csv
return _generate_inventory_csv(files, generation_timestamp, library_root)
def _generate_inventory_csv(
files: list[VideoFile],
timestamp: str,
library_root: Path
) -> str:
"""Generate inventory report in CSV format.
CSV Schema:
- path, filename, size_bytes, modified_timestamp, category, resolution,
codec, duration_seconds, bitrate_kbps
- Timestamps in ISO 8601 format (YYYY-MM-DDTHH:MM:SS) in UTC
- Missing optional values represented as empty strings
- Header row always present
"""
output = StringIO()
# Write metadata as comments
output.write(f"# Generated: {timestamp}\n")
output.write(f"# Library Root: {library_root}\n")
# Define CSV schema
fieldnames = [
'path',
'filename',
'size_bytes',
'modified_timestamp',
'category',
'resolution',
'codec',
'duration_seconds',
'bitrate_kbps'
]
writer = csv.DictWriter(output, fieldnames=fieldnames)
writer.writeheader()
# Write each file
for video_file in files:
# Format timestamp as ISO 8601 in UTC
if video_file.modified_timestamp.tzinfo is None:
# Assume local time, convert to UTC
modified_utc = video_file.modified_timestamp.replace(tzinfo=timezone.utc)
else:
modified_utc = video_file.modified_timestamp.astimezone(timezone.utc)
row = {
'path': str(video_file.path),
'filename': video_file.filename,
'size_bytes': video_file.size_bytes,
'modified_timestamp': modified_utc.strftime("%Y-%m-%dT%H:%M:%S"),
'category': video_file.category,
'resolution': video_file.resolution or '',
'codec': video_file.codec or '',
'duration_seconds': video_file.duration_seconds if video_file.duration_seconds is not None else '',
'bitrate_kbps': video_file.bitrate_kbps if video_file.bitrate_kbps is not None else ''
}
writer.writerow(row)
return output.getvalue()
def _generate_inventory_json(
files: list[VideoFile],
timestamp: str,
library_root: Path
) -> str:
"""Generate inventory report in JSON format."""
inventory_data = {
'metadata': {
'generated': timestamp,
'library_root': str(library_root),
'file_count': len(files)
},
'files': []
}
# Add each file
for video_file in files:
# Format timestamp as ISO 8601 in UTC
if video_file.modified_timestamp.tzinfo is None:
# Assume local time, convert to UTC
modified_utc = video_file.modified_timestamp.replace(tzinfo=timezone.utc)
else:
modified_utc = video_file.modified_timestamp.astimezone(timezone.utc)
file_data = {
'path': str(video_file.path),
'filename': video_file.filename,
'size_bytes': video_file.size_bytes,
'modified_timestamp': modified_utc.strftime("%Y-%m-%dT%H:%M:%S"),
'category': video_file.category,
'resolution': video_file.resolution,
'codec': video_file.codec,
'duration_seconds': video_file.duration_seconds,
'bitrate_kbps': video_file.bitrate_kbps
}
inventory_data['files'].append(file_data)
return json.dumps(inventory_data, indent=2, ensure_ascii=False)
def generate_completeness_report(
analysis: list[SeasonCompleteness],
format: str,
library_root: Path
) -> str:
"""Generate completeness report showing series with episode gaps.
Args:
analysis: List of SeasonCompleteness objects with detected gaps
format: Output format ("text" or "json")
library_root: Root of the library (included in report metadata)
Returns:
Formatted report as string
Raises:
ValueError: If format is not "text" or "json"
"""
if format not in ["text", "json"]:
raise ValueError(f"Invalid format: {format}. Must be 'text' or 'json'")
generation_timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
if format == "json":
return _generate_completeness_json(analysis, generation_timestamp, library_root)
else: # text
return _generate_completeness_text(analysis, generation_timestamp, library_root)
def _generate_completeness_text(
analysis: list[SeasonCompleteness],
timestamp: str,
library_root: Path
) -> str:
"""Generate completeness report in text format."""
lines = []
lines.append("=" * 80)
lines.append("SERIES COMPLETENESS REPORT")
lines.append("=" * 80)
lines.append(f"Generated: {timestamp}")
lines.append(f"Library Root: {library_root}")
lines.append(f"Series with gaps: {len(analysis)}")
lines.append("")
if not analysis:
lines.append("No series with episode gaps detected.")
return "\n".join(lines)
# Group by series title
series_groups = {}
for season_data in analysis:
if season_data.series_title not in series_groups:
series_groups[season_data.series_title] = []
series_groups[season_data.series_title].append(season_data)
# Sort series alphabetically
for series_title in sorted(series_groups.keys()):
lines.append("-" * 80)
lines.append(f"Series: {series_title}")
lines.append("-" * 80)
# Sort seasons by season number
seasons = sorted(series_groups[series_title], key=lambda x: x.season)
for season_data in seasons:
lines.append(f" Season {season_data.season:02d}:")
lines.append(f" Episodes found: {_format_episode_list(season_data.episodes_found)}")
lines.append(f" Episodes missing: {_format_episode_list(season_data.episodes_missing)}")
lines.append("")
return "\n".join(lines)
def _generate_completeness_json(
analysis: list[SeasonCompleteness],
timestamp: str,
library_root: Path
) -> str:
"""Generate completeness report in JSON format."""
report_data = {
"metadata": {
"generated": timestamp,
"library_root": str(library_root),
"series_count": len(set(s.series_title for s in analysis))
},
"series": []
}
# Group by series title
series_groups = {}
for season_data in analysis:
if season_data.series_title not in series_groups:
series_groups[season_data.series_title] = []
series_groups[season_data.series_title].append(season_data)
# Build series data
for series_title in sorted(series_groups.keys()):
seasons_data = []
for season_data in sorted(series_groups[series_title], key=lambda x: x.season):
seasons_data.append({
"season": season_data.season,
"episodes_found": season_data.episodes_found,
"episodes_missing": season_data.episodes_missing
})
report_data["series"].append({
"title": series_title,
"seasons": seasons_data
})
return json.dumps(report_data, indent=2, ensure_ascii=False)
def generate_duplicate_report(
duplicates: list[DuplicateGroup],
format: str,
library_root: Path
) -> str:
"""Generate duplicate report showing duplicate files with quality comparisons.
Args:
duplicates: List of DuplicateGroup objects with duplicate files
format: Output format ("text" or "json")
library_root: Root of the library (included in report metadata)
Returns:
Formatted report as string
Raises:
ValueError: If format is not "text" or "json"
"""
if format not in ["text", "json"]:
raise ValueError(f"Invalid format: {format}. Must be 'text' or 'json'")
generation_timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
if format == "json":
return _generate_duplicate_json(duplicates, generation_timestamp, library_root)
else: # text
return _generate_duplicate_text(duplicates, generation_timestamp, library_root)
def _generate_duplicate_text(
duplicates: list[DuplicateGroup],
timestamp: str,
library_root: Path
) -> str:
"""Generate duplicate report in text format."""
lines = []
lines.append("=" * 80)
lines.append("DUPLICATE FILES REPORT")
lines.append("=" * 80)
lines.append(f"Generated: {timestamp}")
lines.append(f"Library Root: {library_root}")
lines.append(f"Duplicate groups: {len(duplicates)}")
lines.append("")
if not duplicates:
lines.append("No duplicate files detected.")
return "\n".join(lines)
# Sort by largest file size first
sorted_duplicates = sorted(
duplicates,
key=lambda g: max(f.size_bytes for f in g.files),
reverse=True
)
for idx, group in enumerate(sorted_duplicates, 1):
lines.append("-" * 80)
# Format identity
identity = group.identity
if isinstance(identity, MovieIdentity):
lines.append(f"Group {idx}: {identity.title} ({identity.year})")
else: # SeriesIdentity
episodes_str = ", ".join(str(e) for e in identity.episodes)
lines.append(f"Group {idx}: {identity.title} - S{identity.season:02d}E{episodes_str}")
lines.append("-" * 80)
lines.append(f" Files: {len(group.files)}")
lines.append("")
# Show quality comparison for each file
for file_idx, quality_data in enumerate(group.quality_comparison, 1):
lines.append(f" File {file_idx}:")
lines.append(f" Filename: {quality_data['filename']}")
lines.append(f" Path: {quality_data['path']}")
lines.append(f" Size: {_format_size(quality_data['size_bytes'])}")
if 'resolution' in quality_data:
lines.append(f" Resolution: {quality_data['resolution']}")
if 'codec' in quality_data:
lines.append(f" Codec: {quality_data['codec']}")
if 'duration_seconds' in quality_data:
lines.append(f" Duration: {_format_duration(quality_data['duration_seconds'])}")
if 'bitrate_kbps' in quality_data:
lines.append(f" Bitrate: {quality_data['bitrate_kbps']} kbps")
lines.append("")
return "\n".join(lines)
def _generate_duplicate_json(
duplicates: list[DuplicateGroup],
timestamp: str,
library_root: Path
) -> str:
"""Generate duplicate report in JSON format."""
report_data = {
"metadata": {
"generated": timestamp,
"library_root": str(library_root),
"duplicate_groups": len(duplicates)
},
"duplicates": []
}
for group in duplicates:
identity = group.identity
# Format identity
if isinstance(identity, MovieIdentity):
identity_data = {
"type": "movie",
"title": identity.title,
"year": identity.year
}
else: # SeriesIdentity
identity_data = {
"type": "series",
"title": identity.title,
"season": identity.season,
"episodes": identity.episodes
}
group_data = {
"identity": identity_data,
"file_count": len(group.files),
"files": group.quality_comparison
}
report_data["duplicates"].append(group_data)
return json.dumps(report_data, indent=2, ensure_ascii=False)
def generate_summary_report(
files: list[VideoFile],
library_root: Path
) -> str:
"""Generate summary report with library statistics.
Args:
files: List of all VideoFile objects in the library
library_root: Root of the library (included in report metadata)
Returns:
Formatted summary report as text string
"""
generation_timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
lines = []
lines.append("=" * 80)
lines.append("LIBRARY SUMMARY REPORT")
lines.append("=" * 80)
lines.append(f"Generated: {generation_timestamp}")
lines.append(f"Library Root: {library_root}")
lines.append("")
# Calculate total statistics
total_files = len(files)
total_size = sum(f.size_bytes for f in files)
lines.append(f"Total Files: {total_files}")
lines.append(f"Total Size: {_format_size(total_size)}")
lines.append("")
# Category breakdown
lines.append("Category Breakdown:")
lines.append("-" * 40)
category_stats = {}
for file in files:
category = file.category
if category not in category_stats:
category_stats[category] = {"count": 0, "size": 0}
category_stats[category]["count"] += 1
category_stats[category]["size"] += file.size_bytes
# Sort categories alphabetically
for category in sorted(category_stats.keys()):
stats = category_stats[category]
lines.append(f" {category.capitalize()}:")
lines.append(f" Files: {stats['count']}")
lines.append(f" Size: {_format_size(stats['size'])}")
lines.append("")
return "\n".join(lines)
def _format_episode_list(episodes: list[int]) -> str:
"""Format episode list as compact string with ranges.
Examples:
[1, 2, 3, 5, 6, 8] -> "1-3, 5-6, 8"
[1, 3, 5] -> "1, 3, 5"
"""
if not episodes:
return "none"
# Sort episodes
sorted_episodes = sorted(episodes)
# Build ranges
ranges = []
start = sorted_episodes[0]
end = sorted_episodes[0]
for episode in sorted_episodes[1:]:
if episode == end + 1:
# Continue current range
end = episode
else:
# End current range and start new one
if start == end:
ranges.append(str(start))
else:
ranges.append(f"{start}-{end}")
start = episode
end = episode
# Add final range
if start == end:
ranges.append(str(start))
else:
ranges.append(f"{start}-{end}")
return ", ".join(ranges)
def _format_size(size_bytes: int) -> str:
"""Format file size in human-readable format.
Examples:
1024 -> "1.00 KB"
1048576 -> "1.00 MB"
1073741824 -> "1.00 GB"
"""
for unit in ['B', 'KB', 'MB', 'GB', 'TB']:
if size_bytes < 1024.0:
return f"{size_bytes:.2f} {unit}"
size_bytes /= 1024.0
return f"{size_bytes:.2f} PB"
def _format_duration(duration_seconds: float) -> str:
"""Format duration in human-readable format.
Examples:
90 -> "1m 30s"
3665 -> "1h 1m 5s"
"""
hours = int(duration_seconds // 3600)
minutes = int((duration_seconds % 3600) // 60)
seconds = int(duration_seconds % 60)
parts = []
if hours > 0:
parts.append(f"{hours}h")
if minutes > 0:
parts.append(f"{minutes}m")
if seconds > 0 or not parts:
parts.append(f"{seconds}s")
return " ".join(parts)
src/vlm/scanner.py
+460
View File
@@ -0,0 +1,460 @@
"""Inventory scanner for discovering and cataloging video files.
This module implements the core scanning functionality for the Video Library Manager,
including recursive directory traversal, file filtering, metadata extraction, and
categorization based on directory structure.
"""
import csv
import json
import logging
import os
import subprocess
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional
from vlm.config import Config
from vlm.models import VideoFile
logger = logging.getLogger(__name__)
def scan_library(root: Path, config: Config) -> list[VideoFile]:
"""Recursively scan library for video files.
Discovers all video files matching configured extensions within the library root,
records their metadata, and categorizes them based on directory structure.
Args:
root: Root directory to scan
config: Configuration object with video extensions and settings
Returns:
List of VideoFile objects representing discovered files
Note:
- Handles inaccessible files gracefully by logging errors and continuing
- Performs read-only operations without modifying any files or directories
- Categorizes files based on parent directory structure (movie/series/anime/other)
"""
logger.info(f"Starting library scan at: {root}")
if not root.exists():
logger.error(f"Library root does not exist: {root}")
return []
if not root.is_dir():
logger.error(f"Library root is not a directory: {root}")
return []
video_files = []
file_count = 0
error_count = 0
# Recursively scan directory tree
for video_file in _scan_directory_recursive(root, config, root):
video_files.append(video_file)
file_count += 1
if file_count % 100 == 0:
logger.debug(f"Scanned {file_count} files so far...")
logger.info(f"Scan complete. Found {file_count} video files")
if error_count > 0:
logger.warning(f"Encountered {error_count} errors during scan (see log for details)")
return video_files
def _scan_directory_recursive(
directory: Path,
config: Config,
library_root: Path
) -> list[VideoFile]:
"""Recursively scan a directory for video files.
Args:
directory: Directory to scan
config: Configuration object
library_root: Root of the library (for categorization)
Yields:
VideoFile objects for each discovered video file
"""
try:
# Use os.scandir for efficient directory traversal
with os.scandir(directory) as entries:
for entry in entries:
try:
# Skip hidden files and directories (starting with .)
if entry.name.startswith('.'):
continue
if entry.is_file(follow_symlinks=False):
# Check if file has video extension
file_path = Path(entry.path)
if _is_video_file(file_path, config.video_extensions):
video_file = _create_video_file(file_path, library_root)
if video_file:
yield video_file
elif entry.is_dir(follow_symlinks=False):
# Recursively scan subdirectory
subdir_path = Path(entry.path)
yield from _scan_directory_recursive(subdir_path, config, library_root)
except (OSError, PermissionError) as e:
# Handle inaccessible files/directories gracefully
logger.error(f"Cannot access {entry.path}: {e}")
continue
except (OSError, PermissionError) as e:
# Handle inaccessible directory
logger.error(f"Cannot access directory {directory}: {e}")
def _is_video_file(file_path: Path, video_extensions: list[str]) -> bool:
"""Check if file has a video extension.
Args:
file_path: Path to file
video_extensions: List of valid video extensions (e.g., [".mp4", ".mkv"])
Returns:
True if file has a video extension, False otherwise
"""
file_extension = file_path.suffix.lower()
return file_extension in [ext.lower() for ext in video_extensions]
def _create_video_file(file_path: Path, library_root: Path) -> Optional[VideoFile]:
"""Create VideoFile object from file path.
Extracts file metadata and categorizes based on directory structure.
Optionally extracts video metadata using ffprobe if available.
Args:
file_path: Path to video file
library_root: Root of the library (for categorization)
Returns:
VideoFile object or None if file cannot be accessed
"""
try:
# Get file stats
stat = file_path.stat()
size_bytes = stat.st_size
modified_timestamp = datetime.fromtimestamp(stat.st_mtime)
# Categorize based on directory structure
category = categorize_file(file_path, library_root)
# Extract video metadata using ffprobe (optional, non-blocking)
video_metadata = extract_metadata(file_path)
# Create VideoFile object with optional metadata
return VideoFile(
path=file_path,
filename=file_path.name,
size_bytes=size_bytes,
modified_timestamp=modified_timestamp,
category=category,
resolution=video_metadata.get('resolution'),
codec=video_metadata.get('codec'),
duration_seconds=video_metadata.get('duration_seconds'),
bitrate_kbps=video_metadata.get('bitrate_kbps')
)
except (OSError, PermissionError) as e:
logger.error(f"Cannot read file metadata for {file_path}: {e}")
return None
def categorize_file(file_path: Path, library_root: Path) -> str:
"""Determine category based on directory structure.
Categories are determined by the top-level directory within the library root:
- movie/ -> "movie"
- series/ -> "series"
- anime/ -> "anime"
- other/ or anything else -> "other"
Args:
file_path: Path to video file
library_root: Root of the library
Returns:
Category string: "movie", "series", "anime", or "other"
"""
try:
# Get relative path from library root
relative_path = file_path.relative_to(library_root)
# Get the first component of the relative path (top-level directory)
parts = relative_path.parts
if len(parts) > 0:
top_level_dir = parts[0].lower()
if top_level_dir == "movie":
return "movie"
elif top_level_dir == "series":
return "series"
elif top_level_dir == "anime":
return "anime"
else:
return "other"
else:
# File is directly in library root
return "other"
except ValueError:
# File is not within library root
logger.warning(f"File {file_path} is not within library root {library_root}")
return "other"
def extract_metadata(file_path: Path) -> dict:
"""Extract video metadata using ffprobe.
Attempts to extract resolution, codec, duration, and bitrate from video file
using ffprobe. If ffprobe is not available or fails, returns empty dict.
This is a non-blocking operation that gracefully handles failures.
Args:
file_path: Path to video file
Returns:
Dictionary with optional keys:
- resolution: str (e.g., "1920x1080")
- codec: str (e.g., "h264")
- duration_seconds: float
- bitrate_kbps: int
Note:
- Returns empty dict if ffprobe is not available
- Returns empty dict if ffprobe fails to extract metadata
- Logs warnings for failures but does not raise exceptions
"""
try:
# Run ffprobe to get video stream information in JSON format
result = subprocess.run(
[
'ffprobe',
'-v', 'quiet', # Suppress ffprobe output
'-print_format', 'json', # Output as JSON
'-show_streams', # Show stream information
'-show_format', # Show format information
str(file_path)
],
capture_output=True,
text=True,
timeout=10 # 10 second timeout to prevent hanging
)
if result.returncode != 0:
logger.debug(f"ffprobe failed for {file_path.name}: {result.stderr}")
return {}
# Parse JSON output
probe_data = json.loads(result.stdout)
# Extract metadata from the first video stream
metadata = {}
# Find the first video stream
video_stream = None
for stream in probe_data.get('streams', []):
if stream.get('codec_type') == 'video':
video_stream = stream
break
if video_stream:
# Extract resolution
width = video_stream.get('width')
height = video_stream.get('height')
if width and height:
metadata['resolution'] = f"{width}x{height}"
# Extract codec
codec_name = video_stream.get('codec_name')
if codec_name:
metadata['codec'] = codec_name
# Extract duration and bitrate from format section
format_info = probe_data.get('format', {})
# Extract duration
duration = format_info.get('duration')
if duration:
try:
metadata['duration_seconds'] = float(duration)
except (ValueError, TypeError):
pass
# Extract bitrate
bitrate = format_info.get('bit_rate')
if bitrate:
try:
# Convert from bits/sec to kbits/sec
metadata['bitrate_kbps'] = int(float(bitrate) / 1000)
except (ValueError, TypeError):
pass
if metadata:
logger.debug(f"Extracted metadata for {file_path.name}: {metadata}")
return metadata
except FileNotFoundError:
# ffprobe not installed or not in PATH
logger.debug("ffprobe not available - skipping metadata extraction")
return {}
except subprocess.TimeoutExpired:
logger.warning(f"ffprobe timeout for {file_path.name} - skipping metadata")
return {}
except json.JSONDecodeError as e:
logger.warning(f"Failed to parse ffprobe output for {file_path.name}: {e}")
return {}
except Exception as e:
# Catch any other unexpected errors
logger.warning(f"Unexpected error extracting metadata for {file_path.name}: {e}")
return {}
def save_inventory_csv(files: list[VideoFile], output: Path, library_root: Path) -> None:
"""Save inventory to CSV format (primary format).
Generates a CSV file with all file metadata following the defined schema:
path, filename, size_bytes, modified_timestamp, category, resolution, codec,
duration_seconds, bitrate_kbps
Args:
files: List of VideoFile objects to export
output: Path to output CSV file
library_root: Root of the library (included in report metadata)
Note:
- Timestamps are formatted as ISO 8601 (YYYY-MM-DDTHH:MM:SS) in UTC
- Missing optional values are represented as empty strings
- Header row is always present with column names
- Generation timestamp and library root are included as comment lines
"""
logger.info(f"Saving inventory to CSV: {output}")
# Ensure output directory exists
output.parent.mkdir(parents=True, exist_ok=True)
# Get generation timestamp in UTC
generation_timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
with open(output, 'w', newline='', encoding='utf-8') as csvfile:
# Write metadata as comments
csvfile.write(f"# Generated: {generation_timestamp}\n")
csvfile.write(f"# Library Root: {library_root}\n")
# Define CSV schema
fieldnames = [
'path',
'filename',
'size_bytes',
'modified_timestamp',
'category',
'resolution',
'codec',
'duration_seconds',
'bitrate_kbps'
]
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
# Write each file
for video_file in files:
# Format timestamp as ISO 8601 in UTC
if video_file.modified_timestamp.tzinfo is None:
# Assume local time, convert to UTC
modified_utc = video_file.modified_timestamp.replace(tzinfo=timezone.utc)
else:
modified_utc = video_file.modified_timestamp.astimezone(timezone.utc)
row = {
'path': str(video_file.path),
'filename': video_file.filename,
'size_bytes': video_file.size_bytes,
'modified_timestamp': modified_utc.strftime("%Y-%m-%dT%H:%M:%S"),
'category': video_file.category,
'resolution': video_file.resolution or '',
'codec': video_file.codec or '',
'duration_seconds': video_file.duration_seconds if video_file.duration_seconds is not None else '',
'bitrate_kbps': video_file.bitrate_kbps if video_file.bitrate_kbps is not None else ''
}
writer.writerow(row)
logger.info(f"Saved {len(files)} files to CSV inventory")
def save_inventory_json(files: list[VideoFile], output: Path, library_root: Path) -> None:
"""Save inventory to JSON format (optional export format).
Generates a JSON file with all file metadata and report metadata.
Args:
files: List of VideoFile objects to export
output: Path to output JSON file
library_root: Root of the library (included in report metadata)
Note:
- Timestamps are formatted as ISO 8601 strings
- Missing optional values are represented as null
- Generation timestamp and library root are included in metadata section
"""
logger.info(f"Saving inventory to JSON: {output}")
# Ensure output directory exists
output.parent.mkdir(parents=True, exist_ok=True)
# Get generation timestamp in UTC
generation_timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
# Build JSON structure
inventory_data = {
'metadata': {
'generated': generation_timestamp,
'library_root': str(library_root),
'file_count': len(files)
},
'files': []
}
# Add each file
for video_file in files:
# Format timestamp as ISO 8601 in UTC
if video_file.modified_timestamp.tzinfo is None:
# Assume local time, convert to UTC
modified_utc = video_file.modified_timestamp.replace(tzinfo=timezone.utc)
else:
modified_utc = video_file.modified_timestamp.astimezone(timezone.utc)
file_data = {
'path': str(video_file.path),
'filename': video_file.filename,
'size_bytes': video_file.size_bytes,
'modified_timestamp': modified_utc.strftime("%Y-%m-%dT%H:%M:%S"),
'category': video_file.category,
'resolution': video_file.resolution,
'codec': video_file.codec,
'duration_seconds': video_file.duration_seconds,
'bitrate_kbps': video_file.bitrate_kbps
}
inventory_data['files'].append(file_data)
# Write JSON file with pretty formatting
with open(output, 'w', encoding='utf-8') as jsonfile:
json.dump(inventory_data, jsonfile, indent=2, ensure_ascii=False)
logger.info(f"Saved {len(files)} files to JSON inventory")
src/vlm/state.py
+177
View File
@@ -0,0 +1,177 @@
"""State Store for Video Library Manager.
This module provides persistent state management for tracking file statuses
and user decisions throughout the workflow.
"""
import json
from datetime import datetime
from pathlib import Path
from typing import Optional
from vlm.models import FileState, StateStore
# Valid status values
VALID_STATUSES = {"reviewed", "ignored", "planned", "executed", "quarantined"}
def load_state(path: Path) -> StateStore:
"""Load state store from JSON file.
Args:
path: Path to the state store JSON file
Returns:
StateStore object with all file states
Raises:
FileNotFoundError: If the state file doesn't exist
json.JSONDecodeError: If the file contains invalid JSON
"""
with open(path, 'r', encoding='utf-8') as f:
data = json.load(f)
# Parse states dictionary
states = {}
for file_path_str, state_data in data.get('states', {}).items():
states[file_path_str] = FileState(
file_path=Path(state_data['file_path']),
status=state_data['status'],
reason=state_data.get('reason'),
updated_at=datetime.fromisoformat(state_data['updated_at'])
)
return StateStore(
states=states,
version=data.get('version', '1.0'),
last_updated=datetime.fromisoformat(data['last_updated'])
)
def save_state(store: StateStore, path: Path) -> None:
"""Save state store to JSON file.
Args:
store: StateStore object to save
path: Path where the state store should be saved
"""
# Create parent directory if it doesn't exist
path.parent.mkdir(parents=True, exist_ok=True)
# Convert states to serializable format
states_data = {}
for file_path_str, state in store.states.items():
states_data[file_path_str] = {
'file_path': str(state.file_path),
'status': state.status,
'reason': state.reason,
'updated_at': state.updated_at.isoformat()
}
data = {
'states': states_data,
'version': store.version,
'last_updated': store.last_updated.isoformat()
}
with open(path, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2, ensure_ascii=False)
class StateManager:
"""Manager for state store operations.
This class provides a convenient interface for managing file states
with automatic persistence.
"""
def __init__(self, state_path: Path):
"""Initialize the state manager.
Args:
state_path: Path to the state store JSON file
"""
self.state_path = state_path
# Load existing state or create new one
if state_path.exists():
self.store = load_state(state_path)
else:
self.store = StateStore(
states={},
version='1.0',
last_updated=datetime.now()
)
def get_file_state(self, file_path: Path) -> Optional[FileState]:
"""Get state for a specific file.
Args:
file_path: Path to the file
Returns:
FileState object if found, None otherwise
"""
file_path_str = str(file_path)
return self.store.states.get(file_path_str)
def set_file_state(self, file_path: Path, status: str, reason: Optional[str] = None) -> None:
"""Set or update state for a file.
This operation is idempotent - setting the same status multiple times
will update the timestamp and reason.
Args:
file_path: Path to the file
status: Status value (must be one of VALID_STATUSES)
reason: Optional reason for the status
Raises:
ValueError: If status is not valid
"""
if status not in VALID_STATUSES:
raise ValueError(
f"Invalid status '{status}'. Must be one of: {', '.join(sorted(VALID_STATUSES))}"
)
file_path_str = str(file_path)
now = datetime.now()
self.store.states[file_path_str] = FileState(
file_path=file_path,
status=status,
reason=reason,
updated_at=now
)
self.store.last_updated = now
def query_by_status(self, status: str) -> list[FileState]:
"""Get all files with a specific status.
Args:
status: Status value to filter by
Returns:
List of FileState objects with the specified status
"""
return [
state for state in self.store.states.values()
if state.status == status
]
def clear_state(self, file_path: Path) -> None:
"""Remove state for a file.
Args:
file_path: Path to the file
"""
file_path_str = str(file_path)
if file_path_str in self.store.states:
del self.store.states[file_path_str]
self.store.last_updated = datetime.now()
def save(self) -> None:
"""Save the current state store to disk."""
save_state(self.store, self.state_path)
tests/test_analysis.py
+546
View File
@@ -0,0 +1,546 @@
"""Unit tests for the analysis engine.
Tests series completeness analysis, duplicate detection, and quality comparison.
"""
import pytest
from pathlib import Path
from datetime import datetime
from vlm.models import SeriesIdentity, SeasonCompleteness, MovieIdentity, VideoFile, DuplicateGroup
from vlm.analysis import analyze_series_completeness, detect_duplicates, compare_quality
class TestSeriesCompletenessAnalysis:
"""Test series completeness analysis functionality."""
def test_detect_single_gap(self):
"""Test detection of a single missing episode."""
episodes = [
SeriesIdentity("Show Name", 1, [1], 0.9, False, "Show.Name.S01E01.mkv"),
SeriesIdentity("Show Name", 1, [2], 0.9, False, "Show.Name.S01E02.mkv"),
SeriesIdentity("Show Name", 1, [4], 0.9, False, "Show.Name.S01E04.mkv"),
SeriesIdentity("Show Name", 1, [5], 0.9, False, "Show.Name.S01E05.mkv"),
]
result = analyze_series_completeness(episodes)
assert len(result) == 1
assert result[0].series_title == "Show Name"
assert result[0].season == 1
assert result[0].episodes_found == [1, 2, 4, 5]
assert result[0].episodes_missing == [3]
def test_detect_multiple_gaps(self):
"""Test detection of multiple missing episodes."""
episodes = [
SeriesIdentity("Show Name", 1, [1], 0.9, False, "Show.Name.S01E01.mkv"),
SeriesIdentity("Show Name", 1, [3], 0.9, False, "Show.Name.S01E03.mkv"),
SeriesIdentity("Show Name", 1, [5], 0.9, False, "Show.Name.S01E05.mkv"),
SeriesIdentity("Show Name", 1, [7], 0.9, False, "Show.Name.S01E07.mkv"),
]
result = analyze_series_completeness(episodes)
assert len(result) == 1
assert result[0].episodes_found == [1, 3, 5, 7]
assert result[0].episodes_missing == [2, 4, 6]
def test_no_gaps_returns_empty(self):
"""Test that complete seasons are not included in results."""
episodes = [
SeriesIdentity("Show Name", 1, [1], 0.9, False, "Show.Name.S01E01.mkv"),
SeriesIdentity("Show Name", 1, [2], 0.9, False, "Show.Name.S01E02.mkv"),
SeriesIdentity("Show Name", 1, [3], 0.9, False, "Show.Name.S01E03.mkv"),
]
result = analyze_series_completeness(episodes)
assert len(result) == 0
def test_multi_season_independence(self):
"""Test that gap detection for one season doesn't affect others."""
episodes = [
# Season 1 - has gap at episode 2
SeriesIdentity("Show Name", 1, [1], 0.9, False, "Show.Name.S01E01.mkv"),
SeriesIdentity("Show Name", 1, [3], 0.9, False, "Show.Name.S01E03.mkv"),
# Season 2 - complete
SeriesIdentity("Show Name", 2, [1], 0.9, False, "Show.Name.S02E01.mkv"),
SeriesIdentity("Show Name", 2, [2], 0.9, False, "Show.Name.S02E02.mkv"),
# Season 3 - has gap at episode 5
SeriesIdentity("Show Name", 3, [4], 0.9, False, "Show.Name.S03E04.mkv"),
SeriesIdentity("Show Name", 3, [6], 0.9, False, "Show.Name.S03E06.mkv"),
]
result = analyze_series_completeness(episodes)
# Should have 2 results (seasons 1 and 3 with gaps)
assert len(result) == 2
# Find season 1 result
season1 = next(r for r in result if r.season == 1)
assert season1.episodes_found == [1, 3]
assert season1.episodes_missing == [2]
# Find season 3 result
season3 = next(r for r in result if r.season == 3)
assert season3.episodes_found == [4, 6]
assert season3.episodes_missing == [5]
def test_multi_episode_files(self):
"""Test handling of multi-episode files."""
episodes = [
SeriesIdentity("Show Name", 1, [1, 2], 0.9, False, "Show.Name.S01E01-E02.mkv"),
SeriesIdentity("Show Name", 1, [4], 0.9, False, "Show.Name.S01E04.mkv"),
]
result = analyze_series_completeness(episodes)
assert len(result) == 1
assert result[0].episodes_found == [1, 2, 4]
assert result[0].episodes_missing == [3]
def test_different_series_separate_analysis(self):
"""Test that different series are analyzed separately."""
episodes = [
# Series A - has gap
SeriesIdentity("Series A", 1, [1], 0.9, False, "Series.A.S01E01.mkv"),
SeriesIdentity("Series A", 1, [3], 0.9, False, "Series.A.S01E03.mkv"),
# Series B - complete
SeriesIdentity("Series B", 1, [1], 0.9, False, "Series.B.S01E01.mkv"),
SeriesIdentity("Series B", 1, [2], 0.9, False, "Series.B.S01E02.mkv"),
]
result = analyze_series_completeness(episodes)
# Only Series A should be in results
assert len(result) == 1
assert result[0].series_title == "Series A"
assert result[0].episodes_missing == [2]
def test_skip_episodes_without_season(self):
"""Test that episodes with season=None are excluded from analysis."""
episodes = [
SeriesIdentity("Show Name", 1, [1], 0.9, False, "Show.Name.S01E01.mkv"),
SeriesIdentity("Show Name", 1, [2], 0.9, False, "Show.Name.S01E02.mkv"),
SeriesIdentity("Show Name", None, [1], 0.3, True, "Show.Name.Episode.1.mkv"),
]
result = analyze_series_completeness(episodes)
# Should only analyze season 1, which is complete
assert len(result) == 0
def test_skip_episodes_with_empty_episode_list(self):
"""Test that episodes with empty episode list are excluded from analysis."""
episodes = [
SeriesIdentity("Show Name", 1, [1], 0.9, False, "Show.Name.S01E01.mkv"),
SeriesIdentity("Show Name", 1, [3], 0.9, False, "Show.Name.S01E03.mkv"),
SeriesIdentity("Show Name", 1, [], 0.3, True, "Show.Name.S01.mkv"),
]
result = analyze_series_completeness(episodes)
# Should detect gap at episode 2
assert len(result) == 1
assert result[0].episodes_missing == [2]
def test_non_sequential_start(self):
"""Test gap detection when episodes don't start at 1."""
episodes = [
SeriesIdentity("Show Name", 1, [5], 0.9, False, "Show.Name.S01E05.mkv"),
SeriesIdentity("Show Name", 1, [6], 0.9, False, "Show.Name.S01E06.mkv"),
SeriesIdentity("Show Name", 1, [8], 0.9, False, "Show.Name.S01E08.mkv"),
]
result = analyze_series_completeness(episodes)
# Should detect gap at episode 7 in range [5, 8]
assert len(result) == 1
assert result[0].episodes_found == [5, 6, 8]
assert result[0].episodes_missing == [7]
def test_empty_input(self):
"""Test handling of empty episode list."""
result = analyze_series_completeness([])
assert len(result) == 0
def test_single_episode_no_gap(self):
"""Test that a single episode has no gaps."""
episodes = [
SeriesIdentity("Show Name", 1, [1], 0.9, False, "Show.Name.S01E01.mkv"),
]
result = analyze_series_completeness(episodes)
# Single episode has no gaps
assert len(result) == 0
class TestDuplicateDetection:
"""Test duplicate detection functionality."""
def test_detect_movie_duplicates(self):
"""Test detection of duplicate movies with identical title and year."""
identities = [
MovieIdentity("The Matrix", 1999, 0.9, False, "The.Matrix.1999.1080p.mkv"),
MovieIdentity("The Matrix", 1999, 0.9, False, "The.Matrix.1999.720p.mkv"),
MovieIdentity("Inception", 2010, 0.9, False, "Inception.2010.mkv"),
]
files = [
VideoFile(
Path("/movies/The.Matrix.1999.1080p.mkv"),
"The.Matrix.1999.1080p.mkv",
2000000000,
datetime.now(),
"movie",
resolution="1920x1080",
codec="h264"
),
VideoFile(
Path("/movies/The.Matrix.1999.720p.mkv"),
"The.Matrix.1999.720p.mkv",
1000000000,
datetime.now(),
"movie",
resolution="1280x720",
codec="h264"
),
VideoFile(
Path("/movies/Inception.2010.mkv"),
"Inception.2010.mkv",
1500000000,
datetime.now(),
"movie"
),
]
result = detect_duplicates(identities, files)
# Should find one duplicate group (The Matrix)
assert len(result) == 1
assert isinstance(result[0].identity, MovieIdentity)
assert result[0].identity.title == "The Matrix"
assert result[0].identity.year == 1999
assert len(result[0].files) == 2
assert len(result[0].quality_comparison) == 2
def test_detect_series_duplicates(self):
"""Test detection of duplicate series episodes."""
identities = [
SeriesIdentity("Breaking Bad", 1, [1], 0.9, False, "Breaking.Bad.S01E01.1080p.mkv"),
SeriesIdentity("Breaking Bad", 1, [1], 0.9, False, "Breaking.Bad.S01E01.720p.mkv"),
SeriesIdentity("Breaking Bad", 1, [2], 0.9, False, "Breaking.Bad.S01E02.mkv"),
]
files = [
VideoFile(
Path("/series/Breaking.Bad.S01E01.1080p.mkv"),
"Breaking.Bad.S01E01.1080p.mkv",
1500000000,
datetime.now(),
"series",
resolution="1920x1080"
),
VideoFile(
Path("/series/Breaking.Bad.S01E01.720p.mkv"),
"Breaking.Bad.S01E01.720p.mkv",
800000000,
datetime.now(),
"series",
resolution="1280x720"
),
VideoFile(
Path("/series/Breaking.Bad.S01E02.mkv"),
"Breaking.Bad.S01E02.mkv",
1200000000,
datetime.now(),
"series"
),
]
result = detect_duplicates(identities, files)
# Should find one duplicate group (S01E01)
assert len(result) == 1
assert isinstance(result[0].identity, SeriesIdentity)
assert result[0].identity.title == "Breaking Bad"
assert result[0].identity.season == 1
assert 1 in result[0].identity.episodes
assert len(result[0].files) == 2
def test_no_duplicates(self):
"""Test that unique files are not flagged as duplicates."""
identities = [
MovieIdentity("Movie A", 2020, 0.9, False, "Movie.A.2020.mkv"),
MovieIdentity("Movie B", 2021, 0.9, False, "Movie.B.2021.mkv"),
]
files = [
VideoFile(Path("/movies/Movie.A.2020.mkv"), "Movie.A.2020.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie.B.2021.mkv"), "Movie.B.2021.mkv", 1000000000, datetime.now(), "movie"),
]
result = detect_duplicates(identities, files)
assert len(result) == 0
def test_skip_movies_without_year(self):
"""Test that movies without year are excluded from duplicate detection."""
identities = [
MovieIdentity("Unknown Movie", None, 0.3, True, "Unknown.Movie.mkv"),
MovieIdentity("Unknown Movie", None, 0.3, True, "Unknown.Movie.2.mkv"),
]
files = [
VideoFile(Path("/movies/Unknown.Movie.mkv"), "Unknown.Movie.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Unknown.Movie.2.mkv"), "Unknown.Movie.2.mkv", 1000000000, datetime.now(), "movie"),
]
result = detect_duplicates(identities, files)
# Should not detect duplicates for files needing review
assert len(result) == 0
def test_skip_series_without_season(self):
"""Test that series without season are excluded from duplicate detection."""
identities = [
SeriesIdentity("Unknown Show", None, [1], 0.3, True, "Unknown.Show.E01.mkv"),
SeriesIdentity("Unknown Show", None, [1], 0.3, True, "Unknown.Show.Episode.1.mkv"),
]
files = [
VideoFile(Path("/series/Unknown.Show.E01.mkv"), "Unknown.Show.E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Unknown.Show.Episode.1.mkv"), "Unknown.Show.Episode.1.mkv", 1000000000, datetime.now(), "series"),
]
result = detect_duplicates(identities, files)
assert len(result) == 0
def test_skip_series_with_empty_episodes(self):
"""Test that series with empty episode list are excluded."""
identities = [
SeriesIdentity("Show Name", 1, [], 0.3, True, "Show.Name.S01.mkv"),
SeriesIdentity("Show Name", 1, [], 0.3, True, "Show.Name.Season.1.mkv"),
]
files = [
VideoFile(Path("/series/Show.Name.S01.mkv"), "Show.Name.S01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.Name.Season.1.mkv"), "Show.Name.Season.1.mkv", 1000000000, datetime.now(), "series"),
]
result = detect_duplicates(identities, files)
assert len(result) == 0
def test_quality_comparison_includes_all_metadata(self):
"""Test that quality comparison includes all available metadata."""
identities = [
MovieIdentity("Test Movie", 2020, 0.9, False, "Test.Movie.2020.1080p.mkv"),
MovieIdentity("Test Movie", 2020, 0.9, False, "Test.Movie.2020.720p.mkv"),
]
files = [
VideoFile(
Path("/movies/Test.Movie.2020.1080p.mkv"),
"Test.Movie.2020.1080p.mkv",
2000000000,
datetime.now(),
"movie",
resolution="1920x1080",
codec="h264",
duration_seconds=7200.0,
bitrate_kbps=5000
),
VideoFile(
Path("/movies/Test.Movie.2020.720p.mkv"),
"Test.Movie.2020.720p.mkv",
1000000000,
datetime.now(),
"movie",
resolution="1280x720",
codec="h264",
duration_seconds=7200.0,
bitrate_kbps=2500
),
]
result = detect_duplicates(identities, files)
assert len(result) == 1
comparison = result[0].quality_comparison
assert len(comparison) == 2
# Check first file comparison data
assert comparison[0]['filename'] == "Test.Movie.2020.1080p.mkv"
assert comparison[0]['size_bytes'] == 2000000000
assert comparison[0]['resolution'] == "1920x1080"
assert comparison[0]['codec'] == "h264"
assert comparison[0]['duration_seconds'] == 7200.0
assert comparison[0]['bitrate_kbps'] == 5000
# Check second file comparison data
assert comparison[1]['filename'] == "Test.Movie.2020.720p.mkv"
assert comparison[1]['size_bytes'] == 1000000000
assert comparison[1]['resolution'] == "1280x720"
def test_multi_episode_file_duplicates(self):
"""Test duplicate detection for multi-episode files."""
identities = [
SeriesIdentity("Show", 1, [1, 2], 0.9, False, "Show.S01E01-E02.mkv"),
SeriesIdentity("Show", 1, [1], 0.9, False, "Show.S01E01.mkv"),
SeriesIdentity("Show", 1, [2], 0.9, False, "Show.S01E02.mkv"),
]
files = [
VideoFile(Path("/series/Show.S01E01-E02.mkv"), "Show.S01E01-E02.mkv", 2000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"),
]
result = detect_duplicates(identities, files)
# Should find duplicates for both E01 and E02
assert len(result) == 2
def test_different_years_not_duplicates(self):
"""Test that same title with different years are not duplicates."""
identities = [
MovieIdentity("The Thing", 1982, 0.9, False, "The.Thing.1982.mkv"),
MovieIdentity("The Thing", 2011, 0.9, False, "The.Thing.2011.mkv"),
]
files = [
VideoFile(Path("/movies/The.Thing.1982.mkv"), "The.Thing.1982.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/The.Thing.2011.mkv"), "The.Thing.2011.mkv", 1000000000, datetime.now(), "movie"),
]
result = detect_duplicates(identities, files)
assert len(result) == 0
def test_different_seasons_not_duplicates(self):
"""Test that same series/episode in different seasons are not duplicates."""
identities = [
SeriesIdentity("Show", 1, [1], 0.9, False, "Show.S01E01.mkv"),
SeriesIdentity("Show", 2, [1], 0.9, False, "Show.S02E01.mkv"),
]
files = [
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S02E01.mkv"), "Show.S02E01.mkv", 1000000000, datetime.now(), "series"),
]
result = detect_duplicates(identities, files)
assert len(result) == 0
class TestQualityComparison:
"""Test quality comparison functionality."""
def test_compare_quality_with_all_metadata(self):
"""Test quality comparison with all metadata available."""
files = [
VideoFile(
Path("/test/file1.mkv"),
"file1.mkv",
2000000000,
datetime.now(),
"movie",
resolution="1920x1080",
codec="h264",
duration_seconds=7200.0,
bitrate_kbps=5000
),
VideoFile(
Path("/test/file2.mkv"),
"file2.mkv",
1000000000,
datetime.now(),
"movie",
resolution="1280x720",
codec="h265",
duration_seconds=7200.0,
bitrate_kbps=2500
),
]
result = compare_quality(files)
assert len(result) == 2
assert result[0]['filename'] == "file1.mkv"
assert result[0]['size_bytes'] == 2000000000
assert result[0]['resolution'] == "1920x1080"
assert result[0]['codec'] == "h264"
assert result[0]['duration_seconds'] == 7200.0
assert result[0]['bitrate_kbps'] == 5000
assert result[1]['filename'] == "file2.mkv"
assert result[1]['size_bytes'] == 1000000000
assert result[1]['resolution'] == "1280x720"
assert result[1]['codec'] == "h265"
def test_compare_quality_with_partial_metadata(self):
"""Test quality comparison when some metadata is missing."""
files = [
VideoFile(
Path("/test/file1.mkv"),
"file1.mkv",
2000000000,
datetime.now(),
"movie",
resolution="1920x1080"
# codec, duration, bitrate not available
),
VideoFile(
Path("/test/file2.mkv"),
"file2.mkv",
1000000000,
datetime.now(),
"movie"
# No optional metadata
),
]
result = compare_quality(files)
assert len(result) == 2
assert result[0]['filename'] == "file1.mkv"
assert result[0]['size_bytes'] == 2000000000
assert result[0]['resolution'] == "1920x1080"
assert 'codec' not in result[0]
assert 'duration_seconds' not in result[0]
assert 'bitrate_kbps' not in result[0]
assert result[1]['filename'] == "file2.mkv"
assert result[1]['size_bytes'] == 1000000000
assert 'resolution' not in result[1]
assert 'codec' not in result[1]
def test_compare_quality_empty_list(self):
"""Test quality comparison with empty file list."""
result = compare_quality([])
assert len(result) == 0
def test_compare_quality_single_file(self):
"""Test quality comparison with single file."""
files = [
VideoFile(
Path("/test/file.mkv"),
"file.mkv",
1500000000,
datetime.now(),
"movie",
resolution="1920x1080",
codec="h264"
),
]
result = compare_quality(files)
assert len(result) == 1
assert result[0]['filename'] == "file.mkv"
assert result[0]['size_bytes'] == 1500000000
tests/test_analysis_properties.py
+515
View File
@@ -0,0 +1,515 @@
"""Property-based tests for the analysis engine.
Tests universal correctness properties using Hypothesis with minimum 100 iterations.
Each test validates a specific property from the design document.
"""
import pytest
from pathlib import Path
from datetime import datetime
from hypothesis import given, strategies as st, settings
from vlm.models import (
SeriesIdentity, SeasonCompleteness, MovieIdentity, VideoFile, DuplicateGroup
)
from vlm.analysis import analyze_series_completeness, detect_duplicates, compare_quality
from vlm.reports import (
generate_completeness_report, generate_duplicate_report, generate_summary_report
)
# Custom strategies for generating test data
@st.composite
def series_identity_strategy(draw, title=None, season=None):
"""Generate a SeriesIdentity with optional fixed title and season."""
if title is None:
title = draw(st.text(min_size=1, max_size=50, alphabet=st.characters(
whitelist_categories=('Lu', 'Ll', 'Nd'), whitelist_characters=' '
)))
if season is None:
season = draw(st.integers(min_value=1, max_value=20))
# Generate 1-3 episode numbers
episode_count = draw(st.integers(min_value=1, max_value=3))
episodes = draw(st.lists(
st.integers(min_value=1, max_value=50),
min_size=episode_count,
max_size=episode_count,
unique=True
))
confidence = draw(st.floats(min_value=0.5, max_value=1.0))
needs_review = False
original_filename = f"{title.replace(' ', '.')}.S{season:02d}E{episodes[0]:02d}.mkv"
return SeriesIdentity(title, season, sorted(episodes), confidence, needs_review, original_filename)
@st.composite
def movie_identity_strategy(draw, title=None, year=None):
"""Generate a MovieIdentity with optional fixed title and year."""
if title is None:
title = draw(st.text(min_size=1, max_size=50, alphabet=st.characters(
whitelist_categories=('Lu', 'Ll', 'Nd'), whitelist_characters=' '
)))
if year is None:
year = draw(st.integers(min_value=1900, max_value=2030))
confidence = draw(st.floats(min_value=0.5, max_value=1.0))
needs_review = False
original_filename = f"{title.replace(' ', '.')}.{year}.mkv"
return MovieIdentity(title, year, confidence, needs_review, original_filename)
@st.composite
def video_file_strategy(draw, filename=None, category="movie"):
"""Generate a VideoFile with optional fixed filename."""
if filename is None:
filename = draw(st.text(min_size=5, max_size=50, alphabet=st.characters(
whitelist_categories=('Lu', 'Ll', 'Nd'), whitelist_characters='.-_'
))) + ".mkv"
path = Path(f"/{category}/{filename}")
size_bytes = draw(st.integers(min_value=1000000, max_value=10000000000))
modified_timestamp = datetime.now()
# Optional metadata
has_metadata = draw(st.booleans())
if has_metadata:
resolution = draw(st.sampled_from(["1920x1080", "1280x720", "3840x2160", "720x480"]))
codec = draw(st.sampled_from(["h264", "h265", "vp9", "av1"]))
duration_seconds = draw(st.floats(min_value=300, max_value=10800))
bitrate_kbps = draw(st.integers(min_value=500, max_value=20000))
return VideoFile(path, filename, size_bytes, modified_timestamp, category,
resolution, codec, duration_seconds, bitrate_kbps)
else:
return VideoFile(path, filename, size_bytes, modified_timestamp, category)
# Property 10: Gap detection
# Feature: video-library-manager, Property 10: Gap detection
@settings(max_examples=100)
@given(
title=st.text(min_size=1, max_size=30, alphabet=st.characters(
whitelist_categories=('Lu', 'Ll'), whitelist_characters=' '
)),
season=st.integers(min_value=1, max_value=10),
# Generate a list of episode numbers with guaranteed gaps
episodes_data=st.lists(
st.integers(min_value=1, max_value=30),
min_size=3,
max_size=15,
unique=True
)
)
def test_property_10_gap_detection(title, season, episodes_data):
"""Property 10: For any set of episodes within season, analysis SHALL detect
missing episode numbers in range [min, max].
Validates: Requirements 4.1, 4.2
"""
# Sort episodes and ensure there's at least one gap
sorted_episodes = sorted(episodes_data)
# Create episodes, intentionally removing one to create a gap
if len(sorted_episodes) >= 3:
# Remove a middle episode to guarantee a gap
gap_index = len(sorted_episodes) // 2
removed_episode = sorted_episodes[gap_index]
episodes_with_gap = sorted_episodes[:gap_index] + sorted_episodes[gap_index + 1:]
# Create SeriesIdentity objects
episode_identities = [
SeriesIdentity(title, season, [ep], 0.9, False, f"{title}.S{season:02d}E{ep:02d}.mkv")
for ep in episodes_with_gap
]
# Analyze completeness
result = analyze_series_completeness(episode_identities)
# Should detect the gap
if len(result) > 0:
assert result[0].series_title == title
assert result[0].season == season
# The missing episode should be in the detected gaps
min_ep = min(episodes_with_gap)
max_ep = max(episodes_with_gap)
expected_missing = set(range(min_ep, max_ep + 1)) - set(episodes_with_gap)
assert set(result[0].episodes_missing) == expected_missing
assert removed_episode in result[0].episodes_missing
# Property 11: Multi-season independence
# Feature: video-library-manager, Property 11: Multi-season independence
@settings(max_examples=100)
@given(
title=st.text(min_size=1, max_size=30, alphabet=st.characters(
whitelist_categories=('Lu', 'Ll'), whitelist_characters=' '
)),
season1_episodes=st.lists(st.integers(min_value=1, max_value=20), min_size=2, max_size=10, unique=True),
season2_episodes=st.lists(st.integers(min_value=1, max_value=20), min_size=2, max_size=10, unique=True),
)
def test_property_11_multi_season_independence(title, season1_episodes, season2_episodes):
"""Property 11: For any series with multiple seasons, gap detection of one
season SHALL not affect others.
Validates: Requirements 4.4
"""
# Create episodes for season 1 with a gap
s1_sorted = sorted(season1_episodes)
if len(s1_sorted) >= 3:
gap_index = len(s1_sorted) // 2
s1_with_gap = s1_sorted[:gap_index] + s1_sorted[gap_index + 1:]
s1_missing = s1_sorted[gap_index]
else:
s1_with_gap = s1_sorted
s1_missing = None
# Create episodes for season 2 (complete, no gaps)
s2_sorted = sorted(season2_episodes)
s2_complete = list(range(min(s2_sorted), max(s2_sorted) + 1))
# Create SeriesIdentity objects
episode_identities = []
for ep in s1_with_gap:
episode_identities.append(
SeriesIdentity(title, 1, [ep], 0.9, False, f"{title}.S01E{ep:02d}.mkv")
)
for ep in s2_complete:
episode_identities.append(
SeriesIdentity(title, 2, [ep], 0.9, False, f"{title}.S02E{ep:02d}.mkv")
)
# Analyze completeness
result = analyze_series_completeness(episode_identities)
# Season 2 should not appear in results (it's complete)
season2_results = [r for r in result if r.season == 2]
assert len(season2_results) == 0
# Season 1 should appear if there's a gap
if s1_missing is not None:
season1_results = [r for r in result if r.season == 1]
if len(season1_results) > 0:
assert s1_missing in season1_results[0].episodes_missing
# Property 12: Duplicate detection for movies
# Feature: video-library-manager, Property 12: Duplicate detection for movies
@settings(max_examples=100)
@given(
title=st.text(min_size=1, max_size=30, alphabet=st.characters(
whitelist_categories=('Lu', 'Ll'), whitelist_characters=' '
)),
year=st.integers(min_value=1900, max_value=2030),
duplicate_count=st.integers(min_value=2, max_value=5)
)
def test_property_12_duplicate_detection_movies(title, year, duplicate_count):
"""Property 12: For any set of movies with identical normalized titles and years,
all SHALL be grouped as duplicates.
Validates: Requirements 5.1
"""
# Create multiple movie identities with same title and year
identities = []
files = []
for i in range(duplicate_count):
filename = f"{title.replace(' ', '.')}.{year}.{i}.mkv"
identities.append(MovieIdentity(title, year, 0.9, False, filename))
files.append(VideoFile(
Path(f"/movies/{filename}"),
filename,
1000000000 + i * 100000000,
datetime.now(),
"movie"
))
# Detect duplicates
result = detect_duplicates(identities, files)
# Should find exactly one duplicate group
assert len(result) == 1
# The group should contain all files
assert len(result[0].files) == duplicate_count
# Identity should match
assert result[0].identity.title == title
assert result[0].identity.year == year
# Property 13: Duplicate detection for series
# Feature: video-library-manager, Property 13: Duplicate detection for series
@settings(max_examples=100)
@given(
title=st.text(min_size=1, max_size=30, alphabet=st.characters(
whitelist_categories=('Lu', 'Ll'), whitelist_characters=' '
)),
season=st.integers(min_value=1, max_value=10),
episode=st.integers(min_value=1, max_value=30),
duplicate_count=st.integers(min_value=2, max_value=5)
)
def test_property_13_duplicate_detection_series(title, season, episode, duplicate_count):
"""Property 13: For any set of series files with identical normalized titles,
seasons, and episodes, all SHALL be grouped as duplicates.
Validates: Requirements 5.2
"""
# Create multiple series identities with same title, season, and episode
identities = []
files = []
for i in range(duplicate_count):
filename = f"{title.replace(' ', '.')}.S{season:02d}E{episode:02d}.{i}.mkv"
identities.append(SeriesIdentity(title, season, [episode], 0.9, False, filename))
files.append(VideoFile(
Path(f"/series/{filename}"),
filename,
1000000000 + i * 100000000,
datetime.now(),
"series"
))
# Detect duplicates
result = detect_duplicates(identities, files)
# Should find exactly one duplicate group
assert len(result) == 1
# The group should contain all files
assert len(result[0].files) == duplicate_count
# Identity should match
assert result[0].identity.title == title
assert result[0].identity.season == season
assert episode in result[0].identity.episodes
# Property 14: Duplicate quality comparison
# Feature: video-library-manager, Property 14: Duplicate quality comparison
@settings(max_examples=100)
@given(
title=st.text(min_size=1, max_size=30, alphabet=st.characters(
whitelist_categories=('Lu', 'Ll'), whitelist_characters=' '
)),
year=st.integers(min_value=1900, max_value=2030),
file_count=st.integers(min_value=2, max_value=4)
)
def test_property_14_duplicate_quality_comparison(title, year, file_count):
"""Property 14: For any duplicate group, comparison data SHALL include
available metadata for each file.
Validates: Requirements 5.3
"""
# Create movie identities and files with varying metadata
identities = []
files = []
for i in range(file_count):
filename = f"{title.replace(' ', '.')}.{year}.{i}.mkv"
identities.append(MovieIdentity(title, year, 0.9, False, filename))
# Some files have full metadata, some don't
if i % 2 == 0:
files.append(VideoFile(
Path(f"/movies/{filename}"),
filename,
1000000000 + i * 100000000,
datetime.now(),
"movie",
resolution="1920x1080",
codec="h264",
duration_seconds=7200.0,
bitrate_kbps=5000
))
else:
files.append(VideoFile(
Path(f"/movies/{filename}"),
filename,
1000000000 + i * 100000000,
datetime.now(),
"movie"
))
# Detect duplicates
result = detect_duplicates(identities, files)
# Should have quality comparison data
assert len(result) == 1
assert len(result[0].quality_comparison) == file_count
# Each comparison entry should have at least filename and size
for comparison in result[0].quality_comparison:
assert 'filename' in comparison
assert 'size_bytes' in comparison
assert 'path' in comparison
# Files with metadata should have those fields
if comparison['filename'].endswith('.0.mkv') or comparison['filename'].endswith('.2.mkv'):
assert 'resolution' in comparison
assert 'codec' in comparison
assert 'duration_seconds' in comparison
assert 'bitrate_kbps' in comparison
# Property 42: Completeness report
# Feature: video-library-manager, Property 42: Completeness report
@settings(max_examples=100)
@given(
series_count=st.integers(min_value=1, max_value=5),
format=st.sampled_from(["text", "json"])
)
def test_property_42_completeness_report(series_count, format):
"""Property 42: For any set of analyzed series, completeness report SHALL
include all series with detected gaps.
Validates: Requirements 11.2
"""
# Create series with gaps
analysis_results = []
for i in range(series_count):
title = f"Series {i}"
season = 1
episodes_found = [1, 2, 4, 5] # Gap at episode 3
episodes_missing = [3]
analysis_results.append(SeasonCompleteness(
series_title=title,
season=season,
episodes_found=episodes_found,
episodes_missing=episodes_missing
))
# Generate report
library_root = Path("/test/library")
report = generate_completeness_report(analysis_results, format, library_root)
# Report should include all series
for i in range(series_count):
assert f"Series {i}" in report
# Report should include metadata
assert str(library_root) in report
# Property 43: Duplicate report grouping
# Feature: video-library-manager, Property 43: Duplicate report grouping
@settings(max_examples=100)
@given(
duplicate_count=st.integers(min_value=1, max_value=5),
format=st.sampled_from(["text", "json"])
)
def test_property_43_duplicate_report_grouping(duplicate_count, format):
"""Property 43: For any set of detected duplicates, duplicate report SHALL
group files by identity with comparison data.
Validates: Requirements 11.3
"""
# Create duplicate groups
duplicate_groups = []
for i in range(duplicate_count):
title = f"Movie {i}"
year = 2020 + i
# Create 2 files for each duplicate group
files = []
quality_comparison = []
for j in range(2):
filename = f"{title.replace(' ', '.')}.{year}.{j}.mkv"
file = VideoFile(
Path(f"/movies/{filename}"),
filename,
1000000000 + j * 500000000,
datetime.now(),
"movie",
resolution="1920x1080" if j == 0 else "1280x720",
codec="h264"
)
files.append(file)
quality_comparison.append({
'filename': filename,
'path': str(file.path),
'size_bytes': file.size_bytes,
'resolution': file.resolution,
'codec': file.codec
})
identity = MovieIdentity(title, year, 0.9, False, files[0].filename)
duplicate_groups.append(DuplicateGroup(identity, files, quality_comparison))
# Generate report
library_root = Path("/test/library")
report = generate_duplicate_report(duplicate_groups, format, library_root)
# Report should include all duplicate groups
for i in range(duplicate_count):
assert f"Movie {i}" in report
# Report should include comparison data (file sizes, resolutions)
assert "1920x1080" in report or "resolution" in report.lower()
# Report should include metadata
assert str(library_root) in report
# Property 44: Summary report accuracy
# Feature: video-library-manager, Property 44: Summary report accuracy
@settings(max_examples=100)
@given(
file_count=st.integers(min_value=1, max_value=20),
categories=st.lists(
st.sampled_from(["movie", "series", "anime", "other"]),
min_size=1,
max_size=4
)
)
def test_property_44_summary_report_accuracy(file_count, categories):
"""Property 44: For any scanned library, summary report SHALL contain
accurate counts and sizes.
Validates: Requirements 11.4
"""
# Create video files
files = []
total_size = 0
category_counts = {}
for i in range(file_count):
category = categories[i % len(categories)]
size = 1000000000 + i * 100000000
filename = f"file_{i}.mkv"
files.append(VideoFile(
Path(f"/{category}/{filename}"),
filename,
size,
datetime.now(),
category
))
total_size += size
category_counts[category] = category_counts.get(category, 0) + 1
# Generate summary report
library_root = Path("/test/library")
report = generate_summary_report(files, library_root)
# Report should include total file count
assert f"Total Files: {file_count}" in report
# Report should include category breakdown
for category, count in category_counts.items():
assert category.capitalize() in report
assert f"Files: {count}" in report
# Report should include metadata
assert str(library_root) in report
tests/test_cli_quarantine.py
+379
View File
@@ -0,0 +1,379 @@
"""Tests for CLI quarantine commands.
This module tests the CLI interface for quarantine operations.
"""
import json
from pathlib import Path
from click.testing import CliRunner
import pytest
from vlm.cli import main
from vlm.config import Config
@pytest.fixture
def temp_library(tmp_path):
"""Create a temporary library structure for testing."""
library_root = tmp_path / "library"
# Create category directories
(library_root / "movie").mkdir(parents=True)
(library_root / "series").mkdir(parents=True)
(library_root / "anime").mkdir(parents=True)
(library_root / "other").mkdir(parents=True)
return library_root
@pytest.fixture
def config_file(tmp_path, temp_library):
"""Create a temporary config file."""
config_path = tmp_path / "config.yaml"
config_content = f"""
library_root: "{temp_library}"
video_extensions:
- .mp4
- .mkv
- .avi
templates:
movie_dir: "movie/{{title}} ({{year}})/"
series_dir: "series/{{title}}/Season {{season:02d}}/"
movie_filename: "{{title}} ({{year}}){{ext}}"
series_filename: "S{{season:02d}}E{{episode:02d}}{{ext}}"
quarantine_dir: ".quarantine"
log_level: "INFO"
"""
config_path.write_text(config_content)
return config_path
class TestQuarantineListCommand:
"""Tests for 'vlm quarantine list' command."""
def test_list_empty_quarantine(self, config_file):
"""Test listing when no files are quarantined."""
runner = CliRunner()
result = runner.invoke(main, ['--config', str(config_file), 'quarantine', 'list'])
assert result.exit_code == 0
assert "No quarantined files found" in result.output
def test_list_with_quarantined_files(self, config_file, temp_library):
"""Test listing quarantined files."""
runner = CliRunner()
# Create a test movie file
movie_file = temp_library / "movie" / "Test Movie (2020).mkv"
movie_file.write_text("test content")
# Quarantine the file first
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(movie_file)
])
assert result.exit_code == 0
# Now list quarantined files
result = runner.invoke(main, ['--config', str(config_file), 'quarantine', 'list'])
assert result.exit_code == 0
assert "Test Movie (2020).mkv" in result.output
assert "Category: movie" in result.output
def test_list_filter_by_category(self, config_file, temp_library):
"""Test listing with category filter."""
runner = CliRunner()
# Create and quarantine a movie file
movie_file = temp_library / "movie" / "Movie.mkv"
movie_file.write_text("movie content")
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(movie_file)
])
assert result.exit_code == 0
# List only movie category
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'list',
'--category', 'movie'
])
assert result.exit_code == 0
assert "Movie.mkv" in result.output
assert "category 'movie'" in result.output
class TestQuarantineAddCommand:
"""Tests for 'vlm quarantine add' command."""
def test_add_movie_file(self, config_file, temp_library):
"""Test adding a movie file to quarantine."""
runner = CliRunner()
# Create a test movie file
movie_file = temp_library / "movie" / "Test Movie (2020).mkv"
movie_file.write_text("test content")
# Quarantine the file
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(movie_file)
])
assert result.exit_code == 0
assert "successfully quarantined" in result.output
assert not movie_file.exists() # Original file should be moved
# Check quarantine location
quarantine_path = temp_library / "movie" / ".quarantine" / "Test Movie (2020).mkv"
assert quarantine_path.exists()
assert quarantine_path.read_text() == "test content"
def test_add_series_file(self, config_file, temp_library):
"""Test adding a series file to quarantine."""
runner = CliRunner()
# Create a test series file
series_file = temp_library / "series" / "Show Name" / "S01E01.mkv"
series_file.parent.mkdir(parents=True)
series_file.write_text("series content")
# Quarantine the file
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(series_file)
])
assert result.exit_code == 0
assert "successfully quarantined" in result.output
assert not series_file.exists()
def test_add_with_reason(self, config_file, temp_library):
"""Test adding file with a reason."""
runner = CliRunner()
# Create a test movie file
movie_file = temp_library / "movie" / "Duplicate.mkv"
movie_file.write_text("test content")
# Quarantine with reason
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(movie_file),
'--reason', 'duplicate file'
])
assert result.exit_code == 0
assert "Reason: duplicate file" in result.output
assert "successfully quarantined" in result.output
def test_add_anime_file_rejected(self, config_file, temp_library):
"""Test that anime files are rejected."""
runner = CliRunner()
# Create a test anime file
anime_file = temp_library / "anime" / "Anime Show.mkv"
anime_file.write_text("anime content")
# Try to quarantine (should fail)
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(anime_file)
])
assert result.exit_code == 1
assert "not supported" in result.output.lower()
assert anime_file.exists() # File should still exist
def test_add_nonexistent_file(self, config_file, temp_library):
"""Test adding a file that doesn't exist."""
runner = CliRunner()
# Try to quarantine non-existent file
nonexistent = temp_library / "movie" / "DoesNotExist.mkv"
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(nonexistent)
])
# Click should catch this before our code runs
assert result.exit_code != 0
class TestQuarantineRestoreCommand:
"""Tests for 'vlm quarantine restore' command."""
def test_restore_movie_file(self, config_file, temp_library):
"""Test restoring a movie file from quarantine."""
runner = CliRunner()
# Create and quarantine a movie file
movie_file = temp_library / "movie" / "Test Movie.mkv"
movie_file.write_text("test content")
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(movie_file)
])
assert result.exit_code == 0
# Get quarantine path
quarantine_path = temp_library / "movie" / ".quarantine" / "Test Movie.mkv"
assert quarantine_path.exists()
# Restore the file
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'restore',
str(quarantine_path)
])
assert result.exit_code == 0
assert "successfully restored" in result.output
assert movie_file.exists() # Original location should have file
assert not quarantine_path.exists() # Quarantine should be empty
assert movie_file.read_text() == "test content"
def test_restore_series_file(self, config_file, temp_library):
"""Test restoring a series file from quarantine."""
runner = CliRunner()
# Create and quarantine a series file
series_file = temp_library / "series" / "Show" / "S01E01.mkv"
series_file.parent.mkdir(parents=True)
series_file.write_text("series content")
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(series_file)
])
assert result.exit_code == 0
# Get quarantine path
quarantine_path = temp_library / "series" / ".quarantine" / "Show" / "S01E01.mkv"
assert quarantine_path.exists()
# Restore the file
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'restore',
str(quarantine_path)
])
assert result.exit_code == 0
assert "successfully restored" in result.output
assert series_file.exists()
def test_restore_conflict(self, config_file, temp_library):
"""Test restoring when original location is occupied."""
runner = CliRunner()
# Create and quarantine a movie file
movie_file = temp_library / "movie" / "Movie.mkv"
movie_file.write_text("original content")
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(movie_file)
])
assert result.exit_code == 0
# Create a new file at the original location
movie_file.write_text("new content")
# Try to restore (should fail due to conflict)
quarantine_path = temp_library / "movie" / ".quarantine" / "Movie.mkv"
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'restore',
str(quarantine_path)
])
assert result.exit_code == 1
assert "Cannot restore" in result.output or "already exists" in result.output
assert movie_file.read_text() == "new content" # Original location unchanged
assert quarantine_path.exists() # File still in quarantine
class TestQuarantineIntegration:
"""Integration tests for quarantine workflow."""
def test_full_quarantine_workflow(self, config_file, temp_library):
"""Test complete workflow: add -> list -> restore."""
runner = CliRunner()
# Create test files
movie1 = temp_library / "movie" / "Movie1.mkv"
movie2 = temp_library / "movie" / "Movie2.mkv"
movie1.write_text("content1")
movie2.write_text("content2")
# Add both to quarantine
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(movie1),
'--reason', 'duplicate'
])
assert result.exit_code == 0
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'add',
str(movie2),
'--reason', 'low quality'
])
assert result.exit_code == 0
# List quarantined files
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'list'
])
assert result.exit_code == 0
assert "Movie1.mkv" in result.output
assert "Movie2.mkv" in result.output
assert "Total: 2 quarantined file(s)" in result.output
# Restore one file
quarantine_path1 = temp_library / "movie" / ".quarantine" / "Movie1.mkv"
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'restore',
str(quarantine_path1)
])
assert result.exit_code == 0
# List again (should show only 1 file now)
result = runner.invoke(main, [
'--config', str(config_file),
'quarantine', 'list'
])
assert result.exit_code == 0
assert "Movie1.mkv" not in result.output
assert "Movie2.mkv" in result.output
assert "Total: 1 quarantined file(s)" in result.output
# Verify restored file
assert movie1.exists()
assert movie1.read_text() == "content1"
tests/test_cli_reports.py
+343
View File
@@ -0,0 +1,343 @@
"""Integration tests for CLI report commands.
Tests the report commands to ensure they properly wire to the Report Generator.
"""
import json
import csv
from pathlib import Path
from datetime import datetime, timezone
from click.testing import CliRunner
from vlm.cli import main
class TestCLIReports:
"""Test CLI report commands."""
def test_report_inventory_csv(self, tmp_path):
"""Test inventory report generation in CSV format."""
# Create test inventory CSV
inventory_file = tmp_path / "inventory.csv"
with open(inventory_file, 'w', encoding='utf-8') as f:
f.write("# Generated: 2024-01-01T00:00:00\n")
f.write("# Library Root: /test/library\n")
f.write("path,filename,size_bytes,modified_timestamp,category,resolution,codec,duration_seconds,bitrate_kbps\n")
f.write("/test/library/movie/Movie1.mkv,Movie1.mkv,1000000,2024-01-01T00:00:00,movie,1920x1080,h264,7200,5000\n")
f.write("/test/library/series/Show.S01E01.mkv,Show.S01E01.mkv,800000,2024-01-01T00:00:00,series,1280x720,h264,2700,3000\n")
# Run command
runner = CliRunner()
result = runner.invoke(main, ['report', 'inventory', '--input', str(inventory_file), '--format', 'csv'])
# Verify success
assert result.exit_code == 0
assert "Loaded 2 files" in result.output
assert "Movie1.mkv" in result.output
assert "Show.S01E01.mkv" in result.output
def test_report_inventory_json(self, tmp_path):
"""Test inventory report generation in JSON format."""
# Create test inventory CSV
inventory_file = tmp_path / "inventory.csv"
with open(inventory_file, 'w', encoding='utf-8') as f:
f.write("# Generated: 2024-01-01T00:00:00\n")
f.write("# Library Root: /test/library\n")
f.write("path,filename,size_bytes,modified_timestamp,category,resolution,codec,duration_seconds,bitrate_kbps\n")
f.write("/test/library/movie/Movie1.mkv,Movie1.mkv,1000000,2024-01-01T00:00:00,movie,,,\n")
# Run command with output file
output_file = tmp_path / "inventory_report.json"
runner = CliRunner()
result = runner.invoke(main, [
'report', 'inventory',
'--input', str(inventory_file),
'--format', 'json',
'--output', str(output_file)
])
# Verify success
assert result.exit_code == 0
assert output_file.exists()
# Verify JSON content
with open(output_file, 'r') as f:
data = json.load(f)
assert data['metadata']['file_count'] == 1
assert len(data['files']) == 1
assert data['files'][0]['filename'] == 'Movie1.mkv'
def test_report_completeness_text(self, tmp_path):
"""Test completeness report generation in text format."""
# Create test analysis JSON
analysis_file = tmp_path / "analysis.json"
analysis_data = {
"metadata": {
"generated": "2024-01-01T00:00:00",
"source_identities": "identities.json",
"total_movies": 0,
"total_series": 3
},
"completeness": [
{
"series_title": "Breaking Bad",
"season": 1,
"episodes_found": [1, 2, 4],
"episodes_missing": [3]
}
],
"duplicates": []
}
with open(analysis_file, 'w') as f:
json.dump(analysis_data, f)
# Run command
runner = CliRunner()
result = runner.invoke(main, [
'report', 'completeness',
'--input', str(analysis_file),
'--format', 'text'
])
# Verify success
assert result.exit_code == 0
assert "Loaded 1 series with gaps" in result.output
assert "Breaking Bad" in result.output
assert "Episodes missing:" in result.output
def test_report_completeness_json(self, tmp_path):
"""Test completeness report generation in JSON format."""
# Create test analysis JSON
analysis_file = tmp_path / "analysis.json"
analysis_data = {
"metadata": {},
"completeness": [
{
"series_title": "The Wire",
"season": 1,
"episodes_found": [1, 3],
"episodes_missing": [2]
}
],
"duplicates": []
}
with open(analysis_file, 'w') as f:
json.dump(analysis_data, f)
# Run command with output file
output_file = tmp_path / "completeness_report.json"
runner = CliRunner()
result = runner.invoke(main, [
'report', 'completeness',
'--input', str(analysis_file),
'--format', 'json',
'--output', str(output_file)
])
# Verify success
assert result.exit_code == 0
assert output_file.exists()
# Verify JSON content
with open(output_file, 'r') as f:
data = json.load(f)
assert data['metadata']['series_count'] == 1
assert len(data['series']) == 1
assert data['series'][0]['title'] == 'The Wire'
def test_report_duplicates_text(self, tmp_path):
"""Test duplicate report generation in text format."""
# Create test analysis JSON
analysis_file = tmp_path / "analysis.json"
analysis_data = {
"metadata": {},
"completeness": [],
"duplicates": [
{
"identity": {
"type": "movie",
"title": "The Matrix",
"year": 1999
},
"files": [
"/movies/The.Matrix.1999.1080p.mkv",
"/movies/The.Matrix.1999.720p.mkv"
],
"quality_comparison": [
{
"filename": "The.Matrix.1999.1080p.mkv",
"path": "/movies/The.Matrix.1999.1080p.mkv",
"size_bytes": 2000000000,
"resolution": "1920x1080"
},
{
"filename": "The.Matrix.1999.720p.mkv",
"path": "/movies/The.Matrix.1999.720p.mkv",
"size_bytes": 1000000000,
"resolution": "1280x720"
}
]
}
]
}
with open(analysis_file, 'w') as f:
json.dump(analysis_data, f)
# Run command
runner = CliRunner()
result = runner.invoke(main, [
'report', 'duplicates',
'--input', str(analysis_file),
'--format', 'text'
])
# Verify success
assert result.exit_code == 0
assert "Loaded 1 duplicate groups" in result.output
assert "The Matrix (1999)" in result.output
assert "1920x1080" in result.output
def test_report_duplicates_json(self, tmp_path):
"""Test duplicate report generation in JSON format."""
# Create test analysis JSON
analysis_file = tmp_path / "analysis.json"
analysis_data = {
"metadata": {},
"completeness": [],
"duplicates": [
{
"identity": {
"type": "series",
"title": "Breaking Bad",
"season": 1,
"episodes": [1]
},
"files": [
"/series/Breaking.Bad.S01E01.1080p.mkv",
"/series/Breaking.Bad.S01E01.720p.mkv"
],
"quality_comparison": [
{
"filename": "Breaking.Bad.S01E01.1080p.mkv",
"path": "/series/Breaking.Bad.S01E01.1080p.mkv",
"size_bytes": 1500000000
},
{
"filename": "Breaking.Bad.S01E01.720p.mkv",
"path": "/series/Breaking.Bad.S01E01.720p.mkv",
"size_bytes": 800000000
}
]
}
]
}
with open(analysis_file, 'w') as f:
json.dump(analysis_data, f)
# Run command with output file
output_file = tmp_path / "duplicates_report.json"
runner = CliRunner()
result = runner.invoke(main, [
'report', 'duplicates',
'--input', str(analysis_file),
'--format', 'json',
'--output', str(output_file)
])
# Verify success
assert result.exit_code == 0
assert output_file.exists()
# Verify JSON content
with open(output_file, 'r') as f:
data = json.load(f)
assert data['metadata']['duplicate_groups'] == 1
assert len(data['duplicates']) == 1
def test_report_summary(self, tmp_path):
"""Test summary report generation."""
# Create test inventory CSV
inventory_file = tmp_path / "inventory.csv"
with open(inventory_file, 'w', encoding='utf-8') as f:
f.write("# Generated: 2024-01-01T00:00:00\n")
f.write("# Library Root: /test/library\n")
f.write("path,filename,size_bytes,modified_timestamp,category,resolution,codec,duration_seconds,bitrate_kbps\n")
f.write("/test/library/movie/Movie1.mkv,Movie1.mkv,2000000000,2024-01-01T00:00:00,movie,,,\n")
f.write("/test/library/movie/Movie2.mkv,Movie2.mkv,1500000000,2024-01-01T00:00:00,movie,,,\n")
f.write("/test/library/series/Show.S01E01.mkv,Show.S01E01.mkv,1000000000,2024-01-01T00:00:00,series,,,\n")
f.write("/test/library/anime/Anime1.mkv,Anime1.mkv,800000000,2024-01-01T00:00:00,anime,,,\n")
# Run command
runner = CliRunner()
result = runner.invoke(main, [
'report', 'summary',
'--input', str(inventory_file)
])
# Verify success
assert result.exit_code == 0
assert "Loaded 4 files" in result.output
assert "Total Files: 4" in result.output
assert "Movie:" in result.output
assert "Series:" in result.output
assert "Anime:" in result.output
def test_report_summary_with_output_file(self, tmp_path):
"""Test summary report generation with output file."""
# Create test inventory CSV
inventory_file = tmp_path / "inventory.csv"
with open(inventory_file, 'w', encoding='utf-8') as f:
f.write("# Generated: 2024-01-01T00:00:00\n")
f.write("# Library Root: /test/library\n")
f.write("path,filename,size_bytes,modified_timestamp,category,resolution,codec,duration_seconds,bitrate_kbps\n")
f.write("/test/library/movie/Movie1.mkv,Movie1.mkv,1000000,2024-01-01T00:00:00,movie,,,\n")
# Run command with output file
output_file = tmp_path / "summary_report.txt"
runner = CliRunner()
result = runner.invoke(main, [
'report', 'summary',
'--input', str(inventory_file),
'--output', str(output_file)
])
# Verify success
assert result.exit_code == 0
assert output_file.exists()
# Verify file content
with open(output_file, 'r') as f:
content = f.read()
assert "Total Files: 1" in content
assert "Movie:" in content
def test_report_inventory_missing_file(self, tmp_path):
"""Test inventory report with missing input file."""
runner = CliRunner()
result = runner.invoke(main, [
'report', 'inventory',
'--input', str(tmp_path / "nonexistent.csv")
])
# Verify error (Click validates file existence before our code runs)
assert result.exit_code != 0
assert "does not exist" in result.output
def test_report_completeness_missing_file(self, tmp_path):
"""Test completeness report with missing input file."""
runner = CliRunner()
result = runner.invoke(main, [
'report', 'completeness',
'--input', str(tmp_path / "nonexistent.json")
])
# Verify error (Click validates file existence before our code runs)
assert result.exit_code != 0
assert "does not exist" in result.output
tests/test_cli_rollback.py
+196
View File
@@ -0,0 +1,196 @@
"""Tests for CLI rollback command."""
import json
from pathlib import Path
from datetime import datetime
import pytest
from click.testing import CliRunner
from vlm.cli import main
from vlm.models import FileOperation, OperationResult, RollbackLog
@pytest.fixture
def cli_runner():
"""Create a Click CLI test runner."""
return CliRunner()
@pytest.fixture
def sample_rollback_log(tmp_path):
"""Create a sample rollback log file for testing."""
# Create test files
source1 = tmp_path / "source1.txt"
source2 = tmp_path / "source2.txt"
dest1 = tmp_path / "dest1.txt"
dest2 = tmp_path / "dest2.txt"
source1.write_text("content1")
source2.write_text("content2")
# Move files to simulate execution
source1.rename(dest1)
source2.rename(dest2)
# Create rollback log
operations = [
OperationResult(
operation=FileOperation(
operation_type="move",
source_path=source1,
destination_path=dest1,
reason="test move 1",
has_conflict=False,
conflict_reason=None
),
success=True,
error_message=None,
executed_at=datetime.now()
),
OperationResult(
operation=FileOperation(
operation_type="move",
source_path=source2,
destination_path=dest2,
reason="test move 2",
has_conflict=False,
conflict_reason=None
),
success=True,
error_message=None,
executed_at=datetime.now()
)
]
rollback_log = RollbackLog(
log_id="test-log-id",
execution_plan_id="test-plan-id",
executed_at=datetime.now(),
operations=operations
)
# Save rollback log to file
log_path = tmp_path / "rollback_test.json"
log_data = {
"log_id": rollback_log.log_id,
"execution_plan_id": rollback_log.execution_plan_id,
"executed_at": rollback_log.executed_at.isoformat(),
"operations": [
{
"operation_type": op.operation.operation_type,
"source_path": str(op.operation.source_path),
"destination_path": str(op.operation.destination_path),
"reason": op.operation.reason,
"success": op.success,
"error_message": op.error_message,
"executed_at": op.executed_at.isoformat()
}
for op in rollback_log.operations
]
}
with open(log_path, 'w', encoding='utf-8') as f:
json.dump(log_data, f, indent=2)
return {
"log_path": log_path,
"source1": source1,
"source2": source2,
"dest1": dest1,
"dest2": dest2
}
class TestRollbackCommand:
"""Tests for the rollback CLI command."""
def test_rollback_help(self, cli_runner):
"""Test that rollback command shows help text."""
result = cli_runner.invoke(main, ['rollback', '--help'])
assert result.exit_code == 0
assert "Rollback previous execution" in result.output
assert "--log" in result.output
assert "best-effort" in result.output
def test_rollback_with_log_file(self, cli_runner, sample_rollback_log):
"""Test rollback command with explicit log file."""
log_path = sample_rollback_log["log_path"]
dest1 = sample_rollback_log["dest1"]
dest2 = sample_rollback_log["dest2"]
source1 = sample_rollback_log["source1"]
source2 = sample_rollback_log["source2"]
# Verify files are at destination before rollback
assert dest1.exists()
assert dest2.exists()
assert not source1.exists()
assert not source2.exists()
# Run rollback command with auto-confirmation
result = cli_runner.invoke(
main,
['rollback', '--log', str(log_path)],
input='y\n' # Confirm rollback
)
# Check command succeeded
assert result.exit_code == 0
assert "Rollback log loaded" in result.output
assert "Rolling back" in result.output
assert "Rollback Summary" in result.output
# Verify files were moved back to source
assert source1.exists()
assert source2.exists()
assert not dest1.exists()
assert not dest2.exists()
def test_rollback_cancel_confirmation(self, cli_runner, sample_rollback_log):
"""Test that rollback can be cancelled at confirmation prompt."""
log_path = sample_rollback_log["log_path"]
dest1 = sample_rollback_log["dest1"]
dest2 = sample_rollback_log["dest2"]
# Run rollback command and cancel
result = cli_runner.invoke(
main,
['rollback', '--log', str(log_path)],
input='n\n' # Cancel rollback
)
# Check command was cancelled
assert result.exit_code == 0
assert "Rollback cancelled" in result.output
# Verify files were NOT moved (still at destination)
assert dest1.exists()
assert dest2.exists()
def test_rollback_missing_log_file(self, cli_runner, tmp_path):
"""Test rollback command with missing log file."""
missing_log = tmp_path / "nonexistent.json"
result = cli_runner.invoke(
main,
['rollback', '--log', str(missing_log)]
)
# Check command failed with appropriate error
# Click returns exit code 2 for file validation errors
assert result.exit_code == 2
assert "Error" in result.output or "does not exist" in result.output
def test_rollback_no_log_specified_no_logs_exist(self, cli_runner, tmp_path, monkeypatch):
"""Test rollback command without log file when no logs exist."""
# Mock home directory to use tmp_path
fake_home = tmp_path / "fake_home"
fake_home.mkdir()
monkeypatch.setattr(Path, 'home', lambda: fake_home)
result = cli_runner.invoke(main, ['rollback'])
# Check command failed with appropriate error
assert result.exit_code == 1
assert "No rollback logs found" in result.output
tests/test_cli_state.py
+234
View File
@@ -0,0 +1,234 @@
"""Tests for CLI state commands."""
import json
import tempfile
from pathlib import Path
import pytest
from click.testing import CliRunner
from vlm.cli import main
@pytest.fixture
def runner():
"""Create a Click CLI test runner."""
return CliRunner()
@pytest.fixture
def temp_state_file(tmp_path, monkeypatch):
"""Create a temporary state file and set up environment."""
state_dir = tmp_path / ".vlm"
state_dir.mkdir(parents=True, exist_ok=True)
state_file = state_dir / "state.json"
# Mock the home directory to use tmp_path
monkeypatch.setattr(Path, 'home', lambda: tmp_path)
return state_file
@pytest.fixture
def temp_config(tmp_path):
"""Create a temporary config file."""
config_dir = tmp_path / ".vlm"
config_dir.mkdir(parents=True, exist_ok=True)
config_file = config_dir / "config.yaml"
# Create a minimal config
config_content = f"""
library_root: {tmp_path / "videos"}
video_extensions:
- .mp4
- .mkv
templates:
movie_dir: "movie/{{title}} ({{year}})/"
series_dir: "series/{{title}}/Season {{season:02d}}/"
movie_filename: "{{title}} ({{year}}){{ext}}"
series_filename: "S{{season:02d}}E{{episode:02d}}{{ext}}"
quarantine_dir: ".quarantine"
log_level: "INFO"
"""
config_file.write_text(config_content)
return config_file
def test_state_set_and_show(runner, temp_state_file, temp_config):
"""Test setting and showing file state."""
test_file = Path("/test/movie.mkv")
# Set state
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'set', str(test_file),
'--status', 'reviewed',
'--reason', 'checked manually'
])
assert result.exit_code == 0
assert "State updated" in result.output
assert "reviewed" in result.output
# Verify state file was created
assert temp_state_file.exists()
# Show state
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'show', str(test_file)
])
assert result.exit_code == 0
assert "reviewed" in result.output
assert "checked manually" in result.output
def test_state_query(runner, temp_state_file, temp_config):
"""Test querying files by status."""
test_files = [
Path("/test/movie1.mkv"),
Path("/test/movie2.mkv"),
Path("/test/movie3.mkv")
]
# Set states for multiple files
for i, test_file in enumerate(test_files):
status = 'ignored' if i < 2 else 'reviewed'
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'set', str(test_file),
'--status', status
])
assert result.exit_code == 0
# Query for ignored files
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'query',
'--status', 'ignored'
])
assert result.exit_code == 0
assert "movie1.mkv" in result.output
assert "movie2.mkv" in result.output
assert "movie3.mkv" not in result.output
assert "Total: 2 file(s)" in result.output
def test_state_clear(runner, temp_state_file, temp_config):
"""Test clearing file state."""
test_file = Path("/test/movie.mkv")
# Set state
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'set', str(test_file),
'--status', 'reviewed'
])
assert result.exit_code == 0
# Clear state
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'clear', str(test_file)
])
assert result.exit_code == 0
assert "State cleared" in result.output
# Verify state is cleared
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'show', str(test_file)
])
assert result.exit_code == 0
assert "No state found" in result.output
def test_state_set_invalid_status(runner, temp_state_file, temp_config):
"""Test setting state with invalid status."""
test_file = Path("/test/movie.mkv")
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'set', str(test_file),
'--status', 'invalid_status'
])
# Should fail due to invalid choice
assert result.exit_code != 0
def test_state_show_nonexistent(runner, temp_state_file, temp_config):
"""Test showing state for file that doesn't have state."""
test_file = Path("/test/nonexistent.mkv")
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'show', str(test_file)
])
assert result.exit_code == 0
assert "No state found" in result.output
def test_state_clear_nonexistent(runner, temp_state_file, temp_config):
"""Test clearing state for file that doesn't have state."""
test_file = Path("/test/nonexistent.mkv")
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'clear', str(test_file)
])
assert result.exit_code == 0
assert "No state found" in result.output
assert "Nothing to clear" in result.output
def test_state_query_empty(runner, temp_state_file, temp_config):
"""Test querying when no files have the status."""
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'query',
'--status', 'quarantined'
])
assert result.exit_code == 0
assert "No files found" in result.output
def test_state_set_idempotent(runner, temp_state_file, temp_config):
"""Test that setting state multiple times is idempotent."""
test_file = Path("/test/movie.mkv")
# Set state first time
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'set', str(test_file),
'--status', 'reviewed',
'--reason', 'first check'
])
assert result.exit_code == 0
# Set state second time with different reason
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'set', str(test_file),
'--status', 'reviewed',
'--reason', 'second check'
])
assert result.exit_code == 0
# Verify the reason was updated
result = runner.invoke(main, [
'--config', str(temp_config),
'state', 'show', str(test_file)
])
assert result.exit_code == 0
assert "second check" in result.output
assert "first check" not in result.output
tests/test_config.py
+387
View File
@@ -0,0 +1,387 @@
"""Unit tests for Configuration Manager."""
import pytest
import yaml
from pathlib import Path
from vlm.config import Config, load_config, create_default_config, validate_config
class TestConfig:
"""Test Config dataclass."""
def test_config_creation_with_defaults(self):
"""Test creating Config with default values."""
config = Config(library_root=Path("/mnt/nas/videos"))
assert config.library_root == Path("/mnt/nas/videos")
assert ".mp4" in config.video_extensions
assert ".mkv" in config.video_extensions
assert config.movie_template == "movie/{title} ({year})/"
assert config.series_template == "series/{title}/Season {season:02d}/"
assert config.log_level == "INFO"
assert config.quarantine_dir == ".quarantine"
def test_config_creation_with_custom_values(self):
"""Test creating Config with custom values."""
config = Config(
library_root=Path("/custom/path"),
video_extensions=[".mp4", ".avi"],
movie_template="movies/{title}-{year}/",
log_level="DEBUG"
)
assert config.library_root == Path("/custom/path")
assert config.video_extensions == [".mp4", ".avi"]
assert config.movie_template == "movies/{title}-{year}/"
assert config.log_level == "DEBUG"
class TestLoadConfig:
"""Test load_config function."""
def test_load_config_success(self, tmp_path):
"""Test loading valid configuration file."""
config_file = tmp_path / "config.yaml"
config_data = {
'library_root': '/mnt/nas/videos',
'video_extensions': ['.mp4', '.mkv', '.avi'],
'templates': {
'movie_dir': 'movie/{title} ({year})/',
'series_dir': 'series/{title}/Season {season:02d}/',
'movie_filename': '{title} ({year}){ext}',
'series_filename': 'S{season:02d}E{episode:02d}{ext}'
},
'quarantine_dir': '.quarantine',
'log_level': 'DEBUG'
}
with open(config_file, 'w') as f:
yaml.dump(config_data, f)
config = load_config(config_file)
assert config.library_root == Path('/mnt/nas/videos')
assert config.video_extensions == ['.mp4', '.mkv', '.avi']
assert config.movie_template == 'movie/{title} ({year})/'
assert config.series_template == 'series/{title}/Season {season:02d}/'
assert config.log_level == 'DEBUG'
assert config.quarantine_dir == '.quarantine'
def test_load_config_with_home_directory(self, tmp_path):
"""Test loading config with ~ in library_root."""
config_file = tmp_path / "config.yaml"
config_data = {
'library_root': '~/Videos',
'video_extensions': ['.mp4']
}
with open(config_file, 'w') as f:
yaml.dump(config_data, f)
config = load_config(config_file)
# Should expand ~ to home directory
assert config.library_root == Path.home() / "Videos"
def test_load_config_missing_file(self, tmp_path):
"""Test loading non-existent configuration file."""
config_file = tmp_path / "nonexistent.yaml"
with pytest.raises(FileNotFoundError):
load_config(config_file)
def test_load_config_invalid_yaml(self, tmp_path):
"""Test loading configuration with invalid YAML syntax."""
config_file = tmp_path / "config.yaml"
with open(config_file, 'w') as f:
f.write("invalid: yaml: syntax: [unclosed")
with pytest.raises(yaml.YAMLError):
load_config(config_file)
def test_load_config_missing_library_root(self, tmp_path):
"""Test loading configuration without library_root."""
config_file = tmp_path / "config.yaml"
config_data = {
'video_extensions': ['.mp4']
}
with open(config_file, 'w') as f:
yaml.dump(config_data, f)
with pytest.raises(ValueError, match="library_root"):
load_config(config_file)
def test_load_config_empty_file(self, tmp_path):
"""Test loading empty configuration file."""
config_file = tmp_path / "config.yaml"
config_file.write_text("")
with pytest.raises(ValueError, match="library_root"):
load_config(config_file)
def test_load_config_with_defaults(self, tmp_path):
"""Test loading config with minimal settings uses defaults."""
config_file = tmp_path / "config.yaml"
config_data = {
'library_root': '/mnt/nas/videos'
}
with open(config_file, 'w') as f:
yaml.dump(config_data, f)
config = load_config(config_file)
# Should use default values for missing fields
assert config.library_root == Path('/mnt/nas/videos')
assert ".mp4" in config.video_extensions
assert config.movie_template == "movie/{title} ({year})/"
assert config.log_level == "INFO"
class TestCreateDefaultConfig:
"""Test create_default_config function."""
def test_create_default_config(self, tmp_path):
"""Test creating default configuration file."""
config_file = tmp_path / "config.yaml"
config = create_default_config(config_file)
# Check returned config object
assert config.library_root == Path.home() / "Videos"
assert ".mp4" in config.video_extensions
assert ".mkv" in config.video_extensions
assert config.movie_template == "movie/{title} ({year})/"
assert config.series_template == "series/{title}/Season {season:02d}/"
assert config.log_level == "INFO"
assert config.quarantine_dir == ".quarantine"
# Check file was created
assert config_file.exists()
# Check file content
with open(config_file, 'r') as f:
data = yaml.safe_load(f)
assert 'library_root' in data
assert 'video_extensions' in data
assert 'templates' in data
assert 'log_level' in data
assert 'quarantine_dir' in data
def test_create_default_config_creates_parent_dirs(self, tmp_path):
"""Test that create_default_config creates parent directories."""
config_file = tmp_path / "subdir" / "config.yaml"
config = create_default_config(config_file)
assert config_file.exists()
assert config_file.parent.exists()
def test_create_default_config_is_loadable(self, tmp_path):
"""Test that created default config can be loaded."""
config_file = tmp_path / "config.yaml"
created_config = create_default_config(config_file)
loaded_config = load_config(config_file)
# Configs should be equivalent
assert loaded_config.library_root == created_config.library_root
assert loaded_config.video_extensions == created_config.video_extensions
assert loaded_config.movie_template == created_config.movie_template
assert loaded_config.log_level == created_config.log_level
class TestValidateConfig:
"""Test validate_config function."""
def test_validate_valid_config(self):
"""Test validating a valid configuration."""
config = Config(library_root=Path("/mnt/nas/videos"))
errors = validate_config(config)
assert errors == []
def test_validate_empty_library_root(self):
"""Test validating config with empty library_root."""
config = Config(library_root=Path(""))
errors = validate_config(config)
assert len(errors) > 0
assert any("library_root" in err for err in errors)
def test_validate_empty_video_extensions(self):
"""Test validating config with empty video_extensions."""
config = Config(
library_root=Path("/mnt/nas/videos"),
video_extensions=[]
)
errors = validate_config(config)
assert len(errors) > 0
assert any("video_extensions" in err for err in errors)
def test_validate_invalid_video_extension_format(self):
"""Test validating config with invalid video extension format."""
config = Config(
library_root=Path("/mnt/nas/videos"),
video_extensions=["mp4", ".mkv"] # Missing dot on first one
)
errors = validate_config(config)
assert len(errors) > 0
assert any("must start with '.'" in err for err in errors)
def test_validate_empty_templates(self):
"""Test validating config with empty templates."""
config = Config(
library_root=Path("/mnt/nas/videos"),
movie_template="",
series_template=""
)
errors = validate_config(config)
assert len(errors) >= 2
assert any("movie_template" in err for err in errors)
assert any("series_template" in err for err in errors)
def test_validate_invalid_log_level(self):
"""Test validating config with invalid log level."""
config = Config(
library_root=Path("/mnt/nas/videos"),
log_level="INVALID"
)
errors = validate_config(config)
assert len(errors) > 0
assert any("log_level" in err for err in errors)
def test_validate_valid_log_levels(self):
"""Test validating config with all valid log levels."""
valid_levels = ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
for level in valid_levels:
config = Config(
library_root=Path("/mnt/nas/videos"),
log_level=level
)
errors = validate_config(config)
assert errors == [], f"Log level {level} should be valid"
def test_validate_absolute_quarantine_dir(self):
"""Test validating config with absolute quarantine_dir."""
config = Config(
library_root=Path("/mnt/nas/videos"),
quarantine_dir="/absolute/path"
)
errors = validate_config(config)
assert len(errors) > 0
assert any("must be relative" in err for err in errors)
def test_validate_empty_quarantine_dir(self):
"""Test validating config with empty quarantine_dir."""
config = Config(
library_root=Path("/mnt/nas/videos"),
quarantine_dir=""
)
errors = validate_config(config)
assert len(errors) > 0
assert any("quarantine_dir" in err for err in errors)
def test_validate_multiple_errors(self):
"""Test validating config with multiple errors."""
config = Config(
library_root=Path(""),
video_extensions=[],
movie_template="",
log_level="INVALID"
)
errors = validate_config(config)
# Should have multiple errors
assert len(errors) >= 4
class TestConfigIntegration:
"""Integration tests for configuration workflow."""
def test_missing_config_workflow(self, tmp_path):
"""Test workflow: missing config -> create default -> load."""
config_file = tmp_path / "config.yaml"
# Config doesn't exist
assert not config_file.exists()
# Try to load, should raise FileNotFoundError
with pytest.raises(FileNotFoundError):
load_config(config_file)
# Create default config
default_config = create_default_config(config_file)
# Now file exists
assert config_file.exists()
# Load the created config
loaded_config = load_config(config_file)
# Should match default
assert loaded_config.library_root == default_config.library_root
assert loaded_config.video_extensions == default_config.video_extensions
def test_invalid_yaml_workflow(self, tmp_path):
"""Test workflow: invalid YAML -> report error -> use defaults."""
config_file = tmp_path / "config.yaml"
# Create invalid YAML
with open(config_file, 'w') as f:
f.write("invalid: yaml: [unclosed")
# Try to load, should raise YAMLError
with pytest.raises(yaml.YAMLError):
load_config(config_file)
# In real usage, caller would catch this and create default
default_config = create_default_config(config_file)
# Now should be loadable
loaded_config = load_config(config_file)
assert loaded_config.library_root == default_config.library_root
def test_validation_workflow(self, tmp_path):
"""Test workflow: load config -> validate -> report errors."""
config_file = tmp_path / "config.yaml"
# Create config with some invalid values
config_data = {
'library_root': '/mnt/nas/videos',
'video_extensions': ['mp4', '.mkv'], # First one missing dot
'log_level': 'INVALID'
}
with open(config_file, 'w') as f:
yaml.dump(config_data, f)
# Load config
config = load_config(config_file)
# Validate
errors = validate_config(config)
# Should have errors
assert len(errors) > 0
assert any("must start with '.'" in err for err in errors)
assert any("log_level" in err for err in errors)
tests/test_executor.py
+906
View File
@@ -0,0 +1,906 @@
"""Unit tests for execution engine.
Tests execution mode handling, dry-run simulation, and actual file operations.
"""
import logging
from datetime import datetime
from pathlib import Path
from uuid import uuid4
import pytest
from src.vlm.executor import ExecutionEngine
from src.vlm.models import ExecutionPlan, FileOperation
@pytest.fixture
def temp_test_dir(tmp_path):
"""Create a temporary test directory structure."""
# Create source directory with test files
source_dir = tmp_path / "source"
source_dir.mkdir()
# Create test files
test_file1 = source_dir / "test1.mp4"
test_file1.write_text("test content 1")
test_file2 = source_dir / "test2.mkv"
test_file2.write_text("test content 2")
# Create destination directory
dest_dir = tmp_path / "dest"
dest_dir.mkdir()
return {
"source_dir": source_dir,
"dest_dir": dest_dir,
"test_file1": test_file1,
"test_file2": test_file2,
}
@pytest.fixture
def execution_engine():
"""Create an execution engine instance with test logger."""
logger = logging.getLogger("test_executor")
logger.setLevel(logging.DEBUG)
return ExecutionEngine(logger=logger)
@pytest.fixture
def sample_plan(temp_test_dir):
"""Create a sample execution plan."""
operations = [
FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file1"],
destination_path=temp_test_dir["dest_dir"] / "moved1.mp4",
reason="Organize movie",
has_conflict=False,
conflict_reason=None
),
FileOperation(
operation_type="rename",
source_path=temp_test_dir["test_file2"],
destination_path=temp_test_dir["dest_dir"] / "renamed2.mkv",
reason="Rename series episode",
has_conflict=False,
conflict_reason=None
),
]
return ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=operations,
summary={"move": 1, "rename": 1}
)
class TestExecutionModeHandling:
"""Tests for execution mode parameter handling."""
def test_dry_run_mode_default(self, execution_engine, sample_plan, temp_test_dir):
"""Test that dry-run is the default mode."""
results, summary, rollback_log = execution_engine.execute_plan(sample_plan)
# All operations should succeed in dry-run
assert all(r.success for r in results)
assert len(results) == 2
# Check execution summary
assert summary["successful"] == 2
assert summary["failed"] == 0
assert summary["skipped"] == 0
assert summary["total"] == 2
# No rollback log in dry-run mode
assert rollback_log is None
# Files should not be moved (dry-run doesn't modify files)
assert temp_test_dir["test_file1"].exists()
assert temp_test_dir["test_file2"].exists()
assert not (temp_test_dir["dest_dir"] / "moved1.mp4").exists()
assert not (temp_test_dir["dest_dir"] / "renamed2.mkv").exists()
def test_dry_run_mode_explicit(self, execution_engine, sample_plan, temp_test_dir):
"""Test explicit dry-run mode parameter."""
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="dry-run"
)
# All operations should succeed in dry-run
assert all(r.success for r in results)
# Check execution summary
assert summary["successful"] == 2
assert summary["total"] == 2
# No rollback log in dry-run mode
assert rollback_log is None
# Files should not be moved
assert temp_test_dir["test_file1"].exists()
assert not (temp_test_dir["dest_dir"] / "moved1.mp4").exists()
def test_execute_mode_requires_confirmation(self, execution_engine, sample_plan):
"""Test that execute mode requires explicit confirmation."""
with pytest.raises(ValueError, match="requires explicit confirmation"):
execution_engine.execute_plan(sample_plan, mode="execute")
def test_execute_mode_with_confirmation(self, execution_engine, sample_plan, temp_test_dir):
"""Test execute mode with explicit confirmation."""
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# All operations should succeed
assert all(r.success for r in results)
assert len(results) == 2
# Check execution summary
assert summary["successful"] == 2
assert summary["failed"] == 0
assert summary["total"] == 2
# Rollback log should be created in execute mode
assert rollback_log is not None
assert rollback_log.execution_plan_id == sample_plan.plan_id
assert len(rollback_log.operations) == 2
# Files should be moved
assert not temp_test_dir["test_file1"].exists()
assert not temp_test_dir["test_file2"].exists()
assert (temp_test_dir["dest_dir"] / "moved1.mp4").exists()
assert (temp_test_dir["dest_dir"] / "renamed2.mkv").exists()
def test_invalid_mode_raises_error(self, execution_engine, sample_plan):
"""Test that invalid mode parameter raises ValueError."""
with pytest.raises(ValueError, match="Invalid mode"):
execution_engine.execute_plan(sample_plan, mode="invalid")
class TestDryRunSimulation:
"""Tests for dry-run mode simulation."""
def test_dry_run_logs_operations(self, execution_engine, sample_plan, caplog):
"""Test that dry-run mode logs what would happen."""
caplog.set_level(logging.INFO)
execution_engine.execute_plan(sample_plan, mode="dry-run")
# Check that dry-run operations are logged
assert "[DRY-RUN]" in caplog.text
assert "Would move" in caplog.text or "Would rename" in caplog.text
def test_dry_run_never_modifies_files(self, execution_engine, sample_plan, temp_test_dir):
"""Test that dry-run mode never modifies the file system."""
# Record initial state
initial_files = list(temp_test_dir["source_dir"].iterdir())
# Execute in dry-run mode
results, summary, rollback_log = execution_engine.execute_plan(sample_plan, mode="dry-run")
# Verify no files were moved or modified
final_files = list(temp_test_dir["source_dir"].iterdir())
assert set(initial_files) == set(final_files)
# Verify destination directory is still empty
dest_files = list(temp_test_dir["dest_dir"].iterdir())
assert len(dest_files) == 0
def test_dry_run_handles_no_op_operations(self, execution_engine, temp_test_dir):
"""Test that dry-run mode handles no-op operations correctly."""
no_op_operation = FileOperation(
operation_type="no-op",
source_path=temp_test_dir["test_file1"],
destination_path=None,
reason="Anime file - not organized in v1",
has_conflict=False,
conflict_reason=None
)
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=[no_op_operation],
summary={"no-op": 1}
)
results, summary, _ = execution_engine.execute_plan(plan, mode="dry-run")
assert len(results) == 1
assert results[0].success
assert results[0].operation.operation_type == "no-op"
assert summary["skipped"] == 1
def test_dry_run_handles_conflicts(self, execution_engine, temp_test_dir):
"""Test that dry-run mode handles conflicted operations."""
conflicted_operation = FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file1"],
destination_path=temp_test_dir["dest_dir"] / "conflict.mp4",
reason="Organize movie",
has_conflict=True,
conflict_reason="Destination file already exists"
)
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=[conflicted_operation],
summary={"move": 1}
)
results, summary, _ = execution_engine.execute_plan(plan, mode="dry-run")
assert len(results) == 1
assert not results[0].success
assert "Conflict" in results[0].error_message
assert summary["failed"] == 1
assert summary["skipped"] == 1 # Conflicts are also counted as skipped
class TestExecuteMode:
"""Tests for execute mode with actual file operations."""
def test_execute_mode_moves_files(self, execution_engine, sample_plan, temp_test_dir):
"""Test that execute mode actually moves files."""
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Verify operations succeeded
assert all(r.success for r in results)
assert summary["successful"] == 2
# Verify files were moved
assert not temp_test_dir["test_file1"].exists()
assert (temp_test_dir["dest_dir"] / "moved1.mp4").exists()
# Verify file content is preserved
content = (temp_test_dir["dest_dir"] / "moved1.mp4").read_text()
assert content == "test content 1"
def test_execute_mode_creates_directories(self, execution_engine, temp_test_dir):
"""Test that execute mode creates destination directories."""
nested_dest = temp_test_dir["dest_dir"] / "subdir1" / "subdir2" / "file.mp4"
operation = FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file1"],
destination_path=nested_dest,
reason="Organize with nested structure",
has_conflict=False,
conflict_reason=None
)
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=[operation],
summary={"move": 1}
)
results, summary, _ = execution_engine.execute_plan(
plan,
mode="execute",
confirmed=True
)
# Verify operation succeeded
assert results[0].success
assert summary["successful"] == 1
# Verify nested directories were created
assert nested_dest.exists()
assert nested_dest.parent.exists()
def test_execute_mode_handles_missing_source(self, execution_engine, temp_test_dir):
"""Test that execute mode handles missing source files gracefully."""
missing_file = temp_test_dir["source_dir"] / "nonexistent.mp4"
operation = FileOperation(
operation_type="move",
source_path=missing_file,
destination_path=temp_test_dir["dest_dir"] / "dest.mp4",
reason="Move nonexistent file",
has_conflict=False,
conflict_reason=None
)
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=[operation],
summary={"move": 1}
)
results, summary, _ = execution_engine.execute_plan(
plan,
mode="execute",
confirmed=True
)
# Operation should fail gracefully
assert not results[0].success
assert "does not exist" in results[0].error_message
assert summary["failed"] == 1
def test_execute_mode_skips_conflicts(self, execution_engine, temp_test_dir):
"""Test that execute mode skips conflicted operations."""
# Create a file at the destination
dest_file = temp_test_dir["dest_dir"] / "existing.mp4"
dest_file.write_text("existing content")
conflicted_operation = FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file1"],
destination_path=dest_file,
reason="Move to existing location",
has_conflict=True,
conflict_reason="Destination file already exists"
)
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=[conflicted_operation],
summary={"move": 1}
)
results, summary, _ = execution_engine.execute_plan(
plan,
mode="execute",
confirmed=True
)
# Operation should be skipped
assert not results[0].success
assert "Conflict" in results[0].error_message
assert summary["failed"] == 1
assert summary["skipped"] == 1
# Source file should still exist
assert temp_test_dir["test_file1"].exists()
# Destination file should be unchanged
assert dest_file.read_text() == "existing content"
class TestRollbackLog:
"""Tests for rollback log creation."""
def test_rollback_log_created_in_execute_mode(self, execution_engine, sample_plan):
"""Test that rollback log is created in execute mode."""
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
assert rollback_log is not None
assert rollback_log.log_id is not None
assert rollback_log.execution_plan_id == sample_plan.plan_id
assert len(rollback_log.operations) == 2
def test_rollback_log_not_created_in_dry_run(self, execution_engine, sample_plan):
"""Test that rollback log is not created in dry-run mode."""
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="dry-run"
)
assert rollback_log is None
def test_rollback_log_only_includes_successful_operations(
self, execution_engine, temp_test_dir
):
"""Test that rollback log only includes successful operations."""
# Create a plan with one successful and one failed operation
operations = [
FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file1"],
destination_path=temp_test_dir["dest_dir"] / "success.mp4",
reason="This will succeed",
has_conflict=False,
conflict_reason=None
),
FileOperation(
operation_type="move",
source_path=temp_test_dir["source_dir"] / "nonexistent.mp4",
destination_path=temp_test_dir["dest_dir"] / "fail.mp4",
reason="This will fail",
has_conflict=False,
conflict_reason=None
),
]
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=operations,
summary={"move": 2}
)
results, summary, rollback_log = execution_engine.execute_plan(
plan,
mode="execute",
confirmed=True
)
# Rollback log should only include the successful operation
assert rollback_log is not None
assert len(rollback_log.operations) == 1
assert rollback_log.operations[0].success
class TestExecutionSummary:
"""Tests for execution summary logging."""
def test_execution_summary_counts(self, execution_engine, temp_test_dir, caplog):
"""Test that execution summary includes correct counts."""
caplog.set_level(logging.INFO)
operations = [
FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file1"],
destination_path=temp_test_dir["dest_dir"] / "success.mp4",
reason="Successful move",
has_conflict=False,
conflict_reason=None
),
FileOperation(
operation_type="no-op",
source_path=temp_test_dir["test_file2"],
destination_path=None,
reason="Anime file",
has_conflict=False,
conflict_reason=None
),
FileOperation(
operation_type="move",
source_path=temp_test_dir["source_dir"] / "missing.mp4",
destination_path=temp_test_dir["dest_dir"] / "fail.mp4",
reason="This will fail",
has_conflict=False,
conflict_reason=None
),
]
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=operations,
summary={"move": 2, "no-op": 1}
)
results, summary, rollback_log = execution_engine.execute_plan(plan, mode="execute", confirmed=True)
# Check summary structure
assert summary["successful"] == 2 # 1 successful move + 1 no-op
assert summary["failed"] == 1 # 1 failed move
assert summary["skipped"] == 1 # 1 no-op
assert summary["total"] == 3
# Check summary in logs
assert "Execution summary" in caplog.text
assert "successful" in caplog.text
assert "failed" in caplog.text
assert "skipped" in caplog.text
class TestRollbackLogSaving:
"""Tests for saving rollback logs to disk."""
def test_save_rollback_log_creates_file(self, execution_engine, sample_plan, tmp_path):
"""Test that save_rollback_log creates a JSON file."""
# Execute plan to get rollback log
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Save rollback log
output_path = tmp_path / "rollback" / "test_rollback.json"
execution_engine.save_rollback_log(rollback_log, output_path)
# Verify file was created
assert output_path.exists()
assert output_path.is_file()
def test_save_rollback_log_json_structure(self, execution_engine, sample_plan, tmp_path):
"""Test that saved rollback log has correct JSON structure."""
# Execute plan to get rollback log
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Save rollback log
output_path = tmp_path / "rollback" / "test_rollback.json"
execution_engine.save_rollback_log(rollback_log, output_path)
# Load and verify JSON structure
import json
with open(output_path, 'r', encoding='utf-8') as f:
data = json.load(f)
# Check required fields
assert "log_id" in data
assert "execution_plan_id" in data
assert "executed_at" in data
assert "operations" in data
# Check operations structure
assert len(data["operations"]) == 2
for op in data["operations"]:
assert "operation_type" in op
assert "source_path" in op
assert "destination_path" in op
assert "reason" in op
assert "success" in op
assert "executed_at" in op
def test_save_rollback_log_includes_timestamps(self, execution_engine, sample_plan, tmp_path):
"""Test that rollback log includes ISO format timestamps."""
# Execute plan to get rollback log
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Save rollback log
output_path = tmp_path / "rollback" / "test_rollback.json"
execution_engine.save_rollback_log(rollback_log, output_path)
# Load and verify timestamps
import json
with open(output_path, 'r', encoding='utf-8') as f:
data = json.load(f)
# Verify timestamps are in ISO format
from datetime import datetime
executed_at = datetime.fromisoformat(data["executed_at"])
assert executed_at is not None
for op in data["operations"]:
op_executed_at = datetime.fromisoformat(op["executed_at"])
assert op_executed_at is not None
class TestRollbackExecution:
"""Tests for rollback execution functionality."""
def test_load_rollback_log_from_file(self, execution_engine, sample_plan, tmp_path):
"""Test loading a rollback log from a JSON file."""
# Execute plan and save rollback log
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
log_path = tmp_path / "rollback.json"
execution_engine.save_rollback_log(rollback_log, log_path)
# Load the rollback log
loaded_log = execution_engine.load_rollback_log(log_path)
# Verify loaded log matches original
assert loaded_log.log_id == rollback_log.log_id
assert loaded_log.execution_plan_id == rollback_log.execution_plan_id
assert len(loaded_log.operations) == len(rollback_log.operations)
def test_load_rollback_log_missing_file(self, execution_engine, tmp_path):
"""Test that loading a missing rollback log raises FileNotFoundError."""
missing_path = tmp_path / "nonexistent.json"
with pytest.raises(FileNotFoundError, match="Rollback log not found"):
execution_engine.load_rollback_log(missing_path)
def test_load_rollback_log_invalid_json(self, execution_engine, tmp_path):
"""Test that loading invalid JSON raises ValueError."""
invalid_path = tmp_path / "invalid.json"
invalid_path.write_text("not valid json {")
with pytest.raises(ValueError, match="Invalid rollback log format"):
execution_engine.load_rollback_log(invalid_path)
def test_rollback_reverses_operations(self, execution_engine, sample_plan, temp_test_dir):
"""Test that rollback reverses file operations."""
# Execute plan to move files
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Verify files were moved
assert not temp_test_dir["test_file1"].exists()
assert (temp_test_dir["dest_dir"] / "moved1.mp4").exists()
# Perform rollback
rollback_results, rollback_summary = execution_engine.rollback(rollback_log)
# Verify files were moved back
assert temp_test_dir["test_file1"].exists()
assert not (temp_test_dir["dest_dir"] / "moved1.mp4").exists()
# Verify rollback summary
assert rollback_summary["successful"] == 2
assert rollback_summary["failed"] == 0
assert rollback_summary["total"] == 2
def test_rollback_lifo_order(self, execution_engine, temp_test_dir):
"""Test that rollback processes operations in LIFO order."""
# Create a plan with multiple operations
operations = [
FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file1"],
destination_path=temp_test_dir["dest_dir"] / "first.mp4",
reason="First operation",
has_conflict=False,
conflict_reason=None
),
FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file2"],
destination_path=temp_test_dir["dest_dir"] / "second.mkv",
reason="Second operation",
has_conflict=False,
conflict_reason=None
),
]
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=operations,
summary={"move": 2}
)
# Execute and rollback
results, summary, rollback_log = execution_engine.execute_plan(
plan,
mode="execute",
confirmed=True
)
rollback_results, rollback_summary = execution_engine.rollback(rollback_log)
# Verify LIFO order: second operation should be rolled back first
# Both should succeed regardless of order
assert all(r.success for r in rollback_results)
assert len(rollback_results) == 2
def test_rollback_handles_missing_destination(self, execution_engine, sample_plan, temp_test_dir):
"""Test that rollback handles missing destination files gracefully."""
# Execute plan
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Manually delete one of the destination files
(temp_test_dir["dest_dir"] / "moved1.mp4").unlink()
# Perform rollback
rollback_results, rollback_summary = execution_engine.rollback(rollback_log)
# One rollback should fail, one should succeed
assert rollback_summary["successful"] == 1
assert rollback_summary["failed"] == 1
assert rollback_summary["total"] == 2
# The file that wasn't deleted should be rolled back
assert temp_test_dir["test_file2"].exists()
def test_rollback_continues_after_failure(self, execution_engine, sample_plan, temp_test_dir):
"""Test that rollback continues processing after encountering failures."""
# Execute plan
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Delete one destination file to cause a rollback failure
(temp_test_dir["dest_dir"] / "moved1.mp4").unlink()
# Perform rollback
rollback_results, rollback_summary = execution_engine.rollback(rollback_log)
# Verify all operations were attempted (not halted by failure)
assert len(rollback_results) == 2
# One should fail, one should succeed
failed_count = sum(1 for r in rollback_results if not r.success)
success_count = sum(1 for r in rollback_results if r.success)
assert failed_count == 1
assert success_count == 1
def test_rollback_skips_no_op_operations(self, execution_engine, temp_test_dir):
"""Test that rollback skips no-op operations."""
operations = [
FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file1"],
destination_path=temp_test_dir["dest_dir"] / "moved.mp4",
reason="Move file",
has_conflict=False,
conflict_reason=None
),
FileOperation(
operation_type="no-op",
source_path=temp_test_dir["test_file2"],
destination_path=None,
reason="Anime file",
has_conflict=False,
conflict_reason=None
),
]
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=operations,
summary={"move": 1, "no-op": 1}
)
# Execute and rollback
results, summary, rollback_log = execution_engine.execute_plan(
plan,
mode="execute",
confirmed=True
)
rollback_results, rollback_summary = execution_engine.rollback(rollback_log)
# Both operations are in rollback log, but no-op is skipped during rollback
assert len(rollback_results) == 2
assert rollback_summary["skipped"] == 1 # no-op is skipped during rollback
assert rollback_summary["successful"] == 2 # Both succeed (no-op succeeds trivially)
def test_rollback_preserves_file_content(self, execution_engine, sample_plan, temp_test_dir):
"""Test that rollback preserves file content."""
original_content = temp_test_dir["test_file1"].read_text()
# Execute plan
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Perform rollback
rollback_results, rollback_summary = execution_engine.rollback(rollback_log)
# Verify file content is preserved
restored_content = temp_test_dir["test_file1"].read_text()
assert restored_content == original_content
def test_rollback_idempotence(self, execution_engine, sample_plan, temp_test_dir):
"""Test that running rollback multiple times produces the same result."""
# Execute plan
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# First rollback
rollback_results1, rollback_summary1 = execution_engine.rollback(rollback_log)
# Verify files are back
assert temp_test_dir["test_file1"].exists()
assert temp_test_dir["test_file2"].exists()
# Execute plan again
results2, summary2, rollback_log2 = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Second rollback
rollback_results2, rollback_summary2 = execution_engine.rollback(rollback_log2)
# Both rollbacks should have same results
assert rollback_summary1["successful"] == rollback_summary2["successful"]
assert rollback_summary1["failed"] == rollback_summary2["failed"]
# Files should be in same state
assert temp_test_dir["test_file1"].exists()
assert temp_test_dir["test_file2"].exists()
def test_rollback_logs_operations(self, execution_engine, sample_plan, caplog):
"""Test that rollback logs all operations."""
caplog.set_level(logging.INFO)
# Execute plan
results, summary, rollback_log = execution_engine.execute_plan(
sample_plan,
mode="execute",
confirmed=True
)
# Clear logs
caplog.clear()
# Perform rollback
rollback_results, rollback_summary = execution_engine.rollback(rollback_log)
# Verify rollback operations are logged
assert "Starting rollback" in caplog.text
assert "LIFO order" in caplog.text
assert "Successfully rolled back" in caplog.text
assert "Rollback summary" in caplog.text
def test_rollback_summary_accuracy(self, execution_engine, temp_test_dir):
"""Test that rollback summary contains accurate counts."""
# Create a plan with operations that will have mixed results
operations = [
FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file1"],
destination_path=temp_test_dir["dest_dir"] / "file1.mp4",
reason="Move file 1",
has_conflict=False,
conflict_reason=None
),
FileOperation(
operation_type="move",
source_path=temp_test_dir["test_file2"],
destination_path=temp_test_dir["dest_dir"] / "file2.mkv",
reason="Move file 2",
has_conflict=False,
conflict_reason=None
),
]
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
operations=operations,
summary={"move": 2}
)
# Execute plan
results, summary, rollback_log = execution_engine.execute_plan(
plan,
mode="execute",
confirmed=True
)
# Delete one file to cause partial rollback failure
(temp_test_dir["dest_dir"] / "file1.mp4").unlink()
# Perform rollback
rollback_results, rollback_summary = execution_engine.rollback(rollback_log)
# Verify summary accuracy
assert rollback_summary["total"] == 2
assert rollback_summary["successful"] == 1
assert rollback_summary["failed"] == 1
assert rollback_summary["skipped"] == 0
# Verify counts match actual results
actual_success = sum(1 for r in rollback_results if r.success)
actual_failed = sum(1 for r in rollback_results if not r.success)
assert rollback_summary["successful"] == actual_success
assert rollback_summary["failed"] == actual_failed
tests/test_logging.py
+292
View File
@@ -0,0 +1,292 @@
"""Unit tests for logging configuration."""
import logging
import tempfile
from pathlib import Path
import pytest
from vlm.logging_config import (
setup_logging,
get_logger,
log_operation,
MAX_LOG_SIZE,
)
class TestLoggingSetup:
"""Test logging configuration setup."""
def test_setup_logging_creates_logger(self, tmp_path):
"""Test that setup_logging creates a configured logger."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
assert logger is not None
assert logger.name == "vlm"
assert logger.level == logging.DEBUG
def test_setup_logging_creates_log_directory(self, tmp_path):
"""Test that setup_logging creates the log directory."""
log_dir = tmp_path / "logs"
assert not log_dir.exists()
setup_logging(log_level="INFO", log_dir=log_dir)
assert log_dir.exists()
assert log_dir.is_dir()
def test_setup_logging_creates_log_file(self, tmp_path):
"""Test that setup_logging creates the log file."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
# Log a message to ensure file is created
logger.info("Test message")
log_file = tmp_path / "vlm.log"
assert log_file.exists()
def test_setup_logging_with_custom_log_file(self, tmp_path):
"""Test that setup_logging accepts custom log file name."""
logger = setup_logging(
log_level="INFO",
log_dir=tmp_path,
log_file="custom.log"
)
logger.info("Test message")
log_file = tmp_path / "custom.log"
assert log_file.exists()
def test_setup_logging_invalid_level_raises_error(self, tmp_path):
"""Test that invalid log level raises ValueError."""
with pytest.raises(ValueError, match="Invalid log level"):
setup_logging(log_level="INVALID", log_dir=tmp_path)
def test_setup_logging_accepts_valid_levels(self, tmp_path):
"""Test that all valid log levels are accepted."""
valid_levels = ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
for level in valid_levels:
logger = setup_logging(log_level=level, log_dir=tmp_path)
assert logger is not None
class TestDualOutput:
"""Test dual output to console and file."""
def test_console_handler_respects_log_level(self, tmp_path):
"""Test that console handler only logs INFO+ messages."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
logger.debug("Debug message")
logger.info("Info message")
logger.warning("Warning message")
# Check file contains all messages (DEBUG+)
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
assert "Debug message" in log_content
assert "Info message" in log_content
assert "Warning message" in log_content
# Verify console handler has INFO level
console_handler = [h for h in logger.handlers if isinstance(h, logging.StreamHandler) and not isinstance(h, logging.handlers.RotatingFileHandler)][0]
assert console_handler.level == logging.INFO
def test_file_handler_logs_all_levels(self, tmp_path):
"""Test that file handler logs DEBUG+ messages."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
logger.debug("Debug message")
logger.info("Info message")
logger.warning("Warning message")
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
# File should contain all levels
assert "Debug message" in log_content
assert "Info message" in log_content
assert "Warning message" in log_content
class TestLogFormat:
"""Test log message formatting."""
def test_log_includes_timestamp(self, tmp_path):
"""Test that log entries include timestamps."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
logger.info("Test message")
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
# Check for timestamp format (YYYY-MM-DD HH:MM:SS)
import re
timestamp_pattern = r'\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}'
assert re.search(timestamp_pattern, log_content)
def test_log_includes_level(self, tmp_path):
"""Test that log entries include log level."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
logger.info("Test message")
logger.warning("Warning message")
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
assert "INFO" in log_content
assert "WARNING" in log_content
def test_log_includes_operation_type(self, tmp_path):
"""Test that log entries include operation type."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
log_operation(logger, logging.INFO, "Test message", operation_type="scan")
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
assert "[scan]" in log_content
def test_log_includes_file_path(self, tmp_path):
"""Test that log entries include file paths when provided."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
test_path = Path("/test/path/file.mp4")
log_operation(
logger,
logging.INFO,
"Processing file",
operation_type="parse",
file_path=test_path
)
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
assert str(test_path) in log_content
def test_log_without_file_path(self, tmp_path):
"""Test that log entries work without file path."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
log_operation(logger, logging.INFO, "Test message", operation_type="general")
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
assert "Test message" in log_content
assert "[general]" in log_content
class TestLogRotation:
"""Test log rotation at 10MB threshold."""
def test_log_rotation_creates_backup(self, tmp_path):
"""Test that log rotation creates backup files."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
# Write enough data to trigger rotation (slightly over 10MB)
large_message = "x" * 1024 # 1KB message
num_messages = (MAX_LOG_SIZE // 1024) + 100 # Exceed 10MB
for i in range(num_messages):
logger.info(f"{large_message} - {i}")
# Check that backup file was created
log_file = tmp_path / "vlm.log"
backup_file = tmp_path / "vlm.log.1"
assert log_file.exists()
assert backup_file.exists()
def test_log_file_size_stays_under_limit(self, tmp_path):
"""Test that log file size stays under 10MB after rotation."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
# Write enough data to trigger rotation
large_message = "x" * 1024 # 1KB message
num_messages = (MAX_LOG_SIZE // 1024) + 100 # Exceed 10MB
for i in range(num_messages):
logger.info(f"{large_message} - {i}")
log_file = tmp_path / "vlm.log"
# Current log file should be smaller than MAX_LOG_SIZE
assert log_file.stat().st_size < MAX_LOG_SIZE
class TestGetLogger:
"""Test get_logger function."""
def test_get_logger_returns_logger(self):
"""Test that get_logger returns a logger instance."""
logger = get_logger()
assert logger is not None
assert logger.name == "vlm"
def test_get_logger_creates_default_config(self):
"""Test that get_logger creates default configuration if needed."""
# Clear any existing handlers
logger = logging.getLogger("vlm")
logger.handlers.clear()
# Get logger should set up default configuration
logger = get_logger()
assert len(logger.handlers) > 0
class TestLogOperation:
"""Test log_operation helper function."""
def test_log_operation_with_all_parameters(self, tmp_path):
"""Test log_operation with all parameters."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
test_path = Path("/test/file.mp4")
log_operation(
logger,
logging.INFO,
"Processing file",
operation_type="execute",
file_path=test_path
)
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
assert "Processing file" in log_content
assert "[execute]" in log_content
assert str(test_path) in log_content
def test_log_operation_with_minimal_parameters(self, tmp_path):
"""Test log_operation with minimal parameters."""
logger = setup_logging(log_level="INFO", log_dir=tmp_path)
log_operation(logger, logging.INFO, "Simple message")
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
assert "Simple message" in log_content
assert "[general]" in log_content
def test_log_operation_different_levels(self, tmp_path):
"""Test log_operation with different log levels."""
logger = setup_logging(log_level="DEBUG", log_dir=tmp_path)
log_operation(logger, logging.DEBUG, "Debug message", operation_type="scan")
log_operation(logger, logging.INFO, "Info message", operation_type="parse")
log_operation(logger, logging.WARNING, "Warning message", operation_type="execute")
log_operation(logger, logging.ERROR, "Error message", operation_type="rollback")
log_file = tmp_path / "vlm.log"
log_content = log_file.read_text()
assert "DEBUG" in log_content
assert "INFO" in log_content
assert "WARNING" in log_content
assert "ERROR" in log_content
tests/test_parser.py
+841
View File
@@ -0,0 +1,841 @@
"""Tests for the identity parser module."""
import pytest
from vlm.parser import (
parse_movie,
parse_series,
normalize_title,
remove_quality_tags,
remove_release_groups,
)
class TestMovieParser:
"""Tests for movie identity parsing."""
def test_parse_movie_with_parentheses_year(self):
"""Test parsing movie with year in parentheses."""
result = parse_movie("The Matrix (1999).mkv")
assert result.title == "The Matrix"
assert result.year == 1999
assert result.confidence == 0.9
assert result.needs_review is False
assert result.original_filename == "The Matrix (1999).mkv"
def test_parse_movie_with_dot_year(self):
"""Test parsing movie with dot-separated year."""
result = parse_movie("Inception.2010.1080p.BluRay.mkv")
assert result.title == "Inception"
assert result.year == 2010
assert result.confidence == 0.9
assert result.needs_review is False
def test_parse_movie_with_dash_year(self):
"""Test parsing movie with dash-separated year."""
result = parse_movie("The Godfather - 1972.mp4")
assert result.title == "The Godfather"
assert result.year == 1972
assert result.confidence == 0.7
assert result.needs_review is False
def test_parse_movie_with_space_year(self):
"""Test parsing movie with space-separated year."""
result = parse_movie("Pulp Fiction 1994.avi")
assert result.title == "Pulp Fiction"
assert result.year == 1994
assert result.confidence == 0.7
assert result.needs_review is False
def test_parse_movie_with_quality_tags(self):
"""Test parsing movie with quality tags removed."""
result = parse_movie("Interstellar.2014.1080p.BluRay.x264.mkv")
assert result.title == "Interstellar"
assert result.year == 2014
assert "1080p" not in result.title
assert "BluRay" not in result.title
assert "x264" not in result.title
def test_parse_movie_with_release_group(self):
"""Test parsing movie with release group tags removed."""
result = parse_movie("The Shawshank Redemption (1994) [RARBG].mkv")
assert result.title == "The Shawshank Redemption"
assert result.year == 1994
assert "RARBG" not in result.title
def test_parse_movie_with_yts_release_group(self):
"""Test parsing movie with YTS release group."""
result = parse_movie("Fight Club (1999) (YTS).mp4")
assert result.title == "Fight Club"
assert result.year == 1999
# YTS should be removed
def test_parse_movie_without_year(self):
"""Test parsing movie without extractable year."""
result = parse_movie("Some Random Movie.mkv")
assert result.title == "Some Random Movie"
assert result.year is None
assert result.needs_review is True
assert result.confidence == 0.3
def test_parse_movie_with_multiple_quality_tags(self):
"""Test parsing movie with multiple quality indicators."""
result = parse_movie("Avatar.2009.2160p.4K.UHD.BluRay.x265.10bit.mkv")
assert result.title == "Avatar"
assert result.year == 2009
assert "2160p" not in result.title
assert "4K" not in result.title
assert "UHD" not in result.title
def test_parse_movie_with_web_dl(self):
"""Test parsing movie with WEB-DL tag."""
result = parse_movie("The Dark Knight (2008) WEB-DL 1080p.mkv")
assert result.title == "The Dark Knight"
assert result.year == 2008
assert "WEB-DL" not in result.title
def test_parse_movie_with_hdtv(self):
"""Test parsing movie with HDTV tag."""
result = parse_movie("Movie Name 2015 HDTV 720p.avi")
assert result.title == "Movie Name"
assert result.year == 2015
assert "HDTV" not in result.title
def test_parse_movie_with_dots_in_title(self):
"""Test parsing movie with dots in title."""
result = parse_movie("The.Lord.of.the.Rings.2001.mkv")
assert result.title == "The Lord Of The Rings"
assert result.year == 2001
def test_parse_movie_with_underscores(self):
"""Test parsing movie with underscores in title."""
result = parse_movie("Star_Wars_Episode_IV (1977).mp4")
assert result.title == "Star Wars Episode Iv"
assert result.year == 1977
def test_parse_movie_preserves_original_filename(self):
"""Test that original filename is preserved."""
original = "Complex.Movie.Name.2020.1080p.BluRay.x264.[RARBG].mkv"
result = parse_movie(original)
assert result.original_filename == original
class TestNormalizeTitle:
"""Tests for title normalization."""
def test_normalize_removes_dots(self):
"""Test that dots are replaced with spaces."""
assert normalize_title("The.Matrix") == "The Matrix"
def test_normalize_removes_underscores(self):
"""Test that underscores are replaced with spaces."""
assert normalize_title("Star_Wars") == "Star Wars"
def test_normalize_removes_extra_whitespace(self):
"""Test that extra whitespace is removed."""
assert normalize_title("The Matrix Reloaded") == "The Matrix Reloaded"
def test_normalize_applies_title_case(self):
"""Test that title case is applied."""
assert normalize_title("the matrix") == "The Matrix"
assert normalize_title("THE MATRIX") == "The Matrix"
def test_normalize_idempotence(self):
"""Test that normalizing multiple times produces same result."""
title = "The.Matrix.Reloaded"
normalized_once = normalize_title(title)
normalized_twice = normalize_title(normalized_once)
assert normalized_once == normalized_twice
class TestRemoveQualityTags:
"""Tests for quality tag removal."""
def test_remove_resolution_tags(self):
"""Test removal of resolution tags."""
assert "1080p" not in remove_quality_tags("Movie 1080p")
assert "720p" not in remove_quality_tags("Movie 720p")
assert "4K" not in remove_quality_tags("Movie 4K")
def test_remove_source_tags(self):
"""Test removal of source tags."""
assert "BluRay" not in remove_quality_tags("Movie BluRay")
assert "WEB-DL" not in remove_quality_tags("Movie WEB-DL")
assert "HDTV" not in remove_quality_tags("Movie HDTV")
def test_remove_codec_tags(self):
"""Test removal of codec tags."""
assert "x264" not in remove_quality_tags("Movie x264")
assert "x265" not in remove_quality_tags("Movie x265")
assert "HEVC" not in remove_quality_tags("Movie HEVC")
def test_case_insensitive_removal(self):
"""Test that removal is case-insensitive."""
assert "bluray" not in remove_quality_tags("Movie bluray").lower()
assert "BLURAY" not in remove_quality_tags("Movie BLURAY").upper()
class TestRemoveReleaseGroups:
"""Tests for release group removal."""
def test_remove_bracketed_groups(self):
"""Test removal of bracketed release groups."""
assert "[RARBG]" not in remove_release_groups("Movie [RARBG]")
assert "[YTS]" not in remove_release_groups("Movie [YTS]")
def test_preserve_year_in_parentheses(self):
"""Test that years in parentheses are preserved."""
result = remove_release_groups("Movie (2020)")
# Years in parentheses should be preserved - they're handled by the parser
assert "(2020)" in result
class TestSeriesParser:
"""Tests for series identity parsing."""
def test_parse_series_sxxeyy_format(self):
"""Test parsing series with SXXEYY format."""
result = parse_series("Breaking Bad S01E01.mkv")
assert result.title == "Breaking Bad"
assert result.season == 1
assert result.episodes == [1]
assert result.confidence == 0.9
assert result.needs_review is False
assert result.original_filename == "Breaking Bad S01E01.mkv"
def test_parse_series_sxxeyy_lowercase(self):
"""Test parsing series with lowercase sxxeyy format."""
result = parse_series("Game of Thrones s02e05.mkv")
assert result.title == "Game Of Thrones"
assert result.season == 2
assert result.episodes == [5]
assert result.confidence == 0.9
assert result.needs_review is False
def test_parse_series_xxxyy_format(self):
"""Test parsing series with XXxYY format."""
result = parse_series("The Office 1x01.mp4")
assert result.title == "The Office"
assert result.season == 1
assert result.episodes == [1]
assert result.confidence == 0.9
assert result.needs_review is False
def test_parse_series_season_episode_format(self):
"""Test parsing series with Season X Episode Y format."""
result = parse_series("Friends Season 1 Episode 1.avi")
assert result.title == "Friends"
assert result.season == 1
assert result.episodes == [1]
assert result.confidence == 0.7
assert result.needs_review is False
def test_parse_series_multi_episode_dash(self):
"""Test parsing multi-episode file with dash separator."""
result = parse_series("The Wire S01E01-E02.mkv")
assert result.title == "The Wire"
assert result.season == 1
assert result.episodes == [1, 2]
assert result.confidence == 0.9
assert result.needs_review is False
def test_parse_series_multi_episode_no_dash(self):
"""Test parsing multi-episode file without dash."""
result = parse_series("Stranger Things S01E01E02.mkv")
assert result.title == "Stranger Things"
assert result.season == 1
assert result.episodes == [1, 2]
assert result.confidence == 0.9
assert result.needs_review is False
def test_parse_series_multi_episode_three_episodes(self):
"""Test parsing file with three episodes."""
result = parse_series("Show Name S02E01E02E03.mkv")
assert result.title == "Show Name"
assert result.season == 2
assert result.episodes == [1, 2, 3]
assert result.confidence == 0.9
assert result.needs_review is False
def test_parse_series_with_quality_tags(self):
"""Test parsing series with quality tags removed."""
result = parse_series("The Mandalorian S01E01 1080p WEB-DL x264.mkv")
assert result.title == "The Mandalorian"
assert result.season == 1
assert result.episodes == [1]
assert "1080p" not in result.title
assert "WEB-DL" not in result.title
assert "x264" not in result.title
def test_parse_series_with_release_group(self):
"""Test parsing series with release group removed."""
result = parse_series("Westworld S01E01 [RARBG].mkv")
assert result.title == "Westworld"
assert result.season == 1
assert result.episodes == [1]
assert "RARBG" not in result.title
def test_parse_series_with_dots_in_title(self):
"""Test parsing series with dots in title."""
result = parse_series("The.Walking.Dead.S05E10.mkv")
assert result.title == "The Walking Dead"
assert result.season == 5
assert result.episodes == [10]
def test_parse_series_with_underscores(self):
"""Test parsing series with underscores in title."""
result = parse_series("Better_Call_Saul_S02E03.mp4")
assert result.title == "Better Call Saul"
assert result.season == 2
assert result.episodes == [3]
def test_parse_series_without_season(self):
"""Test parsing series without extractable season."""
result = parse_series("Random Show Episode.mkv")
assert result.title == "Random Show Episode"
assert result.season is None
assert result.episodes == []
assert result.needs_review is True
assert result.confidence == 0.3
def test_parse_series_without_episode(self):
"""Test parsing series without extractable episode."""
result = parse_series("Some Series Name.mkv")
assert result.title == "Some Series Name"
assert result.season is None
assert result.episodes == []
assert result.needs_review is True
def test_parse_series_double_digit_season_episode(self):
"""Test parsing series with double-digit season and episode."""
result = parse_series("Doctor Who S12E10.mkv")
assert result.title == "Doctor Who"
assert result.season == 12
assert result.episodes == [10]
assert result.needs_review is False
def test_parse_series_complex_filename(self):
"""Test parsing series with complex filename."""
result = parse_series("The.Expanse.S03E05.1080p.BluRay.x264.[YTS].mkv")
assert result.title == "The Expanse"
assert result.season == 3
assert result.episodes == [5]
assert "1080p" not in result.title
assert "BluRay" not in result.title
assert "YTS" not in result.title
def test_parse_series_preserves_original_filename(self):
"""Test that original filename is preserved."""
original = "Complex.Series.Name.S01E01.1080p.WEB-DL.[RARBG].mkv"
result = parse_series(original)
assert result.original_filename == original
def test_parse_series_with_multiple_quality_tags(self):
"""Test parsing series with multiple quality indicators."""
result = parse_series("Series.Name.S01E01.2160p.4K.UHD.WEB-DL.x265.10bit.mkv")
assert result.title == "Series Name"
assert result.season == 1
assert result.episodes == [1]
assert "2160p" not in result.title
assert "4K" not in result.title
assert "UHD" not in result.title
def test_parse_series_xxxyy_double_digits(self):
"""Test parsing series with XXxYY format and double digits."""
result = parse_series("Show Name 10x15.mp4")
assert result.title == "Show Name"
assert result.season == 10
assert result.episodes == [15]
assert result.confidence == 0.9
class TestEpisodeGrouping:
"""Tests for episode grouping functionality."""
def test_group_episodes_by_title_and_season(self):
"""Test grouping episodes by normalized title and season."""
from vlm.parser import group_episodes
episodes = [
parse_series("Breaking Bad S01E01.mkv"),
parse_series("Breaking Bad S01E02.mkv"),
parse_series("Breaking Bad S02E01.mkv"),
parse_series("Game of Thrones S01E01.mkv"),
]
groups = group_episodes(episodes)
# Should have 3 groups: Breaking Bad S01, Breaking Bad S02, Game of Thrones S01
assert len(groups) == 3
assert ("Breaking Bad", 1) in groups
assert ("Breaking Bad", 2) in groups
assert ("Game Of Thrones", 1) in groups
# Breaking Bad S01 should have 2 episodes
assert len(groups[("Breaking Bad", 1)]) == 2
# Breaking Bad S02 should have 1 episode
assert len(groups[("Breaking Bad", 2)]) == 1
# Game of Thrones S01 should have 1 episode
assert len(groups[("Game Of Thrones", 1)]) == 1
def test_group_episodes_excludes_none_season(self):
"""Test that episodes with season=None are excluded from grouping."""
from vlm.parser import group_episodes
episodes = [
parse_series("Breaking Bad S01E01.mkv"),
parse_series("Random Show Episode.mkv"), # No season
parse_series("Breaking Bad S01E02.mkv"),
]
groups = group_episodes(episodes)
# Should only have 1 group (Breaking Bad S01)
assert len(groups) == 1
assert ("Breaking Bad", 1) in groups
assert len(groups[("Breaking Bad", 1)]) == 2
def test_group_episodes_empty_list(self):
"""Test grouping with empty episode list."""
from vlm.parser import group_episodes
groups = group_episodes([])
assert len(groups) == 0
assert groups == {}
def test_group_episodes_single_episode(self):
"""Test grouping with single episode."""
from vlm.parser import group_episodes
episodes = [parse_series("The Office S01E01.mkv")]
groups = group_episodes(episodes)
assert len(groups) == 1
assert ("The Office", 1) in groups
assert len(groups[("The Office", 1)]) == 1
def test_group_episodes_same_title_different_seasons(self):
"""Test grouping episodes from same series but different seasons."""
from vlm.parser import group_episodes
episodes = [
parse_series("Friends S01E01.mkv"),
parse_series("Friends S01E02.mkv"),
parse_series("Friends S02E01.mkv"),
parse_series("Friends S02E02.mkv"),
parse_series("Friends S03E01.mkv"),
]
groups = group_episodes(episodes)
# Should have 3 groups (one per season)
assert len(groups) == 3
assert ("Friends", 1) in groups
assert ("Friends", 2) in groups
assert ("Friends", 3) in groups
# Check episode counts per season
assert len(groups[("Friends", 1)]) == 2
assert len(groups[("Friends", 2)]) == 2
assert len(groups[("Friends", 3)]) == 1
def test_group_episodes_normalized_titles(self):
"""Test that grouping uses normalized titles."""
from vlm.parser import group_episodes
episodes = [
parse_series("The.Walking.Dead.S01E01.mkv"), # Dots
parse_series("The_Walking_Dead_S01E02.mkv"), # Underscores
parse_series("The Walking Dead S01E03.mkv"), # Spaces
]
groups = group_episodes(episodes)
# All should be grouped together under normalized title
assert len(groups) == 1
assert ("The Walking Dead", 1) in groups
assert len(groups[("The Walking Dead", 1)]) == 3
def test_group_episodes_multi_episode_files(self):
"""Test grouping with multi-episode files."""
from vlm.parser import group_episodes
episodes = [
parse_series("Show Name S01E01.mkv"),
parse_series("Show Name S01E02E03.mkv"), # Multi-episode
parse_series("Show Name S01E04.mkv"),
]
groups = group_episodes(episodes)
# All should be in same group
assert len(groups) == 1
assert ("Show Name", 1) in groups
assert len(groups[("Show Name", 1)]) == 3
def test_group_episodes_preserves_original_objects(self):
"""Test that grouping preserves original SeriesIdentity objects."""
from vlm.parser import group_episodes
episodes = [
parse_series("Breaking Bad S01E01.mkv"),
parse_series("Breaking Bad S01E02.mkv"),
]
groups = group_episodes(episodes)
# Check that original objects are preserved
grouped_episodes = groups[("Breaking Bad", 1)]
assert grouped_episodes[0].original_filename == "Breaking Bad S01E01.mkv"
assert grouped_episodes[1].original_filename == "Breaking Bad S01E02.mkv"
assert grouped_episodes[0].episodes == [1]
assert grouped_episodes[1].episodes == [2]
# ============================================================================
# Property-Based Tests
# ============================================================================
from hypothesis import given, strategies as st, assume, settings
from vlm.parser import group_episodes
import logging
# Custom strategies for generating test data
@st.composite
def movie_filename_strategy(draw):
"""Generate movie filenames matching common patterns."""
# Generate a title (1-5 words)
title_words = draw(st.lists(
st.text(
alphabet=st.characters(whitelist_categories=('Lu', 'Ll'), min_codepoint=65, max_codepoint=122),
min_size=3,
max_size=10
),
min_size=1,
max_size=5
))
title = ' '.join(title_words)
# Generate a year (1900-2030)
year = draw(st.integers(min_value=1900, max_value=2030))
# Choose a pattern
pattern = draw(st.sampled_from([
'parentheses', # Title (Year)
'dot', # Title.Year
'dash', # Title - Year
'space' # Title Year
]))
# Choose optional quality tags
quality_tags = draw(st.lists(
st.sampled_from(['1080p', '720p', '4K', 'BluRay', 'WEB-DL', 'HDTV', 'x264', 'x265']),
max_size=3
))
# Choose optional release group
release_group = draw(st.one_of(
st.none(),
st.sampled_from(['[RARBG]', '[YTS]', '[YIFY]'])
))
# Choose extension
ext = draw(st.sampled_from(['.mp4', '.mkv', '.avi', '.mov']))
# Build filename based on pattern
if pattern == 'parentheses':
filename = f"{title} ({year})"
elif pattern == 'dot':
filename = title.replace(' ', '.') + f".{year}"
elif pattern == 'dash':
filename = f"{title} - {year}"
else: # space
filename = f"{title} {year}"
# Add quality tags
if quality_tags:
filename += ' ' + ' '.join(quality_tags)
# Add release group
if release_group:
filename += ' ' + release_group
# Add extension
filename += ext
return filename, title, year
@st.composite
def series_filename_strategy(draw):
"""Generate series filenames matching common patterns."""
# Generate a title (1-5 words)
title_words = draw(st.lists(
st.text(
alphabet=st.characters(whitelist_categories=('Lu', 'Ll'), min_codepoint=65, max_codepoint=122),
min_size=3,
max_size=10
),
min_size=1,
max_size=5
))
title = ' '.join(title_words)
# Generate season and episode
season = draw(st.integers(min_value=1, max_value=20))
episode = draw(st.integers(min_value=1, max_value=30))
# Choose a pattern
pattern = draw(st.sampled_from([
'SXXEYY', # S01E01
'sxxeyy', # s01e01
'XXxYY', # 1x01
'season_episode' # Season 1 Episode 1
]))
# Choose optional quality tags
quality_tags = draw(st.lists(
st.sampled_from(['1080p', '720p', '4K', 'WEB-DL', 'BluRay', 'x264']),
max_size=2
))
# Choose extension
ext = draw(st.sampled_from(['.mp4', '.mkv', '.avi']))
# Build filename based on pattern
if pattern == 'SXXEYY':
episode_part = f"S{season:02d}E{episode:02d}"
elif pattern == 'sxxeyy':
episode_part = f"s{season:02d}e{episode:02d}"
elif pattern == 'XXxYY':
episode_part = f"{season}x{episode:02d}"
else: # season_episode
episode_part = f"Season {season} Episode {episode}"
# Build filename
filename = f"{title} {episode_part}"
# Add quality tags
if quality_tags:
filename += ' ' + ' '.join(quality_tags)
# Add extension
filename += ext
return filename, title, season, episode
@st.composite
def ambiguous_filename_strategy(draw):
"""Generate filenames without clear season/episode patterns."""
# Generate random text without season/episode patterns
words = draw(st.lists(
st.text(
alphabet=st.characters(whitelist_categories=('Lu', 'Ll'), min_codepoint=65, max_codepoint=122),
min_size=3,
max_size=10
),
min_size=1,
max_size=5
))
filename = ' '.join(words)
# Add extension
ext = draw(st.sampled_from(['.mp4', '.mkv', '.avi']))
filename += ext
return filename
class TestParserProperties:
"""Property-based tests for the Identity Parser."""
# Feature: video-library-manager, Property 5: Movie parsing
@settings(max_examples=100)
@given(movie_filename_strategy())
def test_property_5_movie_parsing(self, movie_data):
"""For any filename matching common movie patterns, parser SHALL extract title and year.
Validates: Requirements 2.1, 2.2
"""
filename, expected_title, expected_year = movie_data
result = parse_movie(filename)
# Parser should extract a title
assert result.title is not None
assert len(result.title) > 0
# Parser should extract the year
assert result.year == expected_year
# Title should be normalized (no dots, underscores, proper case)
assert '.' not in result.title
assert '_' not in result.title
# Quality tags should be removed from title
quality_indicators = ['1080p', '720p', '4K', 'BluRay', 'WEB-DL', 'HDTV', 'x264', 'x265']
for tag in quality_indicators:
assert tag not in result.title
# Release groups should be removed from title
assert '[RARBG]' not in result.title
assert '[YTS]' not in result.title
assert '[YIFY]' not in result.title
# Original filename should be preserved
assert result.original_filename == filename
# Feature: video-library-manager, Property 6: Series parsing
@settings(max_examples=100)
@given(series_filename_strategy())
def test_property_6_series_parsing(self, series_data):
"""For any filename matching common series patterns, parser SHALL extract series title, season, and episodes.
Validates: Requirements 3.1, 3.2
"""
filename, expected_title, expected_season, expected_episode = series_data
result = parse_series(filename)
# Parser should extract a title
assert result.title is not None
assert len(result.title) > 0
# Parser should extract season and episode
assert result.season == expected_season
assert expected_episode in result.episodes
# Title should be normalized
assert '.' not in result.title
assert '_' not in result.title
# Quality tags should be removed from title
quality_indicators = ['1080p', '720p', '4K', 'WEB-DL', 'BluRay', 'x264']
for tag in quality_indicators:
assert tag not in result.title
# Original filename should be preserved
assert result.original_filename == filename
# Feature: video-library-manager, Property 7: Title normalization idempotence
@settings(max_examples=100)
@given(st.text(
alphabet=st.characters(whitelist_categories=('Lu', 'Ll', 'Nd', 'Zs'), min_codepoint=32, max_codepoint=126),
min_size=1,
max_size=100
))
def test_property_7_title_normalization_idempotence(self, title):
"""For any title string, normalizing multiple times SHALL produce same result.
Validates: Requirements 2.6, 3.6
"""
# Filter out empty strings after normalization
assume(len(title.strip()) > 0)
normalized_once = normalize_title(title)
normalized_twice = normalize_title(normalized_once)
normalized_thrice = normalize_title(normalized_twice)
# All normalizations should produce the same result
assert normalized_once == normalized_twice
assert normalized_twice == normalized_thrice
# Feature: video-library-manager, Property 8: Ambiguous filename flagging
@settings(max_examples=100)
@given(ambiguous_filename_strategy())
def test_property_8_ambiguous_filename_flagging(self, filename):
"""For any filename where season/episode cannot be extracted, parser SHALL mark as needs_review.
Validates: Requirements 2.5, 3.5
"""
# Ensure the filename doesn't accidentally match a pattern
# by checking it doesn't contain common episode markers
assume('S' not in filename.upper() or 'E' not in filename.upper())
assume('x' not in filename.lower())
assume('season' not in filename.lower())
assume('episode' not in filename.lower())
result = parse_series(filename)
# If season or episodes cannot be extracted, should be marked for review
if result.season is None or len(result.episodes) == 0:
assert result.needs_review is True
assert result.confidence <= 0.5
# Feature: video-library-manager, Property 9: Episode grouping
@settings(max_examples=100)
@given(st.lists(series_filename_strategy(), min_size=1, max_size=20))
def test_property_9_episode_grouping(self, series_data_list):
"""For any set of parsed episodes, grouping SHALL place episodes with identical normalized titles and seasons in same group.
Validates: Requirements 3.7
"""
# Parse all episodes
episodes = [parse_series(filename) for filename, _, _, _ in series_data_list]
# Group episodes
groups = group_episodes(episodes)
# Verify grouping correctness
for (title, season), group_episodes_list in groups.items():
# All episodes in a group should have the same normalized title and season
for episode in group_episodes_list:
assert episode.title == title
assert episode.season == season
# Verify no episode is in multiple groups
all_grouped_episodes = []
for group_episodes_list in groups.values():
all_grouped_episodes.extend(group_episodes_list)
# Count episodes with valid season (should match grouped count)
valid_episodes = [ep for ep in episodes if ep.season is not None]
assert len(all_grouped_episodes) == len(valid_episodes)
# Feature: video-library-manager, Property 51: Parsing error handling
@settings(max_examples=100)
@given(st.text(
alphabet=st.characters(blacklist_categories=('Cc', 'Cs'), min_codepoint=32, max_codepoint=126),
min_size=1,
max_size=200
))
def test_property_51_parsing_error_handling(self, filename):
"""For any filename that cannot be parsed, system SHALL log error and mark file for review.
Validates: Requirements 13.4
Note: This test verifies that the parser handles unparseable filenames gracefully
by marking them for review, rather than crashing or producing invalid results.
"""
# Ensure filename has an extension
if not any(filename.endswith(ext) for ext in ['.mp4', '.mkv', '.avi', '.mov', '.wmv', '.flv', '.webm', '.m4v']):
filename += '.mkv'
# Try parsing as movie
movie_result = parse_movie(filename)
# Parser should always return a valid MovieIdentity object
assert movie_result is not None
assert movie_result.title is not None
assert movie_result.original_filename == filename
# If year cannot be extracted, should be marked for review
if movie_result.year is None:
assert movie_result.needs_review is True
# Try parsing as series
series_result = parse_series(filename)
# Parser should always return a valid SeriesIdentity object
assert series_result is not None
assert series_result.title is not None
assert series_result.original_filename == filename
# If season/episode cannot be extracted, should be marked for review
if series_result.season is None or len(series_result.episodes) == 0:
assert series_result.needs_review is True
tests/test_planner.py
+923
View File
@@ -0,0 +1,923 @@
"""Unit tests for plan generator."""
import pytest
from datetime import datetime
from pathlib import Path
from vlm.config import Config
from vlm.models import (
ExecutionPlan,
FileOperation,
MovieIdentity,
SeriesIdentity,
VideoFile,
)
from vlm.planner import generate_plan
@pytest.fixture
def config():
"""Create a test configuration."""
return Config(
library_root=Path("/mnt/nas/videos"),
video_extensions=[".mp4", ".mkv", ".avi"],
movie_template="movie/{title} ({year})/",
series_template="series/{title}/Season {season:02d}/",
movie_filename_template="{title} ({year}){ext}",
series_filename_template="S{season:02d}E{episode:02d}{ext}",
log_level="INFO",
quarantine_dir=".quarantine"
)
def test_generate_plan_for_movie_with_year(config):
"""Test plan generation for a movie with year."""
video_file = VideoFile(
path=Path("/mnt/nas/videos/movie/Some.Movie.2020.1080p.mkv"),
filename="Some.Movie.2020.1080p.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Some Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Some.Movie.2020.1080p.mkv"
)
plan = generate_plan([(video_file, identity)], config)
assert isinstance(plan, ExecutionPlan)
assert len(plan.operations) == 1
operation = plan.operations[0]
assert operation.operation_type == "move"
assert operation.source_path == video_file.path
assert operation.destination_path == Path("/mnt/nas/videos/movie/Some Movie (2020)/Some Movie (2020).mkv")
assert not operation.has_conflict
assert "Some Movie (2020)" in operation.reason
def test_generate_plan_for_movie_without_year(config):
"""Test plan generation for a movie without year (needs review)."""
video_file = VideoFile(
path=Path("/mnt/nas/videos/movie/random_movie.mkv"),
filename="random_movie.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Random Movie",
year=None,
confidence=0.3,
needs_review=True,
original_filename="random_movie.mkv"
)
plan = generate_plan([(video_file, identity)], config)
assert len(plan.operations) == 1
operation = plan.operations[0]
assert operation.operation_type == "no-op"
assert operation.destination_path is None
assert "manual review" in operation.reason.lower()
def test_generate_plan_for_series_with_season_and_episode(config):
"""Test plan generation for a series with season and episode."""
video_file = VideoFile(
path=Path("/mnt/nas/videos/series/Show.Name.S01E05.mkv"),
filename="Show.Name.S01E05.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="series"
)
identity = SeriesIdentity(
title="Show Name",
season=1,
episodes=[5],
confidence=0.9,
needs_review=False,
original_filename="Show.Name.S01E05.mkv"
)
plan = generate_plan([(video_file, identity)], config)
assert len(plan.operations) == 1
operation = plan.operations[0]
assert operation.operation_type == "move"
assert operation.source_path == video_file.path
assert operation.destination_path == Path("/mnt/nas/videos/series/Show Name/Season 01/S01E05.mkv")
assert not operation.has_conflict
def test_generate_plan_for_series_without_season(config):
"""Test plan generation for a series without season (needs review)."""
video_file = VideoFile(
path=Path("/mnt/nas/videos/series/ambiguous_show.mkv"),
filename="ambiguous_show.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="series"
)
identity = SeriesIdentity(
title="Ambiguous Show",
season=None,
episodes=[],
confidence=0.3,
needs_review=True,
original_filename="ambiguous_show.mkv"
)
plan = generate_plan([(video_file, identity)], config)
assert len(plan.operations) == 1
operation = plan.operations[0]
assert operation.operation_type == "no-op"
assert operation.destination_path is None
assert "manual review" in operation.reason.lower()
def test_generate_plan_for_anime_category(config):
"""Test plan generation for anime files (no-op in v1)."""
video_file = VideoFile(
path=Path("/mnt/nas/videos/anime/Some.Anime.01.mkv"),
filename="Some.Anime.01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="anime"
)
# Anime files don't get parsed in v1, so identity is None
plan = generate_plan([(video_file, None)], config)
assert len(plan.operations) == 1
operation = plan.operations[0]
assert operation.operation_type == "no-op"
assert operation.destination_path is None
assert "anime" in operation.reason.lower() or "not organized" in operation.reason.lower()
def test_generate_plan_for_other_category(config):
"""Test plan generation for other category files (no-op in v1)."""
video_file = VideoFile(
path=Path("/mnt/nas/videos/other/random.mkv"),
filename="random.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="other"
)
plan = generate_plan([(video_file, None)], config)
assert len(plan.operations) == 1
operation = plan.operations[0]
assert operation.operation_type == "no-op"
assert operation.destination_path is None
assert "other" in operation.reason.lower() or "not organized" in operation.reason.lower()
def test_generate_plan_preserves_category_boundaries(config):
"""Test that plan generation preserves category boundaries."""
movie_file = VideoFile(
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
movie_identity = MovieIdentity(
title="Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Movie.2020.mkv"
)
plan = generate_plan([(movie_file, movie_identity)], config)
operation = plan.operations[0]
# Destination should still be in movie category
assert str(operation.destination_path).startswith(str(config.library_root / "movie"))
def test_generate_plan_for_multi_episode_file(config):
"""Test plan generation for multi-episode files."""
video_file = VideoFile(
path=Path("/mnt/nas/videos/series/Show.S01E01E02.mkv"),
filename="Show.S01E01E02.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="series"
)
identity = SeriesIdentity(
title="Show",
season=1,
episodes=[1, 2], # Multi-episode
confidence=0.9,
needs_review=False,
original_filename="Show.S01E01E02.mkv"
)
plan = generate_plan([(video_file, identity)], config)
operation = plan.operations[0]
# Should use first episode number for filename
assert operation.destination_path == Path("/mnt/nas/videos/series/Show/Season 01/S01E01.mkv")
def test_generate_plan_file_already_at_target(config):
"""Test plan generation when file is already at target location."""
# File already in correct location
video_file = VideoFile(
path=Path("/mnt/nas/videos/movie/Some Movie (2020)/Some Movie (2020).mkv"),
filename="Some Movie (2020).mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Some Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Some Movie (2020).mkv"
)
plan = generate_plan([(video_file, identity)], config)
operation = plan.operations[0]
assert operation.operation_type == "no-op"
assert "already at target" in operation.reason.lower()
def test_generate_plan_rename_vs_move(config):
"""Test that plan distinguishes between rename and move operations."""
# File in correct directory but wrong name (rename)
video_file_rename = VideoFile(
path=Path("/mnt/nas/videos/movie/Some Movie (2020)/old_name.mkv"),
filename="old_name.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Some Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="old_name.mkv"
)
plan = generate_plan([(video_file_rename, identity)], config)
operation = plan.operations[0]
assert operation.operation_type == "rename"
# File in wrong directory (move)
video_file_move = VideoFile(
path=Path("/mnt/nas/videos/movie/wrong_dir/Some Movie (2020).mkv"),
filename="Some Movie (2020).mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
plan = generate_plan([(video_file_move, identity)], config)
operation = plan.operations[0]
assert operation.operation_type == "move"
def test_generate_plan_summary(config):
"""Test that plan summary contains accurate counts."""
files_and_identities = [
# Movie with year (move)
(
VideoFile(
path=Path("/mnt/nas/videos/movie/Movie1.2020.mkv"),
filename="Movie1.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
),
MovieIdentity(
title="Movie1",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Movie1.2020.mkv"
)
),
# Movie without year (no-op)
(
VideoFile(
path=Path("/mnt/nas/videos/movie/Movie2.mkv"),
filename="Movie2.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
),
MovieIdentity(
title="Movie2",
year=None,
confidence=0.3,
needs_review=True,
original_filename="Movie2.mkv"
)
),
# Anime (no-op)
(
VideoFile(
path=Path("/mnt/nas/videos/anime/Anime.01.mkv"),
filename="Anime.01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="anime"
),
None
),
# Series with season/episode (move)
(
VideoFile(
path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"),
filename="Show.S01E01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="series"
),
SeriesIdentity(
title="Show",
season=1,
episodes=[1],
confidence=0.9,
needs_review=False,
original_filename="Show.S01E01.mkv"
)
),
]
plan = generate_plan(files_and_identities, config)
assert plan.summary["total"] == 4
assert plan.summary["move"] == 2
assert plan.summary["no-op"] == 2
assert plan.summary["rename"] == 0
assert plan.summary["quarantine"] == 0
def test_generate_plan_with_different_extensions(config):
"""Test plan generation preserves file extensions."""
extensions = [".mp4", ".mkv", ".avi"]
for ext in extensions:
video_file = VideoFile(
path=Path(f"/mnt/nas/videos/movie/Movie.2020{ext}"),
filename=f"Movie.2020{ext}",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename=f"Movie.2020{ext}"
)
plan = generate_plan([(video_file, identity)], config)
operation = plan.operations[0]
# Check that extension is preserved
assert operation.destination_path.suffix == ext
def test_conflict_detection_for_movie(config, tmp_path):
"""Test conflict detection when destination movie file already exists."""
# Set up config with tmp_path as library root
config.library_root = tmp_path
# Create destination directory and file
dest_dir = tmp_path / "movie" / "Some Movie (2020)"
dest_dir.mkdir(parents=True)
dest_file = dest_dir / "Some Movie (2020).mkv"
dest_file.touch() # Create the file
# Source file in different location
video_file = VideoFile(
path=tmp_path / "movie" / "Some.Movie.2020.1080p.mkv",
filename="Some.Movie.2020.1080p.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Some Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Some.Movie.2020.1080p.mkv"
)
plan = generate_plan([(video_file, identity)], config)
operation = plan.operations[0]
assert operation.has_conflict is True
assert operation.conflict_reason is not None
assert "already exists" in operation.conflict_reason.lower()
assert str(dest_file) in operation.conflict_reason
def test_conflict_detection_for_series(config, tmp_path):
"""Test conflict detection when destination series file already exists."""
# Set up config with tmp_path as library root
config.library_root = tmp_path
# Create destination directory and file
dest_dir = tmp_path / "series" / "Show Name" / "Season 01"
dest_dir.mkdir(parents=True)
dest_file = dest_dir / "S01E05.mkv"
dest_file.touch() # Create the file
# Source file in different location
video_file = VideoFile(
path=tmp_path / "series" / "Show.Name.S01E05.1080p.mkv",
filename="Show.Name.S01E05.1080p.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="series"
)
identity = SeriesIdentity(
title="Show Name",
season=1,
episodes=[5],
confidence=0.9,
needs_review=False,
original_filename="Show.Name.S01E05.1080p.mkv"
)
plan = generate_plan([(video_file, identity)], config)
operation = plan.operations[0]
assert operation.has_conflict is True
assert operation.conflict_reason is not None
assert "already exists" in operation.conflict_reason.lower()
assert str(dest_file) in operation.conflict_reason
def test_no_conflict_when_destination_does_not_exist(config, tmp_path):
"""Test that no conflict is detected when destination file does not exist."""
# Set up config with tmp_path as library root
config.library_root = tmp_path
# Create source directory but NOT destination
source_dir = tmp_path / "movie"
source_dir.mkdir(parents=True)
video_file = VideoFile(
path=source_dir / "Some.Movie.2020.mkv",
filename="Some.Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Some Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Some.Movie.2020.mkv"
)
plan = generate_plan([(video_file, identity)], config)
operation = plan.operations[0]
assert operation.has_conflict is False
assert operation.conflict_reason is None
def test_conflict_detection_with_multiple_files(config, tmp_path):
"""Test conflict detection with multiple files, some with conflicts."""
# Set up config with tmp_path as library root
config.library_root = tmp_path
# Create destination for first movie (conflict)
dest_dir1 = tmp_path / "movie" / "Movie1 (2020)"
dest_dir1.mkdir(parents=True)
(dest_dir1 / "Movie1 (2020).mkv").touch()
# Don't create destination for second movie (no conflict)
files_and_identities = [
# Movie 1 - has conflict
(
VideoFile(
path=tmp_path / "movie" / "Movie1.2020.mkv",
filename="Movie1.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
),
MovieIdentity(
title="Movie1",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Movie1.2020.mkv"
)
),
# Movie 2 - no conflict
(
VideoFile(
path=tmp_path / "movie" / "Movie2.2021.mkv",
filename="Movie2.2021.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
),
MovieIdentity(
title="Movie2",
year=2021,
confidence=0.9,
needs_review=False,
original_filename="Movie2.2021.mkv"
)
),
]
plan = generate_plan(files_and_identities, config)
# First operation should have conflict
assert plan.operations[0].has_conflict is True
assert plan.operations[0].conflict_reason is not None
# Second operation should not have conflict
assert plan.operations[1].has_conflict is False
assert plan.operations[1].conflict_reason is None
def test_save_plan_to_json(config, tmp_path):
"""Test saving execution plan to JSON file."""
from vlm.planner import save_plan
# Create a simple plan
video_file = VideoFile(
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Movie.2020.mkv"
)
plan = generate_plan([(video_file, identity)], config)
# Save to JSON
output_path = tmp_path / "plan.json"
save_plan(plan, output_path)
# Verify file was created
assert output_path.exists()
# Verify JSON is valid and contains expected fields
import json
with open(output_path, 'r') as f:
plan_dict = json.load(f)
assert "plan_id" in plan_dict
assert "created_at" in plan_dict
assert "operations" in plan_dict
assert "summary" in plan_dict
assert plan_dict["plan_id"] == plan.plan_id
assert len(plan_dict["operations"]) == 1
assert plan_dict["summary"]["total"] == 1
def test_load_plan_from_json(config, tmp_path):
"""Test loading execution plan from JSON file."""
from vlm.planner import save_plan, load_plan
# Create and save a plan
video_file = VideoFile(
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Movie.2020.mkv"
)
original_plan = generate_plan([(video_file, identity)], config)
output_path = tmp_path / "plan.json"
save_plan(original_plan, output_path)
# Load the plan
loaded_plan = load_plan(output_path)
# Verify loaded plan matches original
assert loaded_plan.plan_id == original_plan.plan_id
assert loaded_plan.created_at == original_plan.created_at
assert len(loaded_plan.operations) == len(original_plan.operations)
assert loaded_plan.summary == original_plan.summary
# Verify operation details
loaded_op = loaded_plan.operations[0]
original_op = original_plan.operations[0]
assert loaded_op.operation_type == original_op.operation_type
assert loaded_op.source_path == original_op.source_path
assert loaded_op.destination_path == original_op.destination_path
assert loaded_op.reason == original_op.reason
assert loaded_op.has_conflict == original_op.has_conflict
assert loaded_op.conflict_reason == original_op.conflict_reason
def test_save_and_load_plan_with_conflicts(config, tmp_path):
"""Test saving and loading plan with conflict information."""
from vlm.planner import save_plan, load_plan
# Set up config with tmp_path as library root
config.library_root = tmp_path
# Create destination file to trigger conflict
dest_dir = tmp_path / "movie" / "Movie (2020)"
dest_dir.mkdir(parents=True)
(dest_dir / "Movie (2020).mkv").touch()
video_file = VideoFile(
path=tmp_path / "movie" / "Movie.2020.mkv",
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Movie.2020.mkv"
)
original_plan = generate_plan([(video_file, identity)], config)
# Verify conflict was detected
assert original_plan.operations[0].has_conflict is True
# Save and load
output_path = tmp_path / "plan_with_conflict.json"
save_plan(original_plan, output_path)
loaded_plan = load_plan(output_path)
# Verify conflict information is preserved
assert loaded_plan.operations[0].has_conflict is True
assert loaded_plan.operations[0].conflict_reason is not None
assert "already exists" in loaded_plan.operations[0].conflict_reason.lower()
def test_save_and_load_plan_with_no_op_operations(config, tmp_path):
"""Test saving and loading plan with no-op operations."""
from vlm.planner import save_plan, load_plan
# Create files that will generate no-op operations
files_and_identities = [
# Anime (no-op)
(
VideoFile(
path=Path("/mnt/nas/videos/anime/Anime.01.mkv"),
filename="Anime.01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="anime"
),
None
),
# Movie without year (no-op)
(
VideoFile(
path=Path("/mnt/nas/videos/movie/Movie.mkv"),
filename="Movie.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
),
MovieIdentity(
title="Movie",
year=None,
confidence=0.3,
needs_review=True,
original_filename="Movie.mkv"
)
),
]
original_plan = generate_plan(files_and_identities, config)
# Save and load
output_path = tmp_path / "plan_with_noops.json"
save_plan(original_plan, output_path)
loaded_plan = load_plan(output_path)
# Verify no-op operations are preserved
assert len(loaded_plan.operations) == 2
assert all(op.operation_type == "no-op" for op in loaded_plan.operations)
assert all(op.destination_path is None for op in loaded_plan.operations)
def test_save_plan_json_is_human_readable(config, tmp_path):
"""Test that saved JSON is human-readable with proper formatting."""
from vlm.planner import save_plan
video_file = VideoFile(
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Movie.2020.mkv"
)
plan = generate_plan([(video_file, identity)], config)
output_path = tmp_path / "plan.json"
save_plan(plan, output_path)
# Read the raw JSON content
with open(output_path, 'r') as f:
content = f.read()
# Verify it's formatted with indentation (human-readable)
assert "\n" in content # Has newlines
assert " " in content # Has indentation
# Verify it's valid JSON
import json
json.loads(content)
def test_save_plan_with_multiple_operations(config, tmp_path):
"""Test saving plan with multiple operations of different types."""
from vlm.planner import save_plan, load_plan
files_and_identities = [
# Movie (move)
(
VideoFile(
path=Path("/mnt/nas/videos/movie/Movie1.2020.mkv"),
filename="Movie1.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
),
MovieIdentity(
title="Movie1",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Movie1.2020.mkv"
)
),
# Series (move)
(
VideoFile(
path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"),
filename="Show.S01E01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="series"
),
SeriesIdentity(
title="Show",
season=1,
episodes=[1],
confidence=0.9,
needs_review=False,
original_filename="Show.S01E01.mkv"
)
),
# Anime (no-op)
(
VideoFile(
path=Path("/mnt/nas/videos/anime/Anime.01.mkv"),
filename="Anime.01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
category="anime"
),
None
),
]
original_plan = generate_plan(files_and_identities, config)
output_path = tmp_path / "multi_op_plan.json"
save_plan(original_plan, output_path)
loaded_plan = load_plan(output_path)
# Verify all operations are preserved
assert len(loaded_plan.operations) == 3
assert loaded_plan.summary["total"] == 3
assert loaded_plan.summary["move"] == 2
assert loaded_plan.summary["no-op"] == 1
def test_load_plan_file_not_found(tmp_path):
"""Test loading plan from non-existent file raises FileNotFoundError."""
from vlm.planner import load_plan
non_existent_path = tmp_path / "does_not_exist.json"
with pytest.raises(FileNotFoundError):
load_plan(non_existent_path)
def test_load_plan_invalid_json(tmp_path):
"""Test loading plan from invalid JSON raises JSONDecodeError."""
from vlm.planner import load_plan
import json
invalid_json_path = tmp_path / "invalid.json"
with open(invalid_json_path, 'w') as f:
f.write("{ this is not valid json }")
with pytest.raises(json.JSONDecodeError):
load_plan(invalid_json_path)
def test_plan_json_includes_all_required_fields(config, tmp_path):
"""Test that saved JSON includes plan_id, created_at, operations, and summary."""
from vlm.planner import save_plan
import json
video_file = VideoFile(
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
category="movie"
)
identity = MovieIdentity(
title="Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename="Movie.2020.mkv"
)
plan = generate_plan([(video_file, identity)], config)
output_path = tmp_path / "plan.json"
save_plan(plan, output_path)
with open(output_path, 'r') as f:
plan_dict = json.load(f)
# Verify all required fields are present
required_fields = ["plan_id", "created_at", "operations", "summary"]
for field in required_fields:
assert field in plan_dict, f"Missing required field: {field}"
# Verify operations have required fields
operation = plan_dict["operations"][0]
required_op_fields = ["operation_type", "source_path", "destination_path", "reason", "has_conflict", "conflict_reason"]
for field in required_op_fields:
assert field in operation, f"Missing required operation field: {field}"
tests/test_quarantine.py
+909
View File
@@ -0,0 +1,909 @@
"""Tests for quarantine manager."""
import json
import pytest
from pathlib import Path
from datetime import datetime
from src.vlm.quarantine import QuarantineManager
from src.vlm.config import Config
from src.vlm.models import QuarantineEntry, QuarantineManifest
class TestQuarantineManager:
"""Test suite for QuarantineManager."""
@pytest.fixture
def config(self, tmp_path):
"""Create a test configuration."""
library_root = tmp_path / "library"
library_root.mkdir()
# Create category directories
(library_root / "movie").mkdir()
(library_root / "series").mkdir()
(library_root / "anime").mkdir()
(library_root / "other").mkdir()
return Config(
library_root=library_root,
quarantine_dir=".quarantine"
)
@pytest.fixture
def manager(self, config):
"""Create a quarantine manager instance."""
return QuarantineManager(config)
def test_quarantine_movie_file(self, manager, config):
"""Test quarantining a movie file."""
# Create a test movie file
movie_file = config.library_root / "movie" / "Test Movie (2020).mkv"
movie_file.write_text("test content")
# Quarantine the file
result = manager.quarantine_file(movie_file, reason="duplicate")
# Verify operation succeeded
assert result.success is True
assert result.error_message is None
# Verify file was moved
assert not movie_file.exists()
# Verify file is in quarantine
expected_quarantine_path = config.library_root / "movie" / ".quarantine" / "Test Movie (2020).mkv"
assert expected_quarantine_path.exists()
assert expected_quarantine_path.read_text() == "test content"
def test_quarantine_series_file(self, manager, config):
"""Test quarantining a series file."""
# Create a test series file with subdirectory
series_dir = config.library_root / "series" / "Test Show" / "Season 01"
series_dir.mkdir(parents=True)
series_file = series_dir / "S01E01.mkv"
series_file.write_text("test content")
# Quarantine the file
result = manager.quarantine_file(series_file, reason="low quality")
# Verify operation succeeded
assert result.success is True
assert result.error_message is None
# Verify file was moved
assert not series_file.exists()
# Verify file is in quarantine with preserved structure
expected_quarantine_path = (
config.library_root / "series" / ".quarantine" / "Test Show" / "Season 01" / "S01E01.mkv"
)
assert expected_quarantine_path.exists()
assert expected_quarantine_path.read_text() == "test content"
def test_quarantine_anime_file_rejected(self, manager, config):
"""Test that quarantining anime files is rejected."""
# Create a test anime file
anime_file = config.library_root / "anime" / "Test Anime.mkv"
anime_file.write_text("test content")
# Attempt to quarantine should raise ValueError
with pytest.raises(ValueError, match="Quarantine not supported for category 'anime'"):
manager.quarantine_file(anime_file)
# Verify file was not moved
assert anime_file.exists()
def test_quarantine_other_file_rejected(self, manager, config):
"""Test that quarantining other files is rejected."""
# Create a test other file
other_file = config.library_root / "other" / "Test File.mkv"
other_file.write_text("test content")
# Attempt to quarantine should raise ValueError
with pytest.raises(ValueError, match="Quarantine not supported for category 'other'"):
manager.quarantine_file(other_file)
# Verify file was not moved
assert other_file.exists()
def test_quarantine_conflict_handling(self, manager, config):
"""Test that destination conflicts are handled with numeric suffixes."""
# Create a test movie file
movie_file = config.library_root / "movie" / "Test Movie (2020).mkv"
movie_file.write_text("original content")
# Create a conflicting file in quarantine
quarantine_dir = config.library_root / "movie" / ".quarantine"
quarantine_dir.mkdir()
existing_file = quarantine_dir / "Test Movie (2020).mkv"
existing_file.write_text("existing content")
# Quarantine the file
result = manager.quarantine_file(movie_file)
# Verify operation succeeded
assert result.success is True
# Verify original file was moved
assert not movie_file.exists()
# Verify existing file is unchanged
assert existing_file.exists()
assert existing_file.read_text() == "existing content"
# Verify new file has numeric suffix
new_file = quarantine_dir / "Test Movie (2020)_1.mkv"
assert new_file.exists()
assert new_file.read_text() == "original content"
def test_quarantine_multiple_conflicts(self, manager, config):
"""Test handling multiple conflicts with incrementing suffixes."""
# Create quarantine directory with existing files
quarantine_dir = config.library_root / "movie" / ".quarantine"
quarantine_dir.mkdir()
# Create existing files
(quarantine_dir / "Test Movie (2020).mkv").write_text("file 0")
(quarantine_dir / "Test Movie (2020)_1.mkv").write_text("file 1")
(quarantine_dir / "Test Movie (2020)_2.mkv").write_text("file 2")
# Create new file to quarantine
movie_file = config.library_root / "movie" / "Test Movie (2020).mkv"
movie_file.write_text("file 3")
# Quarantine the file
result = manager.quarantine_file(movie_file)
# Verify operation succeeded
assert result.success is True
# Verify new file has suffix _3
new_file = quarantine_dir / "Test Movie (2020)_3.mkv"
assert new_file.exists()
assert new_file.read_text() == "file 3"
def test_quarantine_nonexistent_file(self, manager, config):
"""Test quarantining a file that doesn't exist."""
# Try to quarantine non-existent file
nonexistent_file = config.library_root / "movie" / "Nonexistent.mkv"
result = manager.quarantine_file(nonexistent_file)
# Verify operation failed
assert result.success is False
assert "does not exist" in result.error_message
def test_quarantine_preserves_directory_structure(self, manager, config):
"""Test that quarantine preserves relative directory structure."""
# Create a deeply nested series file
series_path = config.library_root / "series" / "Show" / "Season 02" / "Extras"
series_path.mkdir(parents=True)
series_file = series_path / "Behind the Scenes.mkv"
series_file.write_text("test content")
# Quarantine the file
result = manager.quarantine_file(series_file)
# Verify operation succeeded
assert result.success is True
# Verify structure is preserved in quarantine
expected_path = (
config.library_root / "series" / ".quarantine" /
"Show" / "Season 02" / "Extras" / "Behind the Scenes.mkv"
)
assert expected_path.exists()
assert expected_path.read_text() == "test content"
def test_determine_category_movie(self, manager, config):
"""Test category determination for movie files."""
movie_file = config.library_root / "movie" / "Test.mkv"
category = manager._determine_category(movie_file)
assert category == "movie"
def test_determine_category_series(self, manager, config):
"""Test category determination for series files."""
series_file = config.library_root / "series" / "Show" / "S01E01.mkv"
category = manager._determine_category(series_file)
assert category == "series"
def test_determine_category_anime(self, manager, config):
"""Test category determination for anime files."""
anime_file = config.library_root / "anime" / "Test.mkv"
category = manager._determine_category(anime_file)
assert category == "anime"
def test_determine_category_other(self, manager, config):
"""Test category determination for other files."""
other_file = config.library_root / "other" / "Test.mkv"
category = manager._determine_category(other_file)
assert category == "other"
def test_determine_category_outside_library(self, manager, config):
"""Test category determination for files outside library."""
outside_file = Path("/tmp/Test.mkv")
category = manager._determine_category(outside_file)
assert category == "other"
def test_resolve_conflict_no_conflict(self, manager, config):
"""Test conflict resolution when no conflict exists."""
test_path = config.library_root / "movie" / ".quarantine" / "Test.mkv"
resolved = manager._resolve_conflict(test_path)
assert resolved == test_path
def test_resolve_conflict_with_conflict(self, manager, config):
"""Test conflict resolution when conflict exists."""
quarantine_dir = config.library_root / "movie" / ".quarantine"
quarantine_dir.mkdir(parents=True)
# Create existing file
existing = quarantine_dir / "Test.mkv"
existing.write_text("existing")
# Resolve conflict
resolved = manager._resolve_conflict(existing)
# Should return path with _1 suffix
assert resolved == quarantine_dir / "Test_1.mkv"
assert not resolved.exists()
class TestQuarantineManifest:
"""Test suite for quarantine manifest management."""
@pytest.fixture
def config(self, tmp_path):
"""Create a test configuration."""
library_root = tmp_path / "library"
library_root.mkdir()
# Create category directories
(library_root / "movie").mkdir()
(library_root / "series").mkdir()
return Config(
library_root=library_root,
quarantine_dir=".quarantine"
)
@pytest.fixture
def manager(self, config):
"""Create a quarantine manager instance."""
return QuarantineManager(config)
def test_manifest_created_on_first_quarantine(self, manager, config):
"""Test that manifest is created when first file is quarantined."""
# Create a test movie file
movie_file = config.library_root / "movie" / "Test Movie (2020).mkv"
movie_file.write_text("test content")
# Quarantine the file
result = manager.quarantine_file(movie_file, reason="duplicate")
# Verify operation succeeded
assert result.success is True
# Verify manifest was created
manifest_path = config.library_root / "movie" / ".quarantine" / "manifest.json"
assert manifest_path.exists()
# Load and verify manifest content
with open(manifest_path, 'r') as f:
data = json.load(f)
assert 'entries' in data
assert len(data['entries']) == 1
entry = data['entries'][0]
assert entry['original_path'] == str(movie_file)
assert entry['category'] == "movie"
assert entry['reason'] == "duplicate"
assert entry['size_bytes'] == len("test content")
assert 'quarantined_at' in entry
assert 'quarantine_path' in entry
def test_manifest_updated_on_subsequent_quarantine(self, manager, config):
"""Test that manifest is updated when additional files are quarantined."""
# Create and quarantine first file
movie_file1 = config.library_root / "movie" / "Movie1.mkv"
movie_file1.write_text("content1")
manager.quarantine_file(movie_file1, reason="duplicate")
# Create and quarantine second file
movie_file2 = config.library_root / "movie" / "Movie2.mkv"
movie_file2.write_text("content2")
manager.quarantine_file(movie_file2, reason="low quality")
# Load manifest
manifest_path = config.library_root / "movie" / ".quarantine" / "manifest.json"
with open(manifest_path, 'r') as f:
data = json.load(f)
# Verify both entries are in manifest
assert len(data['entries']) == 2
# Verify first entry
entry1 = data['entries'][0]
assert entry1['original_path'] == str(movie_file1)
assert entry1['reason'] == "duplicate"
# Verify second entry
entry2 = data['entries'][1]
assert entry2['original_path'] == str(movie_file2)
assert entry2['reason'] == "low quality"
def test_manifest_includes_all_required_fields(self, manager, config):
"""Test that manifest entries include all required fields."""
# Create a test series file
series_dir = config.library_root / "series" / "Show" / "Season 01"
series_dir.mkdir(parents=True)
series_file = series_dir / "S01E01.mkv"
series_file.write_text("test content")
# Quarantine the file
manager.quarantine_file(series_file, reason="test reason")
# Load manifest
manifest_path = config.library_root / "series" / ".quarantine" / "manifest.json"
with open(manifest_path, 'r') as f:
data = json.load(f)
entry = data['entries'][0]
# Verify all required fields are present
assert 'original_path' in entry
assert 'quarantine_path' in entry
assert 'quarantined_at' in entry
assert 'reason' in entry
assert 'size_bytes' in entry
assert 'category' in entry
# Verify field values
assert entry['original_path'] == str(series_file)
assert entry['category'] == "series"
assert entry['reason'] == "test reason"
assert entry['size_bytes'] == len("test content")
# Verify timestamp is valid ISO format
datetime.fromisoformat(entry['quarantined_at'])
def test_manifest_separate_per_category(self, manager, config):
"""Test that each category has its own manifest."""
# Create and quarantine movie file
movie_file = config.library_root / "movie" / "Movie.mkv"
movie_file.write_text("movie content")
manager.quarantine_file(movie_file)
# Create and quarantine series file
series_file = config.library_root / "series" / "S01E01.mkv"
series_file.write_text("series content")
manager.quarantine_file(series_file)
# Verify separate manifests exist
movie_manifest_path = config.library_root / "movie" / ".quarantine" / "manifest.json"
series_manifest_path = config.library_root / "series" / ".quarantine" / "manifest.json"
assert movie_manifest_path.exists()
assert series_manifest_path.exists()
# Verify movie manifest contains only movie entry
with open(movie_manifest_path, 'r') as f:
movie_data = json.load(f)
assert len(movie_data['entries']) == 1
assert movie_data['entries'][0]['category'] == "movie"
# Verify series manifest contains only series entry
with open(series_manifest_path, 'r') as f:
series_data = json.load(f)
assert len(series_data['entries']) == 1
assert series_data['entries'][0]['category'] == "series"
def test_manifest_handles_none_reason(self, manager, config):
"""Test that manifest handles files quarantined without a reason."""
# Create and quarantine file without reason
movie_file = config.library_root / "movie" / "Movie.mkv"
movie_file.write_text("content")
manager.quarantine_file(movie_file) # No reason provided
# Load manifest
manifest_path = config.library_root / "movie" / ".quarantine" / "manifest.json"
with open(manifest_path, 'r') as f:
data = json.load(f)
entry = data['entries'][0]
# Verify reason is None/null
assert entry['reason'] is None
def test_load_manifest_empty_when_not_exists(self, manager, config):
"""Test that loading non-existent manifest returns empty manifest."""
manifest = manager._load_manifest("movie")
assert isinstance(manifest, QuarantineManifest)
assert len(manifest.entries) == 0
def test_load_manifest_parses_existing_manifest(self, manager, config):
"""Test that loading existing manifest parses entries correctly."""
# Create manifest manually
manifest_path = config.library_root / "movie" / ".quarantine" / "manifest.json"
manifest_path.parent.mkdir(parents=True)
test_data = {
'entries': [
{
'original_path': '/path/to/original.mkv',
'quarantine_path': '/path/to/quarantine.mkv',
'quarantined_at': '2024-01-15T10:30:00',
'reason': 'test reason',
'size_bytes': 1024,
'category': 'movie'
}
]
}
with open(manifest_path, 'w') as f:
json.dump(test_data, f)
# Load manifest
manifest = manager._load_manifest("movie")
# Verify parsed correctly
assert len(manifest.entries) == 1
entry = manifest.entries[0]
assert entry.original_path == Path('/path/to/original.mkv')
assert entry.quarantine_path == Path('/path/to/quarantine.mkv')
assert entry.quarantined_at == datetime(2024, 1, 15, 10, 30, 0)
assert entry.reason == 'test reason'
assert entry.size_bytes == 1024
assert entry.category == 'movie'
def test_save_manifest_creates_directory(self, manager, config):
"""Test that saving manifest creates quarantine directory if needed."""
# Create manifest
manifest = QuarantineManifest(entries=[])
# Save manifest (directory doesn't exist yet)
manager._save_manifest("movie", manifest)
# Verify directory and file were created
manifest_path = config.library_root / "movie" / ".quarantine" / "manifest.json"
assert manifest_path.exists()
assert manifest_path.parent.is_dir()
def test_manifest_preserves_utf8_characters(self, manager, config):
"""Test that manifest correctly handles UTF-8 characters in paths and reasons."""
# Create file with UTF-8 characters
movie_file = config.library_root / "movie" / "Café Müller (2020).mkv"
movie_file.write_text("content")
# Quarantine with UTF-8 reason
manager.quarantine_file(movie_file, reason="Qualité insuffisante")
# Load manifest
manifest_path = config.library_root / "movie" / ".quarantine" / "manifest.json"
with open(manifest_path, 'r', encoding='utf-8') as f:
data = json.load(f)
entry = data['entries'][0]
# Verify UTF-8 characters are preserved
assert "Café Müller" in entry['original_path']
assert entry['reason'] == "Qualité insuffisante"
class TestQuarantineListing:
"""Test suite for quarantine listing functionality."""
@pytest.fixture
def config(self, tmp_path):
"""Create a test configuration."""
library_root = tmp_path / "library"
library_root.mkdir()
# Create category directories
(library_root / "movie").mkdir()
(library_root / "series").mkdir()
return Config(
library_root=library_root,
quarantine_dir=".quarantine"
)
@pytest.fixture
def manager(self, config):
"""Create a quarantine manager instance."""
return QuarantineManager(config)
def test_list_quarantined_empty(self, manager, config):
"""Test listing quarantined files when none exist."""
entries = manager.list_quarantined()
assert entries == []
def test_list_quarantined_single_category(self, manager, config):
"""Test listing quarantined files from a single category."""
# Create and quarantine movie files
movie1 = config.library_root / "movie" / "Movie1.mkv"
movie1.write_text("content1")
manager.quarantine_file(movie1, reason="duplicate")
movie2 = config.library_root / "movie" / "Movie2.mkv"
movie2.write_text("content2")
manager.quarantine_file(movie2, reason="low quality")
# List quarantined files from movie category
entries = manager.list_quarantined(category="movie")
assert len(entries) == 2
assert all(e.category == "movie" for e in entries)
# Verify entries contain expected data
original_paths = [str(e.original_path) for e in entries]
assert str(movie1) in original_paths
assert str(movie2) in original_paths
def test_list_quarantined_all_categories(self, manager, config):
"""Test listing quarantined files from all categories."""
# Create and quarantine movie file
movie = config.library_root / "movie" / "Movie.mkv"
movie.write_text("movie content")
manager.quarantine_file(movie, reason="duplicate")
# Create and quarantine series file
series = config.library_root / "series" / "S01E01.mkv"
series.write_text("series content")
manager.quarantine_file(series, reason="low quality")
# List all quarantined files
entries = manager.list_quarantined()
assert len(entries) == 2
# Verify both categories are represented
categories = [e.category for e in entries]
assert "movie" in categories
assert "series" in categories
def test_list_quarantined_filter_by_category(self, manager, config):
"""Test filtering quarantined files by category."""
# Create and quarantine files in both categories
movie = config.library_root / "movie" / "Movie.mkv"
movie.write_text("movie content")
manager.quarantine_file(movie)
series = config.library_root / "series" / "S01E01.mkv"
series.write_text("series content")
manager.quarantine_file(series)
# List only movie files
movie_entries = manager.list_quarantined(category="movie")
assert len(movie_entries) == 1
assert movie_entries[0].category == "movie"
# List only series files
series_entries = manager.list_quarantined(category="series")
assert len(series_entries) == 1
assert series_entries[0].category == "series"
def test_list_quarantined_invalid_category(self, manager, config):
"""Test listing with invalid category returns empty list."""
entries = manager.list_quarantined(category="anime")
assert entries == []
entries = manager.list_quarantined(category="other")
assert entries == []
entries = manager.list_quarantined(category="invalid")
assert entries == []
def test_list_quarantined_includes_all_fields(self, manager, config):
"""Test that listed entries include all required fields."""
# Create and quarantine file
movie = config.library_root / "movie" / "Movie.mkv"
movie.write_text("test content")
manager.quarantine_file(movie, reason="test reason")
# List quarantined files
entries = manager.list_quarantined()
assert len(entries) == 1
entry = entries[0]
# Verify all fields are present
assert entry.original_path == movie
assert entry.quarantine_path.exists()
assert entry.quarantined_at is not None
assert entry.reason == "test reason"
assert entry.size_bytes == len("test content")
assert entry.category == "movie"
def test_list_quarantined_preserves_directory_structure(self, manager, config):
"""Test that listing shows preserved directory structure."""
# Create nested series file
series_path = config.library_root / "series" / "Show" / "Season 01"
series_path.mkdir(parents=True)
series_file = series_path / "S01E01.mkv"
series_file.write_text("content")
# Quarantine the file
manager.quarantine_file(series_file)
# List quarantined files
entries = manager.list_quarantined(category="series")
assert len(entries) == 1
entry = entries[0]
# Verify quarantine path preserves structure
expected_quarantine = (
config.library_root / "series" / ".quarantine" /
"Show" / "Season 01" / "S01E01.mkv"
)
assert entry.quarantine_path == expected_quarantine
def test_list_quarantined_multiple_files_same_category(self, manager, config):
"""Test listing multiple files from the same category."""
# Create and quarantine multiple movie files
for i in range(5):
movie = config.library_root / "movie" / f"Movie{i}.mkv"
movie.write_text(f"content{i}")
manager.quarantine_file(movie, reason=f"reason{i}")
# List quarantined files
entries = manager.list_quarantined(category="movie")
assert len(entries) == 5
# Verify all entries are unique
original_paths = [e.original_path for e in entries]
assert len(set(original_paths)) == 5
class TestQuarantineRestoration:
"""Test suite for quarantine restoration functionality."""
@pytest.fixture
def config(self, tmp_path):
"""Create a test configuration."""
library_root = tmp_path / "library"
library_root.mkdir()
# Create category directories
(library_root / "movie").mkdir()
(library_root / "series").mkdir()
return Config(
library_root=library_root,
quarantine_dir=".quarantine"
)
@pytest.fixture
def manager(self, config):
"""Create a quarantine manager instance."""
return QuarantineManager(config)
def test_restore_movie_file(self, manager, config):
"""Test restoring a quarantined movie file."""
# Create and quarantine movie file
original_path = config.library_root / "movie" / "Movie.mkv"
original_path.write_text("test content")
result = manager.quarantine_file(original_path, reason="duplicate")
assert result.success is True
quarantine_path = result.operation.destination_path
# Verify file is in quarantine
assert quarantine_path.exists()
assert not original_path.exists()
# Restore the file
restore_result = manager.restore_from_quarantine(quarantine_path)
# Verify restoration succeeded
assert restore_result.success is True
assert restore_result.error_message is None
# Verify file is back at original location
assert original_path.exists()
assert original_path.read_text() == "test content"
# Verify file is removed from quarantine
assert not quarantine_path.exists()
def test_restore_series_file(self, manager, config):
"""Test restoring a quarantined series file."""
# Create nested series file
series_path = config.library_root / "series" / "Show" / "Season 01"
series_path.mkdir(parents=True)
original_path = series_path / "S01E01.mkv"
original_path.write_text("series content")
# Quarantine the file
result = manager.quarantine_file(original_path)
quarantine_path = result.operation.destination_path
# Restore the file
restore_result = manager.restore_from_quarantine(quarantine_path)
# Verify restoration succeeded
assert restore_result.success is True
# Verify file is back with preserved structure
assert original_path.exists()
assert original_path.read_text() == "series content"
assert not quarantine_path.exists()
def test_restore_removes_manifest_entry(self, manager, config):
"""Test that restoration removes entry from manifest."""
# Create and quarantine file
original_path = config.library_root / "movie" / "Movie.mkv"
original_path.write_text("content")
result = manager.quarantine_file(original_path)
quarantine_path = result.operation.destination_path
# Verify manifest has entry
manifest = manager._load_manifest("movie")
assert len(manifest.entries) == 1
# Restore the file
manager.restore_from_quarantine(quarantine_path)
# Verify manifest entry is removed
manifest = manager._load_manifest("movie")
assert len(manifest.entries) == 0
def test_restore_nonexistent_file(self, manager, config):
"""Test restoring a file that doesn't exist."""
# Try to restore non-existent file
nonexistent = config.library_root / "movie" / ".quarantine" / "Nonexistent.mkv"
result = manager.restore_from_quarantine(nonexistent)
# Verify operation failed
assert result.success is False
assert "does not exist" in result.error_message
def test_restore_conflict_destination_exists(self, manager, config):
"""Test restoration when original location already has a file."""
# Create and quarantine file
original_path = config.library_root / "movie" / "Movie.mkv"
original_path.write_text("original content")
result = manager.quarantine_file(original_path)
quarantine_path = result.operation.destination_path
# Create a new file at original location
original_path.write_text("new content")
# Try to restore
restore_result = manager.restore_from_quarantine(quarantine_path)
# Verify restoration failed due to conflict
assert restore_result.success is False
assert "already exists" in restore_result.error_message
assert restore_result.operation.has_conflict is True
# Verify original file is unchanged
assert original_path.read_text() == "new content"
# Verify quarantine file still exists
assert quarantine_path.exists()
def test_restore_creates_parent_directory(self, manager, config):
"""Test that restoration creates parent directory if needed."""
# Create and quarantine file
series_path = config.library_root / "series" / "Show" / "Season 01"
series_path.mkdir(parents=True)
original_path = series_path / "S01E01.mkv"
original_path.write_text("content")
result = manager.quarantine_file(original_path)
quarantine_path = result.operation.destination_path
# Remove the parent directory
import shutil
shutil.rmtree(series_path)
# Restore the file
restore_result = manager.restore_from_quarantine(quarantine_path)
# Verify restoration succeeded and directory was created
assert restore_result.success is True
assert original_path.exists()
assert original_path.parent.is_dir()
def test_restore_no_manifest_entry(self, manager, config):
"""Test restoring a file that has no manifest entry."""
# Create quarantine file manually without manifest entry
quarantine_dir = config.library_root / "movie" / ".quarantine"
quarantine_dir.mkdir(parents=True)
quarantine_path = quarantine_dir / "Movie.mkv"
quarantine_path.write_text("content")
# Try to restore
result = manager.restore_from_quarantine(quarantine_path)
# Verify operation failed
assert result.success is False
assert "No manifest entry" in result.error_message
def test_restore_invalid_quarantine_path(self, manager, config):
"""Test restoring from an invalid quarantine path."""
# Try to restore from non-quarantine location
invalid_path = config.library_root / "movie" / "Movie.mkv"
invalid_path.write_text("content")
result = manager.restore_from_quarantine(invalid_path)
# Verify operation failed
assert result.success is False
assert "Could not determine category" in result.error_message
def test_restore_multiple_files(self, manager, config):
"""Test restoring multiple quarantined files."""
# Create and quarantine multiple files
files = []
quarantine_paths = []
for i in range(3):
file_path = config.library_root / "movie" / f"Movie{i}.mkv"
file_path.write_text(f"content{i}")
files.append(file_path)
result = manager.quarantine_file(file_path)
quarantine_paths.append(result.operation.destination_path)
# Verify all files are quarantined
for file_path in files:
assert not file_path.exists()
for qpath in quarantine_paths:
assert qpath.exists()
# Restore all files
for qpath in quarantine_paths:
result = manager.restore_from_quarantine(qpath)
assert result.success is True
# Verify all files are restored
for i, file_path in enumerate(files):
assert file_path.exists()
assert file_path.read_text() == f"content{i}"
# Verify all quarantine files are removed
for qpath in quarantine_paths:
assert not qpath.exists()
# Verify manifest is empty
manifest = manager._load_manifest("movie")
assert len(manifest.entries) == 0
def test_determine_category_from_quarantine_movie(self, manager, config):
"""Test determining category from movie quarantine path."""
qpath = config.library_root / "movie" / ".quarantine" / "Movie.mkv"
category = manager._determine_category_from_quarantine(qpath)
assert category == "movie"
def test_determine_category_from_quarantine_series(self, manager, config):
"""Test determining category from series quarantine path."""
qpath = config.library_root / "series" / ".quarantine" / "Show" / "S01E01.mkv"
category = manager._determine_category_from_quarantine(qpath)
assert category == "series"
def test_determine_category_from_quarantine_invalid(self, manager, config):
"""Test determining category from invalid quarantine path."""
# Not in quarantine directory
invalid = config.library_root / "movie" / "Movie.mkv"
category = manager._determine_category_from_quarantine(invalid)
assert category is None
# Outside library root
outside = Path("/tmp/Movie.mkv")
category = manager._determine_category_from_quarantine(outside)
assert category is None
# Unsupported category
anime = config.library_root / "anime" / ".quarantine" / "Anime.mkv"
category = manager._determine_category_from_quarantine(anime)
assert category is None
tests/test_reports.py
+694
View File
@@ -0,0 +1,694 @@
"""Unit tests for report generation.
Tests inventory reports, completeness reports, duplicate reports, and summary reports in various formats.
"""
import csv
import json
import pytest
from io import StringIO
from pathlib import Path
from datetime import datetime
from vlm.models import SeasonCompleteness, DuplicateGroup, VideoFile, MovieIdentity, SeriesIdentity
from vlm.reports import (
generate_inventory_report,
generate_completeness_report,
generate_duplicate_report,
generate_summary_report,
_format_episode_list,
_format_size,
_format_duration
)
class TestInventoryReport:
"""Test inventory report generation."""
def test_generate_csv_report(self):
"""Test generating CSV format inventory report."""
files = [
VideoFile(
Path("/movies/Movie1.mkv"),
"Movie1.mkv",
2000000000,
datetime(2023, 1, 15, 10, 30, 0),
"movie",
resolution="1920x1080",
codec="h264",
duration_seconds=7200.0,
bitrate_kbps=5000
),
VideoFile(
Path("/series/Show.S01E01.mkv"),
"Show.S01E01.mkv",
1000000000,
datetime(2023, 2, 20, 14, 45, 0),
"series",
resolution="1280x720",
codec="h265"
),
]
library_root = Path("/mnt/nas/videos")
report = generate_inventory_report(files, "csv", library_root)
# Check metadata comments
assert "# Generated:" in report
assert f"# Library Root: {library_root}" in report
# Parse CSV
lines = report.strip().split('\n')
# Skip comment lines
csv_lines = [line for line in lines if not line.startswith('#')]
csv_reader = csv.DictReader(csv_lines)
rows = list(csv_reader)
# Check we have 2 data rows
assert len(rows) == 2
# Check first file
assert rows[0]['filename'] == 'Movie1.mkv'
assert rows[0]['size_bytes'] == '2000000000'
assert rows[0]['category'] == 'movie'
assert rows[0]['resolution'] == '1920x1080'
assert rows[0]['codec'] == 'h264'
assert rows[0]['duration_seconds'] == '7200.0'
assert rows[0]['bitrate_kbps'] == '5000'
# Check second file
assert rows[1]['filename'] == 'Show.S01E01.mkv'
assert rows[1]['category'] == 'series'
assert rows[1]['resolution'] == '1280x720'
assert rows[1]['codec'] == 'h265'
# Optional fields not present should be empty
assert rows[1]['duration_seconds'] == ''
assert rows[1]['bitrate_kbps'] == ''
def test_generate_json_report(self):
"""Test generating JSON format inventory report."""
files = [
VideoFile(
Path("/movies/Movie1.mkv"),
"Movie1.mkv",
2000000000,
datetime(2023, 1, 15, 10, 30, 0),
"movie",
resolution="1920x1080",
codec="h264"
),
VideoFile(
Path("/anime/Anime1.mkv"),
"Anime1.mkv",
800000000,
datetime(2023, 3, 10, 8, 15, 0),
"anime"
),
]
library_root = Path("/mnt/nas/videos")
report = generate_inventory_report(files, "json", library_root)
# Parse JSON
data = json.loads(report)
# Check metadata
assert "metadata" in data
assert "generated" in data["metadata"]
assert data["metadata"]["library_root"] == str(library_root)
assert data["metadata"]["file_count"] == 2
# Check files
assert "files" in data
assert len(data["files"]) == 2
# Check first file
file1 = data["files"][0]
assert file1["filename"] == "Movie1.mkv"
assert file1["size_bytes"] == 2000000000
assert file1["category"] == "movie"
assert file1["resolution"] == "1920x1080"
assert file1["codec"] == "h264"
# Check second file
file2 = data["files"][1]
assert file2["filename"] == "Anime1.mkv"
assert file2["category"] == "anime"
# Optional fields should be null
assert file2["resolution"] is None
assert file2["codec"] is None
def test_generate_csv_report_empty(self):
"""Test generating CSV report with no files."""
files = []
library_root = Path("/mnt/nas/videos")
report = generate_inventory_report(files, "csv", library_root)
# Should have metadata and header
assert "# Generated:" in report
assert "path,filename,size_bytes" in report
# Parse CSV
lines = report.strip().split('\n')
csv_lines = [line for line in lines if not line.startswith('#')]
csv_reader = csv.DictReader(csv_lines)
rows = list(csv_reader)
# No data rows
assert len(rows) == 0
def test_generate_json_report_empty(self):
"""Test generating JSON report with no files."""
files = []
library_root = Path("/mnt/nas/videos")
report = generate_inventory_report(files, "json", library_root)
data = json.loads(report)
assert data["metadata"]["file_count"] == 0
assert len(data["files"]) == 0
def test_invalid_format_raises_error(self):
"""Test that invalid format raises ValueError."""
files = []
library_root = Path("/mnt/nas/videos")
with pytest.raises(ValueError, match="Invalid format"):
generate_inventory_report(files, "xml", library_root)
def test_csv_schema_columns(self):
"""Test that CSV has all required columns in correct order."""
files = [
VideoFile(
Path("/test.mkv"),
"test.mkv",
1000,
datetime.now(),
"movie"
)
]
library_root = Path("/test")
report = generate_inventory_report(files, "csv", library_root)
# Parse CSV header
lines = report.strip().split('\n')
csv_lines = [line for line in lines if not line.startswith('#')]
header = csv_lines[0].strip() # Strip to remove any line ending characters
# Check column order
expected_columns = [
'path', 'filename', 'size_bytes', 'modified_timestamp', 'category',
'resolution', 'codec', 'duration_seconds', 'bitrate_kbps'
]
assert header == ','.join(expected_columns)
def test_timestamp_formatting(self):
"""Test that timestamps are formatted as ISO 8601."""
files = [
VideoFile(
Path("/test.mkv"),
"test.mkv",
1000,
datetime(2023, 6, 15, 14, 30, 45),
"movie"
)
]
library_root = Path("/test")
report = generate_inventory_report(files, "csv", library_root)
# Check timestamp format
assert "2023-06-15T14:30:45" in report
class TestCompletenessReport:
"""Test completeness report generation."""
def test_generate_text_report_with_gaps(self):
"""Test generating text format completeness report with gaps."""
analysis = [
SeasonCompleteness("Breaking Bad", 1, [1, 2, 4, 5], [3]),
SeasonCompleteness("Breaking Bad", 2, [1, 3, 5], [2, 4]),
SeasonCompleteness("The Wire", 1, [1, 2, 4], [3]),
]
library_root = Path("/mnt/nas/videos")
report = generate_completeness_report(analysis, "text", library_root)
# Check report structure
assert "SERIES COMPLETENESS REPORT" in report
assert "Generated:" in report
assert str(library_root) in report
assert "Series with gaps: 3" in report
# Check series content
assert "Breaking Bad" in report
assert "The Wire" in report
assert "Season 01:" in report
assert "Season 02:" in report
# Check episode information
assert "Episodes found:" in report
assert "Episodes missing:" in report
def test_generate_json_report_with_gaps(self):
"""Test generating JSON format completeness report with gaps."""
analysis = [
SeasonCompleteness("Breaking Bad", 1, [1, 2, 4, 5], [3]),
SeasonCompleteness("The Wire", 1, [1, 2, 4], [3]),
]
library_root = Path("/mnt/nas/videos")
report = generate_completeness_report(analysis, "json", library_root)
# Parse JSON
data = json.loads(report)
# Check metadata
assert "metadata" in data
assert "generated" in data["metadata"]
assert data["metadata"]["library_root"] == str(library_root)
assert data["metadata"]["series_count"] == 2
# Check series data
assert "series" in data
assert len(data["series"]) == 2
# Check Breaking Bad
breaking_bad = next(s for s in data["series"] if s["title"] == "Breaking Bad")
assert len(breaking_bad["seasons"]) == 1
assert breaking_bad["seasons"][0]["season"] == 1
assert breaking_bad["seasons"][0]["episodes_found"] == [1, 2, 4, 5]
assert breaking_bad["seasons"][0]["episodes_missing"] == [3]
def test_generate_text_report_empty(self):
"""Test generating text report with no gaps."""
analysis = []
library_root = Path("/mnt/nas/videos")
report = generate_completeness_report(analysis, "text", library_root)
assert "SERIES COMPLETENESS REPORT" in report
assert "No series with episode gaps detected." in report
def test_generate_json_report_empty(self):
"""Test generating JSON report with no gaps."""
analysis = []
library_root = Path("/mnt/nas/videos")
report = generate_completeness_report(analysis, "json", library_root)
data = json.loads(report)
assert data["metadata"]["series_count"] == 0
assert len(data["series"]) == 0
def test_invalid_format_raises_error(self):
"""Test that invalid format raises ValueError."""
analysis = []
library_root = Path("/mnt/nas/videos")
with pytest.raises(ValueError, match="Invalid format"):
generate_completeness_report(analysis, "xml", library_root)
def test_multiple_seasons_same_series(self):
"""Test report with multiple seasons of same series."""
analysis = [
SeasonCompleteness("Show Name", 1, [1, 3], [2]),
SeasonCompleteness("Show Name", 2, [1, 2, 4], [3]),
SeasonCompleteness("Show Name", 3, [5, 7], [6]),
]
library_root = Path("/mnt/nas/videos")
report = generate_completeness_report(analysis, "text", library_root)
# Should group all seasons under same series
assert report.count("Show Name") == 1 # Series title appears once
assert "Season 01:" in report
assert "Season 02:" in report
assert "Season 03:" in report
class TestDuplicateReport:
"""Test duplicate report generation."""
def test_generate_text_report_with_duplicates(self):
"""Test generating text format duplicate report."""
identities = [
MovieIdentity("The Matrix", 1999, 0.9, False, "The.Matrix.1999.1080p.mkv"),
]
files = [
VideoFile(
Path("/movies/The.Matrix.1999.1080p.mkv"),
"The.Matrix.1999.1080p.mkv",
2000000000,
datetime.now(),
"movie",
resolution="1920x1080",
codec="h264",
duration_seconds=7200.0,
bitrate_kbps=5000
),
VideoFile(
Path("/movies/The.Matrix.1999.720p.mkv"),
"The.Matrix.1999.720p.mkv",
1000000000,
datetime.now(),
"movie",
resolution="1280x720",
codec="h264"
),
]
quality_comparison = [
{
'filename': 'The.Matrix.1999.1080p.mkv',
'path': '/movies/The.Matrix.1999.1080p.mkv',
'size_bytes': 2000000000,
'resolution': '1920x1080',
'codec': 'h264',
'duration_seconds': 7200.0,
'bitrate_kbps': 5000
},
{
'filename': 'The.Matrix.1999.720p.mkv',
'path': '/movies/The.Matrix.1999.720p.mkv',
'size_bytes': 1000000000,
'resolution': '1280x720',
'codec': 'h264'
}
]
duplicates = [
DuplicateGroup(identities[0], files, quality_comparison)
]
library_root = Path("/mnt/nas/videos")
report = generate_duplicate_report(duplicates, "text", library_root)
# Check report structure
assert "DUPLICATE FILES REPORT" in report
assert "Generated:" in report
assert str(library_root) in report
assert "Duplicate groups: 1" in report
# Check duplicate group content
assert "The Matrix (1999)" in report
assert "Files: 2" in report
# Check file details
assert "The.Matrix.1999.1080p.mkv" in report
assert "The.Matrix.1999.720p.mkv" in report
assert "1920x1080" in report
assert "1280x720" in report
assert "h264" in report
def test_generate_json_report_with_duplicates(self):
"""Test generating JSON format duplicate report."""
identity = MovieIdentity("Inception", 2010, 0.9, False, "Inception.2010.1080p.mkv")
files = [
VideoFile(
Path("/movies/Inception.2010.1080p.mkv"),
"Inception.2010.1080p.mkv",
2000000000,
datetime.now(),
"movie"
),
VideoFile(
Path("/movies/Inception.2010.720p.mkv"),
"Inception.2010.720p.mkv",
1000000000,
datetime.now(),
"movie"
),
]
quality_comparison = [
{'filename': 'Inception.2010.1080p.mkv', 'path': '/movies/Inception.2010.1080p.mkv', 'size_bytes': 2000000000},
{'filename': 'Inception.2010.720p.mkv', 'path': '/movies/Inception.2010.720p.mkv', 'size_bytes': 1000000000}
]
duplicates = [
DuplicateGroup(identity, files, quality_comparison)
]
library_root = Path("/mnt/nas/videos")
report = generate_duplicate_report(duplicates, "json", library_root)
# Parse JSON
data = json.loads(report)
# Check metadata
assert "metadata" in data
assert data["metadata"]["duplicate_groups"] == 1
assert data["metadata"]["library_root"] == str(library_root)
# Check duplicates
assert "duplicates" in data
assert len(data["duplicates"]) == 1
dup = data["duplicates"][0]
assert dup["identity"]["type"] == "movie"
assert dup["identity"]["title"] == "Inception"
assert dup["identity"]["year"] == 2010
assert dup["file_count"] == 2
assert len(dup["files"]) == 2
def test_generate_text_report_series_duplicates(self):
"""Test generating text report with series duplicates."""
identity = SeriesIdentity("Breaking Bad", 1, [1], 0.9, False, "Breaking.Bad.S01E01.mkv")
files = [
VideoFile(
Path("/series/Breaking.Bad.S01E01.1080p.mkv"),
"Breaking.Bad.S01E01.1080p.mkv",
1500000000,
datetime.now(),
"series"
),
VideoFile(
Path("/series/Breaking.Bad.S01E01.720p.mkv"),
"Breaking.Bad.S01E01.720p.mkv",
800000000,
datetime.now(),
"series"
),
]
quality_comparison = [
{'filename': 'Breaking.Bad.S01E01.1080p.mkv', 'path': '/series/Breaking.Bad.S01E01.1080p.mkv', 'size_bytes': 1500000000},
{'filename': 'Breaking.Bad.S01E01.720p.mkv', 'path': '/series/Breaking.Bad.S01E01.720p.mkv', 'size_bytes': 800000000}
]
duplicates = [
DuplicateGroup(identity, files, quality_comparison)
]
library_root = Path("/mnt/nas/videos")
report = generate_duplicate_report(duplicates, "text", library_root)
# Check series format
assert "Breaking Bad - S01E1" in report
assert "Files: 2" in report
def test_generate_text_report_empty(self):
"""Test generating text report with no duplicates."""
duplicates = []
library_root = Path("/mnt/nas/videos")
report = generate_duplicate_report(duplicates, "text", library_root)
assert "DUPLICATE FILES REPORT" in report
assert "No duplicate files detected." in report
def test_generate_json_report_empty(self):
"""Test generating JSON report with no duplicates."""
duplicates = []
library_root = Path("/mnt/nas/videos")
report = generate_duplicate_report(duplicates, "json", library_root)
data = json.loads(report)
assert data["metadata"]["duplicate_groups"] == 0
assert len(data["duplicates"]) == 0
def test_invalid_format_raises_error(self):
"""Test that invalid format raises ValueError."""
duplicates = []
library_root = Path("/mnt/nas/videos")
with pytest.raises(ValueError, match="Invalid format"):
generate_duplicate_report(duplicates, "csv", library_root)
def test_sorted_by_file_size(self):
"""Test that duplicate groups are sorted by largest file size."""
# Create two duplicate groups with different sizes
identity1 = MovieIdentity("Small Movie", 2020, 0.9, False, "Small.Movie.mkv")
files1 = [
VideoFile(Path("/movies/Small.Movie.1.mkv"), "Small.Movie.1.mkv", 500000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Small.Movie.2.mkv"), "Small.Movie.2.mkv", 600000000, datetime.now(), "movie"),
]
quality1 = [
{'filename': 'Small.Movie.1.mkv', 'path': '/movies/Small.Movie.1.mkv', 'size_bytes': 500000000},
{'filename': 'Small.Movie.2.mkv', 'path': '/movies/Small.Movie.2.mkv', 'size_bytes': 600000000}
]
identity2 = MovieIdentity("Large Movie", 2021, 0.9, False, "Large.Movie.mkv")
files2 = [
VideoFile(Path("/movies/Large.Movie.1.mkv"), "Large.Movie.1.mkv", 2000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Large.Movie.2.mkv"), "Large.Movie.2.mkv", 1800000000, datetime.now(), "movie"),
]
quality2 = [
{'filename': 'Large.Movie.1.mkv', 'path': '/movies/Large.Movie.1.mkv', 'size_bytes': 2000000000},
{'filename': 'Large.Movie.2.mkv', 'path': '/movies/Large.Movie.2.mkv', 'size_bytes': 1800000000}
]
duplicates = [
DuplicateGroup(identity1, files1, quality1),
DuplicateGroup(identity2, files2, quality2)
]
library_root = Path("/mnt/nas/videos")
report = generate_duplicate_report(duplicates, "text", library_root)
# Large Movie should appear before Small Movie
large_pos = report.find("Large Movie")
small_pos = report.find("Small Movie")
assert large_pos < small_pos
class TestSummaryReport:
"""Test summary report generation."""
def test_generate_summary_report(self):
"""Test generating summary report with various files."""
files = [
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(), "movie"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(), "anime"),
VideoFile(Path("/other/Random.mkv"), "Random.mkv", 500000000, datetime.now(), "other"),
]
library_root = Path("/mnt/nas/videos")
report = generate_summary_report(files, library_root)
# Check report structure
assert "LIBRARY SUMMARY REPORT" in report
assert "Generated:" in report
assert str(library_root) in report
# Check totals
assert "Total Files: 6" in report
assert "Total Size:" in report
# Check category breakdown
assert "Category Breakdown:" in report
assert "Movie:" in report
assert "Series:" in report
assert "Anime:" in report
assert "Other:" in report
# Check category counts
assert "Files: 2" in report # Movies
def test_generate_summary_report_empty(self):
"""Test generating summary report with no files."""
files = []
library_root = Path("/mnt/nas/videos")
report = generate_summary_report(files, library_root)
assert "LIBRARY SUMMARY REPORT" in report
assert "Total Files: 0" in report
assert "Total Size: 0.00 B" in report
def test_generate_summary_report_single_category(self):
"""Test generating summary report with files in single category."""
files = [
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 2000000000, datetime.now(), "movie"),
]
library_root = Path("/mnt/nas/videos")
report = generate_summary_report(files, library_root)
assert "Total Files: 2" in report
assert "Movie:" in report
assert "Files: 2" in report
class TestFormatHelpers:
"""Test formatting helper functions."""
def test_format_episode_list_single(self):
"""Test formatting single episode."""
assert _format_episode_list([5]) == "5"
def test_format_episode_list_range(self):
"""Test formatting consecutive episode range."""
assert _format_episode_list([1, 2, 3, 4, 5]) == "1-5"
def test_format_episode_list_mixed(self):
"""Test formatting mixed ranges and singles."""
assert _format_episode_list([1, 2, 3, 5, 6, 8]) == "1-3, 5-6, 8"
def test_format_episode_list_non_sequential(self):
"""Test formatting non-sequential episodes."""
assert _format_episode_list([1, 3, 5, 7]) == "1, 3, 5, 7"
def test_format_episode_list_empty(self):
"""Test formatting empty episode list."""
assert _format_episode_list([]) == "none"
def test_format_episode_list_unsorted(self):
"""Test formatting unsorted episode list."""
assert _format_episode_list([5, 1, 3, 2, 4]) == "1-5"
def test_format_size_bytes(self):
"""Test formatting bytes."""
assert _format_size(512) == "512.00 B"
def test_format_size_kilobytes(self):
"""Test formatting kilobytes."""
assert _format_size(1024) == "1.00 KB"
assert _format_size(2048) == "2.00 KB"
def test_format_size_megabytes(self):
"""Test formatting megabytes."""
assert _format_size(1048576) == "1.00 MB"
assert _format_size(5242880) == "5.00 MB"
def test_format_size_gigabytes(self):
"""Test formatting gigabytes."""
assert _format_size(1073741824) == "1.00 GB"
assert _format_size(2147483648) == "2.00 GB"
def test_format_size_terabytes(self):
"""Test formatting terabytes."""
assert _format_size(1099511627776) == "1.00 TB"
def test_format_duration_seconds(self):
"""Test formatting seconds only."""
assert _format_duration(30) == "30s"
assert _format_duration(0) == "0s"
def test_format_duration_minutes(self):
"""Test formatting minutes and seconds."""
assert _format_duration(90) == "1m 30s"
assert _format_duration(120) == "2m"
def test_format_duration_hours(self):
"""Test formatting hours, minutes, and seconds."""
assert _format_duration(3665) == "1h 1m 5s"
assert _format_duration(7200) == "2h"
assert _format_duration(7260) == "2h 1m"
tests/test_reports_integration.py
+145
View File
@@ -0,0 +1,145 @@
"""Integration tests for report generation with analysis engine.
Tests the complete workflow from analysis to report generation.
"""
import json
from pathlib import Path
from datetime import datetime
from vlm.models import SeriesIdentity, VideoFile, MovieIdentity
from vlm.analysis import analyze_series_completeness, detect_duplicates
from vlm.reports import generate_completeness_report, generate_duplicate_report, generate_summary_report
class TestReportsIntegration:
"""Test report generation integrated with analysis engine."""
def test_completeness_workflow(self):
"""Test complete workflow from series analysis to completeness report."""
# Create test episodes with gaps
episodes = [
SeriesIdentity("Breaking Bad", 1, [1], 0.9, False, "Breaking.Bad.S01E01.mkv"),
SeriesIdentity("Breaking Bad", 1, [2], 0.9, False, "Breaking.Bad.S01E02.mkv"),
SeriesIdentity("Breaking Bad", 1, [4], 0.9, False, "Breaking.Bad.S01E04.mkv"),
SeriesIdentity("The Wire", 1, [1], 0.9, False, "The.Wire.S01E01.mkv"),
SeriesIdentity("The Wire", 1, [3], 0.9, False, "The.Wire.S01E03.mkv"),
]
# Analyze completeness
analysis = analyze_series_completeness(episodes)
# Generate text report
library_root = Path("/mnt/nas/videos")
text_report = generate_completeness_report(analysis, "text", library_root)
# Verify report contains expected information
assert "Breaking Bad" in text_report
assert "The Wire" in text_report
assert "Episodes missing:" in text_report
# Generate JSON report
json_report = generate_completeness_report(analysis, "json", library_root)
data = json.loads(json_report)
# Verify JSON structure
assert data["metadata"]["series_count"] == 2
assert len(data["series"]) == 2
def test_duplicate_workflow(self):
"""Test complete workflow from duplicate detection to duplicate report."""
# Create test identities and files
identities = [
MovieIdentity("The Matrix", 1999, 0.9, False, "The.Matrix.1999.1080p.mkv"),
MovieIdentity("The Matrix", 1999, 0.9, False, "The.Matrix.1999.720p.mkv"),
MovieIdentity("Inception", 2010, 0.9, False, "Inception.2010.mkv"),
]
files = [
VideoFile(
Path("/movies/The.Matrix.1999.1080p.mkv"),
"The.Matrix.1999.1080p.mkv",
2000000000,
datetime.now(),
"movie",
resolution="1920x1080",
codec="h264"
),
VideoFile(
Path("/movies/The.Matrix.1999.720p.mkv"),
"The.Matrix.1999.720p.mkv",
1000000000,
datetime.now(),
"movie",
resolution="1280x720",
codec="h264"
),
VideoFile(
Path("/movies/Inception.2010.mkv"),
"Inception.2010.mkv",
1500000000,
datetime.now(),
"movie"
),
]
# Detect duplicates
duplicates = detect_duplicates(identities, files)
# Generate text report
library_root = Path("/mnt/nas/videos")
text_report = generate_duplicate_report(duplicates, "text", library_root)
# Verify report contains expected information
assert "The Matrix (1999)" in text_report
assert "1920x1080" in text_report
assert "1280x720" in text_report
# Generate JSON report
json_report = generate_duplicate_report(duplicates, "json", library_root)
data = json.loads(json_report)
# Verify JSON structure
assert data["metadata"]["duplicate_groups"] == 1
assert len(data["duplicates"]) == 1
assert data["duplicates"][0]["file_count"] == 2
def test_summary_workflow(self):
"""Test summary report generation with mixed file types."""
# Create test files
files = [
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(), "movie"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(), "anime"),
]
# Generate summary report
library_root = Path("/mnt/nas/videos")
report = generate_summary_report(files, library_root)
# Verify report contains expected information
assert "Total Files: 5" in report
assert "Movie:" in report
assert "Series:" in report
assert "Anime:" in report
assert "Category Breakdown:" in report
def test_all_reports_include_metadata(self):
"""Test that all reports include generation timestamp and library root."""
library_root = Path("/mnt/nas/videos")
# Test completeness report
completeness_report = generate_completeness_report([], "text", library_root)
assert "Generated:" in completeness_report
assert str(library_root) in completeness_report
# Test duplicate report
duplicate_report = generate_duplicate_report([], "text", library_root)
assert "Generated:" in duplicate_report
assert str(library_root) in duplicate_report
# Test summary report
summary_report = generate_summary_report([], library_root)
assert "Generated:" in summary_report
assert str(library_root) in summary_report
tests/test_scanner.py
+869
View File
@@ -0,0 +1,869 @@
"""Unit tests for the inventory scanner module."""
import os
import tempfile
from datetime import datetime
from pathlib import Path
from unittest.mock import patch, MagicMock
import subprocess
import json
import pytest
from vlm.config import Config
from vlm.models import VideoFile
from vlm.scanner import (
categorize_file,
scan_library,
extract_metadata,
)
class TestScanLibrary:
"""Tests for the scan_library function."""
def test_scan_empty_directory(self, tmp_path):
"""Test scanning an empty directory returns empty list."""
config = Config(library_root=tmp_path)
result = scan_library(tmp_path, config)
assert result == []
def test_scan_nonexistent_directory(self, tmp_path):
"""Test scanning a nonexistent directory returns empty list."""
nonexistent = tmp_path / "nonexistent"
config = Config(library_root=nonexistent)
result = scan_library(nonexistent, config)
assert result == []
def test_scan_discovers_video_files(self, tmp_path):
"""Test scanning discovers video files with correct extensions."""
# Create test structure
movie_dir = tmp_path / "movie"
movie_dir.mkdir()
# Create video files
video1 = movie_dir / "test1.mp4"
video2 = movie_dir / "test2.mkv"
video1.touch()
video2.touch()
# Create non-video file
text_file = movie_dir / "readme.txt"
text_file.touch()
config = Config(library_root=tmp_path)
result = scan_library(tmp_path, config)
# Should find only video files
assert len(result) == 2
filenames = {vf.filename for vf in result}
assert filenames == {"test1.mp4", "test2.mkv"}
def test_scan_recursive(self, tmp_path):
"""Test scanning recursively discovers files in subdirectories."""
# Create nested structure
movie_dir = tmp_path / "movie"
subdir = movie_dir / "subdir"
subdir.mkdir(parents=True)
# Create files at different levels
video1 = movie_dir / "movie1.mp4"
video2 = subdir / "movie2.mkv"
video1.touch()
video2.touch()
config = Config(library_root=tmp_path)
result = scan_library(tmp_path, config)
assert len(result) == 2
filenames = {vf.filename for vf in result}
assert filenames == {"movie1.mp4", "movie2.mkv"}
def test_scan_filters_by_extension(self, tmp_path):
"""Test scanning filters files by configured extensions."""
movie_dir = tmp_path / "movie"
movie_dir.mkdir()
# Create files with various extensions
mp4_file = movie_dir / "video.mp4"
mkv_file = movie_dir / "video.mkv"
avi_file = movie_dir / "video.avi"
txt_file = movie_dir / "readme.txt"
mp4_file.touch()
mkv_file.touch()
avi_file.touch()
txt_file.touch()
# Configure to only accept .mp4 and .mkv
config = Config(
library_root=tmp_path,
video_extensions=[".mp4", ".mkv"]
)
result = scan_library(tmp_path, config)
assert len(result) == 2
filenames = {vf.filename for vf in result}
assert filenames == {"video.mp4", "video.mkv"}
def test_scan_records_metadata(self, tmp_path):
"""Test scanning records file metadata correctly."""
movie_dir = tmp_path / "movie"
movie_dir.mkdir()
video_file = movie_dir / "test.mp4"
video_file.write_text("test content")
config = Config(library_root=tmp_path)
result = scan_library(tmp_path, config)
assert len(result) == 1
vf = result[0]
# Check metadata
assert vf.filename == "test.mp4"
assert vf.path == video_file
assert vf.size_bytes > 0
assert isinstance(vf.modified_timestamp, datetime)
assert vf.category == "movie"
def test_scan_categorizes_files(self, tmp_path):
"""Test scanning categorizes files based on directory structure."""
# Create category directories
movie_dir = tmp_path / "movie"
series_dir = tmp_path / "series"
anime_dir = tmp_path / "anime"
other_dir = tmp_path / "other"
movie_dir.mkdir()
series_dir.mkdir()
anime_dir.mkdir()
other_dir.mkdir()
# Create files in each category
(movie_dir / "movie.mp4").touch()
(series_dir / "series.mkv").touch()
(anime_dir / "anime.avi").touch()
(other_dir / "other.mov").touch()
config = Config(library_root=tmp_path)
result = scan_library(tmp_path, config)
assert len(result) == 4
# Check categories
categories = {vf.filename: vf.category for vf in result}
assert categories["movie.mp4"] == "movie"
assert categories["series.mkv"] == "series"
assert categories["anime.avi"] == "anime"
assert categories["other.mov"] == "other"
def test_scan_skips_hidden_files(self, tmp_path):
"""Test scanning skips hidden files and directories."""
movie_dir = tmp_path / "movie"
hidden_dir = tmp_path / ".hidden"
movie_dir.mkdir()
hidden_dir.mkdir()
# Create visible and hidden files
visible = movie_dir / "visible.mp4"
hidden_file = movie_dir / ".hidden.mp4"
hidden_dir_file = hidden_dir / "file.mp4"
visible.touch()
hidden_file.touch()
hidden_dir_file.touch()
config = Config(library_root=tmp_path)
result = scan_library(tmp_path, config)
# Should only find visible file
assert len(result) == 1
assert result[0].filename == "visible.mp4"
def test_scan_handles_inaccessible_files(self, tmp_path):
"""Test scanning continues when encountering inaccessible files."""
movie_dir = tmp_path / "movie"
movie_dir.mkdir()
# Create accessible files
video1 = movie_dir / "video1.mp4"
video2 = movie_dir / "video2.mp4"
video1.touch()
video2.touch()
config = Config(library_root=tmp_path)
result = scan_library(tmp_path, config)
# Should find both files (no permission errors in test environment)
assert len(result) == 2
class TestCategorizeFile:
"""Tests for the categorize_file function."""
def test_categorize_movie(self, tmp_path):
"""Test categorizing a file in movie directory."""
movie_dir = tmp_path / "movie"
movie_dir.mkdir()
video_file = movie_dir / "test.mp4"
video_file.touch()
category = categorize_file(video_file, tmp_path)
assert category == "movie"
def test_categorize_series(self, tmp_path):
"""Test categorizing a file in series directory."""
series_dir = tmp_path / "series"
series_dir.mkdir()
video_file = series_dir / "test.mkv"
video_file.touch()
category = categorize_file(video_file, tmp_path)
assert category == "series"
def test_categorize_anime(self, tmp_path):
"""Test categorizing a file in anime directory."""
anime_dir = tmp_path / "anime"
anime_dir.mkdir()
video_file = anime_dir / "test.avi"
video_file.touch()
category = categorize_file(video_file, tmp_path)
assert category == "anime"
def test_categorize_other(self, tmp_path):
"""Test categorizing a file in other directory."""
other_dir = tmp_path / "other"
other_dir.mkdir()
video_file = other_dir / "test.mov"
video_file.touch()
category = categorize_file(video_file, tmp_path)
assert category == "other"
def test_categorize_nested_file(self, tmp_path):
"""Test categorizing a file in nested subdirectory."""
movie_dir = tmp_path / "movie" / "subdir" / "nested"
movie_dir.mkdir(parents=True)
video_file = movie_dir / "test.mp4"
video_file.touch()
category = categorize_file(video_file, tmp_path)
assert category == "movie"
def test_categorize_case_insensitive(self, tmp_path):
"""Test categorization is case-insensitive."""
movie_dir = tmp_path / "Movie"
movie_dir.mkdir()
video_file = movie_dir / "test.mp4"
video_file.touch()
category = categorize_file(video_file, tmp_path)
assert category == "movie"
def test_categorize_file_in_root(self, tmp_path):
"""Test categorizing a file directly in library root."""
video_file = tmp_path / "test.mp4"
video_file.touch()
category = categorize_file(video_file, tmp_path)
assert category == "other"
def test_categorize_unknown_directory(self, tmp_path):
"""Test categorizing a file in unknown directory."""
unknown_dir = tmp_path / "random"
unknown_dir.mkdir()
video_file = unknown_dir / "test.mp4"
video_file.touch()
category = categorize_file(video_file, tmp_path)
assert category == "other"
class TestExtractMetadata:
"""Tests for the extract_metadata function."""
def test_extract_metadata_with_ffprobe_available(self, tmp_path):
"""Test metadata extraction when ffprobe is available and returns valid data."""
video_file = tmp_path / "test.mp4"
video_file.touch()
# Mock ffprobe output
mock_output = {
"streams": [
{
"codec_type": "video",
"codec_name": "h264",
"width": 1920,
"height": 1080
}
],
"format": {
"duration": "120.5",
"bit_rate": "5000000"
}
}
with patch('subprocess.run') as mock_run:
mock_run.return_value = MagicMock(
returncode=0,
stdout=json.dumps(mock_output),
stderr=""
)
result = extract_metadata(video_file)
assert result['resolution'] == "1920x1080"
assert result['codec'] == "h264"
assert result['duration_seconds'] == 120.5
assert result['bitrate_kbps'] == 5000
def test_extract_metadata_ffprobe_not_available(self, tmp_path):
"""Test metadata extraction when ffprobe is not installed."""
video_file = tmp_path / "test.mp4"
video_file.touch()
with patch('subprocess.run', side_effect=FileNotFoundError):
result = extract_metadata(video_file)
assert result == {}
def test_extract_metadata_ffprobe_fails(self, tmp_path):
"""Test metadata extraction when ffprobe fails."""
video_file = tmp_path / "test.mp4"
video_file.touch()
with patch('subprocess.run') as mock_run:
mock_run.return_value = MagicMock(
returncode=1,
stdout="",
stderr="Error processing file"
)
result = extract_metadata(video_file)
assert result == {}
def test_extract_metadata_ffprobe_timeout(self, tmp_path):
"""Test metadata extraction when ffprobe times out."""
video_file = tmp_path / "test.mp4"
video_file.touch()
with patch('subprocess.run', side_effect=subprocess.TimeoutExpired('ffprobe', 10)):
result = extract_metadata(video_file)
assert result == {}
def test_extract_metadata_invalid_json(self, tmp_path):
"""Test metadata extraction when ffprobe returns invalid JSON."""
video_file = tmp_path / "test.mp4"
video_file.touch()
with patch('subprocess.run') as mock_run:
mock_run.return_value = MagicMock(
returncode=0,
stdout="invalid json",
stderr=""
)
result = extract_metadata(video_file)
assert result == {}
def test_extract_metadata_partial_data(self, tmp_path):
"""Test metadata extraction with partial data available."""
video_file = tmp_path / "test.mp4"
video_file.touch()
# Mock ffprobe output with only some fields
mock_output = {
"streams": [
{
"codec_type": "video",
"codec_name": "h264"
# Missing width and height
}
],
"format": {
"duration": "120.5"
# Missing bit_rate
}
}
with patch('subprocess.run') as mock_run:
mock_run.return_value = MagicMock(
returncode=0,
stdout=json.dumps(mock_output),
stderr=""
)
result = extract_metadata(video_file)
assert result['codec'] == "h264"
assert result['duration_seconds'] == 120.5
assert 'resolution' not in result
assert 'bitrate_kbps' not in result
def test_extract_metadata_no_video_stream(self, tmp_path):
"""Test metadata extraction when no video stream is found."""
video_file = tmp_path / "test.mp4"
video_file.touch()
# Mock ffprobe output with only audio stream
mock_output = {
"streams": [
{
"codec_type": "audio",
"codec_name": "aac"
}
],
"format": {
"duration": "120.5",
"bit_rate": "5000000"
}
}
with patch('subprocess.run') as mock_run:
mock_run.return_value = MagicMock(
returncode=0,
stdout=json.dumps(mock_output),
stderr=""
)
result = extract_metadata(video_file)
# Should still extract format-level metadata
assert result['duration_seconds'] == 120.5
assert result['bitrate_kbps'] == 5000
assert 'resolution' not in result
assert 'codec' not in result
def test_scan_library_with_metadata_extraction(self, tmp_path):
"""Test that scan_library integrates metadata extraction."""
movie_dir = tmp_path / "movie"
movie_dir.mkdir()
video_file = movie_dir / "test.mp4"
video_file.touch()
# Mock ffprobe output
mock_output = {
"streams": [
{
"codec_type": "video",
"codec_name": "h264",
"width": 1920,
"height": 1080
}
],
"format": {
"duration": "120.5",
"bit_rate": "5000000"
}
}
with patch('subprocess.run') as mock_run:
mock_run.return_value = MagicMock(
returncode=0,
stdout=json.dumps(mock_output),
stderr=""
)
config = Config(library_root=tmp_path)
result = scan_library(tmp_path, config)
assert len(result) == 1
vf = result[0]
# Check that metadata was extracted
assert vf.resolution == "1920x1080"
assert vf.codec == "h264"
assert vf.duration_seconds == 120.5
assert vf.bitrate_kbps == 5000
def test_scan_library_without_ffprobe(self, tmp_path):
"""Test that scan_library works gracefully without ffprobe."""
movie_dir = tmp_path / "movie"
movie_dir.mkdir()
video_file = movie_dir / "test.mp4"
video_file.touch()
with patch('subprocess.run', side_effect=FileNotFoundError):
config = Config(library_root=tmp_path)
result = scan_library(tmp_path, config)
assert len(result) == 1
vf = result[0]
# Check that file was still scanned without metadata
assert vf.filename == "test.mp4"
assert vf.resolution is None
assert vf.codec is None
assert vf.duration_seconds is None
assert vf.bitrate_kbps is None
class TestInventoryReports:
"""Tests for inventory report generation functions."""
def test_save_inventory_csv_basic(self, tmp_path):
"""Test saving inventory to CSV format with basic data."""
# Create test video files
video_files = [
VideoFile(
path=Path("/library/movie/test1.mp4"),
filename="test1.mp4",
size_bytes=1024000,
modified_timestamp=datetime(2024, 1, 15, 10, 30, 0),
category="movie",
resolution="1920x1080",
codec="h264",
duration_seconds=120.5,
bitrate_kbps=5000
),
VideoFile(
path=Path("/library/series/test2.mkv"),
filename="test2.mkv",
size_bytes=2048000,
modified_timestamp=datetime(2024, 1, 16, 14, 45, 0),
category="series",
resolution="1280x720",
codec="h265",
duration_seconds=45.0,
bitrate_kbps=3000
)
]
output_file = tmp_path / "inventory.csv"
library_root = Path("/library")
from vlm.scanner import save_inventory_csv
save_inventory_csv(video_files, output_file, library_root)
# Verify file was created
assert output_file.exists()
# Read and verify content
with open(output_file, 'r', encoding='utf-8') as f:
content = f.read()
# Check metadata comments
assert "# Generated:" in content
assert "# Library Root: /library" in content
# Check header
assert "path,filename,size_bytes,modified_timestamp,category,resolution,codec,duration_seconds,bitrate_kbps" in content
# Check data rows
assert "test1.mp4" in content
assert "1024000" in content
assert "movie" in content
assert "1920x1080" in content
assert "h264" in content
assert "120.5" in content
assert "5000" in content
assert "test2.mkv" in content
assert "2048000" in content
assert "series" in content
assert "1280x720" in content
assert "h265" in content
assert "45.0" in content
assert "3000" in content
def test_save_inventory_csv_with_missing_metadata(self, tmp_path):
"""Test saving inventory to CSV with missing optional metadata."""
# Create video file without optional metadata
video_files = [
VideoFile(
path=Path("/library/movie/test.mp4"),
filename="test.mp4",
size_bytes=1024000,
modified_timestamp=datetime(2024, 1, 15, 10, 30, 0),
category="movie",
resolution=None,
codec=None,
duration_seconds=None,
bitrate_kbps=None
)
]
output_file = tmp_path / "inventory.csv"
library_root = Path("/library")
from vlm.scanner import save_inventory_csv
save_inventory_csv(video_files, output_file, library_root)
# Verify file was created
assert output_file.exists()
# Read and verify content
import csv
with open(output_file, 'r', encoding='utf-8') as f:
# Skip comment lines
lines = [line for line in f if not line.startswith('#')]
reader = csv.DictReader(lines)
rows = list(reader)
assert len(rows) == 1
row = rows[0]
# Check required fields
assert row['filename'] == 'test.mp4'
assert row['size_bytes'] == '1024000'
assert row['category'] == 'movie'
# Check optional fields are empty strings
assert row['resolution'] == ''
assert row['codec'] == ''
assert row['duration_seconds'] == ''
assert row['bitrate_kbps'] == ''
def test_save_inventory_csv_empty_list(self, tmp_path):
"""Test saving empty inventory to CSV."""
video_files = []
output_file = tmp_path / "inventory.csv"
library_root = Path("/library")
from vlm.scanner import save_inventory_csv
save_inventory_csv(video_files, output_file, library_root)
# Verify file was created
assert output_file.exists()
# Read and verify content
with open(output_file, 'r', encoding='utf-8') as f:
content = f.read()
# Should have metadata and header but no data rows
assert "# Generated:" in content
assert "# Library Root:" in content
assert "path,filename,size_bytes" in content
def test_save_inventory_csv_creates_directory(self, tmp_path):
"""Test that save_inventory_csv creates output directory if needed."""
video_files = [
VideoFile(
path=Path("/library/movie/test.mp4"),
filename="test.mp4",
size_bytes=1024000,
modified_timestamp=datetime(2024, 1, 15, 10, 30, 0),
category="movie"
)
]
# Use nested directory that doesn't exist
output_file = tmp_path / "reports" / "inventory.csv"
library_root = Path("/library")
from vlm.scanner import save_inventory_csv
save_inventory_csv(video_files, output_file, library_root)
# Verify file was created
assert output_file.exists()
def test_save_inventory_json_basic(self, tmp_path):
"""Test saving inventory to JSON format with basic data."""
# Create test video files
video_files = [
VideoFile(
path=Path("/library/movie/test1.mp4"),
filename="test1.mp4",
size_bytes=1024000,
modified_timestamp=datetime(2024, 1, 15, 10, 30, 0),
category="movie",
resolution="1920x1080",
codec="h264",
duration_seconds=120.5,
bitrate_kbps=5000
),
VideoFile(
path=Path("/library/series/test2.mkv"),
filename="test2.mkv",
size_bytes=2048000,
modified_timestamp=datetime(2024, 1, 16, 14, 45, 0),
category="series",
resolution="1280x720",
codec="h265",
duration_seconds=45.0,
bitrate_kbps=3000
)
]
output_file = tmp_path / "inventory.json"
library_root = Path("/library")
from vlm.scanner import save_inventory_json
save_inventory_json(video_files, output_file, library_root)
# Verify file was created
assert output_file.exists()
# Read and verify content
with open(output_file, 'r', encoding='utf-8') as f:
data = json.load(f)
# Check metadata
assert 'metadata' in data
assert 'generated' in data['metadata']
assert data['metadata']['library_root'] == '/library'
assert data['metadata']['file_count'] == 2
# Check files
assert 'files' in data
assert len(data['files']) == 2
# Check first file
file1 = data['files'][0]
assert file1['filename'] == 'test1.mp4'
assert file1['size_bytes'] == 1024000
assert file1['category'] == 'movie'
assert file1['resolution'] == '1920x1080'
assert file1['codec'] == 'h264'
assert file1['duration_seconds'] == 120.5
assert file1['bitrate_kbps'] == 5000
# Check second file
file2 = data['files'][1]
assert file2['filename'] == 'test2.mkv'
assert file2['size_bytes'] == 2048000
assert file2['category'] == 'series'
assert file2['resolution'] == '1280x720'
assert file2['codec'] == 'h265'
assert file2['duration_seconds'] == 45.0
assert file2['bitrate_kbps'] == 3000
def test_save_inventory_json_with_missing_metadata(self, tmp_path):
"""Test saving inventory to JSON with missing optional metadata."""
# Create video file without optional metadata
video_files = [
VideoFile(
path=Path("/library/movie/test.mp4"),
filename="test.mp4",
size_bytes=1024000,
modified_timestamp=datetime(2024, 1, 15, 10, 30, 0),
category="movie",
resolution=None,
codec=None,
duration_seconds=None,
bitrate_kbps=None
)
]
output_file = tmp_path / "inventory.json"
library_root = Path("/library")
from vlm.scanner import save_inventory_json
save_inventory_json(video_files, output_file, library_root)
# Verify file was created
assert output_file.exists()
# Read and verify content
with open(output_file, 'r', encoding='utf-8') as f:
data = json.load(f)
assert len(data['files']) == 1
file_data = data['files'][0]
# Check required fields
assert file_data['filename'] == 'test.mp4'
assert file_data['size_bytes'] == 1024000
assert file_data['category'] == 'movie'
# Check optional fields are null
assert file_data['resolution'] is None
assert file_data['codec'] is None
assert file_data['duration_seconds'] is None
assert file_data['bitrate_kbps'] is None
def test_save_inventory_json_empty_list(self, tmp_path):
"""Test saving empty inventory to JSON."""
video_files = []
output_file = tmp_path / "inventory.json"
library_root = Path("/library")
from vlm.scanner import save_inventory_json
save_inventory_json(video_files, output_file, library_root)
# Verify file was created
assert output_file.exists()
# Read and verify content
with open(output_file, 'r', encoding='utf-8') as f:
data = json.load(f)
# Should have metadata but no files
assert data['metadata']['file_count'] == 0
assert len(data['files']) == 0
def test_save_inventory_json_creates_directory(self, tmp_path):
"""Test that save_inventory_json creates output directory if needed."""
video_files = [
VideoFile(
path=Path("/library/movie/test.mp4"),
filename="test.mp4",
size_bytes=1024000,
modified_timestamp=datetime(2024, 1, 15, 10, 30, 0),
category="movie"
)
]
# Use nested directory that doesn't exist
output_file = tmp_path / "reports" / "inventory.json"
library_root = Path("/library")
from vlm.scanner import save_inventory_json
save_inventory_json(video_files, output_file, library_root)
# Verify file was created
assert output_file.exists()
def test_csv_and_json_consistency(self, tmp_path):
"""Test that CSV and JSON exports contain the same data."""
# Create test video files
video_files = [
VideoFile(
path=Path("/library/movie/test.mp4"),
filename="test.mp4",
size_bytes=1024000,
modified_timestamp=datetime(2024, 1, 15, 10, 30, 0),
category="movie",
resolution="1920x1080",
codec="h264",
duration_seconds=120.5,
bitrate_kbps=5000
)
]
csv_file = tmp_path / "inventory.csv"
json_file = tmp_path / "inventory.json"
library_root = Path("/library")
from vlm.scanner import save_inventory_csv, save_inventory_json
save_inventory_csv(video_files, csv_file, library_root)
save_inventory_json(video_files, json_file, library_root)
# Read CSV data
import csv
with open(csv_file, 'r', encoding='utf-8') as f:
lines = [line for line in f if not line.startswith('#')]
reader = csv.DictReader(lines)
csv_rows = list(reader)
# Read JSON data
with open(json_file, 'r', encoding='utf-8') as f:
json_data = json.load(f)
# Compare data
assert len(csv_rows) == len(json_data['files'])
csv_row = csv_rows[0]
json_file_data = json_data['files'][0]
# Compare key fields
assert csv_row['filename'] == json_file_data['filename']
assert csv_row['size_bytes'] == str(json_file_data['size_bytes'])
assert csv_row['category'] == json_file_data['category']
assert csv_row['resolution'] == json_file_data['resolution']
assert csv_row['codec'] == json_file_data['codec']
tests/test_state.py
+387
View File
@@ -0,0 +1,387 @@
"""Unit tests for State Store operations."""
import json
import pytest
from datetime import datetime
from pathlib import Path
from vlm.state import (
load_state,
save_state,
StateManager,
VALID_STATUSES
)
from vlm.models import FileState, StateStore
class TestLoadSaveState:
"""Tests for load_state and save_state functions."""
def test_save_and_load_empty_state(self, tmp_path):
"""Test saving and loading an empty state store."""
state_path = tmp_path / "state.json"
# Create empty state store
store = StateStore(
states={},
version='1.0',
last_updated=datetime(2024, 1, 1, 12, 0, 0)
)
# Save and load
save_state(store, state_path)
loaded = load_state(state_path)
assert loaded.states == {}
assert loaded.version == '1.0'
assert loaded.last_updated == datetime(2024, 1, 1, 12, 0, 0)
def test_save_and_load_with_states(self, tmp_path):
"""Test saving and loading state store with file states."""
state_path = tmp_path / "state.json"
# Create state store with some states
file1 = Path("/videos/movie1.mp4")
file2 = Path("/videos/series/episode.mkv")
store = StateStore(
states={
str(file1): FileState(
file_path=file1,
status="reviewed",
reason="Checked manually",
updated_at=datetime(2024, 1, 1, 12, 0, 0)
),
str(file2): FileState(
file_path=file2,
status="ignored",
reason=None,
updated_at=datetime(2024, 1, 2, 12, 0, 0)
)
},
version='1.0',
last_updated=datetime(2024, 1, 2, 12, 0, 0)
)
# Save and load
save_state(store, state_path)
loaded = load_state(state_path)
assert len(loaded.states) == 2
assert str(file1) in loaded.states
assert str(file2) in loaded.states
state1 = loaded.states[str(file1)]
assert state1.file_path == file1
assert state1.status == "reviewed"
assert state1.reason == "Checked manually"
assert state1.updated_at == datetime(2024, 1, 1, 12, 0, 0)
state2 = loaded.states[str(file2)]
assert state2.file_path == file2
assert state2.status == "ignored"
assert state2.reason is None
assert state2.updated_at == datetime(2024, 1, 2, 12, 0, 0)
def test_save_creates_parent_directory(self, tmp_path):
"""Test that save_state creates parent directories if needed."""
state_path = tmp_path / "subdir" / "nested" / "state.json"
store = StateStore(
states={},
version='1.0',
last_updated=datetime.now()
)
save_state(store, state_path)
assert state_path.exists()
assert state_path.parent.exists()
def test_load_nonexistent_file_raises_error(self, tmp_path):
"""Test that loading a nonexistent file raises FileNotFoundError."""
state_path = tmp_path / "nonexistent.json"
with pytest.raises(FileNotFoundError):
load_state(state_path)
def test_load_invalid_json_raises_error(self, tmp_path):
"""Test that loading invalid JSON raises JSONDecodeError."""
state_path = tmp_path / "invalid.json"
state_path.write_text("not valid json {")
with pytest.raises(json.JSONDecodeError):
load_state(state_path)
def test_saved_json_is_valid(self, tmp_path):
"""Test that saved JSON is valid and human-readable."""
state_path = tmp_path / "state.json"
file1 = Path("/videos/movie.mp4")
store = StateStore(
states={
str(file1): FileState(
file_path=file1,
status="reviewed",
reason="Test",
updated_at=datetime(2024, 1, 1, 12, 0, 0)
)
},
version='1.0',
last_updated=datetime(2024, 1, 1, 12, 0, 0)
)
save_state(store, state_path)
# Verify JSON is valid by loading it directly
with open(state_path, 'r') as f:
data = json.load(f)
assert 'states' in data
assert 'version' in data
assert 'last_updated' in data
assert data['version'] == '1.0'
class TestStateManager:
"""Tests for StateManager class."""
def test_init_creates_new_state_if_not_exists(self, tmp_path):
"""Test that StateManager creates a new state store if file doesn't exist."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
assert manager.store.states == {}
assert manager.store.version == '1.0'
assert isinstance(manager.store.last_updated, datetime)
def test_init_loads_existing_state(self, tmp_path):
"""Test that StateManager loads existing state store."""
state_path = tmp_path / "state.json"
# Create existing state
file1 = Path("/videos/movie.mp4")
store = StateStore(
states={
str(file1): FileState(
file_path=file1,
status="reviewed",
reason="Test",
updated_at=datetime(2024, 1, 1, 12, 0, 0)
)
},
version='1.0',
last_updated=datetime(2024, 1, 1, 12, 0, 0)
)
save_state(store, state_path)
# Load with manager
manager = StateManager(state_path)
assert len(manager.store.states) == 1
assert str(file1) in manager.store.states
def test_get_file_state_returns_state(self, tmp_path):
"""Test getting state for a file."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
manager.set_file_state(file1, "reviewed", "Test reason")
state = manager.get_file_state(file1)
assert state is not None
assert state.file_path == file1
assert state.status == "reviewed"
assert state.reason == "Test reason"
def test_get_file_state_returns_none_if_not_found(self, tmp_path):
"""Test that get_file_state returns None for unknown files."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
state = manager.get_file_state(file1)
assert state is None
def test_set_file_state_creates_new_state(self, tmp_path):
"""Test setting state for a new file."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
manager.set_file_state(file1, "reviewed", "Checked")
state = manager.get_file_state(file1)
assert state.status == "reviewed"
assert state.reason == "Checked"
assert isinstance(state.updated_at, datetime)
def test_set_file_state_updates_existing_state(self, tmp_path):
"""Test that set_file_state is idempotent and updates timestamp."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
# Set initial state
manager.set_file_state(file1, "reviewed", "First check")
state1 = manager.get_file_state(file1)
# Update state
manager.set_file_state(file1, "reviewed", "Second check")
state2 = manager.get_file_state(file1)
assert state2.status == "reviewed"
assert state2.reason == "Second check"
assert state2.updated_at >= state1.updated_at
def test_set_file_state_validates_status(self, tmp_path):
"""Test that set_file_state validates status values."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
with pytest.raises(ValueError, match="Invalid status"):
manager.set_file_state(file1, "invalid_status")
def test_set_file_state_accepts_all_valid_statuses(self, tmp_path):
"""Test that all valid statuses are accepted."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
for status in VALID_STATUSES:
manager.set_file_state(file1, status)
state = manager.get_file_state(file1)
assert state.status == status
def test_set_file_state_without_reason(self, tmp_path):
"""Test setting state without a reason."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
manager.set_file_state(file1, "ignored")
state = manager.get_file_state(file1)
assert state.status == "ignored"
assert state.reason is None
def test_query_by_status_returns_matching_files(self, tmp_path):
"""Test querying files by status."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie1.mp4")
file2 = Path("/videos/movie2.mp4")
file3 = Path("/videos/movie3.mp4")
manager.set_file_state(file1, "reviewed")
manager.set_file_state(file2, "ignored")
manager.set_file_state(file3, "reviewed")
reviewed = manager.query_by_status("reviewed")
assert len(reviewed) == 2
reviewed_paths = {state.file_path for state in reviewed}
assert file1 in reviewed_paths
assert file3 in reviewed_paths
def test_query_by_status_returns_empty_list_if_none_match(self, tmp_path):
"""Test that query_by_status returns empty list if no matches."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
manager.set_file_state(file1, "reviewed")
quarantined = manager.query_by_status("quarantined")
assert quarantined == []
def test_clear_state_removes_file_state(self, tmp_path):
"""Test clearing state for a file."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
manager.set_file_state(file1, "reviewed")
assert manager.get_file_state(file1) is not None
manager.clear_state(file1)
assert manager.get_file_state(file1) is None
def test_clear_state_on_nonexistent_file_does_nothing(self, tmp_path):
"""Test that clearing state on nonexistent file doesn't raise error."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
# Should not raise error
manager.clear_state(file1)
def test_save_persists_state_to_disk(self, tmp_path):
"""Test that save() persists state to disk."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
file1 = Path("/videos/movie.mp4")
manager.set_file_state(file1, "reviewed", "Test")
# Save to disk
manager.save()
# Load in new manager
manager2 = StateManager(state_path)
state = manager2.get_file_state(file1)
assert state is not None
assert state.status == "reviewed"
assert state.reason == "Test"
def test_state_updates_last_updated_timestamp(self, tmp_path):
"""Test that state operations update last_updated timestamp."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
initial_timestamp = manager.store.last_updated
file1 = Path("/videos/movie.mp4")
manager.set_file_state(file1, "reviewed")
assert manager.store.last_updated >= initial_timestamp
def test_multiple_files_with_different_statuses(self, tmp_path):
"""Test managing multiple files with different statuses."""
state_path = tmp_path / "state.json"
manager = StateManager(state_path)
files = [
(Path("/videos/movie1.mp4"), "reviewed"),
(Path("/videos/movie2.mp4"), "ignored"),
(Path("/videos/movie3.mp4"), "planned"),
(Path("/videos/movie4.mp4"), "executed"),
(Path("/videos/movie5.mp4"), "quarantined"),
]
for file_path, status in files:
manager.set_file_state(file_path, status)
# Verify all statuses
for file_path, expected_status in files:
state = manager.get_file_state(file_path)
assert state.status == expected_status
# Verify queries
for status in VALID_STATUSES:
results = manager.query_by_status(status)
expected_count = sum(1 for _, s in files if s == status)
assert len(results) == expected_count