# Implementation Plan: Video Library Manager ## Overview The Video Library Manager is a Python-based CLI tool for managing personal video collections with a safety-first, human-in-the-loop approach. This implementation plan breaks down the design into discrete coding tasks that build incrementally toward a complete, tested system. **Implementation Approach**: Bottom-up development starting with core data structures, then components, CLI interface, and finally integration testing. **Testing Strategy**: Dual approach using both unit tests (specific examples, edge cases) and property-based tests (universal properties across inputs). Property tests validate the 57 correctness properties defined in the design document. ## Tasks ### 1. Project Foundation - [x] 1.1 Set up Python project structure - Create package structure: `src/vlm/` with `__init__.py` - Create `pyproject.toml` with project metadata and dependencies - Use `uv` for Python package management - Add dependencies: Click, PyYAML, ffmpeg-python, pytest, hypothesis - Create application data directory structure: `~/.vlm/` with subdirectories for logs, plans, rollback - _Requirements: 10.1_ - [x] 1.2 Implement Configuration Manager - Define `Config` dataclass with all configuration fields - Implement `load_config(path: Path) -> Config` with YAML parsing - Implement `create_default_config(path: Path) -> Config` to generate default config - Implement `validate_config(config: Config) -> list[str]` for validation - Handle missing config files by creating defaults - Handle invalid YAML syntax with error reporting and fallback to defaults - _Requirements: 10.1-10.8_ - [ ]* 1.3 Write property tests for Configuration Manager - **Property 38: Configuration validation** - For any configuration with invalid values, validation SHALL detect and report errors - **Property 39: Default configuration validity** - For any missing configuration file, creating default SHALL produce valid configuration - **Property 40: Configuration error handling** - For any configuration with invalid YAML syntax, system SHALL report error and use defaults - _Requirements: 10.6, 10.7, 10.8_ - [x] 1.4 Implement logging infrastructure - Configure Python logging module with configurable log levels (DEBUG, INFO, WARNING, ERROR) - Set up dual output: console (INFO+) and file (DEBUG+) - Include timestamps, operation type, file paths in all log entries - Implement log rotation at 10MB threshold - _Requirements: 13.1-13.7_ - [ ]* 1.5 Write property tests for logging - **Property 49: Error logging** - For any operation with error, log SHALL contain timestamp, operation type, and error details - **Property 52: Log output** - For any logged message, message SHALL appear in console (if level permits) and log file - **Property 53: Log rotation** - For any log file exceeding 10MB, system SHALL rotate the log file - _Requirements: 13.1, 13.6, 13.7_ ### 2. Core Data Structures - [x] 2.1 Define video file and identity data structures - Create `VideoFile` dataclass: path, filename, size_bytes, modified_timestamp, category, optional metadata (resolution, codec, duration, bitrate) - Create `MovieIdentity` dataclass: title, year, confidence, needs_review, original_filename - Create `SeriesIdentity` dataclass: title, season, episodes, confidence, needs_review, original_filename - _Requirements: 1.2, 2.1, 3.1_ - [x] 2.2 Define execution and operation data structures - Create `FileOperation` dataclass: operation_type, source_path, destination_path, reason, has_conflict, conflict_reason - Create `ExecutionPlan` dataclass: plan_id, created_at, operations, summary - Create `OperationResult` dataclass: operation, success, error_message, executed_at - Create `RollbackLog` dataclass: log_id, execution_plan_id, executed_at, operations - _Requirements: 6.1, 7.6, 9.1_ - [x] 2.3 Define quarantine and state data structures - Create `QuarantineEntry` dataclass: original_path, quarantine_path, quarantined_at, reason, size_bytes, category - Create `QuarantineManifest` dataclass: entries - Create `FileState` dataclass: file_path, status, reason, updated_at - Create `StateStore` dataclass: states (dict), version, last_updated - _Requirements: 8.4, State Store design_ ### 3. Inventory Scanner - [x] 3.1 Implement core scanning functionality - Implement recursive directory scanning using `os.scandir()` - Filter files by configured video extensions - Record file metadata: path, filename, size, modification timestamp - Categorize files based on directory structure (movie/series/anime/other) - Handle inaccessible files gracefully: log error and continue scanning - _Requirements: 1.1, 1.2, 1.7_ - [x] 3.2 Add video metadata extraction - Implement optional ffprobe integration for video metadata - Extract resolution, codec, duration, bitrate when ffprobe available - Handle ffprobe failures gracefully with non-blocking fallback - _Requirements: 1.3_ - [x] 3.3 Implement inventory report generation - Generate CSV inventory report with all file metadata (primary format) - Implement CSV schema: path, filename, size_bytes, modified_timestamp, category, resolution, codec, duration_seconds, bitrate_kbps - Support JSON export as optional format - Include generation timestamp and library root in reports - _Requirements: 1.4, 1.8, 11.1, 11.5, 11.6, 11.8_ - [ ]* 3.4 Write property tests for Inventory Scanner - **Property 1: Scan completeness** - For any directory tree, scanning SHALL discover all and only files matching configured extensions - **Property 2: Scan idempotence** - For any directory tree, multiple scans SHALL produce identical results - **Property 3: Scan safety** - For any directory tree, scanning SHALL never modify files or directories - **Property 4: Scan resilience** - For any directory tree with inaccessible files, scanning SHALL discover all accessible files and continue - **Property 47: Anime inventory inclusion** - For any video files in anime directory, scanning SHALL discover and catalog them - _Requirements: 1.1, 1.2, 1.6, 1.7, 12.1, 12.2_ - [ ]* 3.5 Write unit tests for Inventory Scanner - Test scanning with various directory structures - Test file extension filtering - Test metadata extraction with and without ffprobe - Test error handling for inaccessible files - Test CSV and JSON report generation - _Requirements: 1.1-1.8_ ### 4. Identity Parser - [x] 4.1 Implement movie identity parsing - Parse patterns: "Title (Year)", "Title.Year", "Title - Year", "Title Year" - Extract title and year with confidence scoring - Implement quality tag removal: 1080p, 720p, 4K, BluRay, WEB-DL, HDTV, etc. - Implement release group tag removal: [RARBG], (YTS), etc. - Implement title normalization: remove extra whitespace, standardize capitalization - Flag files without extractable year as needs_review - _Requirements: 2.1-2.6_ - [x] 4.2 Implement series identity parsing - Parse patterns: "SXXEYY", "SXXeYY", "SeasonXEpisodeY", "XXxYY" - Extract series title, season number, episode numbers - Handle multi-episode files: extract all episode numbers (e.g., S01E01-E02) - Remove quality tags and release groups from series titles - Normalize series titles - Flag files with season=None or episodes=[] as needs_review (v1 constraint: these files excluded from organization plans) - _Requirements: 3.1-3.6_ - [x] 4.3 Implement episode grouping - Group parsed episodes by normalized series title and season number - _Requirements: 3.7_ - [x] 4.4 Write property tests for Identity Parser - **Property 5: Movie parsing** - For any filename matching common movie patterns, parser SHALL extract title and year - **Property 6: Series parsing** - For any filename matching common series patterns, parser SHALL extract series title, season, and episodes - **Property 7: Title normalization idempotence** - For any title string, normalizing multiple times SHALL produce same result - **Property 8: Ambiguous filename flagging** - For any filename where season/episode cannot be extracted, parser SHALL mark as needs_review - **Property 9: Episode grouping** - For any set of parsed episodes, grouping SHALL place episodes with identical normalized titles and seasons in same group - **Property 51: Parsing error handling** - For any filename that cannot be parsed, system SHALL log error and mark file for review - _Requirements: 2.1, 2.2, 2.5, 2.6, 3.1, 3.2, 3.5, 3.6, 3.7, 13.4_ - [x] 4.5 Write unit tests for Identity Parser - Test various movie filename patterns with different quality tags - Test various series filename patterns including multi-episode files - Test title normalization edge cases - Test ambiguous filename detection - Test episode grouping logic - _Requirements: 2.1-2.7, 3.1-3.7_ ### 5. Analysis Engine - [x] 5.1 Implement series completeness analysis - Group episodes by series title and season - For each season, find min and max episode numbers - Detect gaps in episode sequence [min, max] using heuristic detection - Generate completeness report showing episodes_found and episodes_missing (gaps only) - Do NOT calculate percentages or "complete" status (v1 constraint: no external metadata, heuristic gap detection only) - _Requirements: 4.1-4.6_ - [x] 5.2 Implement duplicate detection - Group files by normalized identity (title+year for movies, title+season+episode for series) - For groups with multiple files, flag as potential duplicates - Extract quality comparison data: file size, resolution, codec - Present comparison data without automatic recommendations - _Requirements: 5.1-5.6_ - [x] 5.3 Implement report generation - Generate completeness report (text/JSON) with series and detected gaps - Generate duplicate report (text/JSON) with quality comparisons - Generate summary report (text) with total files, size, category breakdown, files needing review - Include generation timestamp and library root in all reports - _Requirements: 11.2-11.4, 11.7, 11.8_ - [x] 5.4 Write property tests for Analysis Engine - **Property 10: Gap detection** - For any set of episodes within season, analysis SHALL detect missing episode numbers in range [min, max] - **Property 11: Multi-season independence** - For any series with multiple seasons, gap detection of one season SHALL not affect others - **Property 12: Duplicate detection for movies** - For any set of movies with identical normalized titles and years, all SHALL be grouped as duplicates - **Property 13: Duplicate detection for series** - For any set of series files with identical normalized titles, seasons, and episodes, all SHALL be grouped as duplicates - **Property 14: Duplicate quality comparison** - For any duplicate group, comparison data SHALL include available metadata for each file - **Property 42: Completeness report** - For any set of analyzed series, completeness report SHALL include all series with detected gaps - **Property 43: Duplicate report grouping** - For any set of detected duplicates, duplicate report SHALL group files by identity with comparison data - **Property 44: Summary report accuracy** - For any scanned library, summary report SHALL contain accurate counts and sizes - _Requirements: 4.1, 4.2, 4.4, 5.1, 5.2, 5.3, 11.2, 11.3, 11.4_ - [x] 5.5 Write unit tests for Analysis Engine - Test gap detection with various episode sequences - Test multi-season analysis - Test duplicate detection for movies and series - Test quality comparison data extraction - Test report generation in different formats - _Requirements: 4.1-4.6, 5.1-5.6, 11.2-11.4_ ### 6. Plan Generator - [x] 6.1 Implement plan generation logic - For each parsed identity, determine target directory using config templates - Apply movie template: "movie/{title} ({year})/" - Apply series template: "series/{title}/Season {season:02d}/" - Apply filename templates for movies and series - Create move/rename operations for movies and series with valid identities - Generate no-op operations for anime, other, and series files with season=None or episodes=[] (v1 constraint: flagged for manual review) - Preserve category boundaries (no cross-category moves) - _Requirements: 6.1-6.5, 12.5_ - [x] 6.2 Implement conflict detection - Check if destination file already exists for each operation - Mark operations with conflicts: set has_conflict=True and conflict_reason - _Requirements: 6.8_ - [x] 6.3 Implement plan serialization - Save execution plan to JSON format (human-readable and editable) - Include plan_id (UUID), created_at timestamp, operations list, summary - _Requirements: 6.6, 6.7_ - [ ]* 6.4 Write property tests for Plan Generator - **Property 15: Plan operation completeness** - For any set of video files with parsed identities, execution plan SHALL contain exactly one operation per file - **Property 16: Movie template consistency** - For any movie identity, applying templates SHALL always produce same destination path - **Property 17: Series template consistency** - For any series episode identity, applying templates SHALL always produce same destination path - **Property 18: Category boundary preservation** - For any file in category directory, execution plan SHALL specify destination within same category - **Property 19: Destination path uniqueness** - For any execution plan, no two operations SHALL specify same destination unless marked as conflict - **Property 20: Conflict detection** - For any execution plan where destination file exists, operation SHALL be marked as conflict - **Property 21: Plan JSON validity** - For any generated execution plan, output SHALL be valid JSON - **Property 48: Anime processing deferral** - For any anime files, execution plans SHALL contain only no-op operations - _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5, 6.7, 6.8, 12.3, 12.4, 12.5_ - [ ]* 6.5 Write unit tests for Plan Generator - Test plan generation for various movie and series identities - Test template application with different configurations - Test no-op generation for anime and ambiguous files - Test conflict detection scenarios - Test JSON serialization and deserialization - _Requirements: 6.1-6.9_ ### 7. Execution Engine - [x] 7.1 Implement execution mode handling - Support mode parameter: "dry-run" (default) or "execute" - In dry-run mode: simulate operations and log what would happen without making changes - In execute mode: perform actual file operations - Require --confirm flag for execute mode (v1 constraint: safety-first approach) - _Requirements: 7.1, 7.2_ - [x] 7.2 Implement file operation execution - Create destination directories if they don't exist - Handle conflicts: skip operation and report (don't halt other operations, v1 constraint) - Execute move/rename operations for each FileOperation - Log all operations with timestamps, paths, and results - Continue processing after non-fatal errors - _Requirements: 7.4, 7.5, 7.6, 7.7_ - [x] 7.3 Implement execution summary and rollback log - Generate execution summary with counts: successful, failed, skipped operations - Save rollback log in execute mode only (not in dry-run) - Include operation type, source, destination, timestamp for each operation - _Requirements: 7.8, 9.1, 9.2_ - [ ]* 7.4 Write property tests for Execution Engine - **Property 22: Execution mode safety** - For any execution plan, mode="dry-run" SHALL never modify files - **Property 23: Directory creation** - For any file operation with destination in non-existent directory, executing SHALL create necessary parent directories - **Property 24: Conflict handling** - For any execution plan with conflicts, executing SHALL skip conflicted operations and continue processing - **Property 25: Operation logging** - For any executed operation, log SHALL contain timestamp, operation type, paths, and result - **Property 26: Execution summary** - For any execution, summary SHALL contain accurate counts of successful, failed, and skipped operations - **Property 50: Error resilience** - For any operation sequence with non-fatal errors, system SHALL log errors and continue processing - _Requirements: 7.1, 7.2, 7.4, 7.5, 7.6, 7.7, 7.8, 13.2, 13.3_ - [ ]* 7.5 Write unit tests for Execution Engine - Test dry-run mode (no file modifications) - Test execute mode with actual file operations - Test directory creation - Test conflict handling - Test error handling and resilience - Test execution summary generation - Test rollback log creation - _Requirements: 7.1-7.8_ ### 8. Rollback Functionality - [x] 8.1 Implement rollback execution - Load rollback log from JSON - Reverse operation list (LIFO order for best-effort restoration) - For each operation: attempt to move destination back to source - Handle rollback failures gracefully: log error and continue - Generate rollback summary with counts: successful, failed, skipped - _Requirements: 9.1-9.8_ - [ ]* 8.2 Write property tests for rollback - **Property 27: Rollback best-effort** - For any successful execution, rolling back SHALL attempt to restore all files to original locations - **Property 28: Rollback idempotence** - For any rollback log, executing rollback multiple times SHALL produce same final state - **Property 29: Rollback resilience** - For any rollback with failures, process SHALL log errors and continue processing - **Property 30: Rollback summary** - For any rollback execution, summary SHALL contain accurate counts of successful, failed, and skipped operations - _Requirements: 9.1, 9.2, 9.7, 9.8_ - [ ]* 8.3 Write unit tests for rollback - Test rollback of move operations - Test rollback of rename operations - Test rollback failure handling - Test rollback summary generation - Test rollback idempotence - _Requirements: 9.1-9.8_ ### 9. Quarantine Manager - [x] 9.1 Implement quarantine operations - Verify file is in movie or series category (reject anime/other with error) - Determine relative path from category root - Construct quarantine path: `/.quarantine/` - Handle destination conflicts: append numeric suffix (_1, _2, etc.) - Move file to category-specific quarantine directory - _Requirements: 8.1, 8.2, 8.3, 8.6_ - [x] 9.2 Implement quarantine manifest management - Update category-specific manifest.json with quarantine entry - Include original_path, quarantine_path, timestamp, reason, size_bytes, category - Save manifest to `/.quarantine/manifest.json` - _Requirements: 8.4, 8.5_ - [x] 9.3 Implement quarantine listing and restoration - List quarantined files from category manifests, optionally filtered by category - Restore files from quarantine to original locations (best-effort) - _Requirements: 8.7_ - [ ]* 9.4 Write property tests for Quarantine Manager - **Property 31: Quarantine location** - For any movie or series file marked for quarantine, file SHALL be moved to `/.quarantine/` - **Property 32: Quarantine category restriction** - For any file in anime or other categories, quarantine operations SHALL be rejected with error - **Property 33: Quarantine manifest** - For any quarantined file, category-specific manifest SHALL contain original path, quarantine path, timestamp, and category - **Property 34: Quarantine conflict handling** - For any file being quarantined where destination exists, system SHALL append numeric suffix - **Property 35: Quarantine listing** - For any set of quarantined files, listing SHALL return all entries from category manifests, optionally filtered by category - **Property 36: Quarantine restore** - For any quarantined file, restoring SHALL attempt to move file back to original location - **Property 37: No permanent deletion in v1** - For any file operation in v1, system SHALL never permanently delete files - _Requirements: 8.1, 8.2, 8.4, 8.5, 8.6, 8.7, 8.8_ - [ ]* 9.5 Write unit tests for Quarantine Manager - Test quarantine operations for movie and series files - Test category restriction enforcement - Test manifest creation and updates - Test conflict handling with numeric suffixes - Test listing and restoration - _Requirements: 8.1-8.8_ ### 10. State Store - [x] 10.1 Implement State Store operations - Implement `load_state(path: Path) -> StateStore` to load from JSON - Implement `save_state(store: StateStore, path: Path)` to save to JSON - Implement `get_file_state(file_path: Path) -> Optional[FileState]` for path-based lookup - Implement `set_file_state(file_path: Path, status: str, reason: Optional[str])` with idempotent updates and timestamp - Implement `query_by_status(status: str) -> list[FileState]` to filter by status - Implement `clear_state(file_path: Path)` to remove file state - Validate status values: "reviewed", "ignored", "planned", "executed", "quarantined" - _State Store design_ - [ ]* 10.2 Write property tests for State Store - **Property 54: State persistence** - For any state store saved to disk, loading SHALL produce equivalent StateStore with all file states preserved - **Property 55: State query** - For any status value, querying SHALL return all files with that status - **Property 56: State update** - For any file path, setting state SHALL update store and subsequent queries return new status - **Property 57: State JSON validity** - For any state store, saving SHALL produce valid JSON - _State Store design_ - [ ]* 10.3 Write unit tests for State Store - Test state loading and saving - Test state queries by status - Test state updates and idempotence - Test state clearing - Test status validation - _State Store design_ ### 11. Report Generator - [x] 11.1 Implement report generation - Generate inventory report in CSV/JSON formats with all file metadata - Generate completeness report in text/JSON formats with series gaps - Generate duplicate report in text/JSON formats with quality comparisons - Generate summary report in text format with library statistics - Include generation timestamp and library root in all reports - _Requirements: 11.1-11.8_ - [ ]* 11.2 Write property tests for Report Generator - **Property 41: Inventory report completeness** - For any set of scanned video files, inventory report SHALL include entry for each file - **Property 45: Report format validity** - For any report, output SHALL be valid in specified format (CSV, JSON, or text) - **Property 46: Report metadata** - For any generated report, report SHALL include generation timestamp and library root - _Requirements: 11.1, 11.5, 11.6, 11.7, 11.8_ - [ ]* 11.3 Write unit tests for Report Generator - Test inventory report generation in CSV and JSON - Test completeness report generation - Test duplicate report generation - Test summary report generation - Test report metadata inclusion - _Requirements: 11.1-11.8_ ### 12. CLI Interface - [x] 12.1 Set up CLI framework - Set up Click CLI with main entry point - Implement global options: --config, --log-level - Add help text for main command - Implement error handling for invalid arguments - _Requirements: 14.1, 14.10, 14.11_ - [x] 12.2 Implement scan command - `vlm scan [--output inventory.csv]` - Wire to Inventory Scanner - Display progress and summary - _Requirements: 14.2_ - [x] 12.3 Implement parse command - `vlm parse [--input inventory.csv] [--output identities.json]` - Wire to Identity Parser - Display parsing statistics - _Requirements: 14.3_ - [x] 12.4 Implement analyze command - `vlm analyze [--input identities.json] [--output analysis.json]` - Wire to Analysis Engine - Display analysis summary - _Requirements: 14.4_ - [x] 12.5 Implement plan command - `vlm plan [--input identities.json] [--output plan.json]` - Wire to Plan Generator - Display plan summary with operation counts - _Requirements: 14.5_ - [x] 12.6 Implement execute command - `vlm execute [--plan plan.json] [--confirm]` - Wire to Execution Engine - Default to dry-run mode, require --confirm for actual execution - Display execution progress and summary - _Requirements: 14.6_ - [x] 12.7 Implement quarantine commands - `vlm quarantine list [--category movie|series]` - `vlm quarantine add [--reason "duplicate"]` - `vlm quarantine restore ` - Wire to Quarantine Manager - _Requirements: 14.7_ - [x] 12.8 Implement rollback command - `vlm rollback [--log rollback.json]` - Wire to rollback functionality - Display rollback progress and summary - _Requirements: 14.8_ - [x] 12.9 Implement report commands - `vlm report inventory [--format csv|json|text]` - `vlm report completeness [--format text|json]` - `vlm report duplicates [--format text|json]` - `vlm report summary` - Wire to Report Generator - _Requirements: 14.9_ - [x] 12.10 Implement state commands - `vlm state show ` - `vlm state set --status reviewed [--reason "checked manually"]` - `vlm state query --status ignored` - `vlm state clear ` - Wire to State Store - _Requirements: 14.9_ - [x] 12.11 Implement config commands - `vlm config init` - `vlm config show` - `vlm config validate` - Wire to Configuration Manager - _Requirements: 14.9_ - [ ]* 12.12 Write integration tests for CLI - Test each command with various options - Test error handling for invalid arguments - Test help text display - Test end-to-end workflow through CLI - _Requirements: 14.1-14.11_ ### 13. Integration and Polish - [ ] 13.1 End-to-end integration testing - Test complete workflow: scan → parse → analyze → plan → execute (dry-run) → execute (confirm) → rollback - Test with real-world filename patterns and directory structures - Test error scenarios: inaccessible files, invalid configs, conflicts - Test quarantine operations end-to-end - Test state management throughout workflow - Verify all reports generate correctly with accurate data - _All requirements_ - [ ] 13.2 Error handling validation - Test parsing errors with ambiguous filenames - Test file system errors (permissions, disk full, network issues) - Test configuration errors - Verify error logging and resilience - _Requirements: 13.1-13.7_ - [ ] 13.3 Documentation - Write README with installation instructions - Document typical workflow with examples - Add usage examples for all CLI commands - Document configuration options - Document v1 constraints and limitations - _Requirements: 14.10_ - [ ] 13.4 Final checkpoint - Ensure all tests pass (unit, property-based, integration) - Verify all 57 correctness properties are validated - Review code for security considerations (path validation, filename sanitization) - Verify logging is comprehensive and useful - Test with large library (performance validation) - Ask user if any questions or issues arise ## Notes ### Testing Approach - **Property-based tests** (marked with `*`): Validate universal correctness properties using Hypothesis with minimum 100 iterations per test. Each test references a specific property from the design document. - **Unit tests** (marked with `*`): Test specific examples, edge cases, and error conditions for each component. - **Integration tests**: Validate end-to-end workflows and component interactions. ### Task Dependencies Tasks are ordered to build incrementally: 1. Foundation (config, logging, data structures) 2. Core components (scanner, parser, analysis) 3. Execution components (plan, execute, rollback) 4. Supporting features (quarantine, state, reports) 5. CLI interface 6. Integration and polish ### v1 Constraints - **Quarantine**: Category-level only (movie/.quarantine/, series/.quarantine/), anime/other rejected - **Completeness**: Heuristic gap detection only, no percentages or "complete" status - **Parsing**: Hardcoded patterns, not user-configurable - **Anime**: Inventory only, no organization - **Deletion**: Quarantine only, no permanent deletion - **State Store**: Simple path-based indexing, no fingerprint or move tracking - **Execution**: Defaults to dry-run, requires --confirm for actual changes - **Rollback**: Best-effort, not guaranteed restoration ### Property-Based Testing Configuration - **Framework**: Hypothesis (Python PBT library) - **Iterations**: Minimum 100 per test - **Tagging**: Each test includes comment: `# Feature: video-library-manager, Property N: [description]` - **Coverage**: All 57 correctness properties from design document