- Add core VLM modules (scanner, parser, planner, executor, analysis) - Add CLI with quarantine, reports, rollback, and state management - Add comprehensive test suite - Add project configuration and documentation - Add .gitignore for Python project
28 KiB
Implementation Plan: Video Library Manager
Overview
The Video Library Manager is a Python-based CLI tool for managing personal video collections with a safety-first, human-in-the-loop approach. This implementation plan breaks down the design into discrete coding tasks that build incrementally toward a complete, tested system.
Implementation Approach: Bottom-up development starting with core data structures, then components, CLI interface, and finally integration testing.
Testing Strategy: Dual approach using both unit tests (specific examples, edge cases) and property-based tests (universal properties across inputs). Property tests validate the 57 correctness properties defined in the design document.
Tasks
1. Project Foundation
-
1.1 Set up Python project structure
- Create package structure:
src/vlm/with__init__.py - Create
pyproject.tomlwith project metadata and dependencies - Use
uvfor Python package management - Add dependencies: Click, PyYAML, ffmpeg-python, pytest, hypothesis
- Create application data directory structure:
~/.vlm/with subdirectories for logs, plans, rollback - Requirements: 10.1
- Create package structure:
-
1.2 Implement Configuration Manager
- Define
Configdataclass with all configuration fields - Implement
load_config(path: Path) -> Configwith YAML parsing - Implement
create_default_config(path: Path) -> Configto generate default config - Implement
validate_config(config: Config) -> list[str]for validation - Handle missing config files by creating defaults
- Handle invalid YAML syntax with error reporting and fallback to defaults
- Requirements: 10.1-10.8
- Define
-
* 1.3 Write property tests for Configuration Manager
- Property 38: Configuration validation - For any configuration with invalid values, validation SHALL detect and report errors
- Property 39: Default configuration validity - For any missing configuration file, creating default SHALL produce valid configuration
- Property 40: Configuration error handling - For any configuration with invalid YAML syntax, system SHALL report error and use defaults
- Requirements: 10.6, 10.7, 10.8
-
1.4 Implement logging infrastructure
- Configure Python logging module with configurable log levels (DEBUG, INFO, WARNING, ERROR)
- Set up dual output: console (INFO+) and file (DEBUG+)
- Include timestamps, operation type, file paths in all log entries
- Implement log rotation at 10MB threshold
- Requirements: 13.1-13.7
-
* 1.5 Write property tests for logging
- Property 49: Error logging - For any operation with error, log SHALL contain timestamp, operation type, and error details
- Property 52: Log output - For any logged message, message SHALL appear in console (if level permits) and log file
- Property 53: Log rotation - For any log file exceeding 10MB, system SHALL rotate the log file
- Requirements: 13.1, 13.6, 13.7
2. Core Data Structures
-
2.1 Define video file and identity data structures
- Create
VideoFiledataclass: path, filename, size_bytes, modified_timestamp, category, optional metadata (resolution, codec, duration, bitrate) - Create
MovieIdentitydataclass: title, year, confidence, needs_review, original_filename - Create
SeriesIdentitydataclass: title, season, episodes, confidence, needs_review, original_filename - Requirements: 1.2, 2.1, 3.1
- Create
-
2.2 Define execution and operation data structures
- Create
FileOperationdataclass: operation_type, source_path, destination_path, reason, has_conflict, conflict_reason - Create
ExecutionPlandataclass: plan_id, created_at, operations, summary - Create
OperationResultdataclass: operation, success, error_message, executed_at - Create
RollbackLogdataclass: log_id, execution_plan_id, executed_at, operations - Requirements: 6.1, 7.6, 9.1
- Create
-
2.3 Define quarantine and state data structures
- Create
QuarantineEntrydataclass: original_path, quarantine_path, quarantined_at, reason, size_bytes, category - Create
QuarantineManifestdataclass: entries - Create
FileStatedataclass: file_path, status, reason, updated_at - Create
StateStoredataclass: states (dict), version, last_updated - Requirements: 8.4, State Store design
- Create
3. Inventory Scanner
-
3.1 Implement core scanning functionality
- Implement recursive directory scanning using
os.scandir() - Filter files by configured video extensions
- Record file metadata: path, filename, size, modification timestamp
- Categorize files based on directory structure (movie/series/anime/other)
- Handle inaccessible files gracefully: log error and continue scanning
- Requirements: 1.1, 1.2, 1.7
- Implement recursive directory scanning using
-
3.2 Add video metadata extraction
- Implement optional ffprobe integration for video metadata
- Extract resolution, codec, duration, bitrate when ffprobe available
- Handle ffprobe failures gracefully with non-blocking fallback
- Requirements: 1.3
-
3.3 Implement inventory report generation
- Generate CSV inventory report with all file metadata (primary format)
- Implement CSV schema: path, filename, size_bytes, modified_timestamp, category, resolution, codec, duration_seconds, bitrate_kbps
- Support JSON export as optional format
- Include generation timestamp and library root in reports
- Requirements: 1.4, 1.8, 11.1, 11.5, 11.6, 11.8
-
* 3.4 Write property tests for Inventory Scanner
- Property 1: Scan completeness - For any directory tree, scanning SHALL discover all and only files matching configured extensions
- Property 2: Scan idempotence - For any directory tree, multiple scans SHALL produce identical results
- Property 3: Scan safety - For any directory tree, scanning SHALL never modify files or directories
- Property 4: Scan resilience - For any directory tree with inaccessible files, scanning SHALL discover all accessible files and continue
- Property 47: Anime inventory inclusion - For any video files in anime directory, scanning SHALL discover and catalog them
- Requirements: 1.1, 1.2, 1.6, 1.7, 12.1, 12.2
-
* 3.5 Write unit tests for Inventory Scanner
- Test scanning with various directory structures
- Test file extension filtering
- Test metadata extraction with and without ffprobe
- Test error handling for inaccessible files
- Test CSV and JSON report generation
- Requirements: 1.1-1.8
4. Identity Parser
-
4.1 Implement movie identity parsing
- Parse patterns: "Title (Year)", "Title.Year", "Title - Year", "Title Year"
- Extract title and year with confidence scoring
- Implement quality tag removal: 1080p, 720p, 4K, BluRay, WEB-DL, HDTV, etc.
- Implement release group tag removal: [RARBG], (YTS), etc.
- Implement title normalization: remove extra whitespace, standardize capitalization
- Flag files without extractable year as needs_review
- Requirements: 2.1-2.6
-
4.2 Implement series identity parsing
- Parse patterns: "SXXEYY", "SXXeYY", "SeasonXEpisodeY", "XXxYY"
- Extract series title, season number, episode numbers
- Handle multi-episode files: extract all episode numbers (e.g., S01E01-E02)
- Remove quality tags and release groups from series titles
- Normalize series titles
- Flag files with season=None or episodes=[] as needs_review (v1 constraint: these files excluded from organization plans)
- Requirements: 3.1-3.6
-
4.3 Implement episode grouping
- Group parsed episodes by normalized series title and season number
- Requirements: 3.7
-
4.4 Write property tests for Identity Parser
- Property 5: Movie parsing - For any filename matching common movie patterns, parser SHALL extract title and year
- Property 6: Series parsing - For any filename matching common series patterns, parser SHALL extract series title, season, and episodes
- Property 7: Title normalization idempotence - For any title string, normalizing multiple times SHALL produce same result
- Property 8: Ambiguous filename flagging - For any filename where season/episode cannot be extracted, parser SHALL mark as needs_review
- Property 9: Episode grouping - For any set of parsed episodes, grouping SHALL place episodes with identical normalized titles and seasons in same group
- Property 51: Parsing error handling - For any filename that cannot be parsed, system SHALL log error and mark file for review
- Requirements: 2.1, 2.2, 2.5, 2.6, 3.1, 3.2, 3.5, 3.6, 3.7, 13.4
-
4.5 Write unit tests for Identity Parser
- Test various movie filename patterns with different quality tags
- Test various series filename patterns including multi-episode files
- Test title normalization edge cases
- Test ambiguous filename detection
- Test episode grouping logic
- Requirements: 2.1-2.7, 3.1-3.7
5. Analysis Engine
-
5.1 Implement series completeness analysis
- Group episodes by series title and season
- For each season, find min and max episode numbers
- Detect gaps in episode sequence [min, max] using heuristic detection
- Generate completeness report showing episodes_found and episodes_missing (gaps only)
- Do NOT calculate percentages or "complete" status (v1 constraint: no external metadata, heuristic gap detection only)
- Requirements: 4.1-4.6
-
5.2 Implement duplicate detection
- Group files by normalized identity (title+year for movies, title+season+episode for series)
- For groups with multiple files, flag as potential duplicates
- Extract quality comparison data: file size, resolution, codec
- Present comparison data without automatic recommendations
- Requirements: 5.1-5.6
-
5.3 Implement report generation
- Generate completeness report (text/JSON) with series and detected gaps
- Generate duplicate report (text/JSON) with quality comparisons
- Generate summary report (text) with total files, size, category breakdown, files needing review
- Include generation timestamp and library root in all reports
- Requirements: 11.2-11.4, 11.7, 11.8
-
5.4 Write property tests for Analysis Engine
- Property 10: Gap detection - For any set of episodes within season, analysis SHALL detect missing episode numbers in range [min, max]
- Property 11: Multi-season independence - For any series with multiple seasons, gap detection of one season SHALL not affect others
- Property 12: Duplicate detection for movies - For any set of movies with identical normalized titles and years, all SHALL be grouped as duplicates
- Property 13: Duplicate detection for series - For any set of series files with identical normalized titles, seasons, and episodes, all SHALL be grouped as duplicates
- Property 14: Duplicate quality comparison - For any duplicate group, comparison data SHALL include available metadata for each file
- Property 42: Completeness report - For any set of analyzed series, completeness report SHALL include all series with detected gaps
- Property 43: Duplicate report grouping - For any set of detected duplicates, duplicate report SHALL group files by identity with comparison data
- Property 44: Summary report accuracy - For any scanned library, summary report SHALL contain accurate counts and sizes
- Requirements: 4.1, 4.2, 4.4, 5.1, 5.2, 5.3, 11.2, 11.3, 11.4
-
5.5 Write unit tests for Analysis Engine
- Test gap detection with various episode sequences
- Test multi-season analysis
- Test duplicate detection for movies and series
- Test quality comparison data extraction
- Test report generation in different formats
- Requirements: 4.1-4.6, 5.1-5.6, 11.2-11.4
6. Plan Generator
-
6.1 Implement plan generation logic
- For each parsed identity, determine target directory using config templates
- Apply movie template: "movie/{title} ({year})/"
- Apply series template: "series/{title}/Season {season:02d}/"
- Apply filename templates for movies and series
- Create move/rename operations for movies and series with valid identities
- Generate no-op operations for anime, other, and series files with season=None or episodes=[] (v1 constraint: flagged for manual review)
- Preserve category boundaries (no cross-category moves)
- Requirements: 6.1-6.5, 12.5
-
6.2 Implement conflict detection
- Check if destination file already exists for each operation
- Mark operations with conflicts: set has_conflict=True and conflict_reason
- Requirements: 6.8
-
6.3 Implement plan serialization
- Save execution plan to JSON format (human-readable and editable)
- Include plan_id (UUID), created_at timestamp, operations list, summary
- Requirements: 6.6, 6.7
-
* 6.4 Write property tests for Plan Generator
- Property 15: Plan operation completeness - For any set of video files with parsed identities, execution plan SHALL contain exactly one operation per file
- Property 16: Movie template consistency - For any movie identity, applying templates SHALL always produce same destination path
- Property 17: Series template consistency - For any series episode identity, applying templates SHALL always produce same destination path
- Property 18: Category boundary preservation - For any file in category directory, execution plan SHALL specify destination within same category
- Property 19: Destination path uniqueness - For any execution plan, no two operations SHALL specify same destination unless marked as conflict
- Property 20: Conflict detection - For any execution plan where destination file exists, operation SHALL be marked as conflict
- Property 21: Plan JSON validity - For any generated execution plan, output SHALL be valid JSON
- Property 48: Anime processing deferral - For any anime files, execution plans SHALL contain only no-op operations
- Requirements: 6.1, 6.2, 6.3, 6.4, 6.5, 6.7, 6.8, 12.3, 12.4, 12.5
-
* 6.5 Write unit tests for Plan Generator
- Test plan generation for various movie and series identities
- Test template application with different configurations
- Test no-op generation for anime and ambiguous files
- Test conflict detection scenarios
- Test JSON serialization and deserialization
- Requirements: 6.1-6.9
7. Execution Engine
-
7.1 Implement execution mode handling
- Support mode parameter: "dry-run" (default) or "execute"
- In dry-run mode: simulate operations and log what would happen without making changes
- In execute mode: perform actual file operations
- Require --confirm flag for execute mode (v1 constraint: safety-first approach)
- Requirements: 7.1, 7.2
-
7.2 Implement file operation execution
- Create destination directories if they don't exist
- Handle conflicts: skip operation and report (don't halt other operations, v1 constraint)
- Execute move/rename operations for each FileOperation
- Log all operations with timestamps, paths, and results
- Continue processing after non-fatal errors
- Requirements: 7.4, 7.5, 7.6, 7.7
-
7.3 Implement execution summary and rollback log
- Generate execution summary with counts: successful, failed, skipped operations
- Save rollback log in execute mode only (not in dry-run)
- Include operation type, source, destination, timestamp for each operation
- Requirements: 7.8, 9.1, 9.2
-
* 7.4 Write property tests for Execution Engine
- Property 22: Execution mode safety - For any execution plan, mode="dry-run" SHALL never modify files
- Property 23: Directory creation - For any file operation with destination in non-existent directory, executing SHALL create necessary parent directories
- Property 24: Conflict handling - For any execution plan with conflicts, executing SHALL skip conflicted operations and continue processing
- Property 25: Operation logging - For any executed operation, log SHALL contain timestamp, operation type, paths, and result
- Property 26: Execution summary - For any execution, summary SHALL contain accurate counts of successful, failed, and skipped operations
- Property 50: Error resilience - For any operation sequence with non-fatal errors, system SHALL log errors and continue processing
- Requirements: 7.1, 7.2, 7.4, 7.5, 7.6, 7.7, 7.8, 13.2, 13.3
-
* 7.5 Write unit tests for Execution Engine
- Test dry-run mode (no file modifications)
- Test execute mode with actual file operations
- Test directory creation
- Test conflict handling
- Test error handling and resilience
- Test execution summary generation
- Test rollback log creation
- Requirements: 7.1-7.8
8. Rollback Functionality
-
8.1 Implement rollback execution
- Load rollback log from JSON
- Reverse operation list (LIFO order for best-effort restoration)
- For each operation: attempt to move destination back to source
- Handle rollback failures gracefully: log error and continue
- Generate rollback summary with counts: successful, failed, skipped
- Requirements: 9.1-9.8
-
* 8.2 Write property tests for rollback
- Property 27: Rollback best-effort - For any successful execution, rolling back SHALL attempt to restore all files to original locations
- Property 28: Rollback idempotence - For any rollback log, executing rollback multiple times SHALL produce same final state
- Property 29: Rollback resilience - For any rollback with failures, process SHALL log errors and continue processing
- Property 30: Rollback summary - For any rollback execution, summary SHALL contain accurate counts of successful, failed, and skipped operations
- Requirements: 9.1, 9.2, 9.7, 9.8
-
* 8.3 Write unit tests for rollback
- Test rollback of move operations
- Test rollback of rename operations
- Test rollback failure handling
- Test rollback summary generation
- Test rollback idempotence
- Requirements: 9.1-9.8
9. Quarantine Manager
-
9.1 Implement quarantine operations
- Verify file is in movie or series category (reject anime/other with error)
- Determine relative path from category root
- Construct quarantine path:
<category_root>/.quarantine/<relative_path> - Handle destination conflicts: append numeric suffix (_1, _2, etc.)
- Move file to category-specific quarantine directory
- Requirements: 8.1, 8.2, 8.3, 8.6
-
9.2 Implement quarantine manifest management
- Update category-specific manifest.json with quarantine entry
- Include original_path, quarantine_path, timestamp, reason, size_bytes, category
- Save manifest to
<category_root>/.quarantine/manifest.json - Requirements: 8.4, 8.5
-
9.3 Implement quarantine listing and restoration
- List quarantined files from category manifests, optionally filtered by category
- Restore files from quarantine to original locations (best-effort)
- Requirements: 8.7
-
* 9.4 Write property tests for Quarantine Manager
- Property 31: Quarantine location - For any movie or series file marked for quarantine, file SHALL be moved to
<category_root>/.quarantine/ - Property 32: Quarantine category restriction - For any file in anime or other categories, quarantine operations SHALL be rejected with error
- Property 33: Quarantine manifest - For any quarantined file, category-specific manifest SHALL contain original path, quarantine path, timestamp, and category
- Property 34: Quarantine conflict handling - For any file being quarantined where destination exists, system SHALL append numeric suffix
- Property 35: Quarantine listing - For any set of quarantined files, listing SHALL return all entries from category manifests, optionally filtered by category
- Property 36: Quarantine restore - For any quarantined file, restoring SHALL attempt to move file back to original location
- Property 37: No permanent deletion in v1 - For any file operation in v1, system SHALL never permanently delete files
- Requirements: 8.1, 8.2, 8.4, 8.5, 8.6, 8.7, 8.8
- Property 31: Quarantine location - For any movie or series file marked for quarantine, file SHALL be moved to
-
* 9.5 Write unit tests for Quarantine Manager
- Test quarantine operations for movie and series files
- Test category restriction enforcement
- Test manifest creation and updates
- Test conflict handling with numeric suffixes
- Test listing and restoration
- Requirements: 8.1-8.8
10. State Store
-
10.1 Implement State Store operations
- Implement
load_state(path: Path) -> StateStoreto load from JSON - Implement
save_state(store: StateStore, path: Path)to save to JSON - Implement
get_file_state(file_path: Path) -> Optional[FileState]for path-based lookup - Implement
set_file_state(file_path: Path, status: str, reason: Optional[str])with idempotent updates and timestamp - Implement
query_by_status(status: str) -> list[FileState]to filter by status - Implement
clear_state(file_path: Path)to remove file state - Validate status values: "reviewed", "ignored", "planned", "executed", "quarantined"
- State Store design
- Implement
-
* 10.2 Write property tests for State Store
- Property 54: State persistence - For any state store saved to disk, loading SHALL produce equivalent StateStore with all file states preserved
- Property 55: State query - For any status value, querying SHALL return all files with that status
- Property 56: State update - For any file path, setting state SHALL update store and subsequent queries return new status
- Property 57: State JSON validity - For any state store, saving SHALL produce valid JSON
- State Store design
-
* 10.3 Write unit tests for State Store
- Test state loading and saving
- Test state queries by status
- Test state updates and idempotence
- Test state clearing
- Test status validation
- State Store design
11. Report Generator
-
11.1 Implement report generation
- Generate inventory report in CSV/JSON formats with all file metadata
- Generate completeness report in text/JSON formats with series gaps
- Generate duplicate report in text/JSON formats with quality comparisons
- Generate summary report in text format with library statistics
- Include generation timestamp and library root in all reports
- Requirements: 11.1-11.8
-
* 11.2 Write property tests for Report Generator
- Property 41: Inventory report completeness - For any set of scanned video files, inventory report SHALL include entry for each file
- Property 45: Report format validity - For any report, output SHALL be valid in specified format (CSV, JSON, or text)
- Property 46: Report metadata - For any generated report, report SHALL include generation timestamp and library root
- Requirements: 11.1, 11.5, 11.6, 11.7, 11.8
-
* 11.3 Write unit tests for Report Generator
- Test inventory report generation in CSV and JSON
- Test completeness report generation
- Test duplicate report generation
- Test summary report generation
- Test report metadata inclusion
- Requirements: 11.1-11.8
12. CLI Interface
-
12.1 Set up CLI framework
- Set up Click CLI with main entry point
- Implement global options: --config, --log-level
- Add help text for main command
- Implement error handling for invalid arguments
- Requirements: 14.1, 14.10, 14.11
-
12.2 Implement scan command
vlm scan [--output inventory.csv]- Wire to Inventory Scanner
- Display progress and summary
- Requirements: 14.2
-
12.3 Implement parse command
vlm parse [--input inventory.csv] [--output identities.json]- Wire to Identity Parser
- Display parsing statistics
- Requirements: 14.3
-
12.4 Implement analyze command
vlm analyze [--input identities.json] [--output analysis.json]- Wire to Analysis Engine
- Display analysis summary
- Requirements: 14.4
-
12.5 Implement plan command
vlm plan [--input identities.json] [--output plan.json]- Wire to Plan Generator
- Display plan summary with operation counts
- Requirements: 14.5
-
12.6 Implement execute command
vlm execute [--plan plan.json] [--confirm]- Wire to Execution Engine
- Default to dry-run mode, require --confirm for actual execution
- Display execution progress and summary
- Requirements: 14.6
-
12.7 Implement quarantine commands
vlm quarantine list [--category movie|series]vlm quarantine add <file> [--reason "duplicate"]vlm quarantine restore <file>- Wire to Quarantine Manager
- Requirements: 14.7
-
12.8 Implement rollback command
vlm rollback [--log rollback.json]- Wire to rollback functionality
- Display rollback progress and summary
- Requirements: 14.8
-
12.9 Implement report commands
vlm report inventory [--format csv|json|text]vlm report completeness [--format text|json]vlm report duplicates [--format text|json]vlm report summary- Wire to Report Generator
- Requirements: 14.9
-
12.10 Implement state commands
vlm state show <file>vlm state set <file> --status reviewed [--reason "checked manually"]vlm state query --status ignoredvlm state clear <file>- Wire to State Store
- Requirements: 14.9
-
12.11 Implement config commands
vlm config initvlm config showvlm config validate- Wire to Configuration Manager
- Requirements: 14.9
-
* 12.12 Write integration tests for CLI
- Test each command with various options
- Test error handling for invalid arguments
- Test help text display
- Test end-to-end workflow through CLI
- Requirements: 14.1-14.11
13. Integration and Polish
-
13.1 End-to-end integration testing
- Test complete workflow: scan → parse → analyze → plan → execute (dry-run) → execute (confirm) → rollback
- Test with real-world filename patterns and directory structures
- Test error scenarios: inaccessible files, invalid configs, conflicts
- Test quarantine operations end-to-end
- Test state management throughout workflow
- Verify all reports generate correctly with accurate data
- All requirements
-
13.2 Error handling validation
- Test parsing errors with ambiguous filenames
- Test file system errors (permissions, disk full, network issues)
- Test configuration errors
- Verify error logging and resilience
- Requirements: 13.1-13.7
-
13.3 Documentation
- Write README with installation instructions
- Document typical workflow with examples
- Add usage examples for all CLI commands
- Document configuration options
- Document v1 constraints and limitations
- Requirements: 14.10
-
13.4 Final checkpoint
- Ensure all tests pass (unit, property-based, integration)
- Verify all 57 correctness properties are validated
- Review code for security considerations (path validation, filename sanitization)
- Verify logging is comprehensive and useful
- Test with large library (performance validation)
- Ask user if any questions or issues arise
Notes
Testing Approach
- Property-based tests (marked with
*): Validate universal correctness properties using Hypothesis with minimum 100 iterations per test. Each test references a specific property from the design document. - Unit tests (marked with
*): Test specific examples, edge cases, and error conditions for each component. - Integration tests: Validate end-to-end workflows and component interactions.
Task Dependencies
Tasks are ordered to build incrementally:
- Foundation (config, logging, data structures)
- Core components (scanner, parser, analysis)
- Execution components (plan, execute, rollback)
- Supporting features (quarantine, state, reports)
- CLI interface
- Integration and polish
v1 Constraints
- Quarantine: Category-level only (movie/.quarantine/, series/.quarantine/), anime/other rejected
- Completeness: Heuristic gap detection only, no percentages or "complete" status
- Parsing: Hardcoded patterns, not user-configurable
- Anime: Inventory only, no organization
- Deletion: Quarantine only, no permanent deletion
- State Store: Simple path-based indexing, no fingerprint or move tracking
- Execution: Defaults to dry-run, requires --confirm for actual changes
- Rollback: Best-effort, not guaranteed restoration
Property-Based Testing Configuration
- Framework: Hypothesis (Python PBT library)
- Iterations: Minimum 100 per test
- Tagging: Each test includes comment:
# Feature: video-library-manager, Property N: [description] - Coverage: All 57 correctness properties from design document