1259 lines
43 KiB
Markdown
1259 lines
43 KiB
Markdown
# Design Document: Video Library Manager
|
|||
|
|
|
||
|
|
## 1. System Overview
|
||
|
|
|
||
|
|
The Video Library Manager is a Python-based CLI tool for managing personal video collections stored on network-attached storage. It follows a read-first, human-in-the-loop philosophy where all destructive operations require explicit confirmation and are fully reversible.
|
||
|
|
|
||
|
|
### 1.1 Design Principles
|
||
|
|
|
||
|
|
- **Safety First**: All operations are reversible; no permanent deletion in v1
|
||
|
|
- **Transparency**: All changes are presented as reviewable execution plans
|
||
|
|
- **Read-First**: Extensive analysis before any modifications
|
||
|
|
- **Human-in-the-Loop**: Explicit confirmation required for all file operations
|
||
|
|
- **Incremental**: Support for gradual library cleanup over time
|
||
|
|
|
||
|
|
### 1.2 Architecture
|
||
|
|
|
||
|
|
```text
|
||
|
|
┌─────────────────────────────────────────────────────────────┐
|
||
|
|
│ CLI Interface (Click) │
|
||
|
|
└─────────────────────────────────────────────────────────────┘
|
||
|
|
│
|
||
|
|
┌─────────────────────┼─────────────────────┐
|
||
|
|
│ │ │
|
||
|
|
┌───────▼────────┐ ┌────────▼────────┐ ┌──────▼──────┐
|
||
|
|
│ Inventory │ │ Analysis │ │ Execution │
|
||
|
|
│ Scanner │ │ Engine │ │ Engine │
|
||
|
|
└───────┬────────┘ └────────┬────────┘ └──────┬──────┘
|
||
|
|
│ │ │
|
||
|
|
└─────────────────────┼─────────────────────┘
|
||
|
|
│
|
||
|
|
┌─────────▼─────────┐
|
||
|
|
│ Identity Parser │
|
||
|
|
└─────────┬─────────┘
|
||
|
|
│
|
||
|
|
┌─────────┼─────────┐
|
||
|
|
│ │ │
|
||
|
|
┌─────────▼───┐ ┌──▼──────┐ ┌▼─────────────┐
|
||
|
|
│Configuration│ │ State │ │Report │
|
||
|
|
│Manager │ │ Store │ │Generator │
|
||
|
|
└─────────────┘ └─────────┘ └──────────────┘
|
||
|
|
```
|
||
|
|
|
||
|
|
## 2. Component Design
|
||
|
|
|
||
|
|
### 2.1 Configuration Manager
|
||
|
|
|
||
|
|
**Purpose**: Load, validate, and provide access to user configuration.
|
||
|
|
|
||
|
|
**Data Structure**:
|
||
|
|
|
||
|
|
```python
|
||
|
|
@dataclass
|
||
|
|
class Config:
|
||
|
|
library_root: Path
|
||
|
|
video_extensions: list[str]
|
||
|
|
movie_template: str # "movie/{title} ({year})/"
|
||
|
|
series_template: str # "series/{title}/Season {season:02d}/"
|
||
|
|
movie_filename_template: str # "{title} ({year}){ext}"
|
||
|
|
series_filename_template: str # "S{season:02d}E{episode:02d}{ext}"
|
||
|
|
log_level: str
|
||
|
|
quarantine_dir: str # ".quarantine" - relative to category root
|
||
|
|
```
|
||
|
|
|
||
|
|
**Configuration File Format** (YAML):
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
library_root: "/mnt/nas/videos"
|
||
|
|
video_extensions:
|
||
|
|
- .mp4
|
||
|
|
- .mkv
|
||
|
|
- .avi
|
||
|
|
- .mov
|
||
|
|
- .wmv
|
||
|
|
- .flv
|
||
|
|
- .webm
|
||
|
|
- .m4v
|
||
|
|
|
||
|
|
templates:
|
||
|
|
movie_dir: "movie/{title} ({year})/"
|
||
|
|
series_dir: "series/{title}/Season {season:02d}/"
|
||
|
|
movie_filename: "{title} ({year}){ext}"
|
||
|
|
series_filename: "S{season:02d}E{episode:02d}{ext}"
|
||
|
|
|
||
|
|
quarantine_dir: ".quarantine"
|
||
|
|
log_level: "INFO"
|
||
|
|
```
|
||
|
|
|
||
|
|
**Parsing Patterns** (hardcoded in implementation):
|
||
|
|
|
||
|
|
Movies:
|
||
|
|
|
||
|
|
- `{title} ({year})` - High confidence
|
||
|
|
- `{title}.{year}` - High confidence
|
||
|
|
- `{title} - {year}` - Medium confidence
|
||
|
|
- `{title} {year}` - Medium confidence (if year is 4 digits)
|
||
|
|
|
||
|
|
Series:
|
||
|
|
|
||
|
|
- `S{season:02d}E{episode:02d}` - High confidence
|
||
|
|
- `{season}x{episode}` - High confidence
|
||
|
|
- `Season {season} Episode {episode}` - Medium confidence
|
||
|
|
|
||
|
|
**Note**: Parsing patterns are hardcoded in the implementation and not user-configurable in v1.
|
||
|
|
|
||
|
|
**Operations**:
|
||
|
|
- `load_config(path: Path) -> Config`: Load and validate configuration
|
||
|
|
- `create_default_config(path: Path) -> Config`: Create default configuration file
|
||
|
|
- `validate_config(config: Config) -> list[str]`: Validate configuration and return errors
|
||
|
|
|
||
|
|
### 2.2 Inventory Scanner
|
||
|
|
|
||
|
|
**Purpose**: Discover and catalog all video files in the library.
|
||
|
|
|
||
|
|
**Data Structure**:
|
||
|
|
|
||
|
|
```python
|
||
|
|
@dataclass
|
||
|
|
class VideoFile:
|
||
|
|
path: Path
|
||
|
|
filename: str
|
||
|
|
size_bytes: int
|
||
|
|
modified_timestamp: datetime
|
||
|
|
category: str # "movie", "series", "anime", "other"
|
||
|
|
|
||
|
|
# Optional metadata (if ffprobe available)
|
||
|
|
resolution: Optional[str] # "1920x1080"
|
||
|
|
codec: Optional[str] # "h264"
|
||
|
|
duration_seconds: Optional[float]
|
||
|
|
bitrate_kbps: Optional[int]
|
||
|
|
```
|
||
|
|
|
||
|
|
**Operations**:
|
||
|
|
- `scan_library(root: Path, config: Config) -> list[VideoFile]`: Recursively scan for video files
|
||
|
|
- `extract_metadata(file: Path) -> dict`: Use ffprobe to extract video metadata
|
||
|
|
- `categorize_file(file: Path, root: Path) -> str`: Determine category based on directory structure
|
||
|
|
- `save_inventory_csv(files: list[VideoFile], output: Path)`: Save inventory to CSV (primary format)
|
||
|
|
- `save_inventory_json(files: list[VideoFile], output: Path)`: Save inventory to JSON (optional export)
|
||
|
|
|
||
|
|
**Inventory CSV Schema**:
|
||
|
|
- **Columns**: path, filename, size_bytes, modified_timestamp, category, resolution, codec, duration_seconds, bitrate_kbps
|
||
|
|
- **Required columns**: path, filename, size_bytes, modified_timestamp, category
|
||
|
|
- **Optional columns**: resolution, codec, duration_seconds, bitrate_kbps (present only if ffprobe available)
|
||
|
|
- **Timestamp format**: ISO 8601 (YYYY-MM-DDTHH:MM:SS) in UTC
|
||
|
|
- **Missing optional values**: Empty string
|
||
|
|
- **Header row**: Always present with column names
|
||
|
|
|
||
|
|
**Note**: Implementation details like UTF-8 encoding and line endings are left to standard CSV library defaults.
|
||
|
|
|
||
|
|
**Algorithm**:
|
||
|
|
1. Walk directory tree starting from library root
|
||
|
|
2. For each file, check if extension matches video extensions
|
||
|
|
3. Record file metadata (path, size, timestamp)
|
||
|
|
4. Attempt to extract video metadata using ffprobe (non-blocking)
|
||
|
|
5. Categorize based on parent directory structure
|
||
|
|
6. Handle errors gracefully and continue scanning
|
||
|
|
|
||
|
|
### 2.3 Identity Parser
|
||
|
|
|
||
|
|
**Purpose**: Extract logical identity (title, year, season, episode) from filenames.
|
||
|
|
|
||
|
|
**Data Structures**:
|
||
|
|
|
||
|
|
```python
|
||
|
|
@dataclass
|
||
|
|
class MovieIdentity:
|
||
|
|
title: str
|
||
|
|
year: Optional[int]
|
||
|
|
confidence: float # 0.0 to 1.0
|
||
|
|
needs_review: bool
|
||
|
|
original_filename: str
|
||
|
|
|
||
|
|
@dataclass
|
||
|
|
class SeriesIdentity:
|
||
|
|
title: str
|
||
|
|
season: Optional[int] # None if season cannot be determined
|
||
|
|
episodes: list[int] # Support multi-episode files
|
||
|
|
confidence: float
|
||
|
|
needs_review: bool
|
||
|
|
original_filename: str
|
||
|
|
```
|
||
|
|
|
||
|
|
**Parsing Patterns**:
|
||
|
|
|
||
|
|
Movies:
|
||
|
|
|
||
|
|
- `{title} ({year})` - High confidence
|
||
|
|
- `{title}.{year}` - High confidence
|
||
|
|
- `{title} - {year}` - Medium confidence
|
||
|
|
- `{title} {year}` - Medium confidence (if year is 4 digits)
|
||
|
|
|
||
|
|
Series:
|
||
|
|
|
||
|
|
- `S{season:02d}E{episode:02d}` - High confidence
|
||
|
|
- `{season}x{episode}` - High confidence
|
||
|
|
- `Season {season} Episode {episode}` - Medium confidence
|
||
|
|
|
||
|
|
**Parsing Fallback Rules**:
|
||
|
|
- If season number cannot be extracted, set `season = None` and `needs_review = True`
|
||
|
|
- If episode numbers cannot be extracted, set `episodes = []` and `needs_review = True`
|
||
|
|
- Files with `season = None` or `episodes = []` are flagged for manual review and excluded from organization plans
|
||
|
|
|
||
|
|
**Operations**:
|
||
|
|
- `parse_movie(filename: str) -> MovieIdentity`: Extract movie identity
|
||
|
|
- `parse_series(filename: str) -> SeriesIdentity`: Extract series identity
|
||
|
|
- `normalize_title(title: str) -> str`: Clean and normalize titles
|
||
|
|
- `extract_quality_tags(filename: str) -> list[str]`: Identify quality indicators
|
||
|
|
- `extract_release_group(filename: str) -> Optional[str]`: Identify release group
|
||
|
|
|
||
|
|
**Normalization Rules**:
|
||
|
|
- Remove quality tags: 1080p, 720p, 4K, BluRay, WEB-DL, HDTV, etc.
|
||
|
|
- Remove release group tags: [RARBG], (YTS), etc.
|
||
|
|
- Remove extra whitespace and dots
|
||
|
|
- Standardize capitalization (title case)
|
||
|
|
|
||
|
|
### 2.4 Analysis Engine
|
||
|
|
|
||
|
|
**Purpose**: Detect completeness issues, duplicates, and provide comparison data.
|
||
|
|
|
||
|
|
**Data Structures**:
|
||
|
|
|
||
|
|
```python
|
||
|
|
@dataclass
|
||
|
|
class SeasonCompleteness:
|
||
|
|
series_title: str
|
||
|
|
season: int
|
||
|
|
episodes_found: list[int]
|
||
|
|
episodes_missing: list[int] # Gaps in sequence [min, max]
|
||
|
|
|
||
|
|
@dataclass
|
||
|
|
class DuplicateGroup:
|
||
|
|
identity: Union[MovieIdentity, SeriesIdentity]
|
||
|
|
files: list[VideoFile]
|
||
|
|
quality_comparison: list[dict] # Resolution, codec, size for each file
|
||
|
|
```
|
||
|
|
|
||
|
|
**Operations**:
|
||
|
|
- `analyze_series_completeness(episodes: list[SeriesIdentity]) -> list[SeasonCompleteness]`: Detect missing episodes (heuristic gap detection)
|
||
|
|
- `detect_duplicates(identities: list[Union[MovieIdentity, SeriesIdentity]], files: list[VideoFile]) -> list[DuplicateGroup]`: Find duplicate content
|
||
|
|
- `compare_quality(files: list[VideoFile]) -> list[dict]`: Compare video quality metrics
|
||
|
|
|
||
|
|
**Completeness Algorithm** (Heuristic Gap Detection):
|
||
|
|
1. Group episodes by series title and season
|
||
|
|
2. For each season, find min and max episode numbers
|
||
|
|
3. Generate expected episode range [min, max]
|
||
|
|
4. Identify missing episodes in range as gaps
|
||
|
|
5. Report gaps only (no percentages, no "complete" status)
|
||
|
|
|
||
|
|
**Note**: This is heuristic gap detection only. Missing episodes are detected as gaps in [min, max], not as a measure of season completeness.
|
||
|
|
|
||
|
|
**Duplicate Detection Algorithm**:
|
||
|
|
1. Group files by normalized identity (title+year or title+season+episode)
|
||
|
|
2. For groups with multiple files, extract quality metrics
|
||
|
|
3. Compare resolution, codec, file size
|
||
|
|
4. Present comparison data without recommendations (user decides which to keep)
|
||
|
|
|
||
|
|
### 2.5 Plan Generator
|
||
|
|
|
||
|
|
**Purpose**: Create structured, reviewable execution plans for file operations.
|
||
|
|
|
||
|
|
**Data Structures**:
|
||
|
|
|
||
|
|
```python
|
||
|
|
@dataclass
|
||
|
|
class FileOperation:
|
||
|
|
operation_type: str # "move", "rename", "quarantine", "no-op"
|
||
|
|
source_path: Path
|
||
|
|
destination_path: Optional[Path]
|
||
|
|
reason: str
|
||
|
|
has_conflict: bool
|
||
|
|
conflict_reason: Optional[str]
|
||
|
|
|
||
|
|
@dataclass
|
||
|
|
class ExecutionPlan:
|
||
|
|
plan_id: str # UUID
|
||
|
|
created_at: datetime
|
||
|
|
operations: list[FileOperation]
|
||
|
|
summary: dict # Counts by operation type
|
||
|
|
```
|
||
|
|
|
||
|
|
**Operations**:
|
||
|
|
- `generate_plan(identities: list, files: list[VideoFile], config: Config) -> ExecutionPlan`: Create execution plan
|
||
|
|
- `detect_conflicts(operations: list[FileOperation]) -> list[FileOperation]`: Identify destination conflicts
|
||
|
|
- `save_plan(plan: ExecutionPlan, output: Path)`: Save plan to JSON
|
||
|
|
- `load_plan(path: Path) -> ExecutionPlan`: Load plan from JSON
|
||
|
|
|
||
|
|
**Plan Generation Algorithm**:
|
||
|
|
1. For each parsed identity, determine target directory structure
|
||
|
|
2. Apply directory and filename templates from config
|
||
|
|
3. Check if source and destination differ
|
||
|
|
4. If different, create move/rename operation
|
||
|
|
5. Check if destination already exists (conflict)
|
||
|
|
6. Mark conflicts for user review
|
||
|
|
|
||
|
|
**v1 Constraints**:
|
||
|
|
- Anime files: Generate no-op operations (not organized in v1)
|
||
|
|
- Other files: Generate no-op operations (not organized in v1)
|
||
|
|
- Series files with `season = None` or `episodes = []`: Generate no-op operations (flagged for manual review)
|
||
|
|
- Only movie and series files with valid identities are organized
|
||
|
|
|
||
|
|
### 2.6 Execution Engine
|
||
|
|
|
||
|
|
**Purpose**: Safely execute file operations with dry-run default and rollback capability.
|
||
|
|
|
||
|
|
**Data Structures**:
|
||
|
|
|
||
|
|
```python
|
||
|
|
@dataclass
|
||
|
|
class OperationResult:
|
||
|
|
operation: FileOperation
|
||
|
|
success: bool
|
||
|
|
error_message: Optional[str]
|
||
|
|
executed_at: datetime
|
||
|
|
|
||
|
|
@dataclass
|
||
|
|
class RollbackLog:
|
||
|
|
log_id: str # UUID
|
||
|
|
execution_plan_id: str
|
||
|
|
executed_at: datetime
|
||
|
|
operations: list[OperationResult]
|
||
|
|
```
|
||
|
|
|
||
|
|
**Operations**:
|
||
|
|
- `execute_plan(plan: ExecutionPlan, mode: str) -> list[OperationResult]`: Execute operations (mode: "dry-run" or "execute")
|
||
|
|
- `execute_operation(op: FileOperation, mode: str) -> OperationResult`: Execute single operation
|
||
|
|
- `rollback(log: RollbackLog) -> list[OperationResult]`: Reverse operations (best-effort)
|
||
|
|
- `save_rollback_log(log: RollbackLog, output: Path)`: Save rollback log
|
||
|
|
|
||
|
|
**Execution Algorithm**:
|
||
|
|
1. Load execution plan
|
||
|
|
2. Check mode parameter: "dry-run" (default) or "execute" (requires --confirm flag)
|
||
|
|
3. If mode is "dry-run", simulate operations and log what would happen
|
||
|
|
4. If mode is "execute", perform actual file operations
|
||
|
|
5. For each operation:
|
||
|
|
- Create destination directory if needed
|
||
|
|
- Check for conflicts (destination file already exists)
|
||
|
|
- If conflict exists, report the conflict and skip the operation
|
||
|
|
- Otherwise, execute file operation (move/rename) if mode is "execute"
|
||
|
|
- Log result (success/failure/skipped)
|
||
|
|
6. Save rollback log with all successful operations (only in "execute" mode)
|
||
|
|
7. Generate execution summary with counts of successful, failed, and skipped operations
|
||
|
|
|
||
|
|
**Conflict Handling**: When a destination file already exists, the operation is skipped and reported as a conflict. This prevents accidental overwrites while allowing other operations to proceed.
|
||
|
|
|
||
|
|
**Rollback Algorithm** (Best-Effort):
|
||
|
|
1. Load rollback log
|
||
|
|
2. Reverse operation list (LIFO order)
|
||
|
|
3. For each operation:
|
||
|
|
- If move: attempt to move destination back to source
|
||
|
|
- If rename: attempt to rename destination back to source
|
||
|
|
- If quarantine: attempt to move from quarantine back to original location
|
||
|
|
- Log success or failure for each operation
|
||
|
|
4. Generate rollback report with auditable results
|
||
|
|
|
||
|
|
**Rollback Guarantees**: Rollback is best-effort only. Operations may fail due to file system changes, permissions, or missing files. All rollback attempts are logged with detailed results for audit purposes. Users should verify file system state after rollback.
|
||
|
|
|
||
|
|
### 2.7 Quarantine Manager
|
||
|
|
|
||
|
|
**Purpose**: Safely isolate unwanted files for review before deletion.
|
||
|
|
|
||
|
|
**Data Structures**:
|
||
|
|
|
||
|
|
```python
|
||
|
|
@dataclass
|
||
|
|
class QuarantineEntry:
|
||
|
|
original_path: Path
|
||
|
|
quarantine_path: Path
|
||
|
|
quarantined_at: datetime
|
||
|
|
reason: Optional[str]
|
||
|
|
size_bytes: int
|
||
|
|
category: str # "movie" or "series"
|
||
|
|
|
||
|
|
@dataclass
|
||
|
|
class QuarantineManifest:
|
||
|
|
entries: list[QuarantineEntry]
|
||
|
|
```
|
||
|
|
|
||
|
|
**Operations**:
|
||
|
|
- `quarantine_files(files: list[Path], reason: str, config: Config) -> list[OperationResult]`: Move files to category-specific quarantine
|
||
|
|
- `list_quarantined(category: Optional[str]) -> list[QuarantineEntry]`: List quarantined files, optionally filtered by category
|
||
|
|
- `restore_from_quarantine(entries: list[QuarantineEntry]) -> list[OperationResult]`: Restore files (best-effort)
|
||
|
|
- `save_manifest(manifest: QuarantineManifest, path: Path)`: Save category-specific quarantine manifest
|
||
|
|
|
||
|
|
**Quarantine Structure** (v1 - category-level):
|
||
|
|
|
||
|
|
```text
|
||
|
|
<library_root>/
|
||
|
|
├── movie/
|
||
|
|
│ ├── .quarantine/
|
||
|
|
│ │ ├── manifest.json
|
||
|
|
│ │ └── [quarantined movie files]
|
||
|
|
│ └── [organized movies]
|
||
|
|
└── series/
|
||
|
|
├── .quarantine/
|
||
|
|
│ ├── manifest.json
|
||
|
|
│ └── [quarantined series files]
|
||
|
|
└── [organized series]
|
||
|
|
```
|
||
|
|
|
||
|
|
**v1 Constraints**:
|
||
|
|
- Only movie and series categories support quarantine
|
||
|
|
- Anime and other categories: quarantine operations rejected with error
|
||
|
|
- Each category has its own `.quarantine/` subdirectory and `manifest.json`
|
||
|
|
|
||
|
|
**Quarantine Algorithm**:
|
||
|
|
1. Verify file is in movie or series category (reject anime/other with error)
|
||
|
|
2. Determine relative path from category root
|
||
|
|
3. Construct quarantine path: `<category_root>/.quarantine/<relative_path>`
|
||
|
|
4. Check if quarantine destination exists
|
||
|
|
5. If exists, append numeric suffix (_1, _2, etc.)
|
||
|
|
6. Move file to quarantine
|
||
|
|
7. Update category-specific manifest with original location
|
||
|
|
8. Save manifest
|
||
|
|
|
||
|
|
### 2.8 Report Generator
|
||
|
|
|
||
|
|
**Purpose**: Generate comprehensive reports about library state.
|
||
|
|
|
||
|
|
**Report Types**:
|
||
|
|
|
||
|
|
1. **Inventory Report** (CSV/JSON):
|
||
|
|
- All discovered files with metadata
|
||
|
|
- Columns: path, filename, size, modified, category, resolution, codec, duration
|
||
|
|
|
||
|
|
2. **Completeness Report** (Text/JSON):
|
||
|
|
- Series with episode gaps (heuristic detection)
|
||
|
|
- Grouped by series and season
|
||
|
|
- Shows episodes_found and episodes_missing (gaps in [min, max])
|
||
|
|
|
||
|
|
3. **Duplicate Report** (Text/JSON):
|
||
|
|
- Groups of potential duplicates
|
||
|
|
- Quality comparison data for each group
|
||
|
|
- Sorted by file size (largest first)
|
||
|
|
- No automatic recommendations (user decides)
|
||
|
|
|
||
|
|
4. **Summary Report** (Text):
|
||
|
|
- Total file count and size
|
||
|
|
- Breakdown by category (movie, series, anime, other)
|
||
|
|
- Count of files needing review
|
||
|
|
- Count of potential duplicates
|
||
|
|
|
||
|
|
**Operations**:
|
||
|
|
- `generate_inventory_report(files: list[VideoFile], format: str) -> str`: Generate inventory report
|
||
|
|
- `generate_completeness_report(analysis: list[SeasonCompleteness], format: str) -> str`: Generate completeness report
|
||
|
|
- `generate_duplicate_report(duplicates: list[DuplicateGroup], format: str) -> str`: Generate duplicate report
|
||
|
|
- `generate_summary_report(files: list[VideoFile], analysis: dict) -> str`: Generate summary report
|
||
|
|
|
||
|
|
### 2.9 State Store
|
||
|
|
|
||
|
|
**Purpose**: Persist file statuses and user decisions for tracking workflow progress.
|
||
|
|
|
||
|
|
**Data Structures**:
|
||
|
|
|
||
|
|
```python
|
||
|
|
@dataclass
|
||
|
|
class FileState:
|
||
|
|
file_path: Path
|
||
|
|
status: str # "reviewed", "ignored", "planned", "executed", "quarantined"
|
||
|
|
reason: Optional[str]
|
||
|
|
updated_at: datetime
|
||
|
|
|
||
|
|
@dataclass
|
||
|
|
class StateStore:
|
||
|
|
states: dict[str, FileState] # Key: file path string
|
||
|
|
version: str
|
||
|
|
last_updated: datetime
|
||
|
|
```
|
||
|
|
|
||
|
|
**Operations**:
|
||
|
|
- `load_state(path: Path) -> StateStore`: Load state from JSON file
|
||
|
|
- `save_state(store: StateStore, path: Path)`: Save state to JSON file
|
||
|
|
- `get_file_state(file_path: Path) -> Optional[FileState]`: Get state for a file by path
|
||
|
|
- `set_file_state(file_path: Path, status: str, reason: Optional[str])`: Update file state (idempotent, updates timestamp)
|
||
|
|
- `query_by_status(status: str) -> list[FileState]`: Get all files with a given status
|
||
|
|
- `clear_state(file_path: Path)`: Remove state for a file
|
||
|
|
|
||
|
|
**State Store Location**:
|
||
|
|
|
||
|
|
```json
|
||
|
|
~/.vlm/state.json
|
||
|
|
```
|
||
|
|
|
||
|
|
**State Transitions**:
|
||
|
|
- New file → (no state)
|
||
|
|
- User reviews → "reviewed" (with optional reason)
|
||
|
|
- User ignores → "ignored" (with optional reason)
|
||
|
|
- Plan generated → "planned"
|
||
|
|
- Plan executed → "executed"
|
||
|
|
- File quarantined → "quarantined" (with optional reason)
|
||
|
|
|
||
|
|
**State Update Rules**:
|
||
|
|
- **Idempotent updates**: Setting the same status multiple times updates timestamp and reason
|
||
|
|
- **Status validation**: Only predefined statuses accepted ("reviewed", "ignored", "planned", "executed", "quarantined")
|
||
|
|
- **Audit trail**: State changes logged with timestamp
|
||
|
|
|
||
|
|
**Usage**:
|
||
|
|
- Track which files have been reviewed
|
||
|
|
- Remember user decisions (ignore, quarantine, etc.)
|
||
|
|
- Avoid re-processing handled files
|
||
|
|
- Simple workflow progress tracking
|
||
|
|
|
||
|
|
## 3. CLI Interface Design
|
||
|
|
|
||
|
|
**Framework**: Click (Python CLI framework)
|
||
|
|
|
||
|
|
**Command Structure**:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
vlm [OPTIONS] COMMAND [ARGS]
|
||
|
|
|
||
|
|
Commands:
|
||
|
|
scan Scan library and generate inventory
|
||
|
|
parse Parse identities from filenames
|
||
|
|
analyze Analyze completeness and duplicates
|
||
|
|
plan Generate execution plan
|
||
|
|
execute Execute plan (defaults to dry-run, requires --confirm)
|
||
|
|
quarantine Manage quarantined files (movie/series only in v1)
|
||
|
|
rollback Rollback previous execution (best-effort)
|
||
|
|
report Generate and export reports
|
||
|
|
state Manage file states
|
||
|
|
config Manage configuration
|
||
|
|
```
|
||
|
|
|
||
|
|
**Command Details**:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Scan library
|
||
|
|
vlm scan [--output inventory.csv]
|
||
|
|
|
||
|
|
# Parse identities
|
||
|
|
vlm parse [--input inventory.csv] [--output identities.json]
|
||
|
|
|
||
|
|
# Analyze library
|
||
|
|
vlm analyze [--input identities.json] [--output analysis.json]
|
||
|
|
|
||
|
|
# Generate execution plan
|
||
|
|
vlm plan [--input identities.json] [--output plan.json]
|
||
|
|
|
||
|
|
# Execute plan (defaults to dry-run)
|
||
|
|
vlm execute [--plan plan.json] # Dry-run (default)
|
||
|
|
vlm execute [--plan plan.json] --confirm # Actually execute
|
||
|
|
|
||
|
|
# Quarantine operations (movie/series only in v1)
|
||
|
|
vlm quarantine list [--category movie|series]
|
||
|
|
vlm quarantine add <file> [--reason "duplicate"]
|
||
|
|
vlm quarantine restore <file>
|
||
|
|
|
||
|
|
# Rollback (best-effort with auditable reporting)
|
||
|
|
vlm rollback [--log rollback.json]
|
||
|
|
|
||
|
|
# Reports
|
||
|
|
vlm report inventory [--format csv|json|text]
|
||
|
|
vlm report completeness [--format text|json]
|
||
|
|
vlm report duplicates [--format text|json]
|
||
|
|
vlm report summary
|
||
|
|
|
||
|
|
# State management
|
||
|
|
vlm state show <file>
|
||
|
|
vlm state set <file> --status reviewed [--reason "checked manually"]
|
||
|
|
vlm state query --status ignored
|
||
|
|
vlm state clear <file>
|
||
|
|
|
||
|
|
# Configuration
|
||
|
|
vlm config init
|
||
|
|
vlm config show
|
||
|
|
vlm config validate
|
||
|
|
```
|
||
|
|
|
||
|
|
**Help and Error Handling**:
|
||
|
|
|
||
|
|
Each command provides help text with usage examples via `vlm <command> --help`. When invalid arguments are provided, the system displays an error message and usage help.
|
||
|
|
|
||
|
|
**Validates: Requirements 14.1-14.11**
|
||
|
|
|
||
|
|
## 4. Data Flow
|
||
|
|
|
||
|
|
### 4.1 Typical Workflow
|
||
|
|
|
||
|
|
```text
|
||
|
|
1. Scan Library
|
||
|
|
vlm scan --output inventory.csv
|
||
|
|
→ Discovers all video files
|
||
|
|
→ Saves inventory to CSV
|
||
|
|
|
||
|
|
2. Parse Identities
|
||
|
|
vlm parse --input inventory.csv --output identities.json
|
||
|
|
→ Extracts titles, years, seasons, episodes
|
||
|
|
→ Saves parsed identities
|
||
|
|
|
||
|
|
3. Analyze
|
||
|
|
vlm analyze --input identities.json --output analysis.json
|
||
|
|
→ Detects episode gaps (heuristic)
|
||
|
|
→ Finds duplicates with comparison data
|
||
|
|
→ Saves analysis results
|
||
|
|
|
||
|
|
4. Review Reports
|
||
|
|
vlm report completeness
|
||
|
|
vlm report duplicates
|
||
|
|
→ User reviews findings
|
||
|
|
|
||
|
|
5. Generate Plan
|
||
|
|
vlm plan --input identities.json --output plan.json
|
||
|
|
→ Creates execution plan
|
||
|
|
→ User can edit plan.json manually
|
||
|
|
|
||
|
|
6. Dry Run (Default)
|
||
|
|
vlm execute --plan plan.json
|
||
|
|
→ Simulates operations (default behavior)
|
||
|
|
→ Shows what would happen
|
||
|
|
|
||
|
|
7. Execute with Confirmation
|
||
|
|
vlm execute --plan plan.json --confirm
|
||
|
|
→ Executes file operations
|
||
|
|
→ Saves rollback log
|
||
|
|
|
||
|
|
8. (Optional) Rollback (Best-Effort)
|
||
|
|
vlm rollback --log rollback_<id>.json
|
||
|
|
→ Attempts to reverse operations
|
||
|
|
→ Provides auditable report of results
|
||
|
|
|
||
|
|
9. (Optional) State Management
|
||
|
|
vlm state set <file> --status reviewed --reason "checked manually"
|
||
|
|
vlm state query --status ignored
|
||
|
|
→ Track workflow progress
|
||
|
|
→ Remember user decisions
|
||
|
|
```
|
||
|
|
|
||
|
|
## 5. File System Layout
|
||
|
|
|
||
|
|
### 5.1 Library Structure
|
||
|
|
|
||
|
|
**Before Organization**:
|
||
|
|
|
||
|
|
```text
|
||
|
|
/mnt/nas/videos/
|
||
|
|
├── movie/
|
||
|
|
│ ├── Some.Movie.2020.1080p.BluRay.mkv
|
||
|
|
│ ├── Another Film (2019).mp4
|
||
|
|
│ └── random_movie_file.avi
|
||
|
|
├── series/
|
||
|
|
│ ├── Show.Name.S01E01.mkv
|
||
|
|
│ ├── Show.Name.S01E02.mkv
|
||
|
|
│ └── Different Show 1x01.mp4
|
||
|
|
├── anime/
|
||
|
|
│ └── [Various files - not organized in v1]
|
||
|
|
└── other/
|
||
|
|
└── [Uncategorized files]
|
||
|
|
```
|
||
|
|
|
||
|
|
**After Organization**:
|
||
|
|
|
||
|
|
```text
|
||
|
|
/mnt/nas/videos/
|
||
|
|
├── movie/
|
||
|
|
│ ├── .quarantine/
|
||
|
|
│ │ ├── manifest.json
|
||
|
|
│ │ └── [quarantined movie files]
|
||
|
|
│ ├── Some Movie (2020)/
|
||
|
|
│ │ └── Some Movie (2020).mkv
|
||
|
|
│ ├── Another Film (2019)/
|
||
|
|
│ │ └── Another Film (2019).mp4
|
||
|
|
│ └── Random Movie File/
|
||
|
|
│ └── Random Movie File.avi
|
||
|
|
├── series/
|
||
|
|
│ ├── .quarantine/
|
||
|
|
│ │ ├── manifest.json
|
||
|
|
│ │ └── [quarantined series files]
|
||
|
|
│ ├── Show Name/
|
||
|
|
│ │ └── Season 01/
|
||
|
|
│ │ ├── S01E01.mkv
|
||
|
|
│ │ └── S01E02.mkv
|
||
|
|
│ └── Different Show/
|
||
|
|
│ └── Season 01/
|
||
|
|
│ └── S01E01.mp4
|
||
|
|
├── anime/
|
||
|
|
│ └── [Unchanged - no quarantine in v1]
|
||
|
|
└── other/
|
||
|
|
└── [Unchanged - no quarantine in v1]
|
||
|
|
```
|
||
|
|
|
||
|
|
### 5.2 Application Data
|
||
|
|
|
||
|
|
```text
|
||
|
|
~/.vlm/
|
||
|
|
├── config.yaml
|
||
|
|
├── state.json
|
||
|
|
├── logs/
|
||
|
|
│ ├── vlm.log
|
||
|
|
│ └── vlm.log.1
|
||
|
|
├── plans/
|
||
|
|
│ └── plan_<uuid>.json
|
||
|
|
└── rollback/
|
||
|
|
└── rollback_<uuid>.json
|
||
|
|
```
|
||
|
|
|
||
|
|
## 6. Error Handling Strategy
|
||
|
|
|
||
|
|
### 6.1 Error Categories
|
||
|
|
|
||
|
|
1. **Configuration Errors**: Invalid config, missing library root
|
||
|
|
- Action: Report error, use defaults where possible, exit if critical
|
||
|
|
|
||
|
|
2. **File System Errors**: Permission denied, file not found, disk full
|
||
|
|
- Action: Log error, skip file, continue processing
|
||
|
|
|
||
|
|
3. **Parsing Errors**: Cannot extract identity from filename
|
||
|
|
- Action: Mark for manual review, include in inventory
|
||
|
|
|
||
|
|
4. **Execution Errors**: Cannot move file, destination exists
|
||
|
|
- Action: Log error, skip operation, continue with remaining operations
|
||
|
|
|
||
|
|
5. **Metadata Extraction Errors**: ffprobe not available or fails
|
||
|
|
- Action: Log warning, continue without metadata
|
||
|
|
|
||
|
|
### 6.2 Logging Strategy
|
||
|
|
|
||
|
|
- Use Python `logging` module
|
||
|
|
- Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL
|
||
|
|
- Log to both console (INFO+) and file (DEBUG+)
|
||
|
|
- Include timestamps, operation type, file paths in all log entries
|
||
|
|
- Rotate log files at 10MB
|
||
|
|
|
||
|
|
## 7. Testing Strategy
|
||
|
|
|
||
|
|
*A property is a characteristic or behavior that should hold true across all valid executions of a system—essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
|
||
|
|
|
||
|
|
### 7.1 Dual Testing Approach
|
||
|
|
|
||
|
|
The Video Library Manager uses both unit testing and property-based testing for comprehensive coverage:
|
||
|
|
|
||
|
|
- **Unit tests**: Verify specific examples, edge cases, and error conditions
|
||
|
|
- **Property tests**: Verify universal properties across all inputs
|
||
|
|
|
||
|
|
Both approaches are complementary and necessary. Unit tests catch concrete bugs in specific scenarios, while property tests verify general correctness across a wide range of inputs.
|
||
|
|
|
||
|
|
### 7.2 Unit Tests
|
||
|
|
|
||
|
|
- Test each component independently with specific examples
|
||
|
|
- Mock file system operations for isolation
|
||
|
|
- Test parsing patterns with various filename formats
|
||
|
|
- Test configuration validation with known valid/invalid configs
|
||
|
|
- Test plan generation logic with specific file sets
|
||
|
|
- Test error handling with simulated failures
|
||
|
|
- Test edge cases like empty directories, special characters, long paths
|
||
|
|
|
||
|
|
### 7.3 Integration Tests
|
||
|
|
|
||
|
|
- Test full workflow with test fixtures
|
||
|
|
- Use temporary directories for file operations
|
||
|
|
- Test rollback functionality end-to-end
|
||
|
|
- Test quarantine operations with real file moves
|
||
|
|
- Verify CLI commands work correctly
|
||
|
|
- Test report generation with real data
|
||
|
|
|
||
|
|
### 7.4 Property-Based Tests
|
||
|
|
|
||
|
|
Property-based testing validates correctness properties across many generated inputs.
|
||
|
|
|
||
|
|
**Testing Framework**: Hypothesis (Python PBT library)
|
||
|
|
|
||
|
|
**Configuration**: Each property test runs minimum 100 iterations to ensure comprehensive coverage through randomization.
|
||
|
|
|
||
|
|
**Tagging**: Each property test includes a comment tag: `# Feature: video-library-manager, Property N: [property description]`
|
||
|
|
|
||
|
|
#### 7.4.1 Inventory Scanner Properties
|
||
|
|
|
||
|
|
**Property 1: Scan completeness**
|
||
|
|
|
||
|
|
*For any* directory tree containing video and non-video files, scanning SHALL discover all and only those files with extensions matching the configured video extensions.
|
||
|
|
|
||
|
|
**Validates: Requirements 1.1, 1.2**
|
||
|
|
|
||
|
|
**Property 2: Scan idempotence**
|
||
|
|
|
||
|
|
*For any* directory tree, running the scan operation multiple times SHALL produce identical inventory results.
|
||
|
|
|
||
|
|
**Validates: Requirements 1.6**
|
||
|
|
|
||
|
|
**Property 3: Scan safety**
|
||
|
|
|
||
|
|
*For any* directory tree, scanning SHALL never modify any files or directories.
|
||
|
|
|
||
|
|
**Validates: Requirements 1.6**
|
||
|
|
|
||
|
|
**Property 4: Scan resilience**
|
||
|
|
|
||
|
|
*For any* directory tree containing some inaccessible files, scanning SHALL discover all accessible files and continue processing.
|
||
|
|
|
||
|
|
**Validates: Requirements 1.7**
|
||
|
|
|
||
|
|
#### 7.4.2 Identity Parser Properties
|
||
|
|
|
||
|
|
**Property 5: Movie parsing**
|
||
|
|
|
||
|
|
*For any* filename matching common movie patterns, the parser SHALL extract title and year components.
|
||
|
|
|
||
|
|
**Validates: Requirements 2.1, 2.2**
|
||
|
|
|
||
|
|
**Property 6: Series parsing**
|
||
|
|
|
||
|
|
*For any* filename matching common series patterns, the parser SHALL extract series title, season number, and episode numbers.
|
||
|
|
|
||
|
|
**Validates: Requirements 3.1, 3.2**
|
||
|
|
|
||
|
|
**Property 7: Title normalization idempotence**
|
||
|
|
|
||
|
|
*For any* title string, normalizing it multiple times SHALL produce the same result.
|
||
|
|
|
||
|
|
**Validates: Requirements 2.6, 3.6**
|
||
|
|
|
||
|
|
**Property 8: Ambiguous filename flagging**
|
||
|
|
|
||
|
|
*For any* filename where season or episode information cannot be extracted, the parser SHALL mark the file as needing review.
|
||
|
|
|
||
|
|
**Validates: Requirements 2.5, 3.5**
|
||
|
|
|
||
|
|
**Property 9: Episode grouping**
|
||
|
|
|
||
|
|
*For any* set of parsed series episodes, grouping by series title and season SHALL place episodes with identical normalized titles and season numbers in the same group.
|
||
|
|
|
||
|
|
**Validates: Requirements 3.7**
|
||
|
|
|
||
|
|
#### 7.4.3 Analysis Engine Properties
|
||
|
|
|
||
|
|
**Property 10: Gap detection**
|
||
|
|
|
||
|
|
*For any* set of episodes within a season, the analysis engine SHALL detect missing episode numbers in the range [min, max].
|
||
|
|
|
||
|
|
**Validates: Requirements 4.1, 4.2**
|
||
|
|
|
||
|
|
**Property 11: Multi-season independence**
|
||
|
|
|
||
|
|
*For any* series with multiple seasons, gap detection of one season SHALL not affect other seasons.
|
||
|
|
|
||
|
|
**Validates: Requirements 4.4**
|
||
|
|
|
||
|
|
**Property 12: Duplicate detection for movies**
|
||
|
|
|
||
|
|
*For any* set of movie files with identical normalized titles and years, all such files SHALL be grouped as potential duplicates.
|
||
|
|
|
||
|
|
**Validates: Requirements 5.1**
|
||
|
|
|
||
|
|
**Property 13: Duplicate detection for series**
|
||
|
|
|
||
|
|
*For any* set of series files with identical normalized titles, season numbers, and episode numbers, all such files SHALL be grouped as potential duplicates.
|
||
|
|
|
||
|
|
**Validates: Requirements 5.2**
|
||
|
|
|
||
|
|
**Property 14: Duplicate quality comparison**
|
||
|
|
|
||
|
|
*For any* duplicate group, the comparison data SHALL include available metadata (file size, resolution, codec) for each file.
|
||
|
|
|
||
|
|
**Validates: Requirements 5.3**
|
||
|
|
|
||
|
|
#### 7.4.4 Plan Generator Properties
|
||
|
|
|
||
|
|
**Property 15: Plan operation completeness**
|
||
|
|
|
||
|
|
*For any* set of video files with parsed identities, the execution plan SHALL contain exactly one operation for each file.
|
||
|
|
|
||
|
|
**Validates: Requirements 6.1, 6.2**
|
||
|
|
|
||
|
|
**Property 16: Movie template consistency**
|
||
|
|
|
||
|
|
*For any* movie identity, applying templates SHALL always produce the same destination path.
|
||
|
|
|
||
|
|
**Validates: Requirements 6.3**
|
||
|
|
|
||
|
|
**Property 17: Series template consistency**
|
||
|
|
|
||
|
|
*For any* series episode identity, applying templates SHALL always produce the same destination path.
|
||
|
|
|
||
|
|
**Validates: Requirements 6.4**
|
||
|
|
|
||
|
|
**Property 18: Category boundary preservation**
|
||
|
|
|
||
|
|
*For any* file in a category directory, the execution plan SHALL specify a destination within the same category.
|
||
|
|
|
||
|
|
**Validates: Requirements 6.5**
|
||
|
|
|
||
|
|
**Property 19: Destination path uniqueness**
|
||
|
|
|
||
|
|
*For any* execution plan, no two operations SHALL specify the same destination path unless marked as conflict.
|
||
|
|
|
||
|
|
**Validates: Requirements 6.8**
|
||
|
|
|
||
|
|
**Property 20: Conflict detection**
|
||
|
|
|
||
|
|
*For any* execution plan where a destination file already exists, the operation SHALL be marked as having a conflict.
|
||
|
|
|
||
|
|
**Validates: Requirements 6.8**
|
||
|
|
|
||
|
|
**Property 21: Plan JSON validity**
|
||
|
|
|
||
|
|
*For any* generated execution plan, the output SHALL be valid JSON.
|
||
|
|
|
||
|
|
**Validates: Requirements 6.7**
|
||
|
|
|
||
|
|
#### 7.4.5 Execution Engine Properties
|
||
|
|
|
||
|
|
**Property 22: Execution mode safety**
|
||
|
|
|
||
|
|
*For any* execution plan, mode="dry-run" SHALL never modify the file system, while mode="execute" SHALL perform actual file operations.
|
||
|
|
|
||
|
|
**Validates: Requirements 7.1, 7.2**
|
||
|
|
|
||
|
|
**Property 23: Directory creation**
|
||
|
|
|
||
|
|
*For any* file operation with destination in non-existent directory, executing SHALL create necessary parent directories.
|
||
|
|
|
||
|
|
**Validates: Requirements 7.4**
|
||
|
|
|
||
|
|
**Property 24: Conflict handling**
|
||
|
|
|
||
|
|
*For any* execution plan with conflicts, executing SHALL skip conflicted operations and continue processing.
|
||
|
|
|
||
|
|
**Validates: Requirements 7.5, 7.7**
|
||
|
|
|
||
|
|
**Property 25: Operation logging**
|
||
|
|
|
||
|
|
*For any* executed operation, the log SHALL contain timestamp, operation type, paths, and result.
|
||
|
|
|
||
|
|
**Validates: Requirements 7.6**
|
||
|
|
|
||
|
|
**Property 26: Execution summary**
|
||
|
|
|
||
|
|
*For any* execution, the summary SHALL contain accurate counts of successful, failed, and skipped operations.
|
||
|
|
|
||
|
|
**Validates: Requirements 7.8**
|
||
|
|
|
||
|
|
#### 7.4.6 Rollback Properties
|
||
|
|
|
||
|
|
**Property 27: Rollback best-effort**
|
||
|
|
|
||
|
|
*For any* successful execution, rolling back SHALL attempt to restore all files to original locations.
|
||
|
|
|
||
|
|
**Validates: Requirements 9.1, 9.2**
|
||
|
|
|
||
|
|
**Property 28: Rollback idempotence**
|
||
|
|
|
||
|
|
*For any* rollback log, executing rollback multiple times SHALL produce the same final state.
|
||
|
|
|
||
|
|
**Validates: Requirements 9.1**
|
||
|
|
|
||
|
|
**Property 29: Rollback resilience**
|
||
|
|
|
||
|
|
*For any* rollback with failures, the process SHALL log errors and continue processing.
|
||
|
|
|
||
|
|
**Validates: Requirements 9.7**
|
||
|
|
|
||
|
|
**Property 30: Rollback summary**
|
||
|
|
|
||
|
|
*For any* rollback execution, the summary SHALL contain accurate counts of successful, failed, and skipped operations.
|
||
|
|
|
||
|
|
**Validates: Requirements 9.8**
|
||
|
|
|
||
|
|
#### 7.4.7 Quarantine Properties
|
||
|
|
|
||
|
|
**Property 31: Quarantine location (category-level)**
|
||
|
|
|
||
|
|
*For any* movie or series file marked for quarantine, the file SHALL be moved to `<category_root>/.quarantine/`.
|
||
|
|
|
||
|
|
**Validates: Requirements 8.1, 8.2**
|
||
|
|
|
||
|
|
**Property 32: Quarantine category restriction**
|
||
|
|
|
||
|
|
*For any* file in anime or other categories, quarantine operations SHALL be rejected with error.
|
||
|
|
|
||
|
|
**Validates: Requirements 8.1**
|
||
|
|
|
||
|
|
**Property 33: Quarantine manifest (per-category)**
|
||
|
|
|
||
|
|
*For any* quarantined file, the category-specific manifest SHALL contain original path, quarantine path, timestamp, and category.
|
||
|
|
|
||
|
|
**Validates: Requirements 8.4, 8.5**
|
||
|
|
|
||
|
|
**Property 34: Quarantine conflict handling**
|
||
|
|
|
||
|
|
*For any* file being quarantined where destination exists, the system SHALL append numeric suffix.
|
||
|
|
|
||
|
|
**Validates: Requirements 8.6**
|
||
|
|
|
||
|
|
**Property 35: Quarantine listing**
|
||
|
|
|
||
|
|
*For any* set of quarantined files, listing SHALL return all entries from category manifests, optionally filtered by category.
|
||
|
|
|
||
|
|
**Validates: Requirements 8.7**
|
||
|
|
|
||
|
|
**Property 36: Quarantine restore (best-effort)**
|
||
|
|
|
||
|
|
*For any* quarantined file, restoring SHALL attempt to move file back to original location.
|
||
|
|
|
||
|
|
**Validates: Requirements 8.4**
|
||
|
|
|
||
|
|
**Property 37: No permanent deletion in v1**
|
||
|
|
|
||
|
|
*For any* file operation in v1, the system SHALL never permanently delete files.
|
||
|
|
|
||
|
|
**Validates: Requirements 8.8**
|
||
|
|
|
||
|
|
#### 7.4.8 Configuration Properties
|
||
|
|
|
||
|
|
**Property 38: Configuration validation**
|
||
|
|
|
||
|
|
*For any* configuration with invalid values, validation SHALL detect and report errors.
|
||
|
|
|
||
|
|
**Validates: Requirements 10.8**
|
||
|
|
|
||
|
|
**Property 39: Default configuration validity**
|
||
|
|
|
||
|
|
*For any* missing configuration file, creating default configuration SHALL produce valid configuration.
|
||
|
|
|
||
|
|
**Validates: Requirements 10.6**
|
||
|
|
|
||
|
|
**Property 40: Configuration error handling**
|
||
|
|
|
||
|
|
*For any* configuration file with invalid YAML syntax, the system SHALL report error and use defaults.
|
||
|
|
|
||
|
|
**Validates: Requirements 10.7**
|
||
|
|
|
||
|
|
#### 7.4.9 Reporting Properties
|
||
|
|
|
||
|
|
**Property 41: Inventory report completeness**
|
||
|
|
|
||
|
|
*For any* set of scanned video files, the inventory report SHALL include an entry for each file.
|
||
|
|
|
||
|
|
**Validates: Requirements 11.1**
|
||
|
|
|
||
|
|
**Property 42: Completeness report**
|
||
|
|
|
||
|
|
*For any* set of analyzed series, the completeness report SHALL include all series with detected gaps.
|
||
|
|
|
||
|
|
**Validates: Requirements 11.2**
|
||
|
|
|
||
|
|
**Property 43: Duplicate report grouping**
|
||
|
|
|
||
|
|
*For any* set of detected duplicates, the duplicate report SHALL group files by identity with comparison data.
|
||
|
|
|
||
|
|
**Validates: Requirements 11.3**
|
||
|
|
|
||
|
|
**Property 44: Summary report accuracy**
|
||
|
|
|
||
|
|
*For any* scanned library, the summary report SHALL contain accurate counts and sizes.
|
||
|
|
|
||
|
|
**Validates: Requirements 11.4**
|
||
|
|
|
||
|
|
**Property 45: Report format validity**
|
||
|
|
|
||
|
|
*For any* report, the output SHALL be valid in the specified format (CSV, JSON, or text).
|
||
|
|
|
||
|
|
**Validates: Requirements 11.5, 11.6, 11.7**
|
||
|
|
|
||
|
|
**Property 46: Report metadata**
|
||
|
|
|
||
|
|
*For any* generated report, the report SHALL include generation timestamp and library root.
|
||
|
|
|
||
|
|
**Validates: Requirements 11.8**
|
||
|
|
|
||
|
|
#### 7.4.10 Anime Handling Properties
|
||
|
|
|
||
|
|
**Property 47: Anime inventory inclusion**
|
||
|
|
|
||
|
|
*For any* video files in anime directory, scanning SHALL discover and catalog them.
|
||
|
|
|
||
|
|
**Validates: Requirements 12.1, 12.2**
|
||
|
|
|
||
|
|
**Property 48: Anime processing deferral**
|
||
|
|
|
||
|
|
*For any* anime files, execution plans SHALL contain only no-op operations.
|
||
|
|
|
||
|
|
**Validates: Requirements 12.3, 12.4, 12.5**
|
||
|
|
|
||
|
|
#### 7.4.11 Error Handling Properties
|
||
|
|
|
||
|
|
**Property 49: Error logging**
|
||
|
|
|
||
|
|
*For any* operation that encounters an error, the log SHALL contain timestamp, operation type, and error details.
|
||
|
|
|
||
|
|
**Validates: Requirements 13.1**
|
||
|
|
|
||
|
|
**Property 50: Error resilience**
|
||
|
|
|
||
|
|
*For any* operation sequence with non-fatal errors, the system SHALL log errors and continue processing.
|
||
|
|
|
||
|
|
**Validates: Requirements 13.2, 13.3**
|
||
|
|
|
||
|
|
**Property 51: Parsing error handling**
|
||
|
|
|
||
|
|
*For any* filename that cannot be parsed, the system SHALL log error and mark file for review.
|
||
|
|
|
||
|
|
**Validates: Requirements 13.4**
|
||
|
|
|
||
|
|
**Property 52: Log output**
|
||
|
|
|
||
|
|
*For any* logged message, the message SHALL appear in console (if log level permits) and log file.
|
||
|
|
|
||
|
|
**Validates: Requirements 13.6**
|
||
|
|
|
||
|
|
**Property 53: Log rotation**
|
||
|
|
|
||
|
|
*For any* log file exceeding 10MB, the system SHALL rotate the log file.
|
||
|
|
|
||
|
|
**Validates: Requirements 13.7**
|
||
|
|
|
||
|
|
#### 7.4.12 State Store Properties
|
||
|
|
|
||
|
|
**Property 54: State persistence**
|
||
|
|
|
||
|
|
*For any* state store saved to disk, loading SHALL produce equivalent StateStore with all file states preserved.
|
||
|
|
|
||
|
|
**Validates: State Store design**
|
||
|
|
|
||
|
|
**Property 55: State query**
|
||
|
|
|
||
|
|
*For any* status value, querying SHALL return all files with that status.
|
||
|
|
|
||
|
|
**Validates: State Store design**
|
||
|
|
|
||
|
|
**Property 56: State update**
|
||
|
|
|
||
|
|
*For any* file path, setting state SHALL update the store and subsequent queries return new status.
|
||
|
|
|
||
|
|
**Validates: State Store design**
|
||
|
|
|
||
|
|
**Property 57: State JSON validity**
|
||
|
|
|
||
|
|
*For any* state store, saving SHALL produce valid JSON.
|
||
|
|
|
||
|
|
**Validates: State Store design**
|
||
|
|
|
||
|
|
## 8. Dependencies
|
||
|
|
|
||
|
|
### 8.1 Core Dependencies
|
||
|
|
|
||
|
|
- **Python**: 3.10+ (for dataclasses, type hints, pattern matching)
|
||
|
|
- **Click**: CLI framework
|
||
|
|
- **PyYAML**: Configuration file parsing
|
||
|
|
- **ffmpeg-python**: Video metadata extraction (optional)
|
||
|
|
|
||
|
|
### 8.2 Development Dependencies
|
||
|
|
|
||
|
|
- **pytest**: Unit testing
|
||
|
|
- **hypothesis**: Property-based testing
|
||
|
|
- **black**: Code formatting
|
||
|
|
- **mypy**: Type checking
|
||
|
|
- **ruff**: Linting
|
||
|
|
|
||
|
|
## 9. Performance Considerations
|
||
|
|
|
||
|
|
### 9.1 Scanning Performance
|
||
|
|
|
||
|
|
- Use `os.scandir()` for efficient directory traversal
|
||
|
|
- Parallel metadata extraction using `concurrent.futures`
|
||
|
|
- Skip metadata extraction if ffprobe not available
|
||
|
|
- Cache inventory results to avoid re-scanning
|
||
|
|
|
||
|
|
### 9.2 Memory Management
|
||
|
|
|
||
|
|
- Stream large reports to disk rather than building in memory
|
||
|
|
- Process files in batches for large libraries
|
||
|
|
- Use generators for file iteration
|
||
|
|
|
||
|
|
### 9.3 Network Storage
|
||
|
|
|
||
|
|
- Minimize stat calls on network paths
|
||
|
|
- Batch file operations where possible
|
||
|
|
- Handle network timeouts gracefully
|
||
|
|
|
||
|
|
## 10. Future Enhancements (Post-v1)
|
||
|
|
|
||
|
|
- Anime parsing and organization
|
||
|
|
- Automatic quality-based duplicate resolution
|
||
|
|
- Integration with metadata databases (TMDB, TVDB)
|
||
|
|
- Web UI for plan review and editing
|
||
|
|
- Scheduled scanning and monitoring
|
||
|
|
- Subtitle file handling
|
||
|
|
- NFO file generation
|
||
|
|
- Permanent deletion with confirmation
|
||
|
|
- Multi-library support
|
||
|
|
|
||
|
|
## 11. Security Considerations
|
||
|
|
|
||
|
|
- Validate all file paths to prevent directory traversal
|
||
|
|
- Sanitize filenames to prevent command injection
|
||
|
|
- Limit file operations to library root and quarantine
|
||
|
|
- Log all file operations for audit trail
|
||
|
|
- No network operations in v1 (offline tool)
|
||
|
|
|
||
|
|
## 12. Correctness Properties Summary
|
||
|
|
|
||
|
|
The system maintains 57 formally specified correctness properties validated through property-based testing:
|
||
|
|
|
||
|
|
**Safety Properties** (operations that never cause harm):
|
||
|
|
- **Read-Only Scanning**: Scanning operations never modify files (Property 3)
|
||
|
|
- **Dry-Run Default**: Execution defaults to dry-run mode, requiring --confirm for actual changes (Property 22)
|
||
|
|
- **Category Boundaries**: Files never cross category boundaries during organization (Property 18)
|
||
|
|
- **Quarantine Restrictions**: Only movie/series files can be quarantined in v1 (Property 32)
|
||
|
|
- **No Permanent Deletion**: System never permanently deletes files in v1 (Property 37)
|
||
|
|
|
||
|
|
**Reversibility Properties** (operations that can be undone, best-effort):
|
||
|
|
- **Operation Reversibility**: File operations are reversible via rollback (Property 27)
|
||
|
|
- **Rollback Idempotence**: Multiple rollbacks produce the same result (Property 28)
|
||
|
|
- **Quarantine Reversibility**: Quarantined files can be restored (Property 36)
|
||
|
|
|
||
|
|
**Consistency Properties** (operations that maintain invariants):
|
||
|
|
- **Template Consistency**: Applying templates to the same identity produces the same path (Properties 16, 17)
|
||
|
|
- **Title Normalization**: Normalizing titles is idempotent (Property 7)
|
||
|
|
- **Scan Idempotence**: Multiple scans produce identical results (Property 2)
|
||
|
|
|
||
|
|
**Completeness Properties** (operations that handle all cases):
|
||
|
|
- **Scan Completeness**: All video files are discovered (Property 1)
|
||
|
|
- **Plan Completeness**: Every file has exactly one operation (Property 15)
|
||
|
|
- **Gap Detection**: Missing episodes are identified heuristically (Property 10)
|
||
|
|
- **Duplicate Detection**: Duplicates are flagged with comparison data (Properties 12, 13)
|
||
|
|
|
||
|
|
**Accuracy Properties** (calculations that are correct):
|
||
|
|
- **Summary Accuracy**: Report statistics match actual data (Properties 26, 30, 44)
|
||
|
|
- **Manifest Accuracy**: Quarantine manifests match actual files per category (Property 33)
|
||
|
|
- **State Query Accuracy**: State queries return correct file sets (Property 55)
|
||
|
|
|
||
|
|
**Resilience Properties** (operations that handle errors gracefully):
|
||
|
|
- **Scan Resilience**: Inaccessible files don't halt scanning (Property 4)
|
||
|
|
- **Execution Resilience**: Failed operations don't halt execution (Property 24)
|
||
|
|
- **Rollback Resilience**: Failed rollbacks don't halt rollback process (Property 29)
|
||
|
|
- **Error Resilience**: Errors are logged and processing continues (Property 50)
|
||
|
|
|
||
|
|
**Uniqueness Properties** (no conflicts or duplicates):
|
||
|
|
- **Path Uniqueness**: No two operations target the same destination (Property 19)
|
||
|
|
- **Conflict Detection**: Existing destinations are detected (Property 20)
|
||
|
|
|
||
|
|
**Structure Preservation Properties** (organization is maintained):
|
||
|
|
- **Quarantine Structure**: Category-level quarantine structure (Property 31)
|
||
|
|
- **Anime Preservation**: Anime directory structure remains unchanged (Property 48)
|
||
|
|
|
||
|
|
**State Management Properties** (workflow tracking):
|
||
|
|
- **State Persistence**: State stores are saved and loaded correctly with path-based indexing (Property 54)
|
||
|
|
- **State Updates**: File states are updated and queryable (Properties 55, 56)
|
||
|
|
- **State Validity**: State stores are valid JSON (Property 57)
|
||
|
|
|
||
|
|
**Configuration Properties** (system configuration):
|
||
|
|
- **Configuration Validation**: Invalid configurations are detected (Property 38)
|
||
|
|
- **Default Configuration**: Valid default configuration is created when missing (Property 39)
|
||
|
|
- **Configuration Error Handling**: Invalid syntax is handled gracefully (Property 40)
|
||
|
|
|
||
|
|
All properties are validated using Hypothesis with minimum 100 iterations per test.
|
||
|
|
|
||
|
|
**v1 Design Constraints**:
|
||
|
|
- Quarantine is category-level (movie/.quarantine/, series/.quarantine/) with separate manifests
|
||
|
|
- Config.quarantine_dir is relative to category root (".quarantine")
|
||
|
|
- Only movie and series categories support quarantine (anime/other rejected with error)
|
||
|
|
- Series with season=None or episodes=[] are flagged for manual review and excluded from plans
|
||
|
|
- Completeness analysis is heuristic gap detection only (reports gaps in [min, max])
|
||
|
|
- Analysis provides comparison data, never automatic recommendations
|
||
|
|
- Execution uses mode parameter ("dry-run" or "execute"), defaults to dry-run, requires --confirm
|
||
|
|
- Rollback is best-effort, not guaranteed restoration
|
||
|
|
- Inventory uses CSV as primary format, JSON as optional export
|
||
|
|
- State Store uses simple path-based indexing (no fingerprint, no move tracking)
|
||
|
|
- Plan Generator excludes anime/other files (generates no-op operations)
|
||
|
|
- Parsing patterns are hardcoded in implementation (not configurable)
|