Compare commits

..
3 Commits
Author SHA1 Message Date
windyboyandClaude Sonnet 4.5 8a60aaf9a9 docs: consolidate CLAUDE.md into AGENTS.md and simplify
Merge detailed content from CLAUDE.md into AGENTS.md, then condense
to essential information. Removes Claude-specific documentation in
favor of tool-agnostic guidelines.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-09-25 14:05:44 +08:00
windyboyandClaude Sonnet 4.5 0f0636bf19 fix: complete rate limiter integration and report command structure
- Add RequestRateLimiter class to providers/base.py with wait() method
- Integrate rate limiter through enrichment pipeline (enrich_identities_data → _enrich_record_with_fresh_providers → _build_providers → TMDBProvider)
- Add _cmd wrapper functions to commands/report.py for CLI imports
- Fix SeriesIdentity import in reports.py
- Add sidecar file tracking to planner review_context for move operations
- Update tests to match new rate limiter signature

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-09-25 14:03:01 +08:00
windyboy dfa18ed405 refactor: consolidate skill docs, add anti-drift tests, and apply audit fixes
DLO-13: Restructure vlm-library-workflow skill as safety contract layer.
- Rewrite SKILL.md (69 lines): safety contract, execution threshold semantics,
  six-step high-risk loop, decision rules, phase skeleton
- Delete redundant references (cli-reference, workflow, command-recipes, dev-guide)
- Add triage.md (failure mapping + preflight) and dev-map.md (module→test mapping)
- Add tests/test_docs_consistency.py: 78 parametrized tests verifying documented
  vlm commands exist in CLI registry
- Add CSV path mismatch test to test_plan_review.py (4th safety gate path)
- Delete vlm-expert.skill (Gemini package, 7 months stale) and README Gemini section

DLO-2 audit fixes: rate limiter injection, symmetric quarantine categories,
review-plan safety gates, parser improvements, planner validation.

CLI modularization: commands/ directory with one module per command group.
2026-09-25 13:50:09 +08:00
39 changed files with 1259 additions and 927 deletions
+79 -37
View File
@@ -1,46 +1,88 @@
# Repository Guidelines # Repository Guidelines
## Project Structure & Module Organization ## Project Overview
- Core package lives in `src/vlm/`. Video Library Manager (VLM) - Python CLI for managing personal video collections. Safety-first, human-in-the-loop approach. All operations reversible.
- CLI entrypoint is `src/vlm/cli.py` (`vlm` console script). Shared CLI helpers live in `cli_helpers.py`. Command logic is in `commands/` (scan, parse, enrich, analyze, plan, execute, review_plan, report, quarantine_cmd, state_cmd, config_cmd).
- Functional modules by concern: scanning (`scanner.py`), parsing (`parser.py`), enrichment (`enrichment.py`, `cache.py`, `providers/`), analysis/planning/execution (`analysis.py`, `planner.py`, `executor.py`), I/O helpers (`io.py`), utilities (`utils.py`), state/reporting/logging (`state.py`, `reports.py`, `logging_config.py`), config (`config.py`), models (`models.py`).
- Tests live in `tests/` and mirror feature areas (e.g. `tests/test_scanner.py`, `tests/test_cli_state.py`, `tests/test_enrichment.py`).
- Project metadata and tool config are in `pyproject.toml`.
## Build, Test, and Development Commands ## Quick Start
- `uv pip install -e .` installs the package in editable mode. ```bash
- `uv pip install -e ".[dev]"` installs dev dependencies (`pytest`, `hypothesis`, `pytest-cov`, `ruff`). uv pip install -e ".[dev]" # Install with dev deps
- `uv run pytest -q` runs the full test suite. uv run pytest -q # Run tests
- `uv run pytest tests/test_logging.py` runs a targeted test file during iteration. uv run vlm --help # Verify CLI
- `uv run ruff check src tests` runs the linter (also in CI). ```
- `vlm --help` verifies CLI startup and available commands.
## Coding Style & Naming Conventions ## Core Workflow
- Use Python 3.10+ idioms, 4-space indentation, and PEP 8 naming. 1. `vlm scan` → discover files → `artifacts/inventory.csv`
- Modules/functions/variables: `snake_case`; classes: `PascalCase`; constants: `UPPER_SNAKE_CASE`. 2. `vlm parse` → extract identities → `artifacts/identities.json`
- Keep modules focused on a single responsibility; prefer small pure helpers in domain modules. 3. `vlm enrich` → (optional) add TMDB metadata
- Add type hints for public functions and non-trivial internal APIs. 4. `vlm analyze` → detect gaps/duplicates → `artifacts/analysis.json`
- Use absolute imports in `src/vlm/`: `from vlm.module import ...` (avoid new relative imports). 5. `vlm plan` → generate execution plan → `artifacts/plan.json`
- Ruff (`E`, `F`, `I`) is configured in `pyproject.toml`; CI runs `ruff check src tests`. 6. `vlm review-plan` → preview high-risk operations
7. `vlm execute` → dry-run by default, `--confirm` to execute
8. `vlm rollback` → undo executed operations
## Testing Guidelines ## Project Structure
- Framework: `pytest`; property-based tests use `hypothesis`. - `src/vlm/cli.py` - CLI entrypoint
- Naming (enforced in config): files `test_*.py`, functions `test_*`, classes `Test*`. - `src/vlm/commands/` - Command implementations (scan, parse, enrich, analyze, plan, execute, review_plan, report, quarantine_cmd, state_cmd, config_cmd)
- Add/extend tests with each behavior change, including CLI error paths and edge cases. - `src/vlm/scanner.py` - File discovery + ffprobe metadata
- Prefer narrow unit tests for module logic plus targeted CLI integration tests via `CliRunner`. - `src/vlm/parser.py` - Filename parsing (movies: title+year, series: SxxExx)
- `src/vlm/enrichment.py` - TMDB enrichment pipeline
- `src/vlm/planner.py` - Execution plan generation
- `src/vlm/executor.py` - File operations with rollback
- `src/vlm/models.py` - Data structures (VideoFile, MovieIdentity, SeriesIdentity, etc.)
- `tests/` - Test suite mirroring source modules
## Commit & Pull Request Guidelines ## Key Concepts
- Current history is minimal; use clear, imperative commit subjects (example: `fix logging fallback for unwritable log dir`).
- Keep commits focused; avoid mixing refactors and behavior changes unless tightly coupled.
- PRs should include: summary, rationale, test evidence (`pytest` output), and any CLI-visible output changes.
- Link related issues/tasks when applicable and call out config or migration impacts.
## Security & Configuration Tips ### Safety Protocol
- Do not commit local paths, personal media metadata, or generated state/log artifacts. - NEVER delete files permanently - use quarantine
- Validate config changes against `vlm --help` and at least one end-to-end CLI flow before merging. - All operations create rollback logs with `--confirm`
- Default mode is dry-run
### File Categorization
Based on top-level directory matching `categories` config (case-insensitive). Default: `movie`, `series`, `anime`.
## Documentation baseline ### Schema Versions
- Updated to reflect release 0.2.0 baseline as of 2026-06-01. - **v1** (default): Lightweight, no embedded metadata
- Canonical release notes are tracked in `CHANGELOG.md`. - **v2** (with `--inventory`): Includes video metadata for quality-aware duplicate resolution
- Default workflow artifacts: `artifacts/` (do not commit generated CSV/JSON).
### Parsing Patterns (Hardcoded)
- Movies: `{title} ({year})` or `{title}.{year}`
- Series: `S{season:02d}E{episode:02d}` or `{season}x{episode}`
## Development Commands
```bash
uv run pytest # All tests
uv run pytest tests/test_scanner.py # Specific file
uv run ruff check src tests # Lint
uv run vlm scan # Discover files
uv run vlm parse --inventory artifacts/inventory.csv # Parse with metadata
uv run vlm plan --analysis artifacts/analysis.json # Plan with duplicates
uv run vlm review-plan --tui # Interactive review (requires [tui])
```
## Code Style
- Python 3.10+, 4-space indent, PEP 8
- `snake_case` functions/vars, `PascalCase` classes, `UPPER_SNAKE_CASE` constants
- Type hints for public APIs
- Absolute imports: `from vlm.module import ...`
- Ruff (`E`, `F`, `I`) enforced in CI
## Testing
- pytest + hypothesis for property-based tests
- Naming: `test_*.py`, `test_*()`, `Test*`
- Add tests with behavior changes
- Prefer unit tests + targeted CLI integration via `CliRunner`
## Commit Guidelines
- Imperative subjects: `fix logging fallback for unwritable log dir`
- Keep commits focused (no mixed refactors + behavior changes)
- Include: `Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>`
- PRs: summary, rationale, test evidence, CLI output changes
## Important Notes
- Anime: discovered but not parsed in v1
- State tracking: optional, persisted to `~/.vlm/state.json`
- Quarantine: only movie/series (not anime/other)
- Timestamps: UTC ISO 8601 format
- Logging: `~/.vlm/vlm.log`, falls back to console if unwritable
- Config: `~/.vlm/config.yaml`
+11
View File
@@ -13,6 +13,12 @@
## 2026-05-21 ## 2026-05-21
### CLI line-count reduction (phase 2)
- Shrunk `cli.py` to a thin registrar (~70 lines); Click definitions live beside command implementations in `commands/`.
- Deduplicated report commands via `reports.duplicate_groups_from_analysis` / `completeness_from_analysis` and shared `cli_helpers.run_command` / `emit_report`.
- Archived the completed simplification plan under `docs/archive/2026-pre-baseline/`.
### Functional code simplification ### Functional code simplification
- Removed unused `src/vlm/exceptions.py` and consolidated plan summary helpers (`preferred_plan_summary` everywhere). - Removed unused `src/vlm/exceptions.py` and consolidated plan summary helpers (`preferred_plan_summary` everywhere).
@@ -22,6 +28,11 @@
- Fixed series parser empty title when the only title token is a quality tag (e.g. `UHD S01E01.mp4`). - Fixed series parser empty title when the only title token is a quality tag (e.g. `UHD S01E01.mp4`).
- Verification: `pytest -q` → **517 passed**. - Verification: `pytest -q` → **517 passed**.
### Documentation sync
- Resynchronized `README.md`, `CLAUDE.md`, `AGENTS.md`, and `skills/vlm-library-workflow/` docs to the CLI-modularization baseline; refreshed validation baseline (`pytest -q` → **529 passed**).
- Fixed the archive index pointer to the completed simplification plan (now inside `docs/archive/2026-pre-baseline/`).
## 2026-04-07 ## 2026-04-07
### Review-plan Safety & Validation Hardening ### Review-plan Safety & Validation Hardening
-272
View File
@@ -1,272 +0,0 @@
# CLAUDE.md
## Documentation Status
- Synchronized with release 0.2.0 baseline on 2026-06-01 (see `CHANGELOG.md`).
- Default workflow outputs: `artifacts/` — do not commit generated CSV/JSON at repo root.
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Video Library Manager (VLM) is a Python CLI tool for managing personal video collections with a safety-first, human-in-the-loop approach. All file operations are reversible, require explicit confirmation, and generate reviewable execution plans before making changes.
## Development Commands
**Important:** This project uses `uv` for Python package management. All commands should be run with `uv run` to ensure they execute in the correct project environment with proper dependencies.
### Installation
```bash
# Install package in editable mode
uv pip install -e .
# Install with dev dependencies (pytest, hypothesis)
uv pip install -e ".[dev]"
# Optional Textual UI for `vlm review-plan --tui`
uv pip install -e ".[tui]"
```
The default CLI path does not require Textual; it is imported only when `uv run vlm review-plan --tui` is used.
### Testing
```bash
# Run all tests (use uv run to execute in the project environment)
uv run pytest
# Run specific test file
uv run pytest tests/test_scanner.py
# Run with verbose output
uv run pytest -v
# Run with quiet output (summary only)
uv run pytest -q
# Count total test cases
uv run pytest --collect-only -q
```
### Running the CLI
```bash
# Verify CLI works (use uv run for proper environment)
uv run vlm --help
# Initialize config
uv run vlm config init
# Common workflow
uv run vlm scan # Discover files
uv run vlm parse # Extract identities (v1 schema)
uv run vlm parse --inventory artifacts/inventory.csv # Embed metadata for quality-aware duplicate handling
uv run vlm enrich # (Optional) Enrich titles/reputation via TMDB
uv run vlm enrich --refresh-all # Force full refresh (ignore cache)
uv run vlm analyze # Detect gaps/duplicates
uv run vlm plan # Generate execution plan
uv run vlm plan --analysis artifacts/analysis.json # Generate plan with duplicate resolution
uv run vlm review-plan # Export CSV + terminal review preview
uv run vlm review-plan --tui # Optional full-screen review UI
uv run vlm apply-review # Sync edited CSV decisions back into the plan
uv run vlm execute # Dry-run (default)
uv run vlm execute --confirm # Actually execute
uv run vlm rollback # Undo executed operations
# Reporting
uv run vlm report summary # Overview statistics
uv run vlm report inventory # File inventory
uv run vlm report completeness # Series with missing episodes
uv run vlm report duplicates # Duplicate files with quality comparison
# Quarantine management
uv run vlm quarantine list # List quarantined files
uv run vlm quarantine add <file> --reason "duplicate"
uv run vlm quarantine restore <file>
# State management
uv run vlm state show <file>
uv run vlm state set <file> --status reviewed
```
## Architecture
### Core Workflow
VLM follows a read-first, multi-stage pipeline:
1. **Scan** → discovers video files, extracts metadata via ffprobe (optional), saves to `artifacts/inventory.csv`
2. **Parse** → extracts titles/years/seasons/episodes from filenames, saves to `artifacts/identities.json`
- Use `--inventory artifacts/inventory.csv` to embed video metadata (v2 schema) for accurate duplicate resolution by quality
- Without `--inventory`, produces v1 schema (lightweight, no embedded metadata)
3. **Enrich** (optional) → adds bilingual titles and reputation (TMDB); updates `artifacts/identities.json` in place; uses SQLite cache for incremental runs
4. **Analyze** → detects episode gaps and duplicates, saves to `artifacts/analysis.json`
5. **Plan** → generates reviewable execution plan (`artifacts/plan.json`) with file operations
6. **Review / Apply Review** → previews high-risk operations in terminal or optional TUI, then syncs edited CSV decisions back into the plan when needed
7. **Execute** → performs file operations (dry-run by default, `--confirm` to execute)
8. **Rollback** → reverses executed operations (best-effort)
### Module Organization
- `cli.py` - Click-based CLI interface, global options, command registration
- `context.py` - CLIContext (config, paths) and pass_context for commands
- `commands/` - Command implementations (scan, parse, enrich, analyze, plan, execute/rollback)
- `scanner.py` - File discovery using system `find` command, metadata extraction via ffprobe
- `parser.py` - Filename parsing using regex patterns (movies: title + year, series: SxxExx)
- `enrichment.py` - Enrichment pipeline; `cache.py` - SQLite cache; `providers/` - TMDB etc.
- `io.py` - Unified JSON/CSV I/O helpers, including validated typed plan loading/saving
- `utils.py` - UTC time, format_size, shared helpers
- `analysis.py` - Completeness checking (episode gaps) and duplicate detection
- `duplicate_resolve.py` - Duplicate group resolution with explicit failures for unresolved strategies/data
- `planner.py` - Execution plan generation with conflict detection and manual-review duplicate fallback
- `executor.py` - File operations (move/rename/quarantine) with rollback logging and library-root safety checks
- `quarantine.py` - Quarantine management with manifest tracking
- `state.py` - File state tracking across workflow stages
- `reports.py` - Report generation (inventory, completeness, duplicates, summary)
- `config.py` - YAML configuration loading and validation
- `models.py` - Dataclass definitions for all data structures
- `logging_config.py` - Logging setup with fallback to console if file logging fails
### Key Data Structures
All defined in `models.py`:
- `VideoFile` - represents discovered video file with metadata
- `MovieIdentity` / `SeriesIdentity` - parsed identity with confidence score
- `ExecutionPlan` - collection of file operations with summary
- `FileOperation` - single operation (move/rename/quarantine/no-op) with conflict detection
- `RollbackLog` - log of executed operations for reversal
- `QuarantineEntry` - quarantined file with original location
- `FileState` - workflow state (reviewed/ignored/planned/executed/quarantined)
### File Categorization
Based on top-level directory within library root, matched against configured category mappings (case-insensitive).
Files in unmapped directories are categorized as "other" and skipped by planner/quarantine operations.
### Parsing Patterns (Hardcoded)
**Movies** (high confidence):
- `{title} ({year})`
- `{title}.{year}`
**Series** (high confidence):
- `S{season:02d}E{episode:02d}`
- `{season}x{episode}`
Patterns are hardcoded in parser.py, not user-configurable.
### Configuration
Default location: `~/.vlm/config.yaml`
Key settings:
- `library_root` - root directory to scan (required)
- `video_extensions` - list of extensions to recognize
- `templates.movie_dir` / `templates.series_dir` - directory structure templates
- `templates.movie_filename` / `templates.series_filename` - filename templates
- `quarantine_dir` - name of quarantine directory (default: `.quarantine`)
- `log_level` - logging verbosity
- `categories` - mapping of category names to directory name lists
- `enrichment` (or `enrich`) - TMDB/api_keys, cache_db, translation, reputation; see README for full schema
- `plan.duplicate_keep` - when using `vlm plan --analysis`: `by_quality`, `by_reputation` (default), `by_reputation_quality_time`, `first_seen`, or `manual`
### Category Mappings
Categories are determined by matching the top-level directory name against configured mappings:
**Default mappings:**
```yaml
categories:
movie: [movie]
series: [series]
anime: [anime]
```
**Custom mappings** support multiple directory names per category:
```yaml
categories:
movie: [movie, movies, films]
series: [series, tv, shows, television]
anime: [anime]
```
This allows files in `/library/movies/` or `/library/films/` to be recognized as the "movie" category. Directory matching is case-insensitive.
**Migration Note:** If you have existing directories with non-standard names (like "movies" or "tv"), update your config.yaml and re-run `vlm scan` to fix categorization. No files will be moved.
### Schema Versioning
**identities.json Schema Versions:**
- **v1** (default without `--inventory`): Lightweight schema without embedded metadata
- Records contain: path, filename, category, title, year/season/episodes, confidence, needs_review
- VideoFile objects reconstructed with defaults: size_bytes=0, resolution=None, codec=None
- Suitable for basic organization workflows
- **v2** (with `--inventory`): Enhanced schema with embedded video metadata
- All v1 fields plus `video_metadata` object containing:
- size_bytes, modified_timestamp, resolution, codec, duration_seconds, bitrate_kbps
- Enables accurate duplicate resolution by quality (compare resolution, codec, file size)
- Required for `by_quality` duplicate resolution strategy
- Backward compatible: v1 files load without errors
**Implementation Details:**
- `_video_file_from_record()` in io.py extracts embedded metadata if present
- Parse command with `--inventory` flag loads inventory.csv and embeds metadata in output
- Schema version stored in `vlm_schema_version` field at root level of identities.json
### File Discovery
Uses system `find` command for speed, falls back to Python recursion if unavailable. Hidden paths (starting with `.`) are skipped automatically.
### Timestamp Handling
All timestamps stored in UTC using ISO 8601 format (`YYYY-MM-DDTHH:MM:SS`). The scanner normalizes naive timestamps to UTC using system timezone.
### Logging
Logs to `~/.vlm/vlm.log` by default. If log directory is unwritable, falls back to console-only logging and continues execution (does not fail).
### Error Handling
- Configuration errors: display helpful message, create default config, continue
- Missing ffprobe: skip metadata extraction, log debug message, continue
- File access errors: log error, skip file, continue scanning
- Invalid YAML: display error, use default config
## Testing Guidelines
- Framework: pytest with hypothesis for property-based tests
- Test file naming: `test_*.py`
- Test function naming: `test_*`
- Test class naming: `Test*`
- Use `CliRunner` for CLI integration tests
- Prefer narrow unit tests for module logic plus targeted CLI integration tests
- Add tests with each behavior change, including error paths and edge cases
## Important Implementation Notes
### Safety Protocol
- NEVER permanently delete files - use quarantine instead
- All file operations create rollback logs when executed with --confirm
- Execution plans detect destination conflicts and mark operations
- Default mode is dry-run; --confirm required for actual execution
- Quarantine is reversible via restore command
### Anime Handling
Anime files are discovered and categorized but NOT parsed in v1 (deferred for future implementation).
### State Management
State tracking is optional and allows marking files as reviewed/ignored/planned/executed/quarantined. State is persisted to `~/.vlm/state.json`.
### Quarantine Constraints
Only movie and series files can be quarantined in v1 (anime and other categories rejected).
### Template Variables
Available for path/filename templates:
- Movies: `{title}`, `{year}`, `{ext}`
- Series: `{title}`, `{season}`, `{episode}`, `{ext}`
Format specifiers like `{season:02d}` are supported.
## Code Style
- Python 3.10+ idioms
- 4-space indentation
- PEP 8 naming: `snake_case` for functions/variables, `PascalCase` for classes, `UPPER_SNAKE_CASE` for constants
- Type hints for public functions and non-trivial internal APIs
- Modules focused on single responsibility
- No formatter/linter enforced - maintain consistency with existing files
## Commit Guidelines
- Clear, imperative commit subjects (e.g., "fix logging fallback for unwritable log dir")
- Keep commits focused - avoid mixing refactors and behavior changes
- Include co-author tag: `Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>`
+3 -18
View File
@@ -814,23 +814,6 @@ Always keep backups of important files!
- **Local files only**: Designed for local or mounted network storage - **Local files only**: Designed for local or mounted network storage
- **Best-effort rollback**: Rollback may not succeed if files have been modified - **Best-effort rollback**: Rollback may not succeed if files have been modified
## Agent skills
This project includes workflow guidance under `skills/vlm-library-workflow/` (scan → parse → enrich → analyze → plan → review-plan → apply-review → execute).
### Activation (Gemini CLI)
```bash
activate_skill vlm-library-workflow
```
### Features
- **Context-Aware Guidance**: The Agent understands the VLM safety-first workflow and configuration.
- **Risk Warnings**: Automatic alerts if a plan contains high-risk operations (>20% quarantine).
- **Command Recipes**: Instant access to complex command sequences and troubleshooting steps.
- **Developer Verification**: Built-in test execution recipes for verifying core logic changes.
## Development ## Development
### Running Tests ### Running Tests
@@ -860,7 +843,8 @@ uv run pytest --cov=vlm tests/
``` ```
src/vlm/ src/vlm/
├── cli.py # Click-based CLI interface, global options ├── cli.py # Thin Click registrar; global options and command registration
├── cli_helpers.py # Shared CLI helpers (context init, command runner, report emission)
├── context.py # CLIContext and pass_context for commands ├── context.py # CLIContext and pass_context for commands
├── cli_helpers.py # Shared CLI helpers ├── cli_helpers.py # Shared CLI helpers
├── commands/ # Command implementations ├── commands/ # Command implementations
@@ -894,6 +878,7 @@ src/vlm/
├── review_tui.py # Optional Textual review UI ├── review_tui.py # Optional Textual review UI
├── transaction.py # Execution transaction log ├── transaction.py # Execution transaction log
├── executor.py # File operations and rollback ├── executor.py # File operations and rollback
├── transaction.py # Transaction/rollback logging for executed operations
├── quarantine.py # Quarantine management ├── quarantine.py # Quarantine management
├── state.py # File state tracking ├── state.py # File state tracking
├── reports.py # Report generation ├── reports.py # Report generation
+1 -1
View File
@@ -8,6 +8,6 @@ They are **not** maintained as current project documentation.
- `/README.md` — user guide and workflow - `/README.md` — user guide and workflow
- `/CHANGELOG.md` — release and refactor history - `/CHANGELOG.md` — release and refactor history
- `/CLAUDE.md` / `/AGENTS.md` — agent/developer guidance - `/CLAUDE.md` / `/AGENTS.md` — agent/developer guidance
- `/plans/2026-05-21-functional-code-simplification-plan-v1.md` — code simplification plan - `2026-05-21-functional-code-simplification-plan-v1.md` (in this archive) — completed code simplification plan
Archived on 2026-05-21 as part of the functional code trim (Phase 2). Archived on 2026-05-21 as part of the functional code trim (Phase 2).
+43 -70
View File
@@ -1,96 +1,69 @@
# Documentation Status
- Synced with artifacts/ baseline on 2026-06-01.
--- ---
name: vlm-library-workflow name: vlm-library-workflow
description: Operate and extend the Video Library Manager (`vlm`) with a safety-first, human-in-the-loop workflow across scan, parse, enrich, analyze, plan, review-plan, apply-review, execute, rollback, and developer verification. Use when requests involve organizing a video library, producing or reviewing `artifacts/inventory.csv`, `artifacts/identities.json`, `artifacts/analysis.json`, `artifacts/plan.json`, tuning VLM config templates, resolving duplicates or episode gaps, running dry-run/confirm execution, recovering changes via rollback, explaining VLM CLI usage, or developing/modifying VLM features (parser, providers, planner, executor, commands, tests). description: Operate and extend the Video Library Manager (vlm) CLI. Use for organizing video libraries (scan/parse/enrich/analyze/plan/execute), reviewing plans, resolving duplicates, or developing VLM features.
--- ---
# VLM Library Workflow # VLM Library Workflow
## Overview ## Safety Contract
Run the repository's built-in media organization pipeline with consistent safety checks and explicit output verification. Prefer incremental, reviewable steps and never skip dry-run and plan inspection before destructive operations. 1. Before `execute --confirm`, always run `review-plan` and report `High-risk operations` count.
2. If high-risk > 0: **stop**, wait for explicit user decision.
3. `--confirm` requires `--require-review` to enforce review gate; default does not force.
4. Never run `execute --confirm` without user confirmation.
**Do not commit** generated CSV/JSON under `artifacts/` or at the repository root. ## Execution Threshold Semantics
## Workflow Order - `high_risk_operations`: count of non-no-op operations with risk flags (printed by `review-plan`).
- `review_export_rows`: internal count (not printed); may be larger. Do not use `wc -l` on CSV.
- If export row count is needed, parse CSV data rows or add the count to CLI output.
Run commands in this default sequence unless the user asks for a specific stage: ## High-Risk Closed Loop
1. `vlm config init` then set `library_root` in `~/.vlm/config.yaml` When high-risk > 0, use the same plan and CSV path throughout:
2. `vlm scan` → `artifacts/inventory.csv`
3. `vlm parse --inventory artifacts/inventory.csv` → `artifacts/identities.json` (v2 schema)
4. `vlm enrich` when bilingual titles and reputation signals are needed
5. `vlm analyze --inventory artifacts/inventory.csv` → `artifacts/analysis.json`
6. `vlm plan --analysis artifacts/analysis.json` → `artifacts/plan.json`
7. `vlm review-plan` → `artifacts/plan_manual_review.csv` (use `--tui` only when Textual is installed)
8. Edit CSV if needed, then `vlm apply-review` to sync decisions into `plan.json`
9. `vlm execute` (dry-run) and inspect summary
10. `vlm execute --confirm` only after explicit user confirmation
11. `vlm rollback` if the user requests revert
If `vlm` is not on PATH, prefix commands with `uv run`. 1. `vlm review-plan --input <plan> --output <csv>`
2. Human reviews CSV (modify decision or keep)
3. `vlm apply-review --plan <plan> --csv <csv>` — **required even if no changes** (writes `review_applied_at`)
4. `vlm execute --plan <plan>` (dry-run with updated plan)
5. Get explicit user confirmation
6. `vlm execute --plan <plan> --confirm --require-review --review-csv <csv>`
## Preflight Checks Three rejection paths: missing CSV, missing `review_applied_at`, CSV path mismatch.
Run these checks before executing workflow commands: ## Workflow Phases
1. Confirm current working directory is repository root. | Phase | Purpose | Artifact |
2. Probe command availability: `vlm --help` or `uv run vlm --help`. |-------|---------|----------|
3. Run the target subcommand `--help` when options are uncertain. | config | Set `library_root` | `~/.vlm/config.yaml` |
4. Confirm config validity with `vlm config validate` after config edits. | scan | Discover files | `inventory.csv` |
5. Verify required input artifacts exist under `artifacts/` (or paths passed via flags). | parse | Extract identities | `identities.json` |
6. Treat `execute --confirm` as destructive and require explicit user confirmation. | enrich | Add TMDB metadata | `identities.json` (updated) |
| analyze | Detect gaps/duplicates | `analysis.json` |
| plan | Generate operations | `plan.json` |
| review-plan | Export high-risk + preview | `plan_manual_review.csv` |
| apply-review | Sync manual decisions | `plan.json` (updated) |
| execute | Dry-run, then confirm | rollback log |
| rollback | Revert if needed | restored files |
## Execution Rules ## Decision Rules
Follow these rules while executing tasks: **duplicate_keep strategies** (5): `by_quality`, `by_reputation`, `by_reputation_quality_time`, `first_seen`, `manual`.
1. Prefer read-only stages first: scan, parse, enrich, analyze, plan. **Parser boundary risks**: filenames with resolution-like `1920x1080`/`1440x1080` and `Sample` clips are high-risk; require review-plan output before confirmation.
2. Treat `artifacts/plan.json` as the reviewable contract; summarize counts and conflicts before execution.
3. Run `review-plan` and `apply-review` before `execute --confirm` when manual decisions are required.
4. Run dry-run (`vlm execute`) before `vlm execute --confirm`.
5. If execution is interrupted or results are incorrect, locate rollback logs and run `vlm rollback`.
6. Keep outputs explicit in responses: file path, record counts, and next command.
7. If the user asks for a partial workflow, run only required stages and state skipped dependencies.
8. Before `execute --confirm`, run `vlm review-plan` and report high-risk counts; pause if the user has not reviewed high-risk operations.
## Decision Points **Metadata quality**: prefer `vlm parse --inventory` when duplicate quality ranking matters.
Use these decision policies: **Analysis-assisted planning**: prefer `vlm plan --analysis` for automatic duplicate quarantine.
1. Duplicate handling: choose `plan.duplicate_keep` (`by_quality`, `by_reputation`, `by_reputation_quality_time`, `first_seen`, `manual`) per user preference.
2. Enrichment: skip `vlm enrich` only when translations/reputation are not needed or API keys are unavailable.
3. Metadata quality: prefer `vlm parse --inventory artifacts/inventory.csv` when duplicate quality ranking matters.
4. Analysis-assisted planning: prefer `vlm plan --analysis artifacts/analysis.json` for automatic duplicate quarantine decisions.
5. Parser boundary risk: treat resolution-like tokens (`1920x1080`) and `Sample` clips as high-risk; require review-plan output before confirmation.
## Output Contract ## Output Contract
Return concise, operational summaries: 1. Commands executed and artifacts generated.
2. Key counts (files, identities, duplicates, operations).
1. Commands executed. 3. Risk counts from review-plan; blocking errors with exact remediation command.
2. Artifacts generated or updated (under `artifacts/` by default).
3. Key counts (files scanned, identities parsed, duplicate groups, plan operations).
4. Risk counts from `review-plan` (manual_review / sample_source / high_season / high_episode / conflicts).
5. Blocking errors and exact remediation command.
6. Safe next step.
## Developer Verification
When modifying VLM core logic:
1. **Full suite**: `uv run pytest -q`
2. **Core components**: `uv run pytest tests/test_scanner.py tests/test_planner.py tests/test_executor.py`
3. **Property tests**: `uv run pytest tests/test_analysis_properties.py`
4. **Lint** (with dev extras): `uv run ruff check src tests`
## References ## References
Load these references on demand: - `references/triage.md`: failure triage mapping + preflight checks.
- `references/dev-map.md`: module → test → verification mapping.
1. `references/command-recipes.md` — command syntax, artifact expectations, failure triage. - For CLI options: run `vlm <command> --help`. Do not trust memory.
2. `references/workflow.md` — end-to-end operator workflow.
3. `references/cli-reference.md` — command and config quick reference.
4. `references/dev-guide.md` — architecture and developer modification patterns.
@@ -1,31 +0,0 @@
# Documentation Status
- Updated to current CLI options on 2026-06-01. Default artifact paths are under `artifacts/`.
# VLM CLI Reference
## Configuration
- `vlm config init`: Create default config.
- `vlm config show`: Display current settings.
- `vlm config validate`: Check config for errors.
## Core Commands
- `vlm scan [--output PATH]`: Discover video files.
- `vlm parse [--input CSV] [--output JSON] [--inventory CSV]`: Parse filenames.
- `vlm enrich [--input JSON] [--output JSON] [--refresh-all]`: Fetch TMDB metadata.
- `vlm analyze [--input JSON] [--output JSON]`: Find gaps and duplicates.
- `vlm plan [--input JSON] [--analysis JSON] [--output JSON]`: Generate operations.
- `vlm review-plan [--input JSON] [--output CSV] [--tui]`: Export manual review CSV (optional Textual UI).
- `vlm apply-review [--plan JSON] [--csv PATH]`: Sync edited review CSV into plan.
- `vlm execute [--plan JSON] [--confirm] [--yes] [--verbose-ops] [--safe-mode] [--preserve-directories]`: Move/Rename/Quarantine operations with safety guards.
- `vlm rollback [--log PATH]`: Undo operations.
## Management & Reporting
- `vlm quarantine list|add|restore`: Manage the `.quarantine` directory.
- `vlm report inventory|completeness|duplicates|summary`: Generate human-readable reports.
- `vlm state show|set|query|clear`: Track manual review status for files.
## Important Config Options (`~/.vlm/config.yaml`)
- `library_root`: Path to the video collection.
- `templates`: Naming patterns for movies and series.
- `plan.duplicate_keep`: Strategy for duplicates (`by_quality`, `by_reputation`, `by_reputation_quality_time`, `first_seen`, `manual`).
- `enrichment.api_keys`: TMDB and OpenAI keys.
@@ -1,67 +0,0 @@
# Documentation Status
- Synced with artifacts/ baseline on 2026-06-01.
# VLM Command Recipes
## Baseline
Run from repository root unless the user specifies otherwise.
```bash
vlm --help
vlm config show
vlm config validate
```
If `vlm` is not on PATH:
```bash
uv run vlm --help
uv run vlm config show
uv run vlm config validate
```
## End-to-End Pipeline
Default artifact paths match CLI defaults under `artifacts/`:
```bash
vlm scan
vlm parse --inventory artifacts/inventory.csv
vlm enrich
vlm analyze --inventory artifacts/inventory.csv
vlm plan --analysis artifacts/analysis.json
vlm review-plan --input artifacts/plan.json --output artifacts/plan_manual_review.csv
vlm apply-review
vlm execute
vlm execute --confirm
```
## Artifact Expectations
1. `artifacts/inventory.csv` — discovered files with filesystem and optional ffprobe metadata.
2. `artifacts/identities.json` — parsed identities (v2 when parse used `--inventory`).
3. `artifacts/analysis.json` — completeness gaps and duplicate groups.
4. `artifacts/plan.json` — planned operations (`move`, `rename`, `quarantine`, `no-op`).
5. `artifacts/plan_manual_review.csv` — high-risk rows for human review before `--confirm`.
Do not commit these files to Git.
## Focused Workflows
```bash
vlm parse --inventory artifacts/inventory.csv
vlm analyze --input artifacts/identities.json --output artifacts/analysis.json --inventory artifacts/inventory.csv
vlm plan --input artifacts/identities.json --analysis artifacts/analysis.json --output artifacts/plan.json
vlm rollback
```
## Frequent Failure Triage
1. Missing config: `vlm config init`, set `library_root`.
2. Invalid config: `vlm config validate`.
3. Missing input artifact: run prerequisite stage (scan → parse → analyze → plan).
4. Unexpected duplicate decisions: check `plan.duplicate_keep`, rerun `vlm plan --analysis artifacts/analysis.json`.
5. Unsafe execution: `vlm rollback`, re-run `review-plan` / `apply-review`.
6. `vlm` not found: use `uv run vlm ...`.
7. TMDB rate limits: lower `enrichment.max_concurrency` in config and retry later.
@@ -1,33 +0,0 @@
# Documentation Status
- Updated for the modular command architecture on 2026-02-16.
# VLM Developer Guide
## Project Structure
- `src/vlm/cli.py`: Entry point and thin command wrappers.
- `src/vlm/parser.py`: Regex-based filename parsing logic.
- `src/vlm/enrichment.py`: Pipeline for external metadata fetching.
- `src/vlm/providers/`: API implementations (e.g., TMDB).
- `src/vlm/planner.py`: Logic for mapping identities to filesystem operations.
- `src/vlm/executor.py`: Safe file manipulation and rollback logging.
## Adding a New Command
1. Create a new module in `src/vlm/commands/`.
2. Define the command using `@click.command()`.
3. Register it in `src/vlm/cli.py` with a Click-decorated function that delegates to the module implementation.
## Modifying the Parser
- The parser uses a sequence of regex patterns in `src/vlm/parser.py`.
- Add new patterns to the `PATTERNS` list or improve existing ones.
- Always run `pytest tests/test_parser.py` after changes.
## Data Models
See `src/vlm/models.py` for core data structures:
- `VideoFile`: Basic file metadata.
- `MovieIdentity` / `SeriesIdentity`: Parsed/enriched identity records.
- `FileOperation`: Definition of a move/rename/quarantine/no-op/preserve-directory operation.
## Testing
- **Unit Tests**: `pytest`
- **Property-based Tests**: `pytest tests/test_analysis_properties.py` (uses Hypothesis).
- **Integration Tests**: `pytest tests/test_reports_integration.py`.
@@ -0,0 +1,25 @@
# Developer Map
## Module → Test → Verification
| Module | Tests | When to run |
|--------|-------|-------------|
| `scanner.py` | `tests/test_scanner.py` | File discovery, metadata extraction |
| `parser.py` | `tests/test_parser.py` | Filename parsing changes |
| `enrichment.py`, `providers/` | `tests/test_enrichment.py` | TMDB/API changes |
| `planner.py` | `tests/test_planner.py` | Operation generation, duplicate resolution |
| `executor.py` | `tests/test_executor.py` | File operations, rollback |
| `plan_review.py` | `tests/test_plan_review.py` | Review gate, CSV handling |
| `commands/*.py` | `tests/test_cli_*.py` | CLI flag/behavior changes |
| `analysis.py` | `tests/test_analysis_properties.py` | Property-based tests (Hypothesis) |
| Full integration | `pytest` | Before any commit |
## Adding a New Command
1. Create module in `src/vlm/commands/`.
2. Define with `@click.command()`.
3. Import and register in `src/vlm/cli.py` via `main.add_command(...)`.
## Key Models
See `src/vlm/models.py`: `VideoFile`, `MovieIdentity`, `SeriesIdentity`, `FileOperation`, `ExecutionPlan`.
@@ -0,0 +1,23 @@
# Triage & Preflight
## Preflight Checks
Before executing workflow commands:
1. Confirm working directory is repository root.
2. Run `vlm --help`; if unavailable, switch to `uv run vlm`.
3. Run `<subcommand> --help` when options are uncertain.
4. Run `vlm config validate` after config edits.
5. Verify required input files exist before downstream stages.
6. Treat `execute --confirm` as destructive; require explicit user confirmation.
## Failure Triage
| Problem | Command |
|---------|---------|
| Missing config | `vlm config init`, then set `library_root` |
| Invalid config values | `vlm config validate` and fix reported keys |
| Missing input artifact | Run prerequisite stage (scan → parse → analyze → plan) |
| Unexpected duplicate decisions | Check `plan.duplicate_keep` in config, rerun `vlm plan --analysis` |
| Unsafe execution results | `vlm rollback`, inspect plan, re-run `execute --confirm` |
| `vlm` not found | Use `uv run vlm ...` fallback |
@@ -1,74 +0,0 @@
# Documentation Status
- Synced with artifacts/ baseline on 2026-06-01.
# VLM Workflow Guide
This guide details the standard end-to-end process for organizing a video library using VLM.
## 1. Setup and Discovery
### Initialize Configuration
```bash
vlm config init
```
Edit `~/.vlm/config.yaml` to set `library_root` and optionally `workspace_dir: artifacts`.
### Library Scan
```bash
vlm scan
```
- **Goal**: Create `artifacts/inventory.csv`.
- **Note**: Ensure `ffprobe` is installed for resolution and codec metadata.
## 2. Identification
### Filename Parsing
```bash
vlm parse --inventory artifacts/inventory.csv
```
- **Goal**: Create `artifacts/identities.json`.
- **Why --inventory?**: Embeds video metadata (v2 schema) required for quality-based duplicate resolution.
### Metadata Enrichment
```bash
vlm enrich
```
- **Goal**: Update `artifacts/identities.json` with TMDB data.
- **Troubleshooting**: Check `enrichment.api_keys` in config; never commit `~/.vlm/config.yaml`.
## 3. Analysis and Planning
### Detect Issues
```bash
vlm analyze
```
- **Goal**: Create `artifacts/analysis.json`.
- **Outputs**: Episode gaps and duplicate groups.
### Create Execution Plan
```bash
vlm plan --analysis artifacts/analysis.json
```
- **Goal**: Create `artifacts/plan.json`.
- **Strategy**: Uses `plan.duplicate_keep` (default: `by_reputation`) for duplicate decisions.
## 4. Review and Execution
### Human review
```bash
vlm review-plan
# Optional: vlm review-plan --tui (requires textual extra)
vlm apply-review # after editing artifacts/plan_manual_review.csv
```
### Execute Changes
```bash
vlm execute # Dry-run
vlm execute --confirm # Actual operations (explicit confirmation)
```
### Reverting Changes
```bash
vlm rollback
```
Restores files using the latest log in `~/.vlm/rollback/`.
+62
View File
@@ -2,7 +2,9 @@
from __future__ import annotations from __future__ import annotations
import json
import sys import sys
from collections.abc import Callable
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
@@ -123,3 +125,63 @@ def initialize_cli_context(config: Path, log_level: Optional[str]) -> CLIContext
logger = setup_logging(log_level=cfg.log_level) logger = setup_logging(log_level=cfg.log_level)
return CLIContext(config=cfg, logger=logger) return CLIContext(config=cfg, logger=logger)
def emit_report(content: str, output: Optional[Path]) -> None:
"""Write report content to a file or stdout."""
if output:
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(content, encoding="utf-8")
click.echo(f"Report saved to: {output}")
else:
click.echo()
click.echo(content)
def optional_plan_summary(plan: Optional[Path]):
"""Load optional plan JSON and return a display summary string."""
if plan is None:
return None
if not plan.exists():
click.echo(f"Error: Plan file not found: {plan}", err=True)
raise SystemExit(1)
from vlm.plan_render import preferred_plan_summary
from vlm.planner import load_plan
return preferred_plan_summary(load_plan(plan))
def run_command(
ctx: CLIContext,
fn: Callable[[], None],
*,
stage: str,
json_errors: bool = False,
) -> None:
"""Run a command body with consistent CLI error handling."""
try:
fn()
except FileNotFoundError as e:
command_error(ctx, f"Error: Input file not found: {e}", f"{stage} file not found: {e}")
except json.JSONDecodeError as e:
if json_errors:
command_error(
ctx,
f"Error: Failed to parse JSON file: {e}",
f"{stage} JSON parsing failed: {e}",
exc_info=True,
)
raise
except ValueError as e:
command_error(ctx, f"Error: {e}", f"{stage} validation failed: {e}")
except OSError as e:
command_error(
ctx,
f"Error during {stage}: {e}",
f"{stage} I/O failed: {e}",
exc_info=True,
)
except Exception as e:
click.echo(f"Error during {stage}: {e}", err=True)
ctx.logger.error(f"{stage} failed: {e}", exc_info=True)
raise SystemExit(1) from e
+22 -1
View File
@@ -6,7 +6,8 @@ from typing import Optional
import click import click
from vlm.analysis import analyze_series_completeness, detect_duplicates from vlm.analysis import analyze_series_completeness, detect_duplicates
from vlm.context import CLIContext from vlm.cli_helpers import default_artifact_path, resolve_legacy_default_input_path, run_command
from vlm.context import CLIContext, pass_context
from vlm.io import ( from vlm.io import (
identities_to_analysis_input, identities_to_analysis_input,
load_identities_json, load_identities_json,
@@ -120,3 +121,23 @@ def analyze_cmd(
f"Analysis completed: {len(completeness_results)} incomplete series, " f"Analysis completed: {len(completeness_results)} incomplete series, "
f"{len(duplicate_groups)} duplicate groups, saved to {output}" f"{len(duplicate_groups)} duplicate groups, saved to {output}"
) )
@click.command()
@click.option("--input", type=click.Path(path_type=Path), default=lambda: default_artifact_path("identities.json"))
@click.option("--output", type=click.Path(path_type=Path), default=lambda: default_artifact_path("analysis.json"))
@click.option("--inventory", type=click.Path(exists=True, path_type=Path), default=None)
@pass_context
def analyze(ctx: CLIContext, input: Path, output: Path, inventory: Optional[Path]):
"""Analyze completeness and duplicates."""
run_command(
ctx,
lambda: analyze_cmd(
ctx,
resolve_legacy_default_input_path(input, "input", "identities.json", "--input"),
output,
inventory,
),
stage="analyze",
json_errors=True,
)
+29 -1
View File
@@ -8,7 +8,7 @@ import click
from vlm.cli_helpers import command_error from vlm.cli_helpers import command_error
from vlm.config import create_default_config, validate_config from vlm.config import create_default_config, validate_config
from vlm.context import CLIContext from vlm.context import CLIContext, pass_context
def config_init_cmd(ctx: CLIContext, path: Path) -> None: def config_init_cmd(ctx: CLIContext, path: Path) -> None:
@@ -60,3 +60,31 @@ def config_validate_cmd(ctx: CLIContext) -> None:
for error in errors: for error in errors:
click.echo(f" - {error}", err=True) click.echo(f" - {error}", err=True)
command_error(ctx, "Configuration validation failed.", "Configuration validation failed") command_error(ctx, "Configuration validation failed.", "Configuration validation failed")
@click.group(name="config")
@pass_context
def config_group(ctx: CLIContext):
"""Manage VLM configuration."""
@config_group.command("init")
@click.option("--path", type=click.Path(path_type=Path), default=default_config_path)
@pass_context
def config_init(ctx: CLIContext, path: Path):
"""Create a default config file."""
config_init_cmd(ctx, path)
@config_group.command("show")
@pass_context
def config_show(ctx: CLIContext):
"""Show current configuration."""
config_show_cmd(ctx)
@config_group.command("validate")
@pass_context
def config_validate(ctx: CLIContext):
"""Validate configuration."""
config_validate_cmd(ctx)
+36 -1
View File
@@ -8,7 +8,8 @@ from typing import Optional
import click import click
from vlm.context import CLIContext from vlm.cli_helpers import default_artifact_path, resolve_legacy_default_input_path, run_command
from vlm.context import CLIContext, pass_context
from vlm.enrichment import enrich_identities_data from vlm.enrichment import enrich_identities_data
from vlm.io import load_json_file, save_json_file from vlm.io import load_json_file, save_json_file
@@ -144,3 +145,37 @@ def enrich_cmd(
stats["skipped"], stats["skipped"],
refresh_mode, refresh_mode,
) )
@click.command()
@click.option("--input", type=click.Path(path_type=Path), default=lambda: default_artifact_path("identities.json"))
@click.option("--output", type=click.Path(path_type=Path), default=None)
@click.option("--refresh-changed-only", is_flag=True, default=False)
@click.option("--refresh-all", is_flag=True, default=False)
@click.option("--timeout", type=int, default=6, show_default=True)
@click.option("--retries", type=int, default=2, show_default=True)
@pass_context
def enrich(
ctx: CLIContext,
input: Path,
output: Optional[Path],
refresh_changed_only: bool,
refresh_all: bool,
timeout: int,
retries: int,
):
"""Enrich identities with translation and reputation metadata."""
run_command(
ctx,
lambda: enrich_cmd(
ctx,
resolve_legacy_default_input_path(input, "input", "identities.json", "--input"),
output,
refresh_changed_only,
refresh_all,
timeout,
retries,
),
stage="enrich",
json_errors=True,
)
+49 -1
View File
@@ -7,7 +7,8 @@ from typing import Optional
import click import click
from vlm.context import CLIContext from vlm.cli_helpers import default_artifact_path, resolve_legacy_default_input_path, run_command
from vlm.context import CLIContext, pass_context
from vlm.executor import ExecutionEngine from vlm.executor import ExecutionEngine
from vlm.plan_render import preferred_plan_summary from vlm.plan_render import preferred_plan_summary
from vlm.planner import load_plan from vlm.planner import load_plan
@@ -258,3 +259,50 @@ def rollback_cmd(ctx: CLIContext, log: Optional[Path]) -> None:
rollback_summary["successful"], rollback_summary["successful"],
rollback_summary["failed"], rollback_summary["failed"],
) )
@click.command()
@click.option("--plan", type=click.Path(path_type=Path), default=lambda: default_artifact_path("plan.json"))
@click.option("--confirm", is_flag=True, default=False)
@click.option("--yes", is_flag=True, default=False)
@click.option("--verbose-ops", is_flag=True, default=False)
@click.option("--preserve-directories", is_flag=True, default=False)
@click.option("--safe-mode", is_flag=True, default=False)
@click.option("--require-review", is_flag=True, default=False)
@click.option("--review-csv", type=click.Path(path_type=Path), default=None)
@pass_context
def execute(
ctx: CLIContext,
plan: Path,
confirm: bool,
yes: bool,
verbose_ops: bool,
preserve_directories: bool,
safe_mode: bool,
require_review: bool,
review_csv: Optional[Path],
):
"""Execute plan (dry-run by default; use --confirm to apply)."""
run_command(
ctx,
lambda: execute_cmd(
ctx,
resolve_legacy_default_input_path(plan, "plan", "plan.json", "--plan"),
confirm,
yes,
verbose_ops,
preserve_directories,
safe_mode,
require_review=require_review,
review_csv=review_csv,
),
stage="execute",
)
@click.command()
@click.option("--log", type=click.Path(exists=True, path_type=Path), default=None)
@pass_context
def rollback(ctx: CLIContext, log: Optional[Path]):
"""Rollback previous execution (best-effort)."""
run_command(ctx, lambda: rollback_cmd(ctx, log), stage="rollback")
+32 -1
View File
@@ -8,7 +8,7 @@ from typing import Optional
import click import click
from vlm.cli_helpers import command_error from vlm.cli_helpers import command_error
from vlm.context import CLIContext from vlm.context import CLIContext, pass_context
from vlm.quarantine import QuarantineManager from vlm.quarantine import QuarantineManager
from vlm.utils import format_size from vlm.utils import format_size
@@ -147,3 +147,34 @@ def quarantine_restore_cmd(ctx: CLIContext, file: Path) -> None:
f"Failed to restore file: {e}", f"Failed to restore file: {e}",
exc_info=True, exc_info=True,
) )
@click.group()
@pass_context
def quarantine(ctx: CLIContext):
"""Manage quarantined files (movie and series in v1)."""
@quarantine.command("list")
@click.option("--category", type=click.STRING, default=None, help="Category filter (validated against configured categories)")
@pass_context
def quarantine_list(ctx: CLIContext, category: Optional[str]):
"""List quarantined files."""
quarantine_list_cmd(ctx, category)
@quarantine.command("add")
@click.argument("file", type=click.Path(exists=True, path_type=Path))
@click.option("--reason", type=str, default=None)
@pass_context
def quarantine_add(ctx: CLIContext, file: Path, reason: Optional[str]):
"""Quarantine a file."""
quarantine_add_cmd(ctx, file, reason)
@quarantine.command("restore")
@click.argument("file", type=click.Path(exists=True, path_type=Path))
@pass_context
def quarantine_restore(ctx: CLIContext, file: Path):
"""Restore a file from quarantine."""
quarantine_restore_cmd(ctx, file)
+125 -275
View File
@@ -1,22 +1,25 @@
"""Report CLI command implementations.""" """Report CLI commands."""
from __future__ import annotations from __future__ import annotations
import json
import sys
from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
import click import click
from vlm.cli_helpers import command_error, resolve_legacy_default_input_path from vlm.cli_helpers import (
from vlm.context import CLIContext command_error,
default_artifact_path,
emit_report,
optional_plan_summary,
resolve_legacy_default_input_path,
run_command,
)
from vlm.context import CLIContext, pass_context
from vlm.io import load_analysis_json, load_inventory_csv from vlm.io import load_analysis_json, load_inventory_csv
from vlm.models import DuplicateGroup, MovieIdentity, SeasonCompleteness, SeriesIdentity, VideoFile
from vlm.plan_render import preferred_plan_summary
from vlm.planner import load_plan
from vlm.reports import ( from vlm.reports import (
completeness_from_analysis,
duplicate_groups_from_analysis,
generate_completeness_report, generate_completeness_report,
generate_duplicate_report, generate_duplicate_report,
generate_inventory_report, generate_inventory_report,
@@ -24,290 +27,137 @@ from vlm.reports import (
) )
def report_inventory_cmd( def _run_inventory(ctx: CLIContext, format: str, input: Path, output: Optional[Path]) -> None:
ctx: CLIContext, input = resolve_legacy_default_input_path(input, "input", "inventory.csv", "--input")
format: str, click.echo(f"Loading inventory from: {input}")
input: Path, video_files = load_inventory_csv(input)
output: Optional[Path], click.echo(f"Loaded {len(video_files)} files\n")
report_format = "csv" if format == "text" else format
content = generate_inventory_report(video_files, report_format, ctx.config.library_root)
emit_report(content, output)
ctx.logger.info("inventory report: %s files, format=%s", len(video_files), format)
def _run_completeness(
ctx: CLIContext, format: str, input: Path, output: Optional[Path], plan: Optional[Path]
) -> None: ) -> None:
"""Generate inventory report.""" input = resolve_legacy_default_input_path(input, "input", "analysis.json", "--input")
config = ctx.config plan_summary = optional_plan_summary(plan)
logger = ctx.logger click.echo(f"Loading analysis from: {input}")
analysis_data = load_analysis_json(input)
seasons = completeness_from_analysis(analysis_data)
click.echo(f"Loaded {len(seasons)} series with gaps\n")
content = generate_completeness_report(
seasons, format, ctx.config.library_root, plan_summary=plan_summary
)
emit_report(content, output)
ctx.logger.info("completeness report: %s series", len(seasons))
try:
input = resolve_legacy_default_input_path(input, "input", "inventory.csv", "--input")
click.echo(f"Loading inventory from: {input}")
video_files = load_inventory_csv(input) def _run_duplicates(
ctx: CLIContext, format: str, input: Path, output: Optional[Path], plan: Optional[Path]
) -> None:
input = resolve_legacy_default_input_path(input, "input", "analysis.json", "--input")
plan_summary = optional_plan_summary(plan)
click.echo(f"Loading analysis from: {input}")
analysis_data = load_analysis_json(input)
groups = duplicate_groups_from_analysis(analysis_data)
click.echo(f"Loaded {len(groups)} duplicate groups\n")
content = generate_duplicate_report(
groups, format, ctx.config.library_root, plan_summary=plan_summary
)
emit_report(content, output)
ctx.logger.info("duplicate report: %s groups", len(groups))
click.echo(f"Loaded {len(video_files)} files")
click.echo()
click.echo(f"Generating inventory report in {format} format...") def _run_summary(ctx: CLIContext, input: Path, output: Optional[Path]) -> None:
input = resolve_legacy_default_input_path(input, "input", "inventory.csv", "--input")
click.echo(f"Loading inventory from: {input}")
video_files = load_inventory_csv(input)
click.echo(f"Loaded {len(video_files)} files\n")
content = generate_summary_report(video_files, ctx.config.library_root)
emit_report(content, output)
ctx.logger.info("summary report: %s files", len(video_files))
report_format = "csv" if format == "text" else format
report_content = generate_inventory_report(video_files, report_format, config.library_root)
if output: @click.group()
output.parent.mkdir(parents=True, exist_ok=True) @pass_context
with open(output, "w", encoding="utf-8") as f: def report(ctx: CLIContext):
f.write(report_content) """Generate inventory, completeness, duplicate, and summary reports."""
click.echo(f"Report saved to: {output}")
else:
click.echo()
click.echo(report_content)
logger.info("Generated inventory report in %s format with %s files", format, len(video_files))
except FileNotFoundError: @report.command("inventory")
command_error(ctx, f"Error: Input file not found: {input}", f"Input file not found: {input}") @click.option("--format", type=click.Choice(["csv", "json", "text"], case_sensitive=False), default="text")
@click.option("--input", type=click.Path(path_type=Path), default=lambda: default_artifact_path("inventory.csv"))
@click.option("--output", type=click.Path(path_type=Path), default=None)
@pass_context
def report_inventory(ctx: CLIContext, format: str, input: Path, output: Optional[Path]):
"""List discovered files with metadata."""
run_command(ctx, lambda: _run_inventory(ctx, format, input, output), stage="inventory report")
except Exception as e:
command_error( @report.command("completeness")
ctx, @click.option("--format", type=click.Choice(["text", "json"], case_sensitive=False), default="text")
f"Error generating inventory report: {e}", @click.option("--input", type=click.Path(path_type=Path), default=lambda: default_artifact_path("analysis.json"))
f"Inventory report generation failed: {e}", @click.option("--output", type=click.Path(path_type=Path), default=None)
exc_info=True, @click.option("--plan", type=click.Path(path_type=Path), default=None)
) @pass_context
def report_completeness(ctx: CLIContext, format: str, input: Path, output: Optional[Path], plan: Optional[Path]):
"""Show series with episode gaps."""
run_command(
ctx, lambda: _run_completeness(ctx, format, input, output, plan),
stage="completeness report", json_errors=True,
)
@report.command("duplicates")
@click.option("--format", type=click.Choice(["text", "json"], case_sensitive=False), default="text")
@click.option("--input", type=click.Path(path_type=Path), default=lambda: default_artifact_path("analysis.json"))
@click.option("--output", type=click.Path(path_type=Path), default=None)
@click.option("--plan", type=click.Path(path_type=Path), default=None)
@pass_context
def report_duplicates(ctx: CLIContext, format: str, input: Path, output: Optional[Path], plan: Optional[Path]):
"""Show duplicate groups with quality comparison."""
run_command(
ctx, lambda: _run_duplicates(ctx, format, input, output, plan),
stage="duplicate report", json_errors=True,
)
@report.command("summary")
@click.option("--input", type=click.Path(path_type=Path), default=lambda: default_artifact_path("inventory.csv"))
@click.option("--output", type=click.Path(path_type=Path), default=None)
@pass_context
def report_summary(ctx: CLIContext, input: Path, output: Optional[Path]):
"""Show library statistics."""
run_command(ctx, lambda: _run_summary(ctx, input, output), stage="summary report")
def report_inventory_cmd(ctx: CLIContext, format: str, input: Path, output: Optional[Path]) -> None:
"""Implementation for inventory report command."""
run_command(ctx, lambda: _run_inventory(ctx, format, input, output), stage="inventory report")
def report_completeness_cmd( def report_completeness_cmd(
ctx: CLIContext, ctx: CLIContext, format: str, input: Path, output: Optional[Path], plan: Optional[Path]
format: str,
input: Path,
output: Optional[Path],
plan: Optional[Path],
) -> None: ) -> None:
"""Generate completeness report.""" """Implementation for completeness report command."""
config = ctx.config run_command(
logger = ctx.logger ctx, lambda: _run_completeness(ctx, format, input, output, plan),
stage="completeness report", json_errors=True,
input = resolve_legacy_default_input_path(input, "input", "analysis.json", "--input") )
plan_summary = None
if plan:
if not plan.exists():
click.echo(f"Error: Plan file not found: {plan}", err=True)
sys.exit(1)
execution_plan = load_plan(plan)
plan_summary = preferred_plan_summary(execution_plan)
try:
click.echo(f"Loading analysis from: {input}")
analysis_data = load_analysis_json(input)
completeness_list = analysis_data.get("completeness", [])
season_completeness = []
for c in completeness_list:
season_completeness.append(
SeasonCompleteness(
series_title=c["series_title"],
season=c["season"],
episodes_found=c["episodes_found"],
episodes_missing=c["episodes_missing"],
)
)
click.echo(f"Loaded {len(season_completeness)} series with gaps")
click.echo()
click.echo(f"Generating completeness report in {format} format...")
report_content = generate_completeness_report(
season_completeness, format, config.library_root, plan_summary=plan_summary
)
if output:
output.parent.mkdir(parents=True, exist_ok=True)
with open(output, "w", encoding="utf-8") as f:
f.write(report_content)
click.echo(f"Report saved to: {output}")
else:
click.echo()
click.echo(report_content)
logger.info(
"Generated completeness report in %s format with %s series",
format,
len(season_completeness),
)
except FileNotFoundError:
command_error(ctx, f"Error: Input file not found: {input}", f"Input file not found: {input}")
except json.JSONDecodeError as e:
command_error(
ctx,
f"Error: Failed to parse JSON file: {e}",
f"JSON parsing failed: {e}",
exc_info=True,
)
except Exception as e:
command_error(
ctx,
f"Error generating completeness report: {e}",
f"Completeness report generation failed: {e}",
exc_info=True,
)
def report_duplicates_cmd( def report_duplicates_cmd(
ctx: CLIContext, ctx: CLIContext, format: str, input: Path, output: Optional[Path], plan: Optional[Path]
format: str,
input: Path,
output: Optional[Path],
plan: Optional[Path],
) -> None: ) -> None:
"""Generate duplicate report.""" """Implementation for duplicates report command."""
config = ctx.config run_command(
logger = ctx.logger ctx, lambda: _run_duplicates(ctx, format, input, output, plan),
stage="duplicate report", json_errors=True,
input = resolve_legacy_default_input_path(input, "input", "analysis.json", "--input") )
plan_summary = None
if plan:
if not plan.exists():
click.echo(f"Error: Plan file not found: {plan}", err=True)
sys.exit(1)
execution_plan = load_plan(plan)
plan_summary = preferred_plan_summary(execution_plan)
try:
click.echo(f"Loading analysis from: {input}")
analysis_data = load_analysis_json(input)
duplicates_list = analysis_data.get("duplicates", [])
duplicate_groups = []
for d in duplicates_list:
identity_data = d["identity"]
if identity_data["type"] == "movie":
identity = MovieIdentity(
title=identity_data["title"],
year=identity_data.get("year"),
confidence=1.0,
needs_review=False,
original_filename="",
)
else:
identity = SeriesIdentity(
title=identity_data["title"],
season=identity_data.get("season"),
episodes=identity_data.get("episodes", []),
confidence=1.0,
needs_review=False,
original_filename="",
)
quality_by_path = {
str(item.get("path", "")): item for item in d.get("quality_comparison", [])
}
files = []
for file_path in d["files"]:
quality = quality_by_path.get(str(file_path), {})
files.append(
VideoFile(
path=Path(file_path),
filename=Path(file_path).name,
size_bytes=int(quality.get("size_bytes", 0) or 0),
modified_timestamp=datetime.now(timezone.utc),
category="",
resolution=quality.get("resolution"),
codec=quality.get("codec"),
duration_seconds=quality.get("duration_seconds"),
bitrate_kbps=quality.get("bitrate_kbps"),
)
)
duplicate_groups.append(
DuplicateGroup(
identity=identity,
files=files,
quality_comparison=d["quality_comparison"],
)
)
click.echo(f"Loaded {len(duplicate_groups)} duplicate groups")
click.echo()
click.echo(f"Generating duplicate report in {format} format...")
report_content = generate_duplicate_report(
duplicate_groups, format, config.library_root, plan_summary=plan_summary
)
if output:
output.parent.mkdir(parents=True, exist_ok=True)
with open(output, "w", encoding="utf-8") as f:
f.write(report_content)
click.echo(f"Report saved to: {output}")
else:
click.echo()
click.echo(report_content)
logger.info(
"Generated duplicate report in %s format with %s groups",
format,
len(duplicate_groups),
)
except FileNotFoundError:
command_error(ctx, f"Error: Input file not found: {input}", f"Input file not found: {input}")
except json.JSONDecodeError as e:
command_error(
ctx,
f"Error: Failed to parse JSON file: {e}",
f"JSON parsing failed: {e}",
exc_info=True,
)
except Exception as e:
command_error(
ctx,
f"Error generating duplicate report: {e}",
f"Duplicate report generation failed: {e}",
exc_info=True,
)
def report_summary_cmd(ctx: CLIContext, input: Path, output: Optional[Path]) -> None: def report_summary_cmd(ctx: CLIContext, input: Path, output: Optional[Path]) -> None:
"""Generate summary report.""" """Implementation for summary report command."""
config = ctx.config run_command(ctx, lambda: _run_summary(ctx, input, output), stage="summary report")
logger = ctx.logger
try:
input = resolve_legacy_default_input_path(input, "input", "inventory.csv", "--input")
click.echo(f"Loading inventory from: {input}")
video_files = load_inventory_csv(input)
click.echo(f"Loaded {len(video_files)} files")
click.echo()
click.echo("Generating summary report...")
report_content = generate_summary_report(video_files, config.library_root)
if output:
output.parent.mkdir(parents=True, exist_ok=True)
with open(output, "w", encoding="utf-8") as f:
f.write(report_content)
click.echo(f"Report saved to: {output}")
else:
click.echo()
click.echo(report_content)
logger.info("Generated summary report with %s files", len(video_files))
except FileNotFoundError:
command_error(ctx, f"Error: Input file not found: {input}", f"Input file not found: {input}")
except Exception as e:
command_error(
ctx,
f"Error generating summary report: {e}",
f"Summary report generation failed: {e}",
exc_info=True,
)
+23 -1
View File
@@ -5,7 +5,8 @@ from typing import Optional
import click import click
from vlm.context import CLIContext from vlm.cli_helpers import default_artifact_path, run_command
from vlm.context import CLIContext, pass_context
from vlm.scanner import load_inventory_csv, save_inventory_csv, scan_library from vlm.scanner import load_inventory_csv, save_inventory_csv, scan_library
from vlm.utils import format_size from vlm.utils import format_size
@@ -94,3 +95,24 @@ def scan_cmd(
save_inventory_csv(video_files, output, config.library_root) save_inventory_csv(video_files, output, config.library_root)
click.echo("Inventory saved successfully!") click.echo("Inventory saved successfully!")
logger.info(f"Scan completed: {len(video_files)} files found, saved to {output}") logger.info(f"Scan completed: {len(video_files)} files found, saved to {output}")
@click.command()
@click.option("--output", type=click.Path(path_type=Path), default=lambda: default_artifact_path("inventory.csv"))
@click.option("--metadata/--no-metadata", default=True)
@click.option("--reuse-from", type=click.Path(exists=True, path_type=Path), default=None)
@click.option("--force-refresh-metadata", is_flag=True, default=False)
@pass_context
def scan(
ctx: CLIContext,
output: Path,
metadata: bool,
reuse_from: Optional[Path],
force_refresh_metadata: bool,
):
"""Scan library and write inventory.csv."""
run_command(
ctx,
lambda: scan_cmd(ctx, output, metadata, reuse_from, force_refresh_metadata),
stage="scan",
)
+41 -1
View File
@@ -8,7 +8,7 @@ from typing import Optional
import click import click
from vlm.cli_helpers import command_error from vlm.cli_helpers import command_error
from vlm.context import CLIContext from vlm.context import CLIContext, pass_context
from vlm.state import StateManager from vlm.state import StateManager
@@ -137,3 +137,43 @@ def state_clear_cmd(ctx: CLIContext, file: Path) -> None:
f"Failed to clear file state: {e}", f"Failed to clear file state: {e}",
exc_info=True, exc_info=True,
) )
@click.group()
@pass_context
def state(ctx: CLIContext):
"""Track per-file workflow status."""
@state.command("show")
@click.argument("file", type=click.Path(path_type=Path))
@pass_context
def state_show(ctx: CLIContext, file: Path):
"""Show state for a file."""
state_show_cmd(ctx, file)
@state.command("set")
@click.argument("file", type=click.Path(path_type=Path))
@click.option("--status", type=click.Choice(["reviewed", "ignored", "planned", "executed", "quarantined"], case_sensitive=False), required=True)
@click.option("--reason", type=str, default=None)
@pass_context
def state_set(ctx: CLIContext, file: Path, status: str, reason: Optional[str]):
"""Set state for a file."""
state_set_cmd(ctx, file, status, reason)
@state.command("query")
@click.option("--status", type=click.Choice(["reviewed", "ignored", "planned", "executed", "quarantined"], case_sensitive=False), required=True)
@pass_context
def state_query(ctx: CLIContext, status: str):
"""List files with a given status."""
state_query_cmd(ctx, status)
@state.command("clear")
@click.argument("file", type=click.Path(path_type=Path))
@pass_context
def state_clear(ctx: CLIContext, file: Path):
"""Clear state for a file."""
state_clear_cmd(ctx, file)
+8 -1
View File
@@ -15,6 +15,7 @@ from vlm.cache import EnrichmentCache
from vlm.config import Config from vlm.config import Config
from vlm.parser import normalize_title from vlm.parser import normalize_title
from vlm.providers import ProviderResult, TMDBAuthError, TMDBProvider, TMDBProviderError from vlm.providers import ProviderResult, TMDBAuthError, TMDBProvider, TMDBProviderError
from vlm.providers.base import RequestRateLimiter
from vlm.utils import sanitize_path_component from vlm.utils import sanitize_path_component
RefreshMode = str RefreshMode = str
@@ -62,6 +63,7 @@ def enrich_identities_data(
refresh_all = refresh_mode == "refresh_all" refresh_all = refresh_mode == "refresh_all"
max_workers = max(1, int(config.enrichment_max_concurrency)) max_workers = max(1, int(config.enrichment_max_concurrency))
rate_limiter = RequestRateLimiter(min_interval=0.25)
for section, media_type in (("movies", "movie"), ("series", "series"), ("anime", "anime")): for section, media_type in (("movies", "movie"), ("series", "series"), ("anime", "anime")):
records = identities_data.get(section, []) records = identities_data.get(section, [])
@@ -108,6 +110,7 @@ def enrich_identities_data(
config, config,
request_timeout=request_timeout, request_timeout=request_timeout,
retries=retries, retries=retries,
rate_limiter=rate_limiter,
) )
_apply_payload(record, payload) _apply_payload(record, payload)
cache.put_identity(identity_key, fingerprint, payload) cache.put_identity(identity_key, fingerprint, payload)
@@ -125,6 +128,7 @@ def enrich_identities_data(
config, config,
request_timeout, request_timeout,
retries, retries,
rate_limiter,
) )
future_map[future] = (record, identity_key, fingerprint) future_map[future] = (record, identity_key, fingerprint)
@@ -197,7 +201,7 @@ def _update_stats_after_enrich(
_emit_progress(stats, progress_callback) _emit_progress(stats, progress_callback)
def _build_providers(config: Config, *, request_timeout: int, retries: int) -> list: def _build_providers(config: Config, *, request_timeout: int, retries: int, rate_limiter: RequestRateLimiter) -> list:
providers = [] providers = []
unsupported: list[str] = [] unsupported: list[str] = []
for name in config.enrichment_providers: for name in config.enrichment_providers:
@@ -213,6 +217,7 @@ def _build_providers(config: Config, *, request_timeout: int, retries: int) -> l
timeout_seconds=request_timeout, timeout_seconds=request_timeout,
retries=retries, retries=retries,
min_interval_seconds=0.25, min_interval_seconds=0.25,
rate_limiter=rate_limiter,
) )
) )
else: else:
@@ -232,11 +237,13 @@ def _enrich_record_with_fresh_providers(
config: Config, config: Config,
request_timeout: int, request_timeout: int,
retries: int, retries: int,
rate_limiter: RequestRateLimiter,
) -> tuple[dict, int, list[dict[str, str]], str]: ) -> tuple[dict, int, list[dict[str, str]], str]:
providers = _build_providers( providers = _build_providers(
config, config,
request_timeout=request_timeout, request_timeout=request_timeout,
retries=retries, retries=retries,
rate_limiter=rate_limiter,
) )
return _enrich_record( return _enrich_record(
record, record,
+4 -1
View File
@@ -479,7 +479,10 @@ def identities_to_plan_input(
for s in series_data: for s in series_data:
result.append((_video_file_from_record(s), _series_identity_from_record(s))) result.append((_video_file_from_record(s), _series_identity_from_record(s)))
for a in anime_data: for a in anime_data:
result.append((_video_file_from_record(a), None)) if "title" in a and "needs_review" in a:
result.append((_video_file_from_record(a), _series_identity_from_record(a)))
else:
result.append((_video_file_from_record(a), None))
for o in other_data: for o in other_data:
result.append((_video_file_from_record(o), None)) result.append((_video_file_from_record(o), None))
+1 -1
View File
@@ -333,7 +333,7 @@ class ParsedIdentitiesJSON(TypedDict, total=False):
metadata: dict[str, object] metadata: dict[str, object]
movies: list[MovieIdentityRecord] movies: list[MovieIdentityRecord]
series: list[SeriesIdentityRecord] series: list[SeriesIdentityRecord]
anime: list[IdentityRecord] anime: list[SeriesIdentityRecord]
other: list[IdentityRecord] other: list[IdentityRecord]
+74
View File
@@ -282,6 +282,80 @@ def parse_series(
) )
def parse_anime(
filename: str,
extensions: Optional[list[str]] = None,
) -> SeriesIdentity:
"""Parse an anime filename into a plan-consumable series identity.
Conservative: explicit ``SxxEyy`` / ``Season N - EP`` are organized; an
absolute episode (``Title - NN``) with no season info sets ``needs_review``
so the planner treats it as a manual-review no-op (never silently moved).
"""
if extensions is None:
extensions = DEFAULT_VIDEO_EXTENSIONS
name = filename
for ext in extensions:
if name.lower().endswith(ext.lower()):
name = name[: -len(ext)]
break
# Strip CRC32 hash brackets first so they are not mistaken for a release group
name = re.sub(r'\[[0-9A-Fa-f]{8}\]', '', name)
name = remove_release_groups(name)
name = remove_quality_tags(name)
season = None
episodes: list[int] = []
confidence = 0.0
title_part = name
m = re.search(r'[Ss](\d{1,2})[Ee](\d{1,2})', name)
if m:
season = int(m.group(1))
episodes = [int(m.group(2))]
confidence = 0.9
title_part = name[: m.start()]
else:
m = re.search(r'[Ss]eason\s*(\d{1,2})\s*[-_]\s*(\d{1,3})(?!\d)', name)
if m:
season = int(m.group(1))
episodes = [int(m.group(2))]
confidence = 0.85
title_part = name[: m.start()]
else:
m = re.search(r'\bS(\d{1,2})\s*[-_]\s*(\d{1,3})(?!\d)', name)
if m:
season = int(m.group(1))
episodes = [int(m.group(2))]
confidence = 0.85
title_part = name[: m.start()]
else:
m = re.search(r'\s*[-_]\s*(\d{1,3})(?!\d)\s*$', name)
if m:
episodes = [int(m.group(1))]
confidence = 0.5
title_part = name[: m.start()]
# season remains None -> needs_review
if title_part and title_part.strip():
title = humanize_parsed_title(remove_quality_tags(remove_release_groups(title_part)))
else:
title = humanize_parsed_title(name)
if not title.strip():
title = humanize_parsed_title(filename)
needs_review = season is None or not episodes
return SeriesIdentity(
title=title,
season=season,
episodes=episodes,
confidence=confidence,
needs_review=needs_review,
original_filename=filename,
)
def group_episodes(episodes: list[SeriesIdentity]) -> dict[tuple[str, int], list[SeriesIdentity]]: def group_episodes(episodes: list[SeriesIdentity]) -> dict[tuple[str, int], list[SeriesIdentity]]:
"""Group parsed episodes by normalized series title and season number. """Group parsed episodes by normalized series title and season number.
+6
View File
@@ -35,6 +35,7 @@ REVIEW_CSV_ENRICHED_FIELDS = [
"rel_dest", "rel_dest",
"quality_hint", "quality_hint",
"duplicate_group_id", "duplicate_group_id",
"sidecars",
] ]
REVIEW_CSV_ALL_FIELDS = REVIEW_CSV_BASE_FIELDS + REVIEW_CSV_ENRICHED_FIELDS REVIEW_CSV_ALL_FIELDS = REVIEW_CSV_BASE_FIELDS + REVIEW_CSV_ENRICHED_FIELDS
@@ -246,6 +247,11 @@ def enrich_review_row(
enriched["episode"] = str(ctx["episode"]) enriched["episode"] = str(ctx["episode"])
if ctx.get("duplicate_group_id"): if ctx.get("duplicate_group_id"):
enriched["duplicate_group_id"] = str(ctx["duplicate_group_id"]) enriched["duplicate_group_id"] = str(ctx["duplicate_group_id"])
sidecars = ctx.get("sidecars")
if isinstance(sidecars, list) and sidecars:
names = [str(item.get("name", "")) for item in sidecars if isinstance(item, dict)]
if names:
enriched["sidecars"] = ", ".join(names)
if identity_lookup and source_path: if identity_lookup and source_path:
id_ctx = identity_lookup.get(source_path) or identity_lookup.get( id_ctx = identity_lookup.get(source_path) or identity_lookup.get(
+13
View File
@@ -26,6 +26,7 @@ from vlm.plan_review import (
build_review_context, build_review_context,
normalized_path_key, normalized_path_key,
) )
from vlm.scanner import find_sidecar_companions
from vlm.utils import ( from vlm.utils import (
is_sample_path, is_sample_path,
is_within_root, is_within_root,
@@ -584,6 +585,18 @@ def _stamp_review_context_on_operations(
duplicate_group_id=gid, duplicate_group_id=gid,
keep_candidate=keep_candidate, keep_candidate=keep_candidate,
) )
if op.operation_type != "no-op" and op.destination_path:
sidecars = find_sidecar_companions(vf.path)
if sidecars:
dest_dir = op.destination_path.parent
ctx["sidecars"] = [
{
"name": s.name,
"path": str(s),
"proposed_destination_path": str(dest_dir / s.name),
}
for s in sidecars
]
stamped.append(replace(op, review_context=ctx)) stamped.append(replace(op, review_context=ctx))
continue continue
stamped.append(op) stamped.append(op)
+25
View File
@@ -2,10 +2,35 @@
from __future__ import annotations from __future__ import annotations
import threading
import time
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Optional, Protocol from typing import Optional, Protocol
class RequestRateLimiter:
"""Thread-safe rate limiter for API requests.
Coordinates request timing across concurrent workers to respect
provider rate limits.
"""
def __init__(self, min_interval: float):
"""Initialize with minimum seconds between requests."""
self._min_interval = min_interval
self._last_request_time = 0.0
self._lock = threading.Lock()
def wait(self) -> None:
"""Block until enough time has passed since the last request."""
with self._lock:
now = time.monotonic()
elapsed = now - self._last_request_time
if elapsed < self._min_interval:
time.sleep(self._min_interval - elapsed)
self._last_request_time = time.monotonic()
@dataclass @dataclass
class ProviderResult: class ProviderResult:
"""Normalized provider output used by enrichment pipeline.""" """Normalized provider output used by enrichment pipeline."""
+6 -1
View File
@@ -11,7 +11,7 @@ from urllib.error import HTTPError, URLError
from urllib.parse import urlencode from urllib.parse import urlencode
from urllib.request import Request, urlopen from urllib.request import Request, urlopen
from vlm.providers.base import ProviderResult from vlm.providers.base import ProviderResult, RequestRateLimiter
class TMDBAuthError(RuntimeError): class TMDBAuthError(RuntimeError):
@@ -40,6 +40,7 @@ class TMDBProvider:
min_interval_seconds: float = 0.25, min_interval_seconds: float = 0.25,
backoff_base_seconds: float = 0.5, backoff_base_seconds: float = 0.5,
backoff_max_seconds: float = 4.0, backoff_max_seconds: float = 4.0,
rate_limiter: Optional[RequestRateLimiter] = None,
) -> None: ) -> None:
self.api_key = api_key self.api_key = api_key
self.bearer_token = bearer_token self.bearer_token = bearer_token
@@ -52,6 +53,7 @@ class TMDBProvider:
self.min_interval_seconds = min_interval_seconds self.min_interval_seconds = min_interval_seconds
self.backoff_base_seconds = backoff_base_seconds self.backoff_base_seconds = backoff_base_seconds
self.backoff_max_seconds = backoff_max_seconds self.backoff_max_seconds = backoff_max_seconds
self.rate_limiter = rate_limiter
self._last_request_at = 0.0 self._last_request_at = 0.0
self.last_request_count = 0 self.last_request_count = 0
@@ -181,6 +183,9 @@ class TMDBProvider:
return None return None
def _wait_for_rate_limit(self) -> None: def _wait_for_rate_limit(self) -> None:
if self.rate_limiter is not None:
self.rate_limiter.wait()
return
if self.min_interval_seconds <= 0: if self.min_interval_seconds <= 0:
return return
now = time.monotonic() now = time.monotonic()
+36 -13
View File
@@ -113,11 +113,12 @@ class QuarantineManager:
# Determine category from file path # Determine category from file path
category = self._determine_category(file_path) category = self._determine_category(file_path)
# Reject anime and other categories (v1 constraint) # Reject categories outside the configured quarantine scope
if category not in ("movie", "series"): supported_categories = self._supported_categories()
if category not in supported_categories:
error_msg = ( error_msg = (
f"Quarantine not supported for category '{category}'. " f"Quarantine not supported for category '{category}'. "
f"Only 'movie' and 'series' categories are supported in v1." f"Supported categories: {', '.join(sorted(supported_categories))}."
) )
log_operation( log_operation(
self.logger, self.logger,
@@ -359,6 +360,30 @@ class QuarantineManager:
executed_at=executed_at executed_at=executed_at
) )
def _supported_categories(self) -> set:
"""Return configured category keys supported by the quarantine lifecycle."""
categories_config = self.config.categories or {
"movie": ["movie"],
"series": ["series"],
"anime": ["anime"]
}
return set(categories_config.keys())
def _category_from_directory(self, dir_name: str) -> Optional[str]:
"""Map a category directory name to its configured category key."""
dir_name = dir_name.lower()
if dir_name in self._supported_categories():
return dir_name
categories_config = self.config.categories or {
"movie": ["movie"],
"series": ["series"],
"anime": ["anime"]
}
for category, dir_names in categories_config.items():
if any(dir_name == name.lower() for name in dir_names):
return category
return None
def _determine_category(self, file_path: Path) -> str: def _determine_category(self, file_path: Path) -> str:
"""Determine the category of a file based on its path. """Determine the category of a file based on its path.
@@ -678,20 +703,21 @@ class QuarantineManager:
entries = [] entries = []
# Determine which categories to query # Determine which categories to query
supported = self._supported_categories()
if category is not None: if category is not None:
# Validate category # Validate category
if category not in ("movie", "series"): if category not in supported:
log_operation( log_operation(
self.logger, self.logger,
logging.WARNING, logging.WARNING,
f"Invalid category '{category}' for listing. Only 'movie' and 'series' are supported.", f"Invalid category '{category}' for listing. Supported categories: {', '.join(sorted(supported))}.",
operation_type="quarantine" operation_type="quarantine"
) )
return [] return []
categories = [category] categories = [category]
else: else:
# List from all supported categories # List from all supported categories
categories = ["movie", "series"] categories = sorted(supported)
# Load manifests from each category # Load manifests from each category
for cat in categories: for cat in categories:
@@ -720,7 +746,7 @@ class QuarantineManager:
Returns: Returns:
The path where the file was moved in quarantine, or None if not found The path where the file was moved in quarantine, or None if not found
""" """
for category in ("movie", "series"): for category in sorted(self._supported_categories()):
manifest = self._load_manifest(category) manifest = self._load_manifest(category)
for entry in manifest.entries: for entry in manifest.entries:
if entry.original_path == original_path: if entry.original_path == original_path:
@@ -1014,14 +1040,11 @@ class QuarantineManager:
if len(parts) < 2: if len(parts) < 2:
return None return None
# First part should be category, second should be .quarantine # First part should be category directory, second should be .quarantine
category = parts[0].lower() dir_name = parts[0].lower()
quarantine_dir = parts[1] quarantine_dir = parts[1]
if quarantine_dir != self.config.quarantine_dir: if quarantine_dir != self.config.quarantine_dir:
return None return None
if category in ("movie", "series"): return self._category_from_directory(dir_name)
return category
return None
+68 -1
View File
@@ -14,7 +14,7 @@ from datetime import datetime, timezone
from io import StringIO from io import StringIO
from pathlib import Path from pathlib import Path
from vlm.models import DuplicateGroup, MovieIdentity, SeasonCompleteness, VideoFile from vlm.models import DuplicateGroup, MovieIdentity, SeasonCompleteness, SeriesIdentity, VideoFile
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -565,3 +565,70 @@ def _format_duration(duration_seconds: float) -> str:
parts.append(f"{seconds}s") parts.append(f"{seconds}s")
return " ".join(parts) return " ".join(parts)
def completeness_from_analysis(analysis_data: dict) -> list[SeasonCompleteness]:
"""Build SeasonCompleteness records from analysis JSON."""
result: list[SeasonCompleteness] = []
for row in analysis_data.get("completeness", []) or []:
result.append(
SeasonCompleteness(
series_title=row["series_title"],
season=row["season"],
episodes_found=row["episodes_found"],
episodes_missing=row["episodes_missing"],
)
)
return result
def duplicate_groups_from_analysis(analysis_data: dict) -> list[DuplicateGroup]:
"""Build DuplicateGroup records from analysis JSON."""
groups: list[DuplicateGroup] = []
for dup in analysis_data.get("duplicates", []) or []:
identity_data = dup["identity"]
if identity_data["type"] == "movie":
identity = MovieIdentity(
title=identity_data["title"],
year=identity_data.get("year"),
confidence=1.0,
needs_review=False,
original_filename="",
)
else:
identity = SeriesIdentity(
title=identity_data["title"],
season=identity_data.get("season"),
episodes=identity_data.get("episodes", []),
confidence=1.0,
needs_review=False,
original_filename="",
)
quality_by_path = {
str(item.get("path", "")): item for item in dup.get("quality_comparison", [])
}
files: list[VideoFile] = []
for file_path in dup.get("files", []) or []:
quality = quality_by_path.get(str(file_path), {})
files.append(
VideoFile(
path=Path(file_path),
filename=Path(file_path).name,
size_bytes=int(quality.get("size_bytes", 0) or 0),
modified_timestamp=datetime.now(timezone.utc),
category="",
resolution=quality.get("resolution"),
codec=quality.get("codec"),
duration_seconds=quality.get("duration_seconds"),
bitrate_kbps=quality.get("bitrate_kbps"),
)
)
groups.append(
DuplicateGroup(
identity=identity,
files=files,
quality_comparison=dup.get("quality_comparison", []),
)
)
return groups
+50
View File
@@ -290,6 +290,56 @@ def _is_video_file(file_path: Path, video_extensions: list[str]) -> bool:
return file_extension in [ext.lower() for ext in video_extensions] return file_extension in [ext.lower() for ext in video_extensions]
DEFAULT_SIDECAR_EXTENSIONS = (".srt", ".ass", ".sub", ".idx", ".sup", ".nfo")
def find_sidecar_companions(
video_path: Path,
sidecar_extensions: tuple = DEFAULT_SIDECAR_EXTENSIONS,
) -> list[Path]:
"""Find sidecar files in the same directory that belong to a video file.
Conservative matching: a companion must share the video's exact stem,
optionally followed by dot-separated alphabetic suffix tokens (e.g.
language or track tags such as ``zh`` or ``en.forced``). Numeric or
otherwise non-alphabetic tokens are rejected so unrelated files are
never associated.
Args:
video_path: Path to the video file
sidecar_extensions: Sidecar extensions to consider (case-insensitive)
Returns:
Sorted list of companion paths (empty when none are found).
"""
parent = video_path.parent
try:
with os.scandir(parent) as it:
entries = [e for e in it if e.is_file(follow_symlinks=False)]
except OSError:
return []
stem = video_path.stem
exts = {ext.lower() for ext in sidecar_extensions}
companions: list[Path] = []
for entry in entries:
name = entry.name
suffix = Path(name).suffix.lower()
if suffix not in exts:
continue
base = name[: -len(suffix)]
if base == stem:
companions.append(Path(entry.path))
continue
if not base.startswith(stem + "."):
continue
tokens = base[len(stem) + 1:].split(".")
if tokens and all(token and token.isalpha() for token in tokens):
companions.append(Path(entry.path))
return sorted(companions, key=lambda p: p.name)
def _create_video_file( def _create_video_file(
file_path: Path, file_path: Path,
library_root: Path, library_root: Path,
+6 -6
View File
@@ -177,24 +177,24 @@ class TestQuarantineAddCommand:
assert "Reason: duplicate file" in result.output assert "Reason: duplicate file" in result.output
assert "successfully quarantined" in result.output assert "successfully quarantined" in result.output
def test_add_anime_file_rejected(self, config_file, temp_library): def test_add_anime_file_supported(self, config_file, temp_library):
"""Test that anime files are rejected.""" """Test that anime files can be quarantined (configured category)."""
runner = CliRunner() runner = CliRunner()
# Create a test anime file # Create a test anime file
anime_file = temp_library / "anime" / "Anime Show.mkv" anime_file = temp_library / "anime" / "Anime Show.mkv"
anime_file.write_text("anime content") anime_file.write_text("anime content")
# Try to quarantine (should fail) # Quarantine should succeed
result = runner.invoke(main, [ result = runner.invoke(main, [
'--config', str(config_file), '--config', str(config_file),
'quarantine', 'add', 'quarantine', 'add',
str(anime_file) str(anime_file)
]) ])
assert result.exit_code == 1 assert result.exit_code == 0
assert "not supported" in result.output.lower() assert "successfully quarantined" in result.output
assert anime_file.exists() # File should still exist assert not anime_file.exists()
def test_add_nonexistent_file(self, config_file, temp_library): def test_add_nonexistent_file(self, config_file, temp_library):
"""Test adding a file that doesn't exist.""" """Test adding a file that doesn't exist."""
+82
View File
@@ -0,0 +1,82 @@
"""Verify vlm commands referenced in documentation exist in the CLI registry."""
from __future__ import annotations
import re
from pathlib import Path
import pytest
from click.testing import CliRunner
from vlm.cli import main
REPO_ROOT = Path(__file__).resolve().parent.parent
DOC_GLOBS = [
"README.md",
"CLAUDE.md",
"AGENTS.md",
"skills/**/*.md",
]
COMMAND_PATTERN = re.compile(
r"^(?:uv run )?(?:\$ )?vlm\s+(.+)$", re.MULTILINE
)
def _extract_doc_commands() -> list[str]:
commands: list[str] = []
for pattern in DOC_GLOBS:
for path in REPO_ROOT.glob(pattern):
text = path.read_text(encoding="utf-8")
for match in COMMAND_PATTERN.finditer(text):
raw = match.group(1).strip()
raw = raw.rstrip("\\").strip()
if raw.startswith("#") or not raw:
continue
commands.append(raw)
return commands
def _cli_command_names() -> dict[str, list[str]]:
result: dict[str, list[str]] = {}
for name, cmd in main.commands.items():
result[name] = []
if hasattr(cmd, "commands"):
result[name] = list(cmd.commands.keys())
return result
@pytest.mark.parametrize(
"raw_cmd",
sorted(set(_extract_doc_commands())),
ids=lambda c: c[:60],
)
def test_documented_command_exists(raw_cmd: str):
parts = raw_cmd.split()
if not parts:
pytest.skip("empty command")
first = parts[0]
if first in ("--help", "-h"):
return
registry = _cli_command_names()
if first not in registry:
pytest.fail(f"Documented command 'vlm {first}' not in CLI registry")
if len(parts) > 1:
sub = parts[1]
if sub.startswith("-"):
return
subcommands = registry.get(first, [])
if subcommands and sub not in subcommands:
pytest.fail(
f"Documented subcommand 'vlm {first} {sub}' "
f"not in CLI registry (available: {subcommands})"
)
def test_fake_command_is_caught():
registry = _cli_command_names()
assert "nonexistent-command-xyz" not in registry
+93 -9
View File
@@ -7,7 +7,7 @@ import pytest
from vlm.config import Config from vlm.config import Config
from vlm.enrichment import _build_display_title, _build_providers, enrich_identities_data from vlm.enrichment import _build_display_title, _build_providers, enrich_identities_data
from vlm.providers.base import ProviderResult from vlm.providers.base import ProviderResult, RequestRateLimiter
from vlm.providers.tmdb import TMDBAuthError from vlm.providers.tmdb import TMDBAuthError
@@ -45,7 +45,7 @@ def test_enrich_incremental_cache_hit(tmp_path, monkeypatch):
provider = DummyProvider() provider = DummyProvider()
monkeypatch.setattr( monkeypatch.setattr(
"vlm.enrichment._build_providers", "vlm.enrichment._build_providers",
lambda _config, request_timeout, retries: [provider], lambda _config, request_timeout, retries, **_kwargs: [provider],
) )
identities = { identities = {
@@ -113,7 +113,7 @@ def test_enrich_flags_low_reputation_for_review(tmp_path, monkeypatch):
provider = LowScoreProvider() provider = LowScoreProvider()
monkeypatch.setattr( monkeypatch.setattr(
"vlm.enrichment._build_providers", "vlm.enrichment._build_providers",
lambda _config, request_timeout, retries: [provider], lambda _config, request_timeout, retries, **_kwargs: [provider],
) )
identities = { identities = {
@@ -151,7 +151,7 @@ def test_enrich_refresh_all_bypasses_cache(tmp_path, monkeypatch):
provider = DummyProvider() provider = DummyProvider()
monkeypatch.setattr( monkeypatch.setattr(
"vlm.enrichment._build_providers", "vlm.enrichment._build_providers",
lambda _config, request_timeout, retries: [provider], lambda _config, request_timeout, retries, **_kwargs: [provider],
) )
identities = { identities = {
@@ -188,7 +188,7 @@ def test_build_providers_rejects_unknown_provider(tmp_path):
) )
with pytest.raises(ValueError, match="Unsupported enrichment providers"): with pytest.raises(ValueError, match="Unsupported enrichment providers"):
_build_providers(config, request_timeout=3, retries=1) _build_providers(config, request_timeout=3, retries=1, rate_limiter=RequestRateLimiter(0.25))
def test_refresh_all_clears_stale_enrichment_fields(tmp_path, monkeypatch): def test_refresh_all_clears_stale_enrichment_fields(tmp_path, monkeypatch):
@@ -227,7 +227,7 @@ def test_refresh_all_clears_stale_enrichment_fields(tmp_path, monkeypatch):
provider = FlakyProvider() provider = FlakyProvider()
monkeypatch.setattr( monkeypatch.setattr(
"vlm.enrichment._build_providers", "vlm.enrichment._build_providers",
lambda _config, request_timeout, retries: [provider], lambda _config, request_timeout, retries, **_kwargs: [provider],
) )
identities = { identities = {
@@ -331,7 +331,7 @@ def test_enrich_auth_failure_stops_immediately(tmp_path, monkeypatch):
monkeypatch.setattr( monkeypatch.setattr(
"vlm.enrichment._build_providers", "vlm.enrichment._build_providers",
lambda _config, request_timeout, retries: [AuthFailProvider()], lambda _config, request_timeout, retries, **_kwargs: [AuthFailProvider()],
) )
identities = { identities = {
@@ -387,8 +387,14 @@ def test_enrich_respects_max_concurrency_for_uncached_records(tmp_path, monkeypa
thread_ids: set[int] = set() thread_ids: set[int] = set()
lock = threading.Lock() lock = threading.Lock()
seen_limiters: list[object] = []
def _fake_enrich(record, media_type, config_obj, request_timeout, retries): def _fake_enrich(record, media_type, config_obj, request_timeout, retries, rate_limiter=None):
with lock:
if rate_limiter is not None:
if not seen_limiters:
seen_limiters.append(rate_limiter)
else:
assert rate_limiter is seen_limiters[0]
time.sleep(0.01) time.sleep(0.01)
with lock: with lock:
thread_ids.add(threading.get_ident()) thread_ids.add(threading.get_ident())
@@ -420,3 +426,81 @@ def test_enrich_respects_max_concurrency_for_uncached_records(tmp_path, monkeypa
assert stats["enriched"] == 8 assert stats["enriched"] == 8
assert stats["cache_hits"] == 0 assert stats["cache_hits"] == 0
assert len(thread_ids) > 1 assert len(thread_ids) > 1
class _RateLimitedTMDB:
name = "tmdb"
def __init__(self, api_key, *, rate_limiter=None, record_wait=None, wait_lock=None, **kwargs):
self.rate_limiter = rate_limiter
self.last_request_count = 0
self._record_wait = record_wait
self._wait_lock = wait_lock
def enrich(self, *, title, media_type, year=None):
if self.rate_limiter is not None:
self.rate_limiter.wait()
if self._record_wait is not None and self._wait_lock is not None:
with self._wait_lock:
self._record_wait.append(time.monotonic())
self.last_request_count = 1
return ProviderResult(
provider="tmdb",
canonical_id=f"tmdb:{title}",
title_zh="测试",
title_en=title,
translation_source="tmdb",
)
def test_enrich_concurrent_workers_share_provider_rate_limiter(tmp_path, monkeypatch):
"""Concurrent TMDB providers must admit requests through one limiter."""
config = Config(
library_root=tmp_path,
enrichment_cache_db=tmp_path / "cache.db",
enrichment_providers=["tmdb"],
tmdb_api_key="fake",
translation_fallback_machine=False,
enrichment_max_concurrency=4,
)
identities = {
"metadata": {},
"movies": [
{
"path": f"/library/movie/Test.{i}.mkv",
"filename": f"Test.{i}.mkv",
"category": "movie",
"title": f"Test {i}",
"year": 2020,
"confidence": 0.9,
"needs_review": False,
}
for i in range(3)
],
"series": [],
"anime": [],
"other": [],
}
shared_limiters: dict[str, object] = {}
wait_times: list[float] = []
wait_lock = threading.Lock()
def _factory(api_key, **kwargs):
limiter = kwargs.pop("rate_limiter")
if "limiter" not in shared_limiters:
shared_limiters["limiter"] = limiter
else:
assert limiter is shared_limiters["limiter"]
return _RateLimitedTMDB(api_key, rate_limiter=limiter, record_wait=wait_times, wait_lock=wait_lock)
monkeypatch.setattr("vlm.enrichment.TMDBProvider", _factory)
_, stats = enrich_identities_data(identities, config, refresh_mode="refresh_all")
assert stats["enriched"] == 3
assert len(wait_times) == 3
# Shared limiter must serialize request starts at >= 0.25s apart
# (allow small tolerance for thread scheduling).
for earlier, later in zip(wait_times, wait_times[1:]):
assert later - earlier >= 0.2
+81
View File
@@ -76,6 +76,63 @@ def test_build_identity_lookup_and_enrich(tmp_path):
assert enriched[0]["source_name"] == "Show.S01E01.mkv" assert enriched[0]["source_name"] == "Show.S01E01.mkv"
def test_enrich_review_rows_sidecars_column(tmp_path):
op = FileOperation(
operation_type="move",
source_path=tmp_path / "dl/Show.S01E01.mkv",
destination_path=tmp_path / "lib/series/Show/Season 01/S01E01.mkv",
reason="organize",
has_conflict=False,
review_context={
"title": "Show",
"category": "series",
"season": 1,
"episode": 1,
"sidecars": [
{"name": "Show.S01E01.zh.srt", "source_path": str(tmp_path / "dl/Show.S01E01.zh.srt"), "proposed_destination_path": str(tmp_path / "lib/series/Show/Season 01/Show.S01E01.zh.srt")},
{"name": "Show.S01E01.nfo", "source_path": str(tmp_path / "dl/Show.S01E01.nfo"), "proposed_destination_path": str(tmp_path / "lib/series/Show/Season 01/Show.S01E01.nfo")},
],
},
)
plan = _minimal_plan([op])
rows = [
{
"index": "1",
"operation_type": "move",
"risk_flags": "",
"source_path": str(op.source_path),
"destination_path": str(op.destination_path),
"reason": op.reason,
}
]
enriched = enrich_review_rows(rows, plan, library_root=tmp_path / "lib")
assert enriched[0]["sidecars"] == "Show.S01E01.zh.srt, Show.S01E01.nfo"
def test_enrich_review_rows_sidecars_empty_by_default(tmp_path):
op = FileOperation(
operation_type="move",
source_path=tmp_path / "dl/Show.S01E01.mkv",
destination_path=tmp_path / "lib/series/Show/Season 01/S01E01.mkv",
reason="organize",
has_conflict=False,
review_context={"title": "Show", "category": "series", "season": 1, "episode": 1},
)
plan = _minimal_plan([op])
rows = [
{
"index": "1",
"operation_type": "move",
"risk_flags": "",
"source_path": str(op.source_path),
"destination_path": str(op.destination_path),
"reason": op.reason,
}
]
enriched = enrich_review_rows(rows, plan, library_root=tmp_path / "lib")
assert enriched[0].get("sidecars", "") == ""
def test_check_review_requirements_missing_csv(tmp_path): def test_check_review_requirements_missing_csv(tmp_path):
plan_path = tmp_path / "plan.json" plan_path = tmp_path / "plan.json"
op = FileOperation( op = FileOperation(
@@ -137,6 +194,30 @@ def test_check_review_requirements_passes_after_apply_review(tmp_path):
assert errors == [] assert errors == []
def test_check_review_requirements_detects_csv_path_mismatch(tmp_path):
plan_path = tmp_path / "plan.json"
csv_path = tmp_path / "review.csv"
other_csv = tmp_path / "other.csv"
op = FileOperation(
operation_type="move",
source_path=tmp_path / "a.mkv",
destination_path=tmp_path / "lib/a.mkv",
reason="Series needs manual review (season exceeds configured threshold)",
has_conflict=True,
)
plan = _minimal_plan([op])
save_plan(plan, plan_path)
rows, _ = review_plan(plan)
save_review_csv(rows, csv_path)
save_review_csv(rows, other_csv)
updated = apply_review_to_plan(load_plan(plan_path), csv_path)
save_plan(updated, plan_path)
errors = check_review_requirements(load_plan(plan_path), plan_path, other_csv)
assert any("does not match" in e for e in errors)
def test_planner_noop_manual_review_does_not_block_execute_gate(tmp_path): def test_planner_noop_manual_review_does_not_block_execute_gate(tmp_path):
plan_path = tmp_path / "plan.json" plan_path = tmp_path / "plan.json"
op = FileOperation( op = FileOperation(
+86
View File
@@ -63,6 +63,92 @@ def test_generate_plan_for_movie_with_year(config):
assert "Some Movie (2020)" in operation.reason assert "Some Movie (2020)" in operation.reason
def test_generate_plan_attaches_sidecar_context(tmp_path):
"""Test move operations carry review-visible sidecar associations."""
config = Config(
library_root=tmp_path,
video_extensions=[".mp4", ".mkv", ".avi"],
movie_template="movie/{title} ({year})/",
movie_filename_template="{title} ({year}){ext}",
)
src_dir = tmp_path / "downloads"
src_dir.mkdir()
video = src_dir / "Movie (2020).mkv"
video.touch()
sub = src_dir / "Movie (2020).zh.srt"
sub.touch()
nfo = src_dir / "Movie (2020).nfo"
nfo.touch()
unrelated = src_dir / "Movie (2020).1.srt"
unrelated.touch()
video_file = VideoFile(
path=video,
filename=video.name,
size_bytes=1000000,
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
identity = MovieIdentity(
title="Movie",
year=2020,
confidence=0.9,
needs_review=False,
original_filename=video.name,
)
plan = generate_plan([(video_file, identity)], config)
operation = plan.operations[0]
assert operation.operation_type == "move"
sidecars = operation.review_context.get("sidecars")
assert sidecars is not None
assert [s["name"] for s in sidecars] == ["Movie (2020).nfo", "Movie (2020).zh.srt"]
dest_dir = operation.destination_path.parent
by_name = {s["name"]: s for s in sidecars}
assert by_name["Movie (2020).zh.srt"]["proposed_destination_path"] == str(dest_dir / "Movie (2020).zh.srt")
assert by_name["Movie (2020).nfo"]["proposed_destination_path"] == str(dest_dir / "Movie (2020).nfo")
# Numeric-suffix file must not be associated
assert all(s["name"] != "Movie (2020).1.srt" for s in sidecars)
def test_generate_plan_no_sidecars_for_noop(tmp_path):
"""Test no-op operations carry no sidecar context."""
config = Config(
library_root=tmp_path,
video_extensions=[".mp4", ".mkv", ".avi"],
movie_template="movie/{title} ({year})/",
movie_filename_template="{title} ({year}){ext}",
)
src_dir = tmp_path / "downloads"
src_dir.mkdir()
video = src_dir / "random_movie.mkv"
video.touch()
sub = src_dir / "random_movie.srt"
sub.touch()
video_file = VideoFile(
path=video,
filename=video.name,
size_bytes=1000000,
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
identity = MovieIdentity(
title="Random Movie",
year=None,
confidence=0.3,
needs_review=True,
original_filename=video.name,
)
plan = generate_plan([(video_file, identity)], config)
operation = plan.operations[0]
assert operation.operation_type == "no-op"
assert "sidecars" not in (operation.review_context or {})
def test_generate_plan_for_movie_without_year(config): def test_generate_plan_for_movie_without_year(config):
"""Test plan generation for a movie without year (needs review).""" """Test plan generation for a movie without year (needs review)."""
video_file = VideoFile( video_file = VideoFile(
+16 -9
View File
@@ -82,19 +82,26 @@ class TestQuarantineManager:
assert expected_quarantine_path.exists() assert expected_quarantine_path.exists()
assert expected_quarantine_path.read_text() == "test content" assert expected_quarantine_path.read_text() == "test content"
def test_quarantine_anime_file_rejected(self, manager, config): def test_quarantine_anime_file_round_trip(self, manager, config):
"""Test that quarantining anime files returns a failed result.""" """Test that quarantining and restoring anime files works."""
# Create a test anime file # Create a test anime file
anime_file = config.library_root / "anime" / "Test Anime.mkv" anime_file = config.library_root / "anime" / "Test Anime.mkv"
anime_file.write_text("test content") anime_file.write_text("test content")
result = manager.quarantine_file(anime_file) result = manager.quarantine_file(anime_file)
assert result.success is False assert result.success is True
assert "Quarantine not supported for category 'anime'" in (result.error_message or "") expected_quarantine_path = config.library_root / "anime" / ".quarantine" / "Test Anime.mkv"
assert not anime_file.exists()
assert expected_quarantine_path.exists()
assert expected_quarantine_path.read_text() == "test content"
# Verify file was not moved # Restore and verify round trip
restore_result = manager.restore_from_quarantine(expected_quarantine_path)
assert restore_result.success is True
assert anime_file.exists() assert anime_file.exists()
assert anime_file.read_text() == "test content"
assert not expected_quarantine_path.exists()
def test_quarantine_other_file_rejected(self, manager, config): def test_quarantine_other_file_rejected(self, manager, config):
"""Test that quarantining other files returns a failed result.""" """Test that quarantining other files returns a failed result."""
@@ -590,7 +597,7 @@ class TestQuarantineListing:
def test_list_quarantined_invalid_category(self, manager, config): def test_list_quarantined_invalid_category(self, manager, config):
"""Test listing with invalid category returns empty list.""" """Test listing with invalid category returns empty list."""
entries = manager.list_quarantined(category="anime") entries = manager.list_quarantined(category="unconfigured")
assert entries == [] assert entries == []
entries = manager.list_quarantined(category="other") entries = manager.list_quarantined(category="other")
@@ -906,7 +913,7 @@ class TestQuarantineRestoration:
category = manager._determine_category_from_quarantine(outside) category = manager._determine_category_from_quarantine(outside)
assert category is None assert category is None
# Unsupported category # Unsupported category (directory not in configured categories)
anime = config.library_root / "anime" / ".quarantine" / "Anime.mkv" other = config.library_root / "other" / ".quarantine" / "Other.mkv"
category = manager._determine_category_from_quarantine(anime) category = manager._determine_category_from_quarantine(other)
assert category is None assert category is None
BIN
View File
Binary file not shown.