refactor CLI command modules and synchronize docs

This commit is contained in:
windyboy
2026-02-16 12:31:26 +08:00
parent 0be802eac5
commit 2dd329cba9
41 changed files with 1066 additions and 753 deletions
+6 -1
View File
@@ -2,7 +2,7 @@
## Project Structure & Module Organization
- Core package lives in `src/vlm/`.
- CLI entrypoint is `src/vlm/cli.py` (`vlm` console script). Commands use `pass_context` and `CLIContext` from `context.py`; some command logic lives in `commands/` (e.g. `scan`, `analyze`, `plan`).
- CLI entrypoint is `src/vlm/cli.py` (`vlm` console script). Commands use `pass_context` and `CLIContext` from `context.py`; command logic is modularized in `commands/` (e.g. `scan`, `parse`, `enrich`, `analyze`, `plan`, `execute`).
- Functional modules by concern: scanning (`scanner.py`), parsing (`parser.py`), enrichment (`enrichment.py`, `cache.py`, `providers/`), analysis/planning/execution (`analysis.py`, `planner.py`, `executor.py`), I/O helpers (`io.py`), utilities (`utils.py`), state/reporting/logging (`state.py`, `reports.py`, `logging_config.py`), config (`config.py`), models (`models.py`).
- Tests live in `tests/` and mirror feature areas (e.g. `tests/test_scanner.py`, `tests/test_cli_state.py`, `tests/test_enrichment.py`).
- Project metadata and tool config are in `pyproject.toml`.
@@ -36,3 +36,8 @@
## Security & Configuration Tips
- Do not commit local paths, personal media metadata, or generated state/log artifacts.
- Validate config changes against `vlm --help` and at least one end-to-end CLI flow before merging.
## Documentation baseline
- Updated to reflect refactor results as of 2026-02-16.
- Canonical release notes are tracked in `CHANGELOG.md`.
+3
View File
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# Video Library Manager - Architecture Review
**Review Date**: 2026-02-13
+3
View File
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# Post-Audit Fixes Implementation Plan (2026-02-13)
## Objective
+38
View File
@@ -0,0 +1,38 @@
# Changelog
## 2026-02-16
### Refactor Results
- Modularized CLI command implementations:
- Added `src/vlm/commands/parse.py`
- Added `src/vlm/commands/enrich.py`
- Added `src/vlm/commands/execute.py`
- Refactored `src/vlm/cli.py` to delegate `parse`, `enrich`, `execute`, and `rollback` to command modules
- Unified I/O layer for JSON handling:
- Added `load_json_file`, `save_json_file`, `save_analysis_json` in `src/vlm/io.py`
- Updated analyze/enrich paths to use unified I/O entry points
- Parser/config consistency improvements:
- Added shared `DEFAULT_VIDEO_EXTENSIONS` in `src/vlm/config.py`
- Updated `src/vlm/parser.py` to use the shared config constant instead of local hardcoded defaults
- Execution reliability hardening:
- Updated `src/vlm/executor.py` transaction logging to fail gracefully when `~/.vlm` is not writable (warn + continue)
- Test stability improvements:
- Standardized test timestamps to timezone-aware UTC (`datetime.now(timezone.utc)`)
- Added missing `timezone` imports where required
### Verification
- Full test suite passed after refactor:
- `477 passed`
### Documentation Sync
- Updated repository documentation set to match the refactor baseline:
- Core docs: `README.md`, `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`
- Skill docs: `skills/vlm-library-workflow/SKILL.md` and all references under `skills/vlm-library-workflow/references/`
- Legacy review/plan docs now explicitly marked as historical snapshots and redirected to `CHANGELOG.md` as the current source of truth
+5 -2
View File
@@ -1,5 +1,8 @@
# CLAUDE.md
## Documentation Status
- Synchronized with refactor baseline on 2026-02-16 (see `CHANGELOG.md`).
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
@@ -91,11 +94,11 @@ VLM follows a read-first, multi-stage pipeline:
### Module Organization
- `cli.py` - Click-based CLI interface, global options, command registration
- `context.py` - CLIContext (config, paths) and pass_context for commands
- `commands/` - Command implementations (scan, analyze, plan)
- `commands/` - Command implementations (scan, parse, enrich, analyze, plan, execute/rollback)
- `scanner.py` - File discovery using system `find` command, metadata extraction via ffprobe
- `parser.py` - Filename parsing using regex patterns (movies: title + year, series: SxxExx)
- `enrichment.py` - Enrichment pipeline; `cache.py` - SQLite cache; `providers/` - TMDB etc.
- `io.py` - Load/save identities JSON/CSV, plan/analysis input helpers
- `io.py` - Unified JSON/CSV I/O helpers (including analysis writer and data adapters)
- `utils.py` - UTC time, format_size, shared helpers
- `analysis.py` - Completeness checking (episode gaps) and duplicate detection
- `duplicate_resolve.py` - Duplicate group resolution (by_quality, by_reputation, first_seen, manual)
+3
View File
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# VLM 代码改进清单
本文档记录对 Video Library Manager (VLM) 项目的代码审查发现的问题及对应解决方案。排除 AI/OpenAPI 相关问题。
+3
View File
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# Fix Plan (Verified Issues Only)
## Objective
+13 -10
View File
@@ -1,5 +1,8 @@
# GEMINI.md
## Documentation Status
- Synced with repository refactor baseline on 2026-02-16 (source of truth: `CHANGELOG.md`).
This document provides a comprehensive overview of the Video Library Manager (VLM) project, intended to be used as instructional context for Gemini.
## Project Overview
@@ -48,19 +51,19 @@ The project uses `uv` for dependency management.
The main entry point is the `vlm` command.
* Initialize configuration: `vlm config init`
* Scan the library: `vlm scan`
* Parse filenames: `vlm parse`
* Enrich metadata: `vlm enrich`
* Analyze the library: `vlm analyze`
* Generate a plan: `vlm plan`
* Execute the plan (dry-run): `vlm execute`
* Execute the plan (with confirmation): `vlm execute --confirm`
* Rollback the last execution: `vlm rollback`
* Initialize configuration: `uv run vlm config init`
* Scan the library: `uv run vlm scan`
* Parse filenames: `uv run vlm parse`
* Enrich metadata: `uv run vlm enrich`
* Analyze the library: `uv run vlm analyze`
* Generate a plan: `uv run vlm plan`
* Execute the plan (dry-run): `uv run vlm execute`
* Execute the plan (with confirmation): `uv run vlm execute --confirm`
* Rollback the last execution: `uv run vlm rollback`
**Running tests:**
* Run all tests: `pytest`
* Run all tests: `uv run pytest`
## Development Conventions
+3
View File
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# VLM 改进实施计划(2026-02-13
## 目标
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# VLM 项目与 Skill 改进建议(2026-02-13
## 1. 评估范围与依据
+12 -2
View File
@@ -1,5 +1,12 @@
# Video Library Manager
## Documentation Status
- Last synchronized: **2026-02-16**
- CLI command refactor landed (`parse`, `enrich`, `execute`, `rollback` logic moved to `src/vlm/commands/`).
- Unified JSON I/O interfaces are available in `src/vlm/io.py`.
- Full test baseline after refactor: **477 passed**.
A Python-based CLI tool for managing personal video collections with a safety-first, human-in-the-loop approach.
## Features
@@ -806,8 +813,11 @@ src/vlm/
├── context.py # CLIContext and pass_context for commands
├── commands/ # Command implementations
│ ├── scan.py # Scan command
│ ├── parse.py # Parse command
│ ├── enrich.py # Enrich command
│ ├── analyze.py # Analyze command
── plan.py # Plan command
── plan.py # Plan command
│ └── execute.py # Execute/Rollback commands
├── scanner.py # File discovery and metadata extraction
├── parser.py # Filename parsing (titles, years, episodes)
├── enrichment.py # Title/reputation enrichment pipeline
@@ -815,7 +825,7 @@ src/vlm/
├── providers/ # External metadata providers (TMDB, etc.)
│ ├── base.py # Provider interface
│ └── tmdb.py # TMDB API client
├── io.py # JSON/CSV load/save, load_analysis_json, plan/analysis input helpers
├── io.py # Unified JSON/CSV I/O helpers (load/save JSON, analysis writer, plan/analysis adapters)
├── utils.py # UTC time, format_size, etc.
├── analysis.py # Completeness and duplicate detection
├── duplicate_resolve.py # Duplicate group keep-index (by_quality, by_reputation, by_reputation_quality_time, first_seen, manual)
+3
View File
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# Code Review Report (Verified)
## Scope
+3
View File
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# TMDB Enrichment 重构执行计划
## 1. 目标与范围
+3
View File
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# Video Library Manager (VLM) 深度审计报告
**报告版本**1.0
+3
View File
@@ -1,3 +1,6 @@
> [!NOTE]
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
# Codex 架构复核报告
评审日期: 2026-02-13
+3
View File
@@ -1,3 +1,6 @@
# Documentation Status
- Synced with refactor baseline on 2026-02-16.
---
name: vlm-library-workflow
description: Operate and extend the Video Library Manager (`vlm`) with a safety-first, human-in-the-loop workflow across scan, parse, enrich, analyze, plan, review-plan, execute, rollback, and developer verification. Use when requests involve organizing a video library, producing or reviewing `inventory.csv`/`identities.json`/`analysis.json`/`plan.json`, tuning VLM config templates, resolving duplicates or episode gaps, running dry-run/confirm execution, recovering changes via rollback, explaining VLM CLI usage, or developing/modifying VLM features (parser, providers, planner, executor, commands, tests).
@@ -1,3 +1,6 @@
# Documentation Status
- Updated to current CLI options on 2026-02-16.
# VLM CLI Reference
## Configuration
@@ -11,7 +14,7 @@
- `vlm enrich [--input JSON] [--output JSON] [--refresh-all]`: Fetch TMDB metadata.
- `vlm analyze [--input JSON] [--output JSON]`: Find gaps and duplicates.
- `vlm plan [--input JSON] [--analysis JSON] [--output JSON]`: Generate operations.
- `vlm execute [--plan JSON] [--confirm]`: Move/Rename files.
- `vlm execute [--plan JSON] [--confirm] [--yes] [--verbose-ops] [--safe-mode] [--preserve-directories]`: Move/Rename/Quarantine operations with safety guards.
- `vlm rollback [--log PATH]`: Undo operations.
## Management & Reporting
@@ -22,5 +25,5 @@
## Important Config Options (`~/.vlm/config.yaml`)
- `library_root`: Path to the video collection.
- `templates`: Naming patterns for movies and series.
- `plan.duplicate_keep`: Strategy for duplicates (`by_quality`, `by_reputation`, `first_seen`, `manual`).
- `plan.duplicate_keep`: Strategy for duplicates (`by_quality`, `by_reputation`, `by_reputation_quality_time`, `first_seen`, `manual`).
- `enrichment.api_keys`: TMDB and OpenAI keys.
@@ -1,3 +1,6 @@
# Documentation Status
- Synced with refactor baseline on 2026-02-16.
# VLM Command Recipes
## Baseline
@@ -1,7 +1,10 @@
# Documentation Status
- Updated for the modular command architecture on 2026-02-16.
# VLM Developer Guide
## Project Structure
- `src/vlm/cli.py`: Entry point and command definitions.
- `src/vlm/cli.py`: Entry point and thin command wrappers.
- `src/vlm/parser.py`: Regex-based filename parsing logic.
- `src/vlm/enrichment.py`: Pipeline for external metadata fetching.
- `src/vlm/providers/`: API implementations (e.g., TMDB).
@@ -11,7 +14,7 @@
## Adding a New Command
1. Create a new module in `src/vlm/commands/`.
2. Define the command using `@click.command()`.
3. Register it in `src/vlm/cli.py` using `main.add_command()`.
3. Register it in `src/vlm/cli.py` with a Click-decorated function that delegates to the module implementation.
## Modifying the Parser
- The parser uses a sequence of regex patterns in `src/vlm/parser.py`.
@@ -21,8 +24,8 @@
## Data Models
See `src/vlm/models.py` for core data structures:
- `VideoFile`: Basic file metadata.
- `MediaIdentity`: Parsed and enriched information.
- `PlanOperation`: Definition of a file move/rename/quarantine.
- `MovieIdentity` / `SeriesIdentity`: Parsed/enriched identity records.
- `FileOperation`: Definition of a move/rename/quarantine/no-op/preserve-directory operation.
## Testing
- **Unit Tests**: `pytest`
@@ -1,3 +1,6 @@
# Documentation Status
- Synced with refactor baseline on 2026-02-16.
# VLM Workflow Guide
This guide details the standard end-to-end process for organizing a video library using VLM.
+55 -550
View File
@@ -193,180 +193,20 @@ def parse(ctx: CLIContext, input: Path, output: Path, inventory: Optional[Path])
vlm parse --input my_inventory.csv # Custom input
vlm parse --output parsed_identities.json # Custom output
"""
from datetime import datetime, timezone
from vlm.parser import parse_movie, parse_series
from vlm.io import load_inventory_csv, save_identities_json
config = ctx.config
logger = ctx.logger
try:
# Display parse start message
click.echo(f"Parsing identities from: {input}")
# Load video metadata from inventory if provided
path_to_metadata = {}
if inventory:
click.echo(f"Loading video metadata from: {inventory}")
inventory_files = load_inventory_csv(inventory)
path_to_metadata = {str(vf.path): vf for vf in inventory_files}
click.echo(f"Loaded metadata for {len(path_to_metadata)} files")
click.echo()
# Load inventory via unified I/O layer
inventory_files = load_inventory_csv(input)
video_files = [
{
'path': str(vf.path),
'filename': vf.filename,
'category': vf.category,
}
for vf in inventory_files
]
click.echo(f"Loaded {len(video_files)} files from inventory")
click.echo()
# Parse identities based on category
movie_identities = []
series_identities = []
anime_files = []
other_files = []
def get_video_metadata(file_path: str) -> dict:
"""Extract video metadata from inventory if available."""
if not path_to_metadata:
return {}
vf = path_to_metadata.get(file_path)
if not vf:
return {}
return {
'size_bytes': vf.size_bytes,
'modified_timestamp': vf.modified_timestamp.isoformat(),
'resolution': vf.resolution,
'codec': vf.codec,
'duration_seconds': vf.duration_seconds,
'bitrate_kbps': vf.bitrate_kbps,
}
for vf in video_files:
filename = vf['filename']
category = vf['category']
file_path = vf['path']
video_metadata = get_video_metadata(file_path)
if category == 'movie':
identity = parse_movie(filename, extensions=config.video_extensions)
record = {
'path': file_path,
'filename': filename,
'category': category,
'title': identity.title,
'year': identity.year,
'confidence': identity.confidence,
'needs_review': identity.needs_review
}
if video_metadata:
record['video_metadata'] = video_metadata
movie_identities.append(record)
elif category == 'series':
identity = parse_series(filename, extensions=config.video_extensions)
record = {
'path': file_path,
'filename': filename,
'category': category,
'title': identity.title,
'season': identity.season,
'episodes': identity.episodes,
'confidence': identity.confidence,
'needs_review': identity.needs_review
}
if video_metadata:
record['video_metadata'] = video_metadata
series_identities.append(record)
elif category == 'anime':
# Anime files are not parsed in v1
anime_files.append({
'path': vf['path'],
'filename': filename,
'category': category,
'note': 'Anime parsing deferred in v1'
})
else:
# Other files are not parsed
other_files.append({
'path': vf['path'],
'filename': filename,
'category': category,
'note': 'Not categorized for parsing'
})
# Display parsing statistics
click.echo("Parsing complete!")
click.echo()
click.echo("Results by category:")
click.echo(f" Movies: {len(movie_identities)}")
# Count movies needing review
movies_need_review = sum(1 for m in movie_identities if m['needs_review'])
if movies_need_review > 0:
click.echo(f" - Need review: {movies_need_review}")
click.echo(f" Series: {len(series_identities)}")
# Count series needing review
series_need_review = sum(1 for s in series_identities if s['needs_review'])
if series_need_review > 0:
click.echo(f" - Need review: {series_need_review}")
click.echo(f" Anime: {len(anime_files)} (not parsed in v1)")
click.echo(f" Other: {len(other_files)} (not parsed)")
# Save parsed identities to JSON
click.echo()
click.echo(f"Saving parsed identities to: {output}")
# Ensure output directory exists
output.parent.mkdir(parents=True, exist_ok=True)
# Build JSON structure
generation_timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
# Use v2 schema if video metadata was embedded
schema_version = "2.0" if path_to_metadata else "1.0"
identities_data = {
'vlm_schema_version': schema_version,
'metadata': {
'generated': generation_timestamp,
'source_inventory': str(input),
'total_files': len(video_files)
},
'movies': movie_identities,
'series': series_identities,
'anime': anime_files,
'other': other_files
}
# Write JSON via unified I/O layer
save_identities_json(identities_data, output)
click.echo(f"Parsed identities saved successfully!")
logger.info(f"Parse completed: {len(movie_identities)} movies, {len(series_identities)} series, saved to {output}")
from vlm.commands.parse import parse_cmd
parse_cmd(ctx, input, output, inventory)
except FileNotFoundError:
click.echo(f"Error: Input file not found: {input}", err=True)
logger.error(f"Input file not found: {input}")
ctx.logger.error(f"Input file not found: {input}")
sys.exit(1)
except Exception as e:
click.echo(f"Error during parsing: {e}", err=True)
logger.error(f"Parse failed: {e}", exc_info=True)
except ValueError as e:
click.echo(f"Error: {e}", err=True)
ctx.logger.error(f"Parse failed: {e}")
sys.exit(1)
except OSError as e:
click.echo(f"Error reading/writing files: {e}", err=True)
ctx.logger.error(f"Parse file I/O failed: {e}", exc_info=True)
sys.exit(1)
@@ -424,150 +264,32 @@ def enrich(
Applies incremental cache-backed enrichment to parsed identities and writes
results back into identities JSON.
"""
import json
from vlm.enrichment import enrich_identities_data
config = ctx.config
logger = ctx.logger
if output is None:
output = input
try:
click.echo(f"Enriching identities from: {input}")
click.echo(f"Output file: {output}")
click.echo()
with open(input, 'r', encoding='utf-8') as jsonfile:
identities_data = json.load(jsonfile)
if refresh_all and refresh_changed_only:
click.echo("Error: --refresh-all and --refresh-changed-only are mutually exclusive.", err=True)
sys.exit(1)
if timeout < 1:
click.echo("Error: --timeout must be >= 1", err=True)
sys.exit(1)
if retries < 0:
click.echo("Error: --retries must be >= 0", err=True)
sys.exit(1)
refresh_mode = "incremental"
if refresh_all:
refresh_mode = "refresh_all"
elif refresh_changed_only:
refresh_mode = "refresh_changed_only"
total = (
len(identities_data.get('movies', []))
+ len(identities_data.get('series', []))
+ len(identities_data.get('anime', []))
)
progress_state: dict[str, Optional[click.ProgressBar]] = {"bar": None}
progress_position = {"current": 0}
progress_bucket = {"value": -1}
is_tty = bool(getattr(sys.stderr, "isatty", lambda: False)())
def _enrich_progress(processed: int, total_count: int, metrics: dict[str, int]) -> None:
if total_count <= 0:
return
if is_tty and progress_state["bar"] is None:
bar = click.progressbar(
length=total_count,
label="Enriching records",
show_pos=True,
)
progress_state["bar"] = bar.__enter__()
step = processed - progress_position["current"]
if step > 0 and progress_state["bar"] is not None:
progress_state["bar"].update(step)
progress_position["current"] = processed
if not is_tty:
percent = int(processed * 100 / total_count)
bucket = percent // 5
if bucket > progress_bucket["value"] or processed == total_count:
progress_bucket["value"] = bucket
click.echo(
"Progress: "
f"{processed}/{total_count} ({percent}%) "
f"api_calls={metrics.get('api_calls', 0)} "
f"cache_hits={metrics.get('cache_hits', 0)} "
f"failed={metrics.get('failed', 0)}"
)
try:
if total == 0:
click.echo("No movie/series/anime records found to enrich.")
enriched_data, stats = enrich_identities_data(
identities_data,
config,
refresh_mode=refresh_mode,
request_timeout=timeout,
retries=retries,
logger=logger,
progress_callback=_enrich_progress,
)
finally:
if progress_state["bar"] is not None:
progress_state["bar"].__exit__(None, None, None)
click.echo()
output.parent.mkdir(parents=True, exist_ok=True)
with open(output, 'w', encoding='utf-8') as jsonfile:
json.dump(enriched_data, jsonfile, indent=2, ensure_ascii=False)
total_records = int(stats['total']) if stats['total'] else 0
cache_hits = int(stats['cache_hits'])
hit_rate = (cache_hits / total_records * 100.0) if total_records else 0.0
click.echo("Enrichment complete!")
click.echo(f" Total records: {total_records}")
click.echo(f" Refresh mode: {refresh_mode}")
click.echo(f" Enriched now: {stats['enriched']}")
click.echo(f" Cache hits: {stats['cache_hits']}")
click.echo(f" Cache hit rate: {hit_rate:.1f}%")
click.echo(f" API calls: {stats['api_calls']}")
click.echo(f" Failed requests: {stats['failed']}")
click.echo(f" Skipped: {stats['skipped']}")
click.echo(f" Needs review: {stats['needs_review']}")
skip_reasons = stats.get("skip_reasons", {})
if isinstance(skip_reasons, dict):
non_zero = [f"{name}={count}" for name, count in sorted(skip_reasons.items()) if int(count) > 0]
if non_zero:
click.echo(f" Skip reasons: {' '.join(non_zero)}")
failed_items = stats.get('failed_items', [])
if isinstance(failed_items, list) and failed_items:
click.echo(" Failure sample:")
for item in failed_items[:3]:
click.echo(
f" - [{item.get('provider', 'unknown')}] {item.get('title', '')}: {item.get('reason', '')}"
)
logger.info(
"Enrich completed: total=%s enriched=%s cache_hits=%s failed=%s skipped=%s mode=%s",
stats['total'],
stats['enriched'],
stats['cache_hits'],
stats['failed'],
stats['skipped'],
refresh_mode,
from vlm.commands.enrich import enrich_cmd
enrich_cmd(
ctx,
input,
output,
refresh_changed_only,
refresh_all,
timeout,
retries,
)
except FileNotFoundError:
click.echo(f"Error: Input file not found: {input}", err=True)
logger.error(f"Input file not found: {input}")
ctx.logger.error(f"Input file not found: {input}")
sys.exit(1)
except json.JSONDecodeError as e:
click.echo(f"Error: Failed to parse JSON file: {e}", err=True)
logger.error(f"JSON parsing failed during enrich: {e}", exc_info=True)
ctx.logger.error(f"JSON parsing failed during enrich: {e}", exc_info=True)
sys.exit(1)
except Exception as e:
click.echo(f"Error during enrichment: {e}", err=True)
logger.error(f"Enrich failed: {e}", exc_info=True)
except ValueError as e:
click.echo(f"Error: {e}", err=True)
ctx.logger.error(f"Enrich validation failed: {e}")
sys.exit(1)
except OSError as e:
click.echo(f"Error reading/writing files: {e}", err=True)
ctx.logger.error(f"Enrich file I/O failed: {e}", exc_info=True)
sys.exit(1)
@@ -791,8 +513,20 @@ def review_plan_cmd(
default=False,
help='Print per-operation dry-run logs at INFO level'
)
@click.option(
'--preserve-directories',
is_flag=True,
default=False,
help='Preserve empty source directories instead of allowing them to be destroyed'
)
@click.option(
'--safe-mode',
is_flag=True,
default=False,
help='Enable safe mode: prevent any operations that would destroy directories'
)
@pass_context
def execute(ctx: CLIContext, plan: Path, confirm: bool, yes: bool, verbose_ops: bool):
def execute(ctx: CLIContext, plan: Path, confirm: bool, yes: bool, verbose_ops: bool, preserve_directories: bool, safe_mode: bool):
"""Execute plan (defaults to dry-run, requires --confirm).
Executes file operations from a plan. Defaults to dry-run mode which
@@ -806,155 +540,20 @@ def execute(ctx: CLIContext, plan: Path, confirm: bool, yes: bool, verbose_ops:
vlm execute --confirm # Actually execute operations (with prompt)
vlm execute --confirm --yes # Execute without confirmation prompt
"""
from vlm.planner import load_plan
from vlm.executor import ExecutionEngine
config = ctx.config
logger = ctx.logger
try:
# Determine execution mode
mode = "execute" if confirm else "dry-run"
# Display execution start message
click.echo(f"Loading execution plan from: {plan}")
click.echo()
# Load execution plan
execution_plan = load_plan(plan)
# Display plan summary
click.echo(f"Execution plan loaded: {execution_plan.plan_id}")
click.echo(f"Created at: {execution_plan.created_at}")
click.echo(f"Total operations: {len(execution_plan.operations)}")
if execution_plan.human_summary:
click.echo()
click.echo(execution_plan.human_summary)
elif execution_plan.summary or execution_plan.summary_by_reason:
s = execution_plan.summary or {}
by_r = execution_plan.summary_by_reason or {}
parts = [f"操作统计:共 {s.get('total', len(execution_plan.operations))} 条(move {s.get('move', 0)}rename {s.get('rename', 0)}quarantine {s.get('quarantine', 0)}no-op {s.get('no-op', 0)}"]
if by_r:
parts.append("原因分布:" + "".join(f"{r}: {c}" for r, c in list(by_r.items())[:5]))
click.echo()
click.echo("\n".join(parts))
click.echo()
# Display mode warning
if mode == "dry-run":
click.echo("⚠️ DRY-RUN MODE - No files will be modified")
click.echo(" Use --confirm to actually execute operations")
else:
click.echo("⚠️ EXECUTE MODE - Files will be modified!")
click.echo(" This operation cannot be undone without rollback")
click.echo()
if not yes:
if not click.confirm("Are you sure you want to proceed?"):
click.echo("Execution cancelled.")
return
else:
click.echo("Auto-approved via --yes flag")
click.echo()
click.echo(f"Executing {len(execution_plan.operations)} operations...")
click.echo()
# Load state manager to update file statuses during execution
from vlm.state import StateManager
state_path = Path.home() / ".vlm" / "state.json"
state_manager = StateManager(state_path)
# Create execution engine and execute plan
engine = ExecutionEngine(
logger=logger,
config=config,
verbose_operations=verbose_ops,
state_manager=state_manager
)
results, summary, rollback_log = engine.execute_plan(
execution_plan,
mode=mode,
confirmed=confirm
)
# Display execution progress (show some operations)
if mode == "dry-run":
click.echo("Sample operations (dry-run):")
# Show first 5 operations as examples
for i, result in enumerate(results[:5]):
op = result.operation
if op.operation_type != "no-op":
click.echo(f" [{i+1}] {op.operation_type}: {op.source_path.name}")
if op.destination_path:
click.echo(f" -> {op.destination_path}")
if len(results) > 5:
click.echo(f" ... and {len(results) - 5} more operations")
else:
# In execute mode, show progress for all operations
for i, result in enumerate(results):
op = result.operation
if op.operation_type != "no-op" and not op.has_conflict:
status = "" if result.success else ""
click.echo(f" [{i+1}/{len(results)}] {status} {op.operation_type}: {op.source_path.name}")
if result.error_message:
click.echo(f" Error: {result.error_message}")
# Display execution summary
click.echo()
click.echo("=" * 60)
click.echo(f"Execution Summary ({mode} mode)")
click.echo("=" * 60)
click.echo(f" Total operations: {summary['total']}")
click.echo(f" Successful: {summary['successful']}")
click.echo(f" Failed: {summary['failed']}")
click.echo(f" Skipped: {summary['skipped']}")
click.echo()
# Save rollback log if in execute mode
if mode == "execute" and rollback_log:
# Save to ~/.vlm/rollback/ directory
rollback_dir = Path.home() / ".vlm" / "rollback"
rollback_dir.mkdir(parents=True, exist_ok=True)
rollback_path = rollback_dir / f"rollback_{rollback_log.log_id}.json"
engine.save_rollback_log(rollback_log, rollback_path)
click.echo(f"Rollback log saved to: {rollback_path}")
click.echo()
click.echo("To undo these operations, run:")
click.echo(f" vlm rollback --log {rollback_path}")
click.echo()
# Log completion
if mode == "dry-run":
click.echo("Dry-run complete! No files were modified.")
click.echo("Review the operations above and use --confirm to execute.")
else:
if summary['failed'] > 0:
click.echo(f"⚠️ Execution completed with {summary['failed']} failures.")
click.echo(" Check the log file for details.")
else:
click.echo("✓ Execution completed successfully!")
logger.info(
f"Execution completed in {mode} mode: "
f"{summary['successful']} successful, {summary['failed']} failed, {summary['skipped']} skipped"
)
from vlm.commands.execute import execute_cmd
execute_cmd(ctx, plan, confirm, yes, verbose_ops, preserve_directories, safe_mode)
except FileNotFoundError:
click.echo(f"Error: Plan file not found: {plan}", err=True)
logger.error(f"Plan file not found: {plan}")
click.echo(f"Error: File not found: {plan}", err=True)
ctx.logger.error(f"Execution file not found: {plan}")
sys.exit(1)
except ValueError as e:
click.echo(f"Error: {e}", err=True)
logger.error(f"Execution failed: {e}")
ctx.logger.error(f"Execution validation failed: {e}")
sys.exit(1)
except Exception as e:
except OSError as e:
click.echo(f"Error during execution: {e}", err=True)
logger.error(f"Execution failed: {e}", exc_info=True)
ctx.logger.error(f"Execution I/O failed: {e}", exc_info=True)
sys.exit(1)
@@ -1177,114 +776,20 @@ def rollback(ctx: CLIContext, log: Optional[Path]):
vlm rollback --log rollback_<uuid>.json # Use specific log
vlm rollback --log ~/.vlm/rollback/rollback_*.json
"""
from vlm.executor import ExecutionEngine
config = ctx.config
logger = ctx.logger
try:
# If no log specified, find the most recent rollback log
if log is None:
rollback_dir = Path.home() / ".vlm" / "rollback"
if not rollback_dir.exists():
click.echo("Error: No rollback logs found.", err=True)
click.echo(f"Rollback directory does not exist: {rollback_dir}", err=True)
sys.exit(1)
# Find all rollback log files
rollback_logs = sorted(rollback_dir.glob("rollback_*.json"), key=lambda p: p.stat().st_mtime, reverse=True)
if not rollback_logs:
click.echo("Error: No rollback logs found.", err=True)
click.echo(f"No rollback_*.json files in: {rollback_dir}", err=True)
sys.exit(1)
# Use the most recent log
log = rollback_logs[0]
click.echo(f"Using most recent rollback log: {log}")
click.echo()
# Display rollback start message
click.echo(f"Loading rollback log from: {log}")
click.echo()
# Create execution engine
engine = ExecutionEngine(logger=logger, config=config)
# Load rollback log
rollback_log = engine.load_rollback_log(log)
# Display rollback log info
click.echo(f"Rollback log loaded: {rollback_log.log_id}")
click.echo(f"Original execution: {rollback_log.execution_plan_id}")
click.echo(f"Executed at: {rollback_log.executed_at.strftime('%Y-%m-%d %H:%M:%S')}")
click.echo(f"Operations to rollback: {len(rollback_log.operations)}")
click.echo()
# Display warning
click.echo("⚠️ ROLLBACK OPERATION - Best-effort restoration")
click.echo(" This will attempt to move files back to their original locations.")
click.echo(" Some operations may fail if files have been modified or moved.")
click.echo()
if not click.confirm("Are you sure you want to proceed with rollback?"):
click.echo("Rollback cancelled.")
return
click.echo()
click.echo(f"Rolling back {len(rollback_log.operations)} operations...")
click.echo()
# Perform rollback
results, summary = engine.rollback(rollback_log)
# Display rollback progress
for i, result in enumerate(results):
op = result.operation
if op.operation_type != "no-op":
status = "" if result.success else ""
click.echo(f" [{i+1}/{len(results)}] {status} Rollback: {op.destination_path.name if op.destination_path else op.source_path.name}")
if result.error_message:
click.echo(f" Error: {result.error_message}")
# Display rollback summary
click.echo()
click.echo("=" * 60)
click.echo("Rollback Summary")
click.echo("=" * 60)
click.echo(f" Total operations: {summary['total']}")
click.echo(f" Successful: {summary['successful']}")
click.echo(f" Failed: {summary['failed']}")
click.echo(f" Skipped: {summary['skipped']}")
click.echo()
# Display completion message
if summary['failed'] > 0:
click.echo(f"⚠️ Rollback completed with {summary['failed']} failures.")
click.echo(" Check the log file for details.")
click.echo(" Some files may not have been restored to their original locations.")
else:
click.echo("✓ Rollback completed successfully!")
click.echo(" All files have been restored to their original locations.")
logger.info(
f"Rollback completed: "
f"{summary['successful']} successful, {summary['failed']} failed, {summary['skipped']} skipped"
)
from vlm.commands.execute import rollback_cmd
rollback_cmd(ctx, log)
except FileNotFoundError as e:
click.echo(f"Error: {e}", err=True)
logger.error(f"Rollback log not found: {e}")
ctx.logger.error(f"Rollback log not found: {e}")
sys.exit(1)
except ValueError as e:
click.echo(f"Error: Invalid rollback log format: {e}", err=True)
logger.error(f"Invalid rollback log: {e}")
click.echo(f"Error: {e}", err=True)
ctx.logger.error(f"Rollback failed: {e}")
sys.exit(1)
except Exception as e:
except OSError as e:
click.echo(f"Error during rollback: {e}", err=True)
logger.error(f"Rollback failed: {e}", exc_info=True)
ctx.logger.error(f"Rollback failed: {e}", exc_info=True)
sys.exit(1)
+14 -17
View File
@@ -1,6 +1,5 @@
"""Analyze command implementation."""
import json
from pathlib import Path
from typing import Optional
@@ -8,9 +7,13 @@ import click
from vlm.analysis import analyze_series_completeness, detect_duplicates
from vlm.context import CLIContext
from vlm.io import identities_to_analysis_input, load_identities_json, load_inventory_csv
from vlm.io import (
identities_to_analysis_input,
load_identities_json,
load_inventory_csv,
save_analysis_json,
)
from vlm.models import MovieIdentity, SeriesIdentity
from vlm.utils import utc_now
def analyze_cmd(
@@ -68,7 +71,6 @@ def analyze_cmd(
click.echo(f"Saving analysis results to: {output}")
output.parent.mkdir(parents=True, exist_ok=True)
generation_timestamp = utc_now().strftime("%Y-%m-%dT%H:%M:%S")
completeness_list = [
{
"series_title": c.series_title,
@@ -96,19 +98,14 @@ def analyze_cmd(
"quality_comparison": d.quality_comparison,
}
)
analysis_data = {
"vlm_schema_version": "1.0",
"metadata": {
"generated": generation_timestamp,
"source_identities": str(input),
"total_movies": len(movies_data),
"total_series": len(series_data),
},
"completeness": completeness_list,
"duplicates": duplicates_list,
}
with open(output, "w", encoding="utf-8") as jsonfile:
json.dump(analysis_data, jsonfile, indent=2, ensure_ascii=False)
save_analysis_json(
completeness=completeness_list,
duplicates=duplicates_list,
source_identities=input,
total_movies=len(movies_data),
total_series=len(series_data),
output=output,
)
click.echo("Analysis results saved successfully!")
logger.info(
+146
View File
@@ -0,0 +1,146 @@
"""Enrich command implementation."""
from __future__ import annotations
import sys
from pathlib import Path
from typing import Optional
import click
from vlm.context import CLIContext
from vlm.enrichment import enrich_identities_data
from vlm.io import load_json_file, save_json_file
def enrich_cmd(
ctx: CLIContext,
input: Path,
output: Optional[Path],
refresh_changed_only: bool,
refresh_all: bool,
timeout: int,
retries: int,
) -> None:
"""Enrich identities with translation and reputation metadata."""
logger = ctx.logger
config = ctx.config
if output is None:
output = input
click.echo(f"Enriching identities from: {input}")
click.echo(f"Output file: {output}")
click.echo()
identities_data = load_json_file(input)
if refresh_all and refresh_changed_only:
raise ValueError("--refresh-all and --refresh-changed-only are mutually exclusive.")
if timeout < 1:
raise ValueError("--timeout must be >= 1")
if retries < 0:
raise ValueError("--retries must be >= 0")
refresh_mode = "incremental"
if refresh_all:
refresh_mode = "refresh_all"
elif refresh_changed_only:
refresh_mode = "refresh_changed_only"
total = (
len(identities_data.get("movies", []))
+ len(identities_data.get("series", []))
+ len(identities_data.get("anime", []))
)
progress_state: dict[str, Optional[click.ProgressBar]] = {"bar": None}
progress_position = {"current": 0}
progress_bucket = {"value": -1}
is_tty = bool(getattr(sys.stderr, "isatty", lambda: False)())
def _enrich_progress(processed: int, total_count: int, metrics: dict[str, int]) -> None:
if total_count <= 0:
return
if is_tty and progress_state["bar"] is None:
bar = click.progressbar(
length=total_count,
label="Enriching records",
show_pos=True,
)
progress_state["bar"] = bar.__enter__()
step = processed - progress_position["current"]
if step > 0 and progress_state["bar"] is not None:
progress_state["bar"].update(step)
progress_position["current"] = processed
if not is_tty:
percent = int(processed * 100 / total_count)
bucket = percent // 5
if bucket > progress_bucket["value"] or processed == total_count:
progress_bucket["value"] = bucket
click.echo(
"Progress: "
f"{processed}/{total_count} ({percent}%) "
f"api_calls={metrics.get('api_calls', 0)} "
f"cache_hits={metrics.get('cache_hits', 0)} "
f"failed={metrics.get('failed', 0)}"
)
try:
if total == 0:
click.echo("No movie/series/anime records found to enrich.")
enriched_data, stats = enrich_identities_data(
identities_data,
config,
refresh_mode=refresh_mode,
request_timeout=timeout,
retries=retries,
logger=logger,
progress_callback=_enrich_progress,
)
finally:
if progress_state["bar"] is not None:
progress_state["bar"].__exit__(None, None, None)
click.echo()
save_json_file(enriched_data, output)
total_records = int(stats["total"]) if stats["total"] else 0
cache_hits = int(stats["cache_hits"])
hit_rate = (cache_hits / total_records * 100.0) if total_records else 0.0
click.echo("Enrichment complete!")
click.echo(f" Total records: {total_records}")
click.echo(f" Refresh mode: {refresh_mode}")
click.echo(f" Enriched now: {stats['enriched']}")
click.echo(f" Cache hits: {stats['cache_hits']}")
click.echo(f" Cache hit rate: {hit_rate:.1f}%")
click.echo(f" API calls: {stats['api_calls']}")
click.echo(f" Failed requests: {stats['failed']}")
click.echo(f" Skipped: {stats['skipped']}")
click.echo(f" Needs review: {stats['needs_review']}")
skip_reasons = stats.get("skip_reasons", {})
if isinstance(skip_reasons, dict):
non_zero = [f"{name}={count}" for name, count in sorted(skip_reasons.items()) if int(count) > 0]
if non_zero:
click.echo(f" Skip reasons: {' '.join(non_zero)}")
failed_items = stats.get("failed_items", [])
if isinstance(failed_items, list) and failed_items:
click.echo(" Failure sample:")
for item in failed_items[:3]:
click.echo(f" - [{item.get('provider', 'unknown')}] {item.get('title', '')}: {item.get('reason', '')}")
logger.info(
"Enrich completed: total=%s enriched=%s cache_hits=%s failed=%s skipped=%s mode=%s",
stats["total"],
stats["enriched"],
stats["cache_hits"],
stats["failed"],
stats["skipped"],
refresh_mode,
)
+257
View File
@@ -0,0 +1,257 @@
"""Execute and rollback command implementations."""
from __future__ import annotations
from pathlib import Path
from typing import Optional
import click
from vlm.context import CLIContext
from vlm.executor import ExecutionEngine
from vlm.planner import load_plan
from vlm.state import StateManager
def _validate_plan_structure(execution_plan) -> list[str]:
"""Validate source/destination paths for move/rename operations."""
validation_errors: list[str] = []
for operation in execution_plan.operations:
if operation.operation_type in ("move", "rename"):
if not operation.source_path.exists():
validation_errors.append(f"Source file does not exist: {operation.source_path}")
elif not operation.source_path.is_file():
validation_errors.append(f"Source path is not a file: {operation.source_path}")
for operation in execution_plan.operations:
if operation.operation_type in ("move", "rename") and operation.destination_path:
dest_dir = operation.destination_path.parent
if not dest_dir.exists() and not dest_dir.parent.exists():
validation_errors.append(f"Cannot create destination directory (parent missing): {dest_dir}")
return validation_errors
def execute_cmd(
ctx: CLIContext,
plan: Path,
confirm: bool,
yes: bool,
verbose_ops: bool,
preserve_directories: bool,
safe_mode: bool,
) -> None:
"""Execute an execution plan in dry-run or execute mode."""
config = ctx.config
logger = ctx.logger
mode = "execute" if confirm else "dry-run"
click.echo(f"Loading execution plan from: {plan}")
click.echo()
execution_plan = load_plan(plan)
if preserve_directories:
click.echo("Directory preservation is enabled (plan metadata/operations will be honored).")
click.echo()
if safe_mode:
click.echo("Safe mode enabled - validating plan for directory preservation...")
emptied_dirs = execution_plan.metadata.get("emptied_directories", [])
if emptied_dirs:
raise ValueError(
"SAFE MODE VIOLATION: plan would empty directories. "
"Regenerate/adjust the plan to preserve directory structure."
)
click.echo("Safe mode validation passed - no directories would be destroyed")
click.echo()
click.echo("Validating directory structure...")
validation_errors = _validate_plan_structure(execution_plan)
if validation_errors:
formatted = "\n".join(f" - {error}" for error in validation_errors[:10])
if len(validation_errors) > 10:
formatted += f"\n ... and {len(validation_errors) - 10} more errors"
raise ValueError(f"DIRECTORY STRUCTURE VALIDATION FAILED:\n{formatted}")
click.echo("Directory structure validation passed")
click.echo()
click.echo(f"Execution plan loaded: {execution_plan.plan_id}")
click.echo(f"Created at: {execution_plan.created_at}")
click.echo(f"Total operations: {len(execution_plan.operations)}")
if execution_plan.human_summary:
click.echo()
click.echo(execution_plan.human_summary)
elif execution_plan.summary or execution_plan.summary_by_reason:
s = execution_plan.summary or {}
by_r = execution_plan.summary_by_reason or {}
parts = [
"操作统计:共 "
f"{s.get('total', len(execution_plan.operations))}"
f"move {s.get('move', 0)}rename {s.get('rename', 0)}"
f"quarantine {s.get('quarantine', 0)}no-op {s.get('no-op', 0)}"
]
if by_r:
parts.append("原因分布:" + "".join(f"{r}: {c}" for r, c in list(by_r.items())[:5]))
click.echo()
click.echo("\n".join(parts))
click.echo()
if mode == "dry-run":
click.echo("DRY-RUN MODE - No files will be modified")
click.echo("Use --confirm to actually execute operations")
else:
click.echo("EXECUTE MODE - Files will be modified")
click.echo()
if not yes and not click.confirm("Are you sure you want to proceed?"):
click.echo("Execution cancelled.")
return
if yes:
click.echo("Auto-approved via --yes flag")
click.echo()
click.echo(f"Executing {len(execution_plan.operations)} operations...")
click.echo()
state_path = Path.home() / ".vlm" / "state.json"
state_manager = StateManager(state_path)
engine = ExecutionEngine(
logger=logger,
config=config,
verbose_operations=verbose_ops,
state_manager=state_manager,
)
results, summary, rollback_log = engine.execute_plan(
execution_plan,
mode=mode,
confirmed=confirm,
)
if mode == "dry-run":
click.echo("Sample operations (dry-run):")
for i, result in enumerate(results[:5]):
op = result.operation
if op.operation_type != "no-op":
click.echo(f" [{i + 1}] {op.operation_type}: {op.source_path.name}")
if op.destination_path:
click.echo(f" -> {op.destination_path}")
if len(results) > 5:
click.echo(f" ... and {len(results) - 5} more operations")
else:
for i, result in enumerate(results):
op = result.operation
if op.operation_type != "no-op" and not op.has_conflict:
status = "OK" if result.success else "FAIL"
click.echo(f" [{i + 1}/{len(results)}] {status} {op.operation_type}: {op.source_path.name}")
if result.error_message:
click.echo(f" Error: {result.error_message}")
click.echo()
click.echo("=" * 60)
click.echo(f"Execution Summary ({mode} mode)")
click.echo("=" * 60)
click.echo(f" Total operations: {summary['total']}")
click.echo(f" Successful: {summary['successful']}")
click.echo(f" Failed: {summary['failed']}")
click.echo(f" Skipped: {summary['skipped']}")
click.echo()
if mode == "execute" and rollback_log:
rollback_dir = Path.home() / ".vlm" / "rollback"
rollback_dir.mkdir(parents=True, exist_ok=True)
rollback_path = rollback_dir / f"rollback_{rollback_log.log_id}.json"
engine.save_rollback_log(rollback_log, rollback_path)
click.echo(f"Rollback log saved to: {rollback_path}")
click.echo()
click.echo("To undo these operations, run:")
click.echo(f" vlm rollback --log {rollback_path}")
click.echo()
if mode == "dry-run":
click.echo("Dry-run complete! No files were modified.")
click.echo("Review the operations above and use --confirm to execute.")
elif summary["failed"] > 0:
click.echo(f"Execution completed with {summary['failed']} failures.")
click.echo("Check the log file for details.")
else:
click.echo("Execution completed successfully!")
logger.info(
"Execution completed in %s mode: %s successful, %s failed, %s skipped",
mode,
summary["successful"],
summary["failed"],
summary["skipped"],
)
def rollback_cmd(ctx: CLIContext, log: Optional[Path]) -> None:
"""Rollback a prior execution from rollback log."""
config = ctx.config
logger = ctx.logger
if log is None:
rollback_dir = Path.home() / ".vlm" / "rollback"
if not rollback_dir.exists():
raise FileNotFoundError(f"No rollback logs found. Rollback directory does not exist: {rollback_dir}")
rollback_logs = sorted(
rollback_dir.glob("rollback_*.json"), key=lambda p: p.stat().st_mtime, reverse=True
)
if not rollback_logs:
raise FileNotFoundError(f"No rollback logs found. No rollback_*.json files in: {rollback_dir}")
log = rollback_logs[0]
click.echo(f"Loading rollback log: {log}")
engine = ExecutionEngine(logger=logger, config=config)
rollback_log = engine.load_rollback_log(log)
click.echo(f"Rollback log loaded: {rollback_log.log_id}")
click.echo(f"Plan ID: {rollback_log.execution_plan_id}")
click.echo(f"Executed at: {rollback_log.executed_at}")
click.echo(f"Operations to rollback: {len(rollback_log.operations)}")
click.echo()
if not click.confirm("Proceed with rollback?"):
click.echo("Rollback cancelled.")
return
click.echo(f"Rolling back {len(rollback_log.operations)} operations...")
click.echo()
click.echo("Performing rollback...")
rollback_results, rollback_summary = engine.rollback(rollback_log)
click.echo()
click.echo("Rollback Summary")
click.echo("=" * 60)
click.echo(f" Total operations: {rollback_summary['total']}")
click.echo(f" Successful: {rollback_summary['successful']}")
click.echo(f" Failed: {rollback_summary['failed']}")
click.echo(f" Skipped: {rollback_summary['skipped']}")
click.echo()
if rollback_summary["failed"] > 0:
click.echo("Failed operations:")
for result in rollback_results:
if not result.success:
op = result.operation
click.echo(f" - {op.operation_type}: {op.source_path}")
if result.error_message:
click.echo(f" Error: {result.error_message}")
click.echo()
if rollback_summary["failed"] == 0:
click.echo("Rollback completed successfully!")
else:
click.echo(f"Rollback completed with {rollback_summary['failed']} failures.")
logger.info(
"Rollback completed: %s successful, %s failed",
rollback_summary["successful"],
rollback_summary["failed"],
)
+165
View File
@@ -0,0 +1,165 @@
"""Parse command implementation."""
from __future__ import annotations
from pathlib import Path
from typing import Optional
import click
from vlm.context import CLIContext
from vlm.io import load_inventory_csv, save_identities_json
from vlm.parser import parse_movie, parse_series
from vlm.utils import utc_now
def parse_cmd(ctx: CLIContext, input: Path, output: Path, inventory: Optional[Path]) -> None:
"""Parse identities from scanned inventory."""
config = ctx.config
logger = ctx.logger
click.echo(f"Parsing identities from: {input}")
path_to_metadata: dict[str, object] = {}
if inventory:
click.echo(f"Loading video metadata from: {inventory}")
inventory_files = load_inventory_csv(inventory)
path_to_metadata = {str(vf.path): vf for vf in inventory_files}
click.echo(f"Loaded metadata for {len(path_to_metadata)} files")
click.echo()
inventory_files = load_inventory_csv(input)
video_files = [
{
"path": str(vf.path),
"filename": vf.filename,
"category": vf.category,
}
for vf in inventory_files
]
click.echo(f"Loaded {len(video_files)} files from inventory")
click.echo()
movie_identities: list[dict] = []
series_identities: list[dict] = []
anime_files: list[dict] = []
other_files: list[dict] = []
def get_video_metadata(file_path: str) -> dict:
"""Extract video metadata from inventory if available."""
if not path_to_metadata:
return {}
vf = path_to_metadata.get(file_path)
if not vf:
return {}
return {
"size_bytes": vf.size_bytes,
"modified_timestamp": vf.modified_timestamp.isoformat(),
"resolution": vf.resolution,
"codec": vf.codec,
"duration_seconds": vf.duration_seconds,
"bitrate_kbps": vf.bitrate_kbps,
}
for vf in video_files:
filename = vf["filename"]
category = vf["category"]
file_path = vf["path"]
video_metadata = get_video_metadata(file_path)
if category == "movie":
identity = parse_movie(filename, extensions=config.video_extensions)
record = {
"path": file_path,
"filename": filename,
"category": category,
"title": identity.title,
"year": identity.year,
"confidence": identity.confidence,
"needs_review": identity.needs_review,
}
if video_metadata:
record["video_metadata"] = video_metadata
movie_identities.append(record)
elif category == "series":
identity = parse_series(filename, extensions=config.video_extensions)
record = {
"path": file_path,
"filename": filename,
"category": category,
"title": identity.title,
"season": identity.season,
"episodes": identity.episodes,
"confidence": identity.confidence,
"needs_review": identity.needs_review,
}
if video_metadata:
record["video_metadata"] = video_metadata
series_identities.append(record)
elif category == "anime":
anime_files.append(
{
"path": vf["path"],
"filename": filename,
"category": category,
"note": "Anime parsing deferred in v1",
}
)
else:
other_files.append(
{
"path": vf["path"],
"filename": filename,
"category": category,
"note": "Not categorized for parsing",
}
)
click.echo("Parsing complete!")
click.echo()
click.echo("Results by category:")
click.echo(f" Movies: {len(movie_identities)}")
movies_need_review = sum(1 for m in movie_identities if m["needs_review"])
if movies_need_review > 0:
click.echo(f" - Need review: {movies_need_review}")
click.echo(f" Series: {len(series_identities)}")
series_need_review = sum(1 for s in series_identities if s["needs_review"])
if series_need_review > 0:
click.echo(f" - Need review: {series_need_review}")
click.echo(f" Anime: {len(anime_files)} (not parsed in v1)")
click.echo(f" Other: {len(other_files)} (not parsed)")
click.echo()
click.echo(f"Saving parsed identities to: {output}")
generation_timestamp = utc_now().strftime("%Y-%m-%dT%H:%M:%S")
schema_version = "2.0" if path_to_metadata else "1.0"
identities_data = {
"vlm_schema_version": schema_version,
"metadata": {
"generated": generation_timestamp,
"source_inventory": str(input),
"total_files": len(video_files),
},
"movies": movie_identities,
"series": series_identities,
"anime": anime_files,
"other": other_files,
}
save_identities_json(identities_data, output)
click.echo("Parsed identities saved successfully!")
logger.info(
"Parse completed: %s movies, %s series, saved to %s",
len(movie_identities),
len(series_identities),
output,
)
+9
View File
@@ -81,6 +81,15 @@ def plan_cmd(ctx: CLIContext, input: Path, output: Path, analysis: Optional[Path
click.echo(f" ⚠️ Conflicts detected: {conflicts}")
click.echo(" Review the plan file for details on conflicting operations.")
# Check for directory warnings
directory_warning = execution_plan.metadata.get("directory_warning", False)
if directory_warning:
emptied_dirs = execution_plan.metadata.get("emptied_directories", [])
click.echo()
click.echo(f" ⚠️ Directory preservation warning: {len(emptied_dirs)} directories will be emptied")
click.echo(" These directories will be preserved but may be empty after execution.")
click.echo(" Review the plan file for details on preserved directories.")
click.echo()
click.echo(f"Saving execution plan to: {output}")
output.parent.mkdir(parents=True, exist_ok=True)
+7 -7
View File
@@ -5,15 +5,17 @@ from pathlib import Path
from typing import Optional
import yaml
DEFAULT_VIDEO_EXTENSIONS = [
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
]
@dataclass
class Config:
"""Configuration for Video Library Manager."""
library_root: Path
video_extensions: list[str] = field(default_factory=lambda: [
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
])
video_extensions: list[str] = field(default_factory=lambda: list(DEFAULT_VIDEO_EXTENSIONS))
movie_template: str = "movie/{title} ({year})/"
series_template: str = "series/{title}/Season {season:02d}/"
movie_filename_template: str = "{title} ({year}){ext}"
@@ -74,9 +76,7 @@ def load_config(path: Path) -> Config:
library_root = Path(library_root_str).expanduser()
video_extensions = data.get("video_extensions", [
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
])
video_extensions = data.get("video_extensions", list(DEFAULT_VIDEO_EXTENSIONS))
templates = data.get("templates", {})
movie_template = templates.get("movie_dir", "movie/{title} ({year})/")
@@ -149,7 +149,7 @@ def create_default_config(path: Path) -> Config:
"""Create a default configuration file and return the Config object."""
default_config = Config(
library_root=Path.home() / "Videos",
video_extensions=[".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"],
video_extensions=list(DEFAULT_VIDEO_EXTENSIONS),
)
enrichment_content = {
+48 -6
View File
@@ -93,8 +93,17 @@ class ExecutionEngine:
transaction_log = None
if mode == "execute":
log_path = Path.home() / ".vlm" / "transaction.json"
transaction_log = TransactionLog(log_path)
transaction_log.start_transaction(plan)
try:
transaction_log = TransactionLog(log_path)
transaction_log.start_transaction(plan)
except OSError as exc:
log_operation(
self.logger,
logging.WARNING,
f"Transaction log disabled (cannot write {log_path}): {exc}",
operation_type="execute",
)
transaction_log = None
# Execute all operations
results = []
@@ -105,9 +114,18 @@ class ExecutionEngine:
# Update transaction and state logs in execute mode
if mode == "execute":
if transaction_log:
transaction_log.mark_operation_complete(
i, result.success, result.error_message
)
try:
transaction_log.mark_operation_complete(
i, result.success, result.error_message
)
except OSError as exc:
log_operation(
self.logger,
logging.WARNING,
f"Failed to update transaction log: {exc}",
operation_type="execute",
)
transaction_log = None
# Update file state if successful and not a no-op
if result.success and operation.operation_type != "no-op" and self.state_manager:
@@ -126,7 +144,15 @@ class ExecutionEngine:
if mode == "execute":
if transaction_log:
status = "completed" if all(r.success for r in results) else "failed"
transaction_log.complete_transaction(status=status)
try:
transaction_log.complete_transaction(status=status)
except OSError as exc:
log_operation(
self.logger,
logging.WARNING,
f"Failed to finalize transaction log: {exc}",
operation_type="execute",
)
if self.state_manager:
self.state_manager.save()
@@ -189,6 +215,22 @@ class ExecutionEngine:
executed_at=executed_at
)
# Handle preserve-directory operations
if operation.operation_type == "preserve-directory":
log_operation(
self.logger,
logging.DEBUG,
f"Preserving directory: {operation.reason}",
operation_type="execute",
file_path=operation.source_path
)
return OperationResult(
operation=operation,
success=True,
error_message=None,
executed_at=executed_at
)
# Handle quarantine operations (no destination_path; use QuarantineManager)
if operation.operation_type == "quarantine":
if not self._quarantine_manager:
+44 -7
View File
@@ -15,34 +15,71 @@ from vlm.scanner import load_inventory_csv, save_inventory_csv
__all__ = [
"load_inventory_csv",
"save_inventory_csv",
"load_json_file",
"save_json_file",
"load_identities_json",
"save_identities_json",
"load_analysis_json",
"save_analysis_json",
"identities_to_plan_input",
"identities_to_analysis_input",
]
def load_json_file(path: Path) -> dict:
"""Load a JSON object from disk."""
with open(path, "r", encoding="utf-8") as f:
return json.load(f)
def save_json_file(data: dict, path: Path) -> None:
"""Save a JSON object to disk."""
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
json.dump(data, f, indent=2, ensure_ascii=False)
def load_analysis_json(path: Path) -> dict:
"""Load analysis result from JSON file (metadata, completeness, duplicates).
Caller should check file existence and handle missing/invalid keys.
"""
with open(path, "r", encoding="utf-8") as f:
return json.load(f)
return load_json_file(path)
def load_identities_json(path: Path) -> dict:
"""Load identities from JSON file."""
with open(path, "r", encoding="utf-8") as f:
return json.load(f)
return load_json_file(path)
def save_identities_json(data: dict, path: Path) -> None:
"""Save identities dict to JSON file."""
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
json.dump(data, f, indent=2, ensure_ascii=False)
save_json_file(data, path)
def save_analysis_json(
*,
completeness: list[dict],
duplicates: list[dict],
source_identities: Path,
total_movies: int,
total_series: int,
output: Path,
) -> None:
"""Save analysis result JSON using the canonical schema."""
generation_timestamp = utc_now().strftime("%Y-%m-%dT%H:%M:%S")
analysis_data = {
"vlm_schema_version": "1.0",
"metadata": {
"generated": generation_timestamp,
"source_identities": str(source_identities),
"total_movies": total_movies,
"total_series": total_series,
},
"completeness": completeness,
"duplicates": duplicates,
}
save_json_file(analysis_data, output)
def _video_file_from_record(record: dict) -> VideoFile:
+1 -1
View File
@@ -115,7 +115,7 @@ class FileOperation:
"""Represents a single file operation in an execution plan.
Attributes:
operation_type: Type of operation ("move", "rename", "quarantine", "no-op")
operation_type: Type of operation ("move", "rename", "quarantine", "no-op", "preserve-directory")
source_path: Source file path
destination_path: Destination file path (None for no-op operations)
reason: Human-readable reason for the operation
+1 -6
View File
@@ -7,14 +7,9 @@ logical identities such as movie titles/years and series titles/seasons/episodes
import re
from typing import Optional
from vlm.config import DEFAULT_VIDEO_EXTENSIONS
from vlm.models import MovieIdentity, SeriesIdentity
# Default extensions used when extensions param is not provided (matches config default)
DEFAULT_VIDEO_EXTENSIONS = [
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
]
# Quality tags to remove from titles
QUALITY_TAGS = [
r'\b1080p\b', r'\b720p\b', r'\b480p\b', r'\b2160p\b',
+57 -1
View File
@@ -40,6 +40,42 @@ def _is_sample_path(path: Path) -> bool:
return bool(re.search(r"(^|[\s._-])sample($|[\s._-])", stem))
def _analyze_directory_impact(operations: list[FileOperation]) -> dict:
"""Analyze which directories will be emptied by the plan."""
# Get all source directories that have files being moved/renamed
source_dirs = set()
moved_files = set()
for operation in operations:
if operation.operation_type in ("move", "rename"):
source_dirs.add(operation.source_path.parent)
moved_files.add(operation.source_path)
emptied_dirs = []
for dir_path in source_dirs:
# Only analyze directories that actually exist
if not dir_path.exists():
continue
# Count files that will remain in this directory after operations
remaining_count = 0
try:
for item in dir_path.iterdir():
if item.is_file() and item not in moved_files:
remaining_count += 1
except (OSError, PermissionError):
# If we can't read the directory, skip analysis
continue
# If no files will remain, this directory will be emptied
if remaining_count == 0:
emptied_dirs.append(dir_path)
return {
"emptied_directories": emptied_dirs,
"warning_required": len(emptied_dirs) > 0
}
def generate_plan(
identities: list[tuple[VideoFile, Union[MovieIdentity, SeriesIdentity, None]]],
config: Config,
@@ -120,10 +156,29 @@ def generate_plan(
conflict_reason=None,
)
# Analyze directory impact and add preservation operations
directory_analysis = _analyze_directory_impact(operations)
if directory_analysis["warning_required"]:
# Add directory preservation operations for emptied directories
for emptied_dir in directory_analysis["emptied_directories"]:
operations.append(FileOperation(
operation_type="preserve-directory",
source_path=emptied_dir,
destination_path=None,
reason=f"Preserve empty source directory: {emptied_dir.name}",
has_conflict=False,
conflict_reason=None
))
summary = _generate_summary(operations)
summary_by_reason = _generate_summary_by_reason(operations)
human_summary = _generate_human_summary(operations, summary, summary_by_reason, metadata)
# Add directory warnings to metadata
if directory_analysis["warning_required"]:
metadata["emptied_directories"] = [str(d) for d in directory_analysis["emptied_directories"]]
metadata["directory_warning"] = True
return ExecutionPlan(
plan_id=str(uuid.uuid4()),
created_at=utc_now(),
@@ -445,7 +500,8 @@ def _generate_summary(operations: list[FileOperation]) -> dict:
"move": 0,
"rename": 0,
"quarantine": 0,
"no-op": 0
"no-op": 0,
"preserve-directory": 0
}
for operation in operations:
+29 -29
View File
@@ -5,7 +5,7 @@ Tests series completeness analysis, duplicate detection, and quality comparison.
import pytest
from pathlib import Path
from datetime import datetime
from datetime import datetime, timezone
from vlm.models import SeriesIdentity, SeasonCompleteness, MovieIdentity, VideoFile, DuplicateGroup
from vlm.analysis import analyze_series_completeness, detect_duplicates, compare_quality
@@ -193,7 +193,7 @@ class TestDuplicateDetection:
Path("/movies/The.Matrix.1999.1080p.mkv"),
"The.Matrix.1999.1080p.mkv",
2000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1920x1080",
codec="h264"
@@ -202,7 +202,7 @@ class TestDuplicateDetection:
Path("/movies/The.Matrix.1999.720p.mkv"),
"The.Matrix.1999.720p.mkv",
1000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1280x720",
codec="h264"
@@ -211,7 +211,7 @@ class TestDuplicateDetection:
Path("/movies/Inception.2010.mkv"),
"Inception.2010.mkv",
1500000000,
datetime.now(),
datetime.now(timezone.utc),
"movie"
),
]
@@ -239,7 +239,7 @@ class TestDuplicateDetection:
Path("/series/Breaking.Bad.S01E01.1080p.mkv"),
"Breaking.Bad.S01E01.1080p.mkv",
1500000000,
datetime.now(),
datetime.now(timezone.utc),
"series",
resolution="1920x1080"
),
@@ -247,7 +247,7 @@ class TestDuplicateDetection:
Path("/series/Breaking.Bad.S01E01.720p.mkv"),
"Breaking.Bad.S01E01.720p.mkv",
800000000,
datetime.now(),
datetime.now(timezone.utc),
"series",
resolution="1280x720"
),
@@ -255,7 +255,7 @@ class TestDuplicateDetection:
Path("/series/Breaking.Bad.S01E02.mkv"),
"Breaking.Bad.S01E02.mkv",
1200000000,
datetime.now(),
datetime.now(timezone.utc),
"series"
),
]
@@ -278,8 +278,8 @@ class TestDuplicateDetection:
]
files = [
VideoFile(Path("/movies/Movie.A.2020.mkv"), "Movie.A.2020.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie.B.2021.mkv"), "Movie.B.2021.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie.A.2020.mkv"), "Movie.A.2020.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/Movie.B.2021.mkv"), "Movie.B.2021.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
]
result = detect_duplicates(list(zip(identities, files)))
@@ -294,8 +294,8 @@ class TestDuplicateDetection:
]
files = [
VideoFile(Path("/movies/Unknown.Movie.mkv"), "Unknown.Movie.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Unknown.Movie.2.mkv"), "Unknown.Movie.2.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Unknown.Movie.mkv"), "Unknown.Movie.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/Unknown.Movie.2.mkv"), "Unknown.Movie.2.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
]
result = detect_duplicates(list(zip(identities, files)))
@@ -311,8 +311,8 @@ class TestDuplicateDetection:
]
files = [
VideoFile(Path("/series/Unknown.Show.E01.mkv"), "Unknown.Show.E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Unknown.Show.Episode.1.mkv"), "Unknown.Show.Episode.1.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Unknown.Show.E01.mkv"), "Unknown.Show.E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
VideoFile(Path("/series/Unknown.Show.Episode.1.mkv"), "Unknown.Show.Episode.1.mkv", 1000000000, datetime.now(timezone.utc), "series"),
]
result = detect_duplicates(list(zip(identities, files)))
@@ -327,8 +327,8 @@ class TestDuplicateDetection:
]
files = [
VideoFile(Path("/series/Show.Name.S01.mkv"), "Show.Name.S01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.Name.Season.1.mkv"), "Show.Name.Season.1.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.Name.S01.mkv"), "Show.Name.S01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
VideoFile(Path("/series/Show.Name.Season.1.mkv"), "Show.Name.Season.1.mkv", 1000000000, datetime.now(timezone.utc), "series"),
]
result = detect_duplicates(list(zip(identities, files)))
@@ -347,7 +347,7 @@ class TestDuplicateDetection:
Path("/movies/Test.Movie.2020.1080p.mkv"),
"Test.Movie.2020.1080p.mkv",
2000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1920x1080",
codec="h264",
@@ -358,7 +358,7 @@ class TestDuplicateDetection:
Path("/movies/Test.Movie.2020.720p.mkv"),
"Test.Movie.2020.720p.mkv",
1000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1280x720",
codec="h264",
@@ -395,9 +395,9 @@ class TestDuplicateDetection:
]
files = [
VideoFile(Path("/series/Show.S01E01-E02.mkv"), "Show.S01E01-E02.mkv", 2000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E01-E02.mkv"), "Show.S01E01-E02.mkv", 2000000000, datetime.now(timezone.utc), "series"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(timezone.utc), "series"),
]
result = detect_duplicates(list(zip(identities, files)))
@@ -413,8 +413,8 @@ class TestDuplicateDetection:
]
files = [
VideoFile(Path("/movies/The.Thing.1982.mkv"), "The.Thing.1982.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/The.Thing.2011.mkv"), "The.Thing.2011.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/The.Thing.1982.mkv"), "The.Thing.1982.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/The.Thing.2011.mkv"), "The.Thing.2011.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
]
result = detect_duplicates(list(zip(identities, files)))
@@ -429,8 +429,8 @@ class TestDuplicateDetection:
]
files = [
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S02E01.mkv"), "Show.S02E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
VideoFile(Path("/series/Show.S02E01.mkv"), "Show.S02E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
]
result = detect_duplicates(list(zip(identities, files)))
@@ -448,7 +448,7 @@ class TestQualityComparison:
Path("/test/file1.mkv"),
"file1.mkv",
2000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1920x1080",
codec="h264",
@@ -459,7 +459,7 @@ class TestQualityComparison:
Path("/test/file2.mkv"),
"file2.mkv",
1000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1280x720",
codec="h265",
@@ -490,7 +490,7 @@ class TestQualityComparison:
Path("/test/file1.mkv"),
"file1.mkv",
2000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1920x1080"
# codec, duration, bitrate not available
@@ -499,7 +499,7 @@ class TestQualityComparison:
Path("/test/file2.mkv"),
"file2.mkv",
1000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie"
# No optional metadata
),
@@ -532,7 +532,7 @@ class TestQualityComparison:
Path("/test/file.mkv"),
"file.mkv",
1500000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1920x1080",
codec="h264"
+8 -8
View File
@@ -6,7 +6,7 @@ Each test validates a specific property from the design document.
import pytest
from pathlib import Path
from datetime import datetime
from datetime import datetime, timezone
from hypothesis import given, strategies as st, settings
from vlm.models import (
SeriesIdentity, SeasonCompleteness, MovieIdentity, VideoFile, DuplicateGroup
@@ -74,7 +74,7 @@ def video_file_strategy(draw, filename=None, category="movie"):
path = Path(f"/{category}/{filename}")
size_bytes = draw(st.integers(min_value=1000000, max_value=10000000000))
modified_timestamp = datetime.now()
modified_timestamp = datetime.now(timezone.utc)
# Optional metadata
has_metadata = draw(st.booleans())
@@ -226,7 +226,7 @@ def test_property_12_duplicate_detection_movies(title, year, duplicate_count):
Path(f"/movies/{filename}"),
filename,
1000000000 + i * 100000000,
datetime.now(),
datetime.now(timezone.utc),
"movie"
))
@@ -272,7 +272,7 @@ def test_property_13_duplicate_detection_series(title, season, episode, duplicat
Path(f"/series/{filename}"),
filename,
1000000000 + i * 100000000,
datetime.now(),
datetime.now(timezone.utc),
"series"
))
@@ -321,7 +321,7 @@ def test_property_14_duplicate_quality_comparison(title, year, file_count):
Path(f"/movies/{filename}"),
filename,
1000000000 + i * 100000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1920x1080",
codec="h264",
@@ -333,7 +333,7 @@ def test_property_14_duplicate_quality_comparison(title, year, file_count):
Path(f"/movies/{filename}"),
filename,
1000000000 + i * 100000000,
datetime.now(),
datetime.now(timezone.utc),
"movie"
))
@@ -429,7 +429,7 @@ def test_property_43_duplicate_report_grouping(duplicate_count, format):
Path(f"/movies/{filename}"),
filename,
1000000000 + j * 500000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1920x1080" if j == 0 else "1280x720",
codec="h264"
@@ -492,7 +492,7 @@ def test_property_44_summary_report_accuracy(file_count, categories):
Path(f"/{category}/{filename}"),
filename,
size,
datetime.now(),
datetime.now(timezone.utc),
category
))
+4 -4
View File
@@ -2,7 +2,7 @@
import json
from pathlib import Path
from datetime import datetime
from datetime import datetime, timezone
import pytest
from click.testing import CliRunner
@@ -46,7 +46,7 @@ def sample_rollback_log(tmp_path):
),
success=True,
error_message=None,
executed_at=datetime.now()
executed_at=datetime.now(timezone.utc)
),
OperationResult(
operation=FileOperation(
@@ -59,14 +59,14 @@ def sample_rollback_log(tmp_path):
),
success=True,
error_message=None,
executed_at=datetime.now()
executed_at=datetime.now(timezone.utc)
)
]
rollback_log = RollbackLog(
log_id="test-log-id",
execution_plan_id="test-plan-id",
executed_at=datetime.now(),
executed_at=datetime.now(timezone.utc),
operations=operations
)
+14 -14
View File
@@ -4,7 +4,7 @@ Tests execution mode handling, dry-run simulation, and actual file operations.
"""
import logging
from datetime import datetime
from datetime import datetime, timezone
from pathlib import Path
from uuid import uuid4
@@ -73,7 +73,7 @@ def sample_plan(temp_test_dir):
return ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=operations,
summary={"move": 1, "rename": 1}
)
@@ -208,7 +208,7 @@ class TestDryRunSimulation:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=[no_op_operation],
summary={"no-op": 1}
)
@@ -233,7 +233,7 @@ class TestDryRunSimulation:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=[conflicted_operation],
summary={"move": 1}
)
@@ -285,7 +285,7 @@ class TestExecuteMode:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=[operation],
summary={"move": 1}
)
@@ -319,7 +319,7 @@ class TestExecuteMode:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=[operation],
summary={"move": 1}
)
@@ -352,7 +352,7 @@ class TestExecuteMode:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=[conflicted_operation],
summary={"move": 1}
)
@@ -427,7 +427,7 @@ class TestRollbackLog:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=operations,
summary={"move": 2}
)
@@ -480,7 +480,7 @@ class TestExecutionSummary:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=operations,
summary={"move": 2, "no-op": 1}
)
@@ -573,7 +573,7 @@ class TestRollbackLogSaving:
data = json.load(f)
# Verify timestamps are in ISO format
from datetime import datetime
from datetime import datetime, timezone
executed_at = datetime.fromisoformat(data["executed_at"])
assert executed_at is not None
@@ -670,7 +670,7 @@ class TestRollbackExecution:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=operations,
summary={"move": 2}
)
@@ -759,7 +759,7 @@ class TestRollbackExecution:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=operations,
summary={"move": 1, "no-op": 1}
)
@@ -877,7 +877,7 @@ class TestRollbackExecution:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=operations,
summary={"move": 2}
)
@@ -920,7 +920,7 @@ class TestRollbackExecution:
plan = ExecutionPlan(
plan_id=str(uuid4()),
created_at=datetime.now(),
created_at=datetime.now(timezone.utc),
operations=[
FileOperation(
operation_type="quarantine",
+46 -46
View File
@@ -1,7 +1,7 @@
"""Unit tests for plan generator."""
import pytest
from datetime import datetime
from datetime import datetime, timezone
from pathlib import Path
from vlm.config import Config
@@ -36,7 +36,7 @@ def test_generate_plan_for_movie_with_year(config):
path=Path("/mnt/nas/videos/movie/Some.Movie.2020.1080p.mkv"),
filename="Some.Movie.2020.1080p.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -67,7 +67,7 @@ def test_generate_plan_for_movie_without_year(config):
path=Path("/mnt/nas/videos/movie/random_movie.mkv"),
filename="random_movie.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -94,7 +94,7 @@ def test_generate_plan_for_series_with_season_and_episode(config):
path=Path("/mnt/nas/videos/series/Show.Name.S01E05.mkv"),
filename="Show.Name.S01E05.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
)
@@ -123,7 +123,7 @@ def test_generate_plan_blocks_series_with_high_season(config):
path=Path("/mnt/nas/videos/series/Show.Name.S20E01.mkv"),
filename="Show.Name.S20E01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
)
identity = SeriesIdentity(
@@ -147,7 +147,7 @@ def test_generate_plan_blocks_series_with_high_episode(config):
path=Path("/mnt/nas/videos/series/Show.Name.S01E120.mkv"),
filename="Show.Name.S01E120.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
)
identity = SeriesIdentity(
@@ -171,7 +171,7 @@ def test_generate_plan_sample_is_noop_by_default(config):
path=Path("/mnt/nas/videos/series/Show.Name.Sample.S01E01.mkv"),
filename="Show.Name.Sample.S01E01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
)
identity = SeriesIdentity(
@@ -196,7 +196,7 @@ def test_generate_plan_sample_can_be_included_via_config(config):
path=Path("/mnt/nas/videos/series/Show.Name.Sample.S01E01.mkv"),
filename="Show.Name.Sample.S01E01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
)
identity = SeriesIdentity(
@@ -219,7 +219,7 @@ def test_generate_plan_for_series_without_season(config):
path=Path("/mnt/nas/videos/series/ambiguous_show.mkv"),
filename="ambiguous_show.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
)
@@ -247,7 +247,7 @@ def test_generate_plan_for_anime_category(config):
path=Path("/mnt/nas/videos/anime/Some.Anime.01.mkv"),
filename="Some.Anime.01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="anime"
)
@@ -267,7 +267,7 @@ def test_generate_plan_for_other_category(config):
path=Path("/mnt/nas/videos/other/random.mkv"),
filename="random.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="other"
)
@@ -286,7 +286,7 @@ def test_generate_plan_preserves_category_boundaries(config):
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -311,7 +311,7 @@ def test_generate_plan_for_multi_episode_file(config):
path=Path("/mnt/nas/videos/series/Show.S01E01E02.mkv"),
filename="Show.S01E01E02.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
)
@@ -338,7 +338,7 @@ def test_generate_plan_file_already_at_target(config):
path=Path("/mnt/nas/videos/movie/Some Movie (2020)/Some Movie (2020).mkv"),
filename="Some Movie (2020).mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -364,7 +364,7 @@ def test_generate_plan_rename_vs_move(config):
path=Path("/mnt/nas/videos/movie/Some Movie (2020)/old_name.mkv"),
filename="old_name.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -385,7 +385,7 @@ def test_generate_plan_rename_vs_move(config):
path=Path("/mnt/nas/videos/movie/wrong_dir/Some Movie (2020).mkv"),
filename="Some Movie (2020).mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -403,7 +403,7 @@ def test_generate_plan_summary(config):
path=Path("/mnt/nas/videos/movie/Movie1.2020.mkv"),
filename="Movie1.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
),
MovieIdentity(
@@ -420,7 +420,7 @@ def test_generate_plan_summary(config):
path=Path("/mnt/nas/videos/movie/Movie2.mkv"),
filename="Movie2.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
),
MovieIdentity(
@@ -437,7 +437,7 @@ def test_generate_plan_summary(config):
path=Path("/mnt/nas/videos/anime/Anime.01.mkv"),
filename="Anime.01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="anime"
),
None
@@ -448,7 +448,7 @@ def test_generate_plan_summary(config):
path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"),
filename="Show.S01E01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
),
SeriesIdentity(
@@ -480,14 +480,14 @@ def test_generate_plan_with_analysis_by_quality(config):
path=p_720,
filename="Test.2020.720p.WEB-DL.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
vf_1080 = VideoFile(
path=p_1080,
filename="Test.2020.1080p.BluRay.mkv",
size_bytes=2000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
identity = MovieIdentity(
@@ -528,14 +528,14 @@ def test_generate_plan_with_analysis_by_reputation_quality_time_reason(config):
path=p_old,
filename="Test.2020.720p.WEB-DL.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
vf_new = VideoFile(
path=p_new,
filename="Test.2020.1080p.BluRay.mkv",
size_bytes=2000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
identity = MovieIdentity(
@@ -574,14 +574,14 @@ def test_generate_plan_with_analysis_by_reputation_missing_scores_uses_fallback_
path=p_web,
filename="Test.2020.2160p.WEB-DL.mkv",
size_bytes=3000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
vf_bluray = VideoFile(
path=p_bluray,
filename="Test.2020.1080p.BluRay.mkv",
size_bytes=2000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
identity = MovieIdentity(
@@ -622,14 +622,14 @@ def test_generate_plan_human_summary_marks_disc_files_as_high_risk(config):
path=p_disc1,
filename="The.Best.of.Youth.DISC1.mkv",
size_bytes=2000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
vf2 = VideoFile(
path=p_disc2,
filename="The.Best.of.Youth.DISC2.mkv",
size_bytes=1800000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie",
)
identity = MovieIdentity(
@@ -669,7 +669,7 @@ def test_generate_plan_with_different_extensions(config):
path=Path(f"/mnt/nas/videos/movie/Movie.2020{ext}"),
filename=f"Movie.2020{ext}",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -704,7 +704,7 @@ def test_conflict_detection_for_movie(config, tmp_path):
path=tmp_path / "movie" / "Some.Movie.2020.1080p.mkv",
filename="Some.Movie.2020.1080p.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -741,7 +741,7 @@ def test_conflict_detection_for_series(config, tmp_path):
path=tmp_path / "series" / "Show.Name.S01E05.1080p.mkv",
filename="Show.Name.S01E05.1080p.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
)
@@ -776,7 +776,7 @@ def test_no_conflict_when_destination_does_not_exist(config, tmp_path):
path=source_dir / "Some.Movie.2020.mkv",
filename="Some.Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -814,7 +814,7 @@ def test_conflict_detection_with_multiple_files(config, tmp_path):
path=tmp_path / "movie" / "Movie1.2020.mkv",
filename="Movie1.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
),
MovieIdentity(
@@ -831,7 +831,7 @@ def test_conflict_detection_with_multiple_files(config, tmp_path):
path=tmp_path / "movie" / "Movie2.2021.mkv",
filename="Movie2.2021.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
),
MovieIdentity(
@@ -864,7 +864,7 @@ def test_save_plan_to_json(config, tmp_path):
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -909,7 +909,7 @@ def test_load_plan_from_json(config, tmp_path):
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -963,7 +963,7 @@ def test_save_and_load_plan_with_conflicts(config, tmp_path):
path=tmp_path / "movie" / "Movie.2020.mkv",
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -1003,7 +1003,7 @@ def test_save_and_load_plan_with_no_op_operations(config, tmp_path):
path=Path("/mnt/nas/videos/anime/Anime.01.mkv"),
filename="Anime.01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="anime"
),
None
@@ -1014,7 +1014,7 @@ def test_save_and_load_plan_with_no_op_operations(config, tmp_path):
path=Path("/mnt/nas/videos/movie/Movie.mkv"),
filename="Movie.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
),
MovieIdentity(
@@ -1048,7 +1048,7 @@ def test_save_plan_json_is_human_readable(config, tmp_path):
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -1089,7 +1089,7 @@ def test_save_plan_with_multiple_operations(config, tmp_path):
path=Path("/mnt/nas/videos/movie/Movie1.2020.mkv"),
filename="Movie1.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
),
MovieIdentity(
@@ -1106,7 +1106,7 @@ def test_save_plan_with_multiple_operations(config, tmp_path):
path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"),
filename="Show.S01E01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
),
SeriesIdentity(
@@ -1124,7 +1124,7 @@ def test_save_plan_with_multiple_operations(config, tmp_path):
path=Path("/mnt/nas/videos/anime/Anime.01.mkv"),
filename="Anime.01.mkv",
size_bytes=500000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="anime"
),
None
@@ -1176,7 +1176,7 @@ def test_plan_json_includes_all_required_fields(config, tmp_path):
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -1214,7 +1214,7 @@ def test_movie_rejected_by_review_generates_noop(config):
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
filename="Movie.2020.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="movie"
)
@@ -1238,7 +1238,7 @@ def test_series_rejected_by_review_generates_noop(config):
path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"),
filename="Show.S01E01.mkv",
size_bytes=1000000,
modified_timestamp=datetime.now(),
modified_timestamp=datetime.now(timezone.utc),
category="series"
)
+1 -1
View File
@@ -3,7 +3,7 @@
import json
import pytest
from pathlib import Path
from datetime import datetime
from datetime import datetime, timezone
from vlm.quarantine import QuarantineManager
from vlm.config import Config
+23 -23
View File
@@ -185,7 +185,7 @@ class TestInventoryReport:
Path("/test.mkv"),
"test.mkv",
1000,
datetime.now(),
datetime.now(timezone.utc),
"movie"
)
]
@@ -377,7 +377,7 @@ class TestDuplicateReport:
Path("/movies/The.Matrix.1999.1080p.mkv"),
"The.Matrix.1999.1080p.mkv",
2000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1920x1080",
codec="h264",
@@ -388,7 +388,7 @@ class TestDuplicateReport:
Path("/movies/The.Matrix.1999.720p.mkv"),
"The.Matrix.1999.720p.mkv",
1000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1280x720",
codec="h264"
@@ -448,14 +448,14 @@ class TestDuplicateReport:
Path("/movies/Inception.2010.1080p.mkv"),
"Inception.2010.1080p.mkv",
2000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie"
),
VideoFile(
Path("/movies/Inception.2010.720p.mkv"),
"Inception.2010.720p.mkv",
1000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie"
),
]
@@ -501,14 +501,14 @@ class TestDuplicateReport:
Path("/series/Breaking.Bad.S01E01.1080p.mkv"),
"Breaking.Bad.S01E01.1080p.mkv",
1500000000,
datetime.now(),
datetime.now(timezone.utc),
"series"
),
VideoFile(
Path("/series/Breaking.Bad.S01E01.720p.mkv"),
"Breaking.Bad.S01E01.720p.mkv",
800000000,
datetime.now(),
datetime.now(timezone.utc),
"series"
),
]
@@ -564,8 +564,8 @@ class TestDuplicateReport:
# Create two duplicate groups with different sizes
identity1 = MovieIdentity("Small Movie", 2020, 0.9, False, "Small.Movie.mkv")
files1 = [
VideoFile(Path("/movies/Small.Movie.1.mkv"), "Small.Movie.1.mkv", 500000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Small.Movie.2.mkv"), "Small.Movie.2.mkv", 600000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Small.Movie.1.mkv"), "Small.Movie.1.mkv", 500000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/Small.Movie.2.mkv"), "Small.Movie.2.mkv", 600000000, datetime.now(timezone.utc), "movie"),
]
quality1 = [
{'filename': 'Small.Movie.1.mkv', 'path': '/movies/Small.Movie.1.mkv', 'size_bytes': 500000000},
@@ -574,8 +574,8 @@ class TestDuplicateReport:
identity2 = MovieIdentity("Large Movie", 2021, 0.9, False, "Large.Movie.mkv")
files2 = [
VideoFile(Path("/movies/Large.Movie.1.mkv"), "Large.Movie.1.mkv", 2000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Large.Movie.2.mkv"), "Large.Movie.2.mkv", 1800000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Large.Movie.1.mkv"), "Large.Movie.1.mkv", 2000000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/Large.Movie.2.mkv"), "Large.Movie.2.mkv", 1800000000, datetime.now(timezone.utc), "movie"),
]
quality2 = [
{'filename': 'Large.Movie.1.mkv', 'path': '/movies/Large.Movie.1.mkv', 'size_bytes': 2000000000},
@@ -600,8 +600,8 @@ class TestDuplicateReport:
"""Sort order should use quality_comparison sizes when VideoFile sizes are zero."""
identity1 = MovieIdentity("Tiny", 2020, 0.9, False, "Tiny.mkv")
files1 = [
VideoFile(Path("/movies/Tiny.1.mkv"), "Tiny.1.mkv", 0, datetime.now(), "movie"),
VideoFile(Path("/movies/Tiny.2.mkv"), "Tiny.2.mkv", 0, datetime.now(), "movie"),
VideoFile(Path("/movies/Tiny.1.mkv"), "Tiny.1.mkv", 0, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/Tiny.2.mkv"), "Tiny.2.mkv", 0, datetime.now(timezone.utc), "movie"),
]
quality1 = [
{"filename": "Tiny.1.mkv", "path": "/movies/Tiny.1.mkv", "size_bytes": 600000000},
@@ -610,8 +610,8 @@ class TestDuplicateReport:
identity2 = MovieIdentity("Huge", 2021, 0.9, False, "Huge.mkv")
files2 = [
VideoFile(Path("/movies/Huge.1.mkv"), "Huge.1.mkv", 0, datetime.now(), "movie"),
VideoFile(Path("/movies/Huge.2.mkv"), "Huge.2.mkv", 0, datetime.now(), "movie"),
VideoFile(Path("/movies/Huge.1.mkv"), "Huge.1.mkv", 0, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/Huge.2.mkv"), "Huge.2.mkv", 0, datetime.now(timezone.utc), "movie"),
]
quality2 = [
{"filename": "Huge.1.mkv", "path": "/movies/Huge.1.mkv", "size_bytes": 3000000000},
@@ -632,12 +632,12 @@ class TestSummaryReport:
def test_generate_summary_report(self):
"""Test generating summary report with various files."""
files = [
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(), "movie"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(), "anime"),
VideoFile(Path("/other/Random.mkv"), "Random.mkv", 500000000, datetime.now(), "other"),
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(timezone.utc), "series"),
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(timezone.utc), "anime"),
VideoFile(Path("/other/Random.mkv"), "Random.mkv", 500000000, datetime.now(timezone.utc), "other"),
]
library_root = Path("/mnt/nas/videos")
@@ -677,8 +677,8 @@ class TestSummaryReport:
def test_generate_summary_report_single_category(self):
"""Test generating summary report with files in single category."""
files = [
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 1000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 2000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 2000000000, datetime.now(timezone.utc), "movie"),
]
library_root = Path("/mnt/nas/videos")
+9 -9
View File
@@ -5,7 +5,7 @@ Tests the complete workflow from analysis to report generation.
import json
from pathlib import Path
from datetime import datetime
from datetime import datetime, timezone
from vlm.models import SeriesIdentity, VideoFile, MovieIdentity
from vlm.analysis import analyze_series_completeness, detect_duplicates
from vlm.reports import generate_completeness_report, generate_duplicate_report, generate_summary_report
@@ -59,7 +59,7 @@ class TestReportsIntegration:
Path("/movies/The.Matrix.1999.1080p.mkv"),
"The.Matrix.1999.1080p.mkv",
2000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1920x1080",
codec="h264"
@@ -68,7 +68,7 @@ class TestReportsIntegration:
Path("/movies/The.Matrix.1999.720p.mkv"),
"The.Matrix.1999.720p.mkv",
1000000000,
datetime.now(),
datetime.now(timezone.utc),
"movie",
resolution="1280x720",
codec="h264"
@@ -77,7 +77,7 @@ class TestReportsIntegration:
Path("/movies/Inception.2010.mkv"),
"Inception.2010.mkv",
1500000000,
datetime.now(),
datetime.now(timezone.utc),
"movie"
),
]
@@ -107,11 +107,11 @@ class TestReportsIntegration:
"""Test summary report generation with mixed file types."""
# Create test files
files = [
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(), "movie"),
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(), "movie"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"),
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(), "anime"),
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(timezone.utc), "movie"),
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(timezone.utc), "series"),
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(timezone.utc), "anime"),
]
# Generate summary report
+1 -1
View File
@@ -92,7 +92,7 @@ class TestLoadSaveState:
store = StateStore(
states={},
version='1.0',
last_updated=datetime.now()
last_updated=datetime.now(timezone.utc)
)
save_state(store, state_path)