diff --git a/AGENTS.md b/AGENTS.md index fd933d7..87b2e79 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,7 +2,7 @@ ## Project Structure & Module Organization - Core package lives in `src/vlm/`. -- CLI entrypoint is `src/vlm/cli.py` (`vlm` console script). Commands use `pass_context` and `CLIContext` from `context.py`; some command logic lives in `commands/` (e.g. `scan`, `analyze`, `plan`). +- CLI entrypoint is `src/vlm/cli.py` (`vlm` console script). Commands use `pass_context` and `CLIContext` from `context.py`; command logic is modularized in `commands/` (e.g. `scan`, `parse`, `enrich`, `analyze`, `plan`, `execute`). - Functional modules by concern: scanning (`scanner.py`), parsing (`parser.py`), enrichment (`enrichment.py`, `cache.py`, `providers/`), analysis/planning/execution (`analysis.py`, `planner.py`, `executor.py`), I/O helpers (`io.py`), utilities (`utils.py`), state/reporting/logging (`state.py`, `reports.py`, `logging_config.py`), config (`config.py`), models (`models.py`). - Tests live in `tests/` and mirror feature areas (e.g. `tests/test_scanner.py`, `tests/test_cli_state.py`, `tests/test_enrichment.py`). - Project metadata and tool config are in `pyproject.toml`. @@ -36,3 +36,8 @@ ## Security & Configuration Tips - Do not commit local paths, personal media metadata, or generated state/log artifacts. - Validate config changes against `vlm --help` and at least one end-to-end CLI flow before merging. + + +## Documentation baseline +- Updated to reflect refactor results as of 2026-02-16. +- Canonical release notes are tracked in `CHANGELOG.md`. diff --git a/ARCHITECTURE_REVIEW.md b/ARCHITECTURE_REVIEW.md index 5126500..ff49fe5 100644 --- a/ARCHITECTURE_REVIEW.md +++ b/ARCHITECTURE_REVIEW.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # Video Library Manager - Architecture Review **Review Date**: 2026-02-13 diff --git a/AUDIT_FIX_PLAN.md b/AUDIT_FIX_PLAN.md index 1dafa29..5d2b086 100644 --- a/AUDIT_FIX_PLAN.md +++ b/AUDIT_FIX_PLAN.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # Post-Audit Fixes Implementation Plan (2026-02-13) ## Objective diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..3fc0154 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,38 @@ +# Changelog + +## 2026-02-16 + +### Refactor Results + +- Modularized CLI command implementations: + - Added `src/vlm/commands/parse.py` + - Added `src/vlm/commands/enrich.py` + - Added `src/vlm/commands/execute.py` + - Refactored `src/vlm/cli.py` to delegate `parse`, `enrich`, `execute`, and `rollback` to command modules + +- Unified I/O layer for JSON handling: + - Added `load_json_file`, `save_json_file`, `save_analysis_json` in `src/vlm/io.py` + - Updated analyze/enrich paths to use unified I/O entry points + +- Parser/config consistency improvements: + - Added shared `DEFAULT_VIDEO_EXTENSIONS` in `src/vlm/config.py` + - Updated `src/vlm/parser.py` to use the shared config constant instead of local hardcoded defaults + +- Execution reliability hardening: + - Updated `src/vlm/executor.py` transaction logging to fail gracefully when `~/.vlm` is not writable (warn + continue) + +- Test stability improvements: + - Standardized test timestamps to timezone-aware UTC (`datetime.now(timezone.utc)`) + - Added missing `timezone` imports where required + +### Verification + +- Full test suite passed after refactor: + - `477 passed` + +### Documentation Sync + +- Updated repository documentation set to match the refactor baseline: + - Core docs: `README.md`, `AGENTS.md`, `CLAUDE.md`, `GEMINI.md` + - Skill docs: `skills/vlm-library-workflow/SKILL.md` and all references under `skills/vlm-library-workflow/references/` + - Legacy review/plan docs now explicitly marked as historical snapshots and redirected to `CHANGELOG.md` as the current source of truth diff --git a/CLAUDE.md b/CLAUDE.md index 7b002c2..5bc37be 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,5 +1,8 @@ # CLAUDE.md +## Documentation Status +- Synchronized with refactor baseline on 2026-02-16 (see `CHANGELOG.md`). + This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Project Overview @@ -91,11 +94,11 @@ VLM follows a read-first, multi-stage pipeline: ### Module Organization - `cli.py` - Click-based CLI interface, global options, command registration - `context.py` - CLIContext (config, paths) and pass_context for commands -- `commands/` - Command implementations (scan, analyze, plan) +- `commands/` - Command implementations (scan, parse, enrich, analyze, plan, execute/rollback) - `scanner.py` - File discovery using system `find` command, metadata extraction via ffprobe - `parser.py` - Filename parsing using regex patterns (movies: title + year, series: SxxExx) - `enrichment.py` - Enrichment pipeline; `cache.py` - SQLite cache; `providers/` - TMDB etc. -- `io.py` - Load/save identities JSON/CSV, plan/analysis input helpers +- `io.py` - Unified JSON/CSV I/O helpers (including analysis writer and data adapters) - `utils.py` - UTC time, format_size, shared helpers - `analysis.py` - Completeness checking (episode gaps) and duplicate detection - `duplicate_resolve.py` - Duplicate group resolution (by_quality, by_reputation, first_seen, manual) diff --git a/CODE_IMPROVEMENTS.md b/CODE_IMPROVEMENTS.md index 77109d6..6671ec4 100644 --- a/CODE_IMPROVEMENTS.md +++ b/CODE_IMPROVEMENTS.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # VLM 代码改进清单 本文档记录对 Video Library Manager (VLM) 项目的代码审查发现的问题及对应解决方案。排除 AI/OpenAPI 相关问题。 diff --git a/FIX_PLAN.md b/FIX_PLAN.md index 7910e96..24d6f0d 100644 --- a/FIX_PLAN.md +++ b/FIX_PLAN.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # Fix Plan (Verified Issues Only) ## Objective diff --git a/GEMINI.md b/GEMINI.md index d6bf4f4..e7d0dd8 100644 --- a/GEMINI.md +++ b/GEMINI.md @@ -1,5 +1,8 @@ # GEMINI.md +## Documentation Status +- Synced with repository refactor baseline on 2026-02-16 (source of truth: `CHANGELOG.md`). + This document provides a comprehensive overview of the Video Library Manager (VLM) project, intended to be used as instructional context for Gemini. ## Project Overview @@ -48,19 +51,19 @@ The project uses `uv` for dependency management. The main entry point is the `vlm` command. -* Initialize configuration: `vlm config init` -* Scan the library: `vlm scan` -* Parse filenames: `vlm parse` -* Enrich metadata: `vlm enrich` -* Analyze the library: `vlm analyze` -* Generate a plan: `vlm plan` -* Execute the plan (dry-run): `vlm execute` -* Execute the plan (with confirmation): `vlm execute --confirm` -* Rollback the last execution: `vlm rollback` +* Initialize configuration: `uv run vlm config init` +* Scan the library: `uv run vlm scan` +* Parse filenames: `uv run vlm parse` +* Enrich metadata: `uv run vlm enrich` +* Analyze the library: `uv run vlm analyze` +* Generate a plan: `uv run vlm plan` +* Execute the plan (dry-run): `uv run vlm execute` +* Execute the plan (with confirmation): `uv run vlm execute --confirm` +* Rollback the last execution: `uv run vlm rollback` **Running tests:** -* Run all tests: `pytest` +* Run all tests: `uv run pytest` ## Development Conventions diff --git a/IMPLEMENTATION_PLAN_2026-02-13.md b/IMPLEMENTATION_PLAN_2026-02-13.md index 7ba6cd3..de3c42a 100644 --- a/IMPLEMENTATION_PLAN_2026-02-13.md +++ b/IMPLEMENTATION_PLAN_2026-02-13.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # VLM 改进实施计划（2026-02-13） ## 目标 diff --git a/IMPROVEMENT_RECOMMENDATIONS_2026-02-13.md b/IMPROVEMENT_RECOMMENDATIONS_2026-02-13.md index 7cded0c..c6a32fa 100644 --- a/IMPROVEMENT_RECOMMENDATIONS_2026-02-13.md +++ b/IMPROVEMENT_RECOMMENDATIONS_2026-02-13.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # VLM 项目与 Skill 改进建议（2026-02-13） ## 1. 评估范围与依据 diff --git a/README.md b/README.md index 84d47ac..32ca8a6 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,12 @@ # Video Library Manager +## Documentation Status + +- Last synchronized: **2026-02-16** +- CLI command refactor landed (`parse`, `enrich`, `execute`, `rollback` logic moved to `src/vlm/commands/`). +- Unified JSON I/O interfaces are available in `src/vlm/io.py`. +- Full test baseline after refactor: **477 passed**. + A Python-based CLI tool for managing personal video collections with a safety-first, human-in-the-loop approach. ## Features @@ -806,8 +813,11 @@ src/vlm/ ├── context.py # CLIContext and pass_context for commands ├── commands/ # Command implementations │ ├── scan.py # Scan command +│ ├── parse.py # Parse command +│ ├── enrich.py # Enrich command │ ├── analyze.py # Analyze command -│ └── plan.py # Plan command +│ ├── plan.py # Plan command +│ └── execute.py # Execute/Rollback commands ├── scanner.py # File discovery and metadata extraction ├── parser.py # Filename parsing (titles, years, episodes) ├── enrichment.py # Title/reputation enrichment pipeline @@ -815,7 +825,7 @@ src/vlm/ ├── providers/ # External metadata providers (TMDB, etc.) │ ├── base.py # Provider interface │ └── tmdb.py # TMDB API client -├── io.py # JSON/CSV load/save, load_analysis_json, plan/analysis input helpers +├── io.py # Unified JSON/CSV I/O helpers (load/save JSON, analysis writer, plan/analysis adapters) ├── utils.py # UTC time, format_size, etc. ├── analysis.py # Completeness and duplicate detection ├── duplicate_resolve.py # Duplicate group keep-index (by_quality, by_reputation, by_reputation_quality_time, first_seen, manual) diff --git a/REVIEW_REPORT.md b/REVIEW_REPORT.md index 5e55c7d..4a4b56c 100644 --- a/REVIEW_REPORT.md +++ b/REVIEW_REPORT.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # Code Review Report (Verified) ## Scope diff --git a/TMDB_REFACTOR_PLAN.md b/TMDB_REFACTOR_PLAN.md index dc8113c..870a6ca 100644 --- a/TMDB_REFACTOR_PLAN.md +++ b/TMDB_REFACTOR_PLAN.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # TMDB Enrichment 重构执行计划 ## 1. 目标与范围 diff --git a/VLM_PROJECT_AUDIT_REPORT.md b/VLM_PROJECT_AUDIT_REPORT.md index d184470..f42bf95 100644 --- a/VLM_PROJECT_AUDIT_REPORT.md +++ b/VLM_PROJECT_AUDIT_REPORT.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # Video Library Manager (VLM) 深度审计报告 **报告版本**：1.0 diff --git a/codex_review.md b/codex_review.md index 412428d..c65acc7 100644 --- a/codex_review.md +++ b/codex_review.md @@ -1,3 +1,6 @@ +> [!NOTE] +> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16). + # Codex 架构复核报告 评审日期: 2026-02-13 diff --git a/skills/vlm-library-workflow/SKILL.md b/skills/vlm-library-workflow/SKILL.md index 703624c..5b9c87a 100644 --- a/skills/vlm-library-workflow/SKILL.md +++ b/skills/vlm-library-workflow/SKILL.md @@ -1,3 +1,6 @@ +# Documentation Status +- Synced with refactor baseline on 2026-02-16. + --- name: vlm-library-workflow description: Operate and extend the Video Library Manager (`vlm`) with a safety-first, human-in-the-loop workflow across scan, parse, enrich, analyze, plan, review-plan, execute, rollback, and developer verification. Use when requests involve organizing a video library, producing or reviewing `inventory.csv`/`identities.json`/`analysis.json`/`plan.json`, tuning VLM config templates, resolving duplicates or episode gaps, running dry-run/confirm execution, recovering changes via rollback, explaining VLM CLI usage, or developing/modifying VLM features (parser, providers, planner, executor, commands, tests). diff --git a/skills/vlm-library-workflow/references/cli-reference.md b/skills/vlm-library-workflow/references/cli-reference.md index 00f4234..dcb4042 100644 --- a/skills/vlm-library-workflow/references/cli-reference.md +++ b/skills/vlm-library-workflow/references/cli-reference.md @@ -1,3 +1,6 @@ +# Documentation Status +- Updated to current CLI options on 2026-02-16. + # VLM CLI Reference ## Configuration @@ -11,7 +14,7 @@ - `vlm enrich [--input JSON] [--output JSON] [--refresh-all]`: Fetch TMDB metadata. - `vlm analyze [--input JSON] [--output JSON]`: Find gaps and duplicates. - `vlm plan [--input JSON] [--analysis JSON] [--output JSON]`: Generate operations. -- `vlm execute [--plan JSON] [--confirm]`: Move/Rename files. +- `vlm execute [--plan JSON] [--confirm] [--yes] [--verbose-ops] [--safe-mode] [--preserve-directories]`: Move/Rename/Quarantine operations with safety guards. - `vlm rollback [--log PATH]`: Undo operations. ## Management & Reporting @@ -22,5 +25,5 @@ ## Important Config Options (`~/.vlm/config.yaml`) - `library_root`: Path to the video collection. - `templates`: Naming patterns for movies and series. -- `plan.duplicate_keep`: Strategy for duplicates (`by_quality`, `by_reputation`, `first_seen`, `manual`). +- `plan.duplicate_keep`: Strategy for duplicates (`by_quality`, `by_reputation`, `by_reputation_quality_time`, `first_seen`, `manual`). - `enrichment.api_keys`: TMDB and OpenAI keys. diff --git a/skills/vlm-library-workflow/references/command-recipes.md b/skills/vlm-library-workflow/references/command-recipes.md index 1864e80..35d57d9 100644 --- a/skills/vlm-library-workflow/references/command-recipes.md +++ b/skills/vlm-library-workflow/references/command-recipes.md @@ -1,3 +1,6 @@ +# Documentation Status +- Synced with refactor baseline on 2026-02-16. + # VLM Command Recipes ## Baseline diff --git a/skills/vlm-library-workflow/references/dev-guide.md b/skills/vlm-library-workflow/references/dev-guide.md index c8b6df2..c7b8802 100644 --- a/skills/vlm-library-workflow/references/dev-guide.md +++ b/skills/vlm-library-workflow/references/dev-guide.md @@ -1,7 +1,10 @@ +# Documentation Status +- Updated for the modular command architecture on 2026-02-16. + # VLM Developer Guide ## Project Structure -- `src/vlm/cli.py`: Entry point and command definitions. +- `src/vlm/cli.py`: Entry point and thin command wrappers. - `src/vlm/parser.py`: Regex-based filename parsing logic. - `src/vlm/enrichment.py`: Pipeline for external metadata fetching. - `src/vlm/providers/`: API implementations (e.g., TMDB). @@ -11,7 +14,7 @@ ## Adding a New Command 1. Create a new module in `src/vlm/commands/`. 2. Define the command using `@click.command()`. -3. Register it in `src/vlm/cli.py` using `main.add_command()`. +3. Register it in `src/vlm/cli.py` with a Click-decorated function that delegates to the module implementation. ## Modifying the Parser - The parser uses a sequence of regex patterns in `src/vlm/parser.py`. @@ -21,8 +24,8 @@ ## Data Models See `src/vlm/models.py` for core data structures: - `VideoFile`: Basic file metadata. -- `MediaIdentity`: Parsed and enriched information. -- `PlanOperation`: Definition of a file move/rename/quarantine. +- `MovieIdentity` / `SeriesIdentity`: Parsed/enriched identity records. +- `FileOperation`: Definition of a move/rename/quarantine/no-op/preserve-directory operation. ## Testing - **Unit Tests**: `pytest` diff --git a/skills/vlm-library-workflow/references/workflow.md b/skills/vlm-library-workflow/references/workflow.md index 4a0152e..19ba72b 100644 --- a/skills/vlm-library-workflow/references/workflow.md +++ b/skills/vlm-library-workflow/references/workflow.md @@ -1,3 +1,6 @@ +# Documentation Status +- Synced with refactor baseline on 2026-02-16. + # VLM Workflow Guide This guide details the standard end-to-end process for organizing a video library using VLM. diff --git a/src/vlm/cli.py b/src/vlm/cli.py index 61f0bd8..57e290c 100644 --- a/src/vlm/cli.py +++ b/src/vlm/cli.py @@ -193,180 +193,20 @@ def parse(ctx: CLIContext, input: Path, output: Path, inventory: Optional[Path]) vlm parse --input my_inventory.csv # Custom input vlm parse --output parsed_identities.json # Custom output """ - from datetime import datetime, timezone - from vlm.parser import parse_movie, parse_series - from vlm.io import load_inventory_csv, save_identities_json - - config = ctx.config - logger = ctx.logger - try: - # Display parse start message - click.echo(f"Parsing identities from: {input}") - - # Load video metadata from inventory if provided - path_to_metadata = {} - if inventory: - click.echo(f"Loading video metadata from: {inventory}") - inventory_files = load_inventory_csv(inventory) - path_to_metadata = {str(vf.path): vf for vf in inventory_files} - click.echo(f"Loaded metadata for {len(path_to_metadata)} files") - - click.echo() - - # Load inventory via unified I/O layer - inventory_files = load_inventory_csv(input) - video_files = [ - { - 'path': str(vf.path), - 'filename': vf.filename, - 'category': vf.category, - } - for vf in inventory_files - ] - - click.echo(f"Loaded {len(video_files)} files from inventory") - click.echo() - - # Parse identities based on category - movie_identities = [] - series_identities = [] - anime_files = [] - other_files = [] - - def get_video_metadata(file_path: str) -> dict: - """Extract video metadata from inventory if available.""" - if not path_to_metadata: - return {} - vf = path_to_metadata.get(file_path) - if not vf: - return {} - return { - 'size_bytes': vf.size_bytes, - 'modified_timestamp': vf.modified_timestamp.isoformat(), - 'resolution': vf.resolution, - 'codec': vf.codec, - 'duration_seconds': vf.duration_seconds, - 'bitrate_kbps': vf.bitrate_kbps, - } - - for vf in video_files: - filename = vf['filename'] - category = vf['category'] - file_path = vf['path'] - video_metadata = get_video_metadata(file_path) - - if category == 'movie': - identity = parse_movie(filename, extensions=config.video_extensions) - record = { - 'path': file_path, - 'filename': filename, - 'category': category, - 'title': identity.title, - 'year': identity.year, - 'confidence': identity.confidence, - 'needs_review': identity.needs_review - } - if video_metadata: - record['video_metadata'] = video_metadata - movie_identities.append(record) - - elif category == 'series': - identity = parse_series(filename, extensions=config.video_extensions) - record = { - 'path': file_path, - 'filename': filename, - 'category': category, - 'title': identity.title, - 'season': identity.season, - 'episodes': identity.episodes, - 'confidence': identity.confidence, - 'needs_review': identity.needs_review - } - if video_metadata: - record['video_metadata'] = video_metadata - series_identities.append(record) - - elif category == 'anime': - # Anime files are not parsed in v1 - anime_files.append({ - 'path': vf['path'], - 'filename': filename, - 'category': category, - 'note': 'Anime parsing deferred in v1' - }) - - else: - # Other files are not parsed - other_files.append({ - 'path': vf['path'], - 'filename': filename, - 'category': category, - 'note': 'Not categorized for parsing' - }) - - # Display parsing statistics - click.echo("Parsing complete!") - click.echo() - click.echo("Results by category:") - click.echo(f" Movies: {len(movie_identities)}") - - # Count movies needing review - movies_need_review = sum(1 for m in movie_identities if m['needs_review']) - if movies_need_review > 0: - click.echo(f" - Need review: {movies_need_review}") - - click.echo(f" Series: {len(series_identities)}") - - # Count series needing review - series_need_review = sum(1 for s in series_identities if s['needs_review']) - if series_need_review > 0: - click.echo(f" - Need review: {series_need_review}") - - click.echo(f" Anime: {len(anime_files)} (not parsed in v1)") - click.echo(f" Other: {len(other_files)} (not parsed)") - - # Save parsed identities to JSON - click.echo() - click.echo(f"Saving parsed identities to: {output}") - - # Ensure output directory exists - output.parent.mkdir(parents=True, exist_ok=True) - - # Build JSON structure - generation_timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S") - - # Use v2 schema if video metadata was embedded - schema_version = "2.0" if path_to_metadata else "1.0" - - identities_data = { - 'vlm_schema_version': schema_version, - 'metadata': { - 'generated': generation_timestamp, - 'source_inventory': str(input), - 'total_files': len(video_files) - }, - 'movies': movie_identities, - 'series': series_identities, - 'anime': anime_files, - 'other': other_files - } - - # Write JSON via unified I/O layer - save_identities_json(identities_data, output) - - click.echo(f"Parsed identities saved successfully!") - - logger.info(f"Parse completed: {len(movie_identities)} movies, {len(series_identities)} series, saved to {output}") - + from vlm.commands.parse import parse_cmd + parse_cmd(ctx, input, output, inventory) except FileNotFoundError: click.echo(f"Error: Input file not found: {input}", err=True) - logger.error(f"Input file not found: {input}") + ctx.logger.error(f"Input file not found: {input}") sys.exit(1) - - except Exception as e: - click.echo(f"Error during parsing: {e}", err=True) - logger.error(f"Parse failed: {e}", exc_info=True) + except ValueError as e: + click.echo(f"Error: {e}", err=True) + ctx.logger.error(f"Parse failed: {e}") + sys.exit(1) + except OSError as e: + click.echo(f"Error reading/writing files: {e}", err=True) + ctx.logger.error(f"Parse file I/O failed: {e}", exc_info=True) sys.exit(1) @@ -424,150 +264,32 @@ def enrich( Applies incremental cache-backed enrichment to parsed identities and writes results back into identities JSON. """ - import json - from vlm.enrichment import enrich_identities_data - - config = ctx.config - logger = ctx.logger - - if output is None: - output = input - try: - click.echo(f"Enriching identities from: {input}") - click.echo(f"Output file: {output}") - click.echo() - - with open(input, 'r', encoding='utf-8') as jsonfile: - identities_data = json.load(jsonfile) - - if refresh_all and refresh_changed_only: - click.echo("Error: --refresh-all and --refresh-changed-only are mutually exclusive.", err=True) - sys.exit(1) - - if timeout < 1: - click.echo("Error: --timeout must be >= 1", err=True) - sys.exit(1) - - if retries < 0: - click.echo("Error: --retries must be >= 0", err=True) - sys.exit(1) - - refresh_mode = "incremental" - if refresh_all: - refresh_mode = "refresh_all" - elif refresh_changed_only: - refresh_mode = "refresh_changed_only" - - total = ( - len(identities_data.get('movies', [])) - + len(identities_data.get('series', [])) - + len(identities_data.get('anime', [])) - ) - - progress_state: dict[str, Optional[click.ProgressBar]] = {"bar": None} - progress_position = {"current": 0} - progress_bucket = {"value": -1} - is_tty = bool(getattr(sys.stderr, "isatty", lambda: False)()) - - def _enrich_progress(processed: int, total_count: int, metrics: dict[str, int]) -> None: - if total_count <= 0: - return - - if is_tty and progress_state["bar"] is None: - bar = click.progressbar( - length=total_count, - label="Enriching records", - show_pos=True, - ) - progress_state["bar"] = bar.__enter__() - - step = processed - progress_position["current"] - if step > 0 and progress_state["bar"] is not None: - progress_state["bar"].update(step) - progress_position["current"] = processed - - if not is_tty: - percent = int(processed * 100 / total_count) - bucket = percent // 5 - if bucket > progress_bucket["value"] or processed == total_count: - progress_bucket["value"] = bucket - click.echo( - "Progress: " - f"{processed}/{total_count} ({percent}%) " - f"api_calls={metrics.get('api_calls', 0)} " - f"cache_hits={metrics.get('cache_hits', 0)} " - f"failed={metrics.get('failed', 0)}" - ) - - try: - if total == 0: - click.echo("No movie/series/anime records found to enrich.") - enriched_data, stats = enrich_identities_data( - identities_data, - config, - refresh_mode=refresh_mode, - request_timeout=timeout, - retries=retries, - logger=logger, - progress_callback=_enrich_progress, - ) - finally: - if progress_state["bar"] is not None: - progress_state["bar"].__exit__(None, None, None) - click.echo() - - output.parent.mkdir(parents=True, exist_ok=True) - with open(output, 'w', encoding='utf-8') as jsonfile: - json.dump(enriched_data, jsonfile, indent=2, ensure_ascii=False) - - total_records = int(stats['total']) if stats['total'] else 0 - cache_hits = int(stats['cache_hits']) - hit_rate = (cache_hits / total_records * 100.0) if total_records else 0.0 - - click.echo("Enrichment complete!") - click.echo(f" Total records: {total_records}") - click.echo(f" Refresh mode: {refresh_mode}") - click.echo(f" Enriched now: {stats['enriched']}") - click.echo(f" Cache hits: {stats['cache_hits']}") - click.echo(f" Cache hit rate: {hit_rate:.1f}%") - click.echo(f" API calls: {stats['api_calls']}") - click.echo(f" Failed requests: {stats['failed']}") - click.echo(f" Skipped: {stats['skipped']}") - click.echo(f" Needs review: {stats['needs_review']}") - skip_reasons = stats.get("skip_reasons", {}) - if isinstance(skip_reasons, dict): - non_zero = [f"{name}={count}" for name, count in sorted(skip_reasons.items()) if int(count) > 0] - if non_zero: - click.echo(f" Skip reasons: {' '.join(non_zero)}") - failed_items = stats.get('failed_items', []) - if isinstance(failed_items, list) and failed_items: - click.echo(" Failure sample:") - for item in failed_items[:3]: - click.echo( - f" - [{item.get('provider', 'unknown')}] {item.get('title', '')}: {item.get('reason', '')}" - ) - - logger.info( - "Enrich completed: total=%s enriched=%s cache_hits=%s failed=%s skipped=%s mode=%s", - stats['total'], - stats['enriched'], - stats['cache_hits'], - stats['failed'], - stats['skipped'], - refresh_mode, + from vlm.commands.enrich import enrich_cmd + enrich_cmd( + ctx, + input, + output, + refresh_changed_only, + refresh_all, + timeout, + retries, ) except FileNotFoundError: click.echo(f"Error: Input file not found: {input}", err=True) - logger.error(f"Input file not found: {input}") + ctx.logger.error(f"Input file not found: {input}") sys.exit(1) except json.JSONDecodeError as e: click.echo(f"Error: Failed to parse JSON file: {e}", err=True) - logger.error(f"JSON parsing failed during enrich: {e}", exc_info=True) + ctx.logger.error(f"JSON parsing failed during enrich: {e}", exc_info=True) sys.exit(1) - except Exception as e: - click.echo(f"Error during enrichment: {e}", err=True) - logger.error(f"Enrich failed: {e}", exc_info=True) + except ValueError as e: + click.echo(f"Error: {e}", err=True) + ctx.logger.error(f"Enrich validation failed: {e}") + sys.exit(1) + except OSError as e: + click.echo(f"Error reading/writing files: {e}", err=True) + ctx.logger.error(f"Enrich file I/O failed: {e}", exc_info=True) sys.exit(1) @@ -791,8 +513,20 @@ def review_plan_cmd( default=False, help='Print per-operation dry-run logs at INFO level' ) +@click.option( + '--preserve-directories', + is_flag=True, + default=False, + help='Preserve empty source directories instead of allowing them to be destroyed' +) +@click.option( + '--safe-mode', + is_flag=True, + default=False, + help='Enable safe mode: prevent any operations that would destroy directories' +) @pass_context -def execute(ctx: CLIContext, plan: Path, confirm: bool, yes: bool, verbose_ops: bool): +def execute(ctx: CLIContext, plan: Path, confirm: bool, yes: bool, verbose_ops: bool, preserve_directories: bool, safe_mode: bool): """Execute plan (defaults to dry-run, requires --confirm). Executes file operations from a plan. Defaults to dry-run mode which @@ -806,155 +540,20 @@ def execute(ctx: CLIContext, plan: Path, confirm: bool, yes: bool, verbose_ops: vlm execute --confirm # Actually execute operations (with prompt) vlm execute --confirm --yes # Execute without confirmation prompt """ - from vlm.planner import load_plan - from vlm.executor import ExecutionEngine - - config = ctx.config - logger = ctx.logger - try: - # Determine execution mode - mode = "execute" if confirm else "dry-run" - - # Display execution start message - click.echo(f"Loading execution plan from: {plan}") - click.echo() - - # Load execution plan - execution_plan = load_plan(plan) - - # Display plan summary - click.echo(f"Execution plan loaded: {execution_plan.plan_id}") - click.echo(f"Created at: {execution_plan.created_at}") - click.echo(f"Total operations: {len(execution_plan.operations)}") - if execution_plan.human_summary: - click.echo() - click.echo(execution_plan.human_summary) - elif execution_plan.summary or execution_plan.summary_by_reason: - s = execution_plan.summary or {} - by_r = execution_plan.summary_by_reason or {} - parts = [f"操作统计：共 {s.get('total', len(execution_plan.operations))} 条（move {s.get('move', 0)}，rename {s.get('rename', 0)}，quarantine {s.get('quarantine', 0)}，no-op {s.get('no-op', 0)}）"] - if by_r: - parts.append("原因分布：" + "；".join(f"{r}: {c}" for r, c in list(by_r.items())[:5])) - click.echo() - click.echo("\n".join(parts)) - click.echo() - - # Display mode warning - if mode == "dry-run": - click.echo("⚠️ DRY-RUN MODE - No files will be modified") - click.echo(" Use --confirm to actually execute operations") - else: - click.echo("⚠️ EXECUTE MODE - Files will be modified!") - click.echo(" This operation cannot be undone without rollback") - click.echo() - if not yes: - if not click.confirm("Are you sure you want to proceed?"): - click.echo("Execution cancelled.") - return - else: - click.echo("Auto-approved via --yes flag") - - click.echo() - click.echo(f"Executing {len(execution_plan.operations)} operations...") - click.echo() - - # Load state manager to update file statuses during execution - from vlm.state import StateManager - state_path = Path.home() / ".vlm" / "state.json" - state_manager = StateManager(state_path) - - # Create execution engine and execute plan - engine = ExecutionEngine( - logger=logger, - config=config, - verbose_operations=verbose_ops, - state_manager=state_manager - ) - results, summary, rollback_log = engine.execute_plan( - execution_plan, - mode=mode, - confirmed=confirm - ) - - # Display execution progress (show some operations) - if mode == "dry-run": - click.echo("Sample operations (dry-run):") - # Show first 5 operations as examples - for i, result in enumerate(results[:5]): - op = result.operation - if op.operation_type != "no-op": - click.echo(f" [{i+1}] {op.operation_type}: {op.source_path.name}") - if op.destination_path: - click.echo(f" -> {op.destination_path}") - - if len(results) > 5: - click.echo(f" ... and {len(results) - 5} more operations") - else: - # In execute mode, show progress for all operations - for i, result in enumerate(results): - op = result.operation - if op.operation_type != "no-op" and not op.has_conflict: - status = "✓" if result.success else "✗" - click.echo(f" [{i+1}/{len(results)}] {status} {op.operation_type}: {op.source_path.name}") - if result.error_message: - click.echo(f" Error: {result.error_message}") - - # Display execution summary - click.echo() - click.echo("=" * 60) - click.echo(f"Execution Summary ({mode} mode)") - click.echo("=" * 60) - click.echo(f" Total operations: {summary['total']}") - click.echo(f" Successful: {summary['successful']}") - click.echo(f" Failed: {summary['failed']}") - click.echo(f" Skipped: {summary['skipped']}") - click.echo() - - # Save rollback log if in execute mode - if mode == "execute" and rollback_log: - # Save to ~/.vlm/rollback/ directory - rollback_dir = Path.home() / ".vlm" / "rollback" - rollback_dir.mkdir(parents=True, exist_ok=True) - rollback_path = rollback_dir / f"rollback_{rollback_log.log_id}.json" - - engine.save_rollback_log(rollback_log, rollback_path) - - click.echo(f"Rollback log saved to: {rollback_path}") - click.echo() - click.echo("To undo these operations, run:") - click.echo(f" vlm rollback --log {rollback_path}") - click.echo() - - # Log completion - if mode == "dry-run": - click.echo("Dry-run complete! No files were modified.") - click.echo("Review the operations above and use --confirm to execute.") - else: - if summary['failed'] > 0: - click.echo(f"⚠️ Execution completed with {summary['failed']} failures.") - click.echo(" Check the log file for details.") - else: - click.echo("✓ Execution completed successfully!") - - logger.info( - f"Execution completed in {mode} mode: " - f"{summary['successful']} successful, {summary['failed']} failed, {summary['skipped']} skipped" - ) - + from vlm.commands.execute import execute_cmd + execute_cmd(ctx, plan, confirm, yes, verbose_ops, preserve_directories, safe_mode) except FileNotFoundError: - click.echo(f"Error: Plan file not found: {plan}", err=True) - logger.error(f"Plan file not found: {plan}") + click.echo(f"Error: File not found: {plan}", err=True) + ctx.logger.error(f"Execution file not found: {plan}") sys.exit(1) - except ValueError as e: click.echo(f"Error: {e}", err=True) - logger.error(f"Execution failed: {e}") + ctx.logger.error(f"Execution validation failed: {e}") sys.exit(1) - - except Exception as e: + except OSError as e: click.echo(f"Error during execution: {e}", err=True) - logger.error(f"Execution failed: {e}", exc_info=True) + ctx.logger.error(f"Execution I/O failed: {e}", exc_info=True) sys.exit(1) @@ -1177,114 +776,20 @@ def rollback(ctx: CLIContext, log: Optional[Path]): vlm rollback --log rollback_.json # Use specific log vlm rollback --log ~/.vlm/rollback/rollback_*.json """ - from vlm.executor import ExecutionEngine - - config = ctx.config - logger = ctx.logger - try: - # If no log specified, find the most recent rollback log - if log is None: - rollback_dir = Path.home() / ".vlm" / "rollback" - if not rollback_dir.exists(): - click.echo("Error: No rollback logs found.", err=True) - click.echo(f"Rollback directory does not exist: {rollback_dir}", err=True) - sys.exit(1) - - # Find all rollback log files - rollback_logs = sorted(rollback_dir.glob("rollback_*.json"), key=lambda p: p.stat().st_mtime, reverse=True) - - if not rollback_logs: - click.echo("Error: No rollback logs found.", err=True) - click.echo(f"No rollback_*.json files in: {rollback_dir}", err=True) - sys.exit(1) - - # Use the most recent log - log = rollback_logs[0] - click.echo(f"Using most recent rollback log: {log}") - click.echo() - - # Display rollback start message - click.echo(f"Loading rollback log from: {log}") - click.echo() - - # Create execution engine - engine = ExecutionEngine(logger=logger, config=config) - - # Load rollback log - rollback_log = engine.load_rollback_log(log) - - # Display rollback log info - click.echo(f"Rollback log loaded: {rollback_log.log_id}") - click.echo(f"Original execution: {rollback_log.execution_plan_id}") - click.echo(f"Executed at: {rollback_log.executed_at.strftime('%Y-%m-%d %H:%M:%S')}") - click.echo(f"Operations to rollback: {len(rollback_log.operations)}") - click.echo() - - # Display warning - click.echo("⚠️ ROLLBACK OPERATION - Best-effort restoration") - click.echo(" This will attempt to move files back to their original locations.") - click.echo(" Some operations may fail if files have been modified or moved.") - click.echo() - - if not click.confirm("Are you sure you want to proceed with rollback?"): - click.echo("Rollback cancelled.") - return - - click.echo() - click.echo(f"Rolling back {len(rollback_log.operations)} operations...") - click.echo() - - # Perform rollback - results, summary = engine.rollback(rollback_log) - - # Display rollback progress - for i, result in enumerate(results): - op = result.operation - if op.operation_type != "no-op": - status = "✓" if result.success else "✗" - click.echo(f" [{i+1}/{len(results)}] {status} Rollback: {op.destination_path.name if op.destination_path else op.source_path.name}") - if result.error_message: - click.echo(f" Error: {result.error_message}") - - # Display rollback summary - click.echo() - click.echo("=" * 60) - click.echo("Rollback Summary") - click.echo("=" * 60) - click.echo(f" Total operations: {summary['total']}") - click.echo(f" Successful: {summary['successful']}") - click.echo(f" Failed: {summary['failed']}") - click.echo(f" Skipped: {summary['skipped']}") - click.echo() - - # Display completion message - if summary['failed'] > 0: - click.echo(f"⚠️ Rollback completed with {summary['failed']} failures.") - click.echo(" Check the log file for details.") - click.echo(" Some files may not have been restored to their original locations.") - else: - click.echo("✓ Rollback completed successfully!") - click.echo(" All files have been restored to their original locations.") - - logger.info( - f"Rollback completed: " - f"{summary['successful']} successful, {summary['failed']} failed, {summary['skipped']} skipped" - ) - + from vlm.commands.execute import rollback_cmd + rollback_cmd(ctx, log) except FileNotFoundError as e: click.echo(f"Error: {e}", err=True) - logger.error(f"Rollback log not found: {e}") + ctx.logger.error(f"Rollback log not found: {e}") sys.exit(1) - except ValueError as e: - click.echo(f"Error: Invalid rollback log format: {e}", err=True) - logger.error(f"Invalid rollback log: {e}") + click.echo(f"Error: {e}", err=True) + ctx.logger.error(f"Rollback failed: {e}") sys.exit(1) - - except Exception as e: + except OSError as e: click.echo(f"Error during rollback: {e}", err=True) - logger.error(f"Rollback failed: {e}", exc_info=True) + ctx.logger.error(f"Rollback failed: {e}", exc_info=True) sys.exit(1) diff --git a/src/vlm/commands/analyze.py b/src/vlm/commands/analyze.py index 3c0996f..faf0cad 100644 --- a/src/vlm/commands/analyze.py +++ b/src/vlm/commands/analyze.py @@ -1,6 +1,5 @@ """Analyze command implementation.""" -import json from pathlib import Path from typing import Optional @@ -8,9 +7,13 @@ import click from vlm.analysis import analyze_series_completeness, detect_duplicates from vlm.context import CLIContext -from vlm.io import identities_to_analysis_input, load_identities_json, load_inventory_csv +from vlm.io import ( + identities_to_analysis_input, + load_identities_json, + load_inventory_csv, + save_analysis_json, +) from vlm.models import MovieIdentity, SeriesIdentity -from vlm.utils import utc_now def analyze_cmd( @@ -68,7 +71,6 @@ def analyze_cmd( click.echo(f"Saving analysis results to: {output}") output.parent.mkdir(parents=True, exist_ok=True) - generation_timestamp = utc_now().strftime("%Y-%m-%dT%H:%M:%S") completeness_list = [ { "series_title": c.series_title, @@ -96,19 +98,14 @@ def analyze_cmd( "quality_comparison": d.quality_comparison, } ) - analysis_data = { - "vlm_schema_version": "1.0", - "metadata": { - "generated": generation_timestamp, - "source_identities": str(input), - "total_movies": len(movies_data), - "total_series": len(series_data), - }, - "completeness": completeness_list, - "duplicates": duplicates_list, - } - with open(output, "w", encoding="utf-8") as jsonfile: - json.dump(analysis_data, jsonfile, indent=2, ensure_ascii=False) + save_analysis_json( + completeness=completeness_list, + duplicates=duplicates_list, + source_identities=input, + total_movies=len(movies_data), + total_series=len(series_data), + output=output, + ) click.echo("Analysis results saved successfully!") logger.info( diff --git a/src/vlm/commands/enrich.py b/src/vlm/commands/enrich.py new file mode 100644 index 0000000..3d90a01 --- /dev/null +++ b/src/vlm/commands/enrich.py @@ -0,0 +1,146 @@ +"""Enrich command implementation.""" + +from __future__ import annotations + +import sys +from pathlib import Path +from typing import Optional + +import click + +from vlm.context import CLIContext +from vlm.enrichment import enrich_identities_data +from vlm.io import load_json_file, save_json_file + + +def enrich_cmd( + ctx: CLIContext, + input: Path, + output: Optional[Path], + refresh_changed_only: bool, + refresh_all: bool, + timeout: int, + retries: int, +) -> None: + """Enrich identities with translation and reputation metadata.""" + logger = ctx.logger + config = ctx.config + + if output is None: + output = input + + click.echo(f"Enriching identities from: {input}") + click.echo(f"Output file: {output}") + click.echo() + + identities_data = load_json_file(input) + + if refresh_all and refresh_changed_only: + raise ValueError("--refresh-all and --refresh-changed-only are mutually exclusive.") + if timeout < 1: + raise ValueError("--timeout must be >= 1") + if retries < 0: + raise ValueError("--retries must be >= 0") + + refresh_mode = "incremental" + if refresh_all: + refresh_mode = "refresh_all" + elif refresh_changed_only: + refresh_mode = "refresh_changed_only" + + total = ( + len(identities_data.get("movies", [])) + + len(identities_data.get("series", [])) + + len(identities_data.get("anime", [])) + ) + + progress_state: dict[str, Optional[click.ProgressBar]] = {"bar": None} + progress_position = {"current": 0} + progress_bucket = {"value": -1} + is_tty = bool(getattr(sys.stderr, "isatty", lambda: False)()) + + def _enrich_progress(processed: int, total_count: int, metrics: dict[str, int]) -> None: + if total_count <= 0: + return + + if is_tty and progress_state["bar"] is None: + bar = click.progressbar( + length=total_count, + label="Enriching records", + show_pos=True, + ) + progress_state["bar"] = bar.__enter__() + + step = processed - progress_position["current"] + if step > 0 and progress_state["bar"] is not None: + progress_state["bar"].update(step) + progress_position["current"] = processed + + if not is_tty: + percent = int(processed * 100 / total_count) + bucket = percent // 5 + if bucket > progress_bucket["value"] or processed == total_count: + progress_bucket["value"] = bucket + click.echo( + "Progress: " + f"{processed}/{total_count} ({percent}%) " + f"api_calls={metrics.get('api_calls', 0)} " + f"cache_hits={metrics.get('cache_hits', 0)} " + f"failed={metrics.get('failed', 0)}" + ) + + try: + if total == 0: + click.echo("No movie/series/anime records found to enrich.") + enriched_data, stats = enrich_identities_data( + identities_data, + config, + refresh_mode=refresh_mode, + request_timeout=timeout, + retries=retries, + logger=logger, + progress_callback=_enrich_progress, + ) + finally: + if progress_state["bar"] is not None: + progress_state["bar"].__exit__(None, None, None) + click.echo() + + save_json_file(enriched_data, output) + + total_records = int(stats["total"]) if stats["total"] else 0 + cache_hits = int(stats["cache_hits"]) + hit_rate = (cache_hits / total_records * 100.0) if total_records else 0.0 + + click.echo("Enrichment complete!") + click.echo(f" Total records: {total_records}") + click.echo(f" Refresh mode: {refresh_mode}") + click.echo(f" Enriched now: {stats['enriched']}") + click.echo(f" Cache hits: {stats['cache_hits']}") + click.echo(f" Cache hit rate: {hit_rate:.1f}%") + click.echo(f" API calls: {stats['api_calls']}") + click.echo(f" Failed requests: {stats['failed']}") + click.echo(f" Skipped: {stats['skipped']}") + click.echo(f" Needs review: {stats['needs_review']}") + + skip_reasons = stats.get("skip_reasons", {}) + if isinstance(skip_reasons, dict): + non_zero = [f"{name}={count}" for name, count in sorted(skip_reasons.items()) if int(count) > 0] + if non_zero: + click.echo(f" Skip reasons: {' '.join(non_zero)}") + + failed_items = stats.get("failed_items", []) + if isinstance(failed_items, list) and failed_items: + click.echo(" Failure sample:") + for item in failed_items[:3]: + click.echo(f" - [{item.get('provider', 'unknown')}] {item.get('title', '')}: {item.get('reason', '')}") + + logger.info( + "Enrich completed: total=%s enriched=%s cache_hits=%s failed=%s skipped=%s mode=%s", + stats["total"], + stats["enriched"], + stats["cache_hits"], + stats["failed"], + stats["skipped"], + refresh_mode, + ) diff --git a/src/vlm/commands/execute.py b/src/vlm/commands/execute.py new file mode 100644 index 0000000..d1cc84a --- /dev/null +++ b/src/vlm/commands/execute.py @@ -0,0 +1,257 @@ +"""Execute and rollback command implementations.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Optional + +import click + +from vlm.context import CLIContext +from vlm.executor import ExecutionEngine +from vlm.planner import load_plan +from vlm.state import StateManager + + +def _validate_plan_structure(execution_plan) -> list[str]: + """Validate source/destination paths for move/rename operations.""" + validation_errors: list[str] = [] + + for operation in execution_plan.operations: + if operation.operation_type in ("move", "rename"): + if not operation.source_path.exists(): + validation_errors.append(f"Source file does not exist: {operation.source_path}") + elif not operation.source_path.is_file(): + validation_errors.append(f"Source path is not a file: {operation.source_path}") + + for operation in execution_plan.operations: + if operation.operation_type in ("move", "rename") and operation.destination_path: + dest_dir = operation.destination_path.parent + if not dest_dir.exists() and not dest_dir.parent.exists(): + validation_errors.append(f"Cannot create destination directory (parent missing): {dest_dir}") + + return validation_errors + + +def execute_cmd( + ctx: CLIContext, + plan: Path, + confirm: bool, + yes: bool, + verbose_ops: bool, + preserve_directories: bool, + safe_mode: bool, +) -> None: + """Execute an execution plan in dry-run or execute mode.""" + config = ctx.config + logger = ctx.logger + mode = "execute" if confirm else "dry-run" + + click.echo(f"Loading execution plan from: {plan}") + click.echo() + + execution_plan = load_plan(plan) + + if preserve_directories: + click.echo("Directory preservation is enabled (plan metadata/operations will be honored).") + click.echo() + + if safe_mode: + click.echo("Safe mode enabled - validating plan for directory preservation...") + emptied_dirs = execution_plan.metadata.get("emptied_directories", []) + if emptied_dirs: + raise ValueError( + "SAFE MODE VIOLATION: plan would empty directories. " + "Regenerate/adjust the plan to preserve directory structure." + ) + click.echo("Safe mode validation passed - no directories would be destroyed") + click.echo() + + click.echo("Validating directory structure...") + validation_errors = _validate_plan_structure(execution_plan) + if validation_errors: + formatted = "\n".join(f" - {error}" for error in validation_errors[:10]) + if len(validation_errors) > 10: + formatted += f"\n ... and {len(validation_errors) - 10} more errors" + raise ValueError(f"DIRECTORY STRUCTURE VALIDATION FAILED:\n{formatted}") + + click.echo("Directory structure validation passed") + click.echo() + + click.echo(f"Execution plan loaded: {execution_plan.plan_id}") + click.echo(f"Created at: {execution_plan.created_at}") + click.echo(f"Total operations: {len(execution_plan.operations)}") + + if execution_plan.human_summary: + click.echo() + click.echo(execution_plan.human_summary) + elif execution_plan.summary or execution_plan.summary_by_reason: + s = execution_plan.summary or {} + by_r = execution_plan.summary_by_reason or {} + parts = [ + "操作统计：共 " + f"{s.get('total', len(execution_plan.operations))} 条" + f"（move {s.get('move', 0)}，rename {s.get('rename', 0)}，" + f"quarantine {s.get('quarantine', 0)}，no-op {s.get('no-op', 0)}）" + ] + if by_r: + parts.append("原因分布：" + "；".join(f"{r}: {c}" for r, c in list(by_r.items())[:5])) + click.echo() + click.echo("\n".join(parts)) + click.echo() + + if mode == "dry-run": + click.echo("DRY-RUN MODE - No files will be modified") + click.echo("Use --confirm to actually execute operations") + else: + click.echo("EXECUTE MODE - Files will be modified") + click.echo() + if not yes and not click.confirm("Are you sure you want to proceed?"): + click.echo("Execution cancelled.") + return + if yes: + click.echo("Auto-approved via --yes flag") + + click.echo() + click.echo(f"Executing {len(execution_plan.operations)} operations...") + click.echo() + + state_path = Path.home() / ".vlm" / "state.json" + state_manager = StateManager(state_path) + + engine = ExecutionEngine( + logger=logger, + config=config, + verbose_operations=verbose_ops, + state_manager=state_manager, + ) + results, summary, rollback_log = engine.execute_plan( + execution_plan, + mode=mode, + confirmed=confirm, + ) + + if mode == "dry-run": + click.echo("Sample operations (dry-run):") + for i, result in enumerate(results[:5]): + op = result.operation + if op.operation_type != "no-op": + click.echo(f" [{i + 1}] {op.operation_type}: {op.source_path.name}") + if op.destination_path: + click.echo(f" -> {op.destination_path}") + if len(results) > 5: + click.echo(f" ... and {len(results) - 5} more operations") + else: + for i, result in enumerate(results): + op = result.operation + if op.operation_type != "no-op" and not op.has_conflict: + status = "OK" if result.success else "FAIL" + click.echo(f" [{i + 1}/{len(results)}] {status} {op.operation_type}: {op.source_path.name}") + if result.error_message: + click.echo(f" Error: {result.error_message}") + + click.echo() + click.echo("=" * 60) + click.echo(f"Execution Summary ({mode} mode)") + click.echo("=" * 60) + click.echo(f" Total operations: {summary['total']}") + click.echo(f" Successful: {summary['successful']}") + click.echo(f" Failed: {summary['failed']}") + click.echo(f" Skipped: {summary['skipped']}") + click.echo() + + if mode == "execute" and rollback_log: + rollback_dir = Path.home() / ".vlm" / "rollback" + rollback_dir.mkdir(parents=True, exist_ok=True) + rollback_path = rollback_dir / f"rollback_{rollback_log.log_id}.json" + engine.save_rollback_log(rollback_log, rollback_path) + + click.echo(f"Rollback log saved to: {rollback_path}") + click.echo() + click.echo("To undo these operations, run:") + click.echo(f" vlm rollback --log {rollback_path}") + click.echo() + + if mode == "dry-run": + click.echo("Dry-run complete! No files were modified.") + click.echo("Review the operations above and use --confirm to execute.") + elif summary["failed"] > 0: + click.echo(f"Execution completed with {summary['failed']} failures.") + click.echo("Check the log file for details.") + else: + click.echo("Execution completed successfully!") + + logger.info( + "Execution completed in %s mode: %s successful, %s failed, %s skipped", + mode, + summary["successful"], + summary["failed"], + summary["skipped"], + ) + + +def rollback_cmd(ctx: CLIContext, log: Optional[Path]) -> None: + """Rollback a prior execution from rollback log.""" + config = ctx.config + logger = ctx.logger + + if log is None: + rollback_dir = Path.home() / ".vlm" / "rollback" + if not rollback_dir.exists(): + raise FileNotFoundError(f"No rollback logs found. Rollback directory does not exist: {rollback_dir}") + rollback_logs = sorted( + rollback_dir.glob("rollback_*.json"), key=lambda p: p.stat().st_mtime, reverse=True + ) + if not rollback_logs: + raise FileNotFoundError(f"No rollback logs found. No rollback_*.json files in: {rollback_dir}") + log = rollback_logs[0] + + click.echo(f"Loading rollback log: {log}") + + engine = ExecutionEngine(logger=logger, config=config) + rollback_log = engine.load_rollback_log(log) + + click.echo(f"Rollback log loaded: {rollback_log.log_id}") + click.echo(f"Plan ID: {rollback_log.execution_plan_id}") + click.echo(f"Executed at: {rollback_log.executed_at}") + click.echo(f"Operations to rollback: {len(rollback_log.operations)}") + click.echo() + + if not click.confirm("Proceed with rollback?"): + click.echo("Rollback cancelled.") + return + + click.echo(f"Rolling back {len(rollback_log.operations)} operations...") + click.echo() + click.echo("Performing rollback...") + rollback_results, rollback_summary = engine.rollback(rollback_log) + + click.echo() + click.echo("Rollback Summary") + click.echo("=" * 60) + click.echo(f" Total operations: {rollback_summary['total']}") + click.echo(f" Successful: {rollback_summary['successful']}") + click.echo(f" Failed: {rollback_summary['failed']}") + click.echo(f" Skipped: {rollback_summary['skipped']}") + click.echo() + + if rollback_summary["failed"] > 0: + click.echo("Failed operations:") + for result in rollback_results: + if not result.success: + op = result.operation + click.echo(f" - {op.operation_type}: {op.source_path}") + if result.error_message: + click.echo(f" Error: {result.error_message}") + click.echo() + + if rollback_summary["failed"] == 0: + click.echo("Rollback completed successfully!") + else: + click.echo(f"Rollback completed with {rollback_summary['failed']} failures.") + + logger.info( + "Rollback completed: %s successful, %s failed", + rollback_summary["successful"], + rollback_summary["failed"], + ) diff --git a/src/vlm/commands/parse.py b/src/vlm/commands/parse.py new file mode 100644 index 0000000..cbdaab6 --- /dev/null +++ b/src/vlm/commands/parse.py @@ -0,0 +1,165 @@ +"""Parse command implementation.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Optional + +import click + +from vlm.context import CLIContext +from vlm.io import load_inventory_csv, save_identities_json +from vlm.parser import parse_movie, parse_series +from vlm.utils import utc_now + + +def parse_cmd(ctx: CLIContext, input: Path, output: Path, inventory: Optional[Path]) -> None: + """Parse identities from scanned inventory.""" + config = ctx.config + logger = ctx.logger + + click.echo(f"Parsing identities from: {input}") + + path_to_metadata: dict[str, object] = {} + if inventory: + click.echo(f"Loading video metadata from: {inventory}") + inventory_files = load_inventory_csv(inventory) + path_to_metadata = {str(vf.path): vf for vf in inventory_files} + click.echo(f"Loaded metadata for {len(path_to_metadata)} files") + + click.echo() + + inventory_files = load_inventory_csv(input) + video_files = [ + { + "path": str(vf.path), + "filename": vf.filename, + "category": vf.category, + } + for vf in inventory_files + ] + + click.echo(f"Loaded {len(video_files)} files from inventory") + click.echo() + + movie_identities: list[dict] = [] + series_identities: list[dict] = [] + anime_files: list[dict] = [] + other_files: list[dict] = [] + + def get_video_metadata(file_path: str) -> dict: + """Extract video metadata from inventory if available.""" + if not path_to_metadata: + return {} + vf = path_to_metadata.get(file_path) + if not vf: + return {} + return { + "size_bytes": vf.size_bytes, + "modified_timestamp": vf.modified_timestamp.isoformat(), + "resolution": vf.resolution, + "codec": vf.codec, + "duration_seconds": vf.duration_seconds, + "bitrate_kbps": vf.bitrate_kbps, + } + + for vf in video_files: + filename = vf["filename"] + category = vf["category"] + file_path = vf["path"] + video_metadata = get_video_metadata(file_path) + + if category == "movie": + identity = parse_movie(filename, extensions=config.video_extensions) + record = { + "path": file_path, + "filename": filename, + "category": category, + "title": identity.title, + "year": identity.year, + "confidence": identity.confidence, + "needs_review": identity.needs_review, + } + if video_metadata: + record["video_metadata"] = video_metadata + movie_identities.append(record) + elif category == "series": + identity = parse_series(filename, extensions=config.video_extensions) + record = { + "path": file_path, + "filename": filename, + "category": category, + "title": identity.title, + "season": identity.season, + "episodes": identity.episodes, + "confidence": identity.confidence, + "needs_review": identity.needs_review, + } + if video_metadata: + record["video_metadata"] = video_metadata + series_identities.append(record) + elif category == "anime": + anime_files.append( + { + "path": vf["path"], + "filename": filename, + "category": category, + "note": "Anime parsing deferred in v1", + } + ) + else: + other_files.append( + { + "path": vf["path"], + "filename": filename, + "category": category, + "note": "Not categorized for parsing", + } + ) + + click.echo("Parsing complete!") + click.echo() + click.echo("Results by category:") + click.echo(f" Movies: {len(movie_identities)}") + + movies_need_review = sum(1 for m in movie_identities if m["needs_review"]) + if movies_need_review > 0: + click.echo(f" - Need review: {movies_need_review}") + + click.echo(f" Series: {len(series_identities)}") + + series_need_review = sum(1 for s in series_identities if s["needs_review"]) + if series_need_review > 0: + click.echo(f" - Need review: {series_need_review}") + + click.echo(f" Anime: {len(anime_files)} (not parsed in v1)") + click.echo(f" Other: {len(other_files)} (not parsed)") + + click.echo() + click.echo(f"Saving parsed identities to: {output}") + + generation_timestamp = utc_now().strftime("%Y-%m-%dT%H:%M:%S") + schema_version = "2.0" if path_to_metadata else "1.0" + + identities_data = { + "vlm_schema_version": schema_version, + "metadata": { + "generated": generation_timestamp, + "source_inventory": str(input), + "total_files": len(video_files), + }, + "movies": movie_identities, + "series": series_identities, + "anime": anime_files, + "other": other_files, + } + + save_identities_json(identities_data, output) + + click.echo("Parsed identities saved successfully!") + logger.info( + "Parse completed: %s movies, %s series, saved to %s", + len(movie_identities), + len(series_identities), + output, + ) diff --git a/src/vlm/commands/plan.py b/src/vlm/commands/plan.py index de6b265..b38748d 100644 --- a/src/vlm/commands/plan.py +++ b/src/vlm/commands/plan.py @@ -81,6 +81,15 @@ def plan_cmd(ctx: CLIContext, input: Path, output: Path, analysis: Optional[Path click.echo(f" ⚠️ Conflicts detected: {conflicts}") click.echo(" Review the plan file for details on conflicting operations.") + # Check for directory warnings + directory_warning = execution_plan.metadata.get("directory_warning", False) + if directory_warning: + emptied_dirs = execution_plan.metadata.get("emptied_directories", []) + click.echo() + click.echo(f" ⚠️ Directory preservation warning: {len(emptied_dirs)} directories will be emptied") + click.echo(" These directories will be preserved but may be empty after execution.") + click.echo(" Review the plan file for details on preserved directories.") + click.echo() click.echo(f"Saving execution plan to: {output}") output.parent.mkdir(parents=True, exist_ok=True) diff --git a/src/vlm/config.py b/src/vlm/config.py index b7042af..d7f0176 100644 --- a/src/vlm/config.py +++ b/src/vlm/config.py @@ -5,15 +5,17 @@ from pathlib import Path from typing import Optional import yaml +DEFAULT_VIDEO_EXTENSIONS = [ + ".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v" +] + @dataclass class Config: """Configuration for Video Library Manager.""" library_root: Path - video_extensions: list[str] = field(default_factory=lambda: [ - ".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v" - ]) + video_extensions: list[str] = field(default_factory=lambda: list(DEFAULT_VIDEO_EXTENSIONS)) movie_template: str = "movie/{title} ({year})/" series_template: str = "series/{title}/Season {season:02d}/" movie_filename_template: str = "{title} ({year}){ext}" @@ -74,9 +76,7 @@ def load_config(path: Path) -> Config: library_root = Path(library_root_str).expanduser() - video_extensions = data.get("video_extensions", [ - ".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v" - ]) + video_extensions = data.get("video_extensions", list(DEFAULT_VIDEO_EXTENSIONS)) templates = data.get("templates", {}) movie_template = templates.get("movie_dir", "movie/{title} ({year})/") @@ -149,7 +149,7 @@ def create_default_config(path: Path) -> Config: """Create a default configuration file and return the Config object.""" default_config = Config( library_root=Path.home() / "Videos", - video_extensions=[".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"], + video_extensions=list(DEFAULT_VIDEO_EXTENSIONS), ) enrichment_content = { diff --git a/src/vlm/executor.py b/src/vlm/executor.py index bb4ee63..61ba2ce 100644 --- a/src/vlm/executor.py +++ b/src/vlm/executor.py @@ -93,8 +93,17 @@ class ExecutionEngine: transaction_log = None if mode == "execute": log_path = Path.home() / ".vlm" / "transaction.json" - transaction_log = TransactionLog(log_path) - transaction_log.start_transaction(plan) + try: + transaction_log = TransactionLog(log_path) + transaction_log.start_transaction(plan) + except OSError as exc: + log_operation( + self.logger, + logging.WARNING, + f"Transaction log disabled (cannot write {log_path}): {exc}", + operation_type="execute", + ) + transaction_log = None # Execute all operations results = [] @@ -105,9 +114,18 @@ class ExecutionEngine: # Update transaction and state logs in execute mode if mode == "execute": if transaction_log: - transaction_log.mark_operation_complete( - i, result.success, result.error_message - ) + try: + transaction_log.mark_operation_complete( + i, result.success, result.error_message + ) + except OSError as exc: + log_operation( + self.logger, + logging.WARNING, + f"Failed to update transaction log: {exc}", + operation_type="execute", + ) + transaction_log = None # Update file state if successful and not a no-op if result.success and operation.operation_type != "no-op" and self.state_manager: @@ -126,7 +144,15 @@ class ExecutionEngine: if mode == "execute": if transaction_log: status = "completed" if all(r.success for r in results) else "failed" - transaction_log.complete_transaction(status=status) + try: + transaction_log.complete_transaction(status=status) + except OSError as exc: + log_operation( + self.logger, + logging.WARNING, + f"Failed to finalize transaction log: {exc}", + operation_type="execute", + ) if self.state_manager: self.state_manager.save() @@ -189,6 +215,22 @@ class ExecutionEngine: executed_at=executed_at ) + # Handle preserve-directory operations + if operation.operation_type == "preserve-directory": + log_operation( + self.logger, + logging.DEBUG, + f"Preserving directory: {operation.reason}", + operation_type="execute", + file_path=operation.source_path + ) + return OperationResult( + operation=operation, + success=True, + error_message=None, + executed_at=executed_at + ) + # Handle quarantine operations (no destination_path; use QuarantineManager) if operation.operation_type == "quarantine": if not self._quarantine_manager: diff --git a/src/vlm/io.py b/src/vlm/io.py index 4a15ad1..c9195a0 100644 --- a/src/vlm/io.py +++ b/src/vlm/io.py @@ -15,34 +15,71 @@ from vlm.scanner import load_inventory_csv, save_inventory_csv __all__ = [ "load_inventory_csv", "save_inventory_csv", + "load_json_file", + "save_json_file", "load_identities_json", "save_identities_json", "load_analysis_json", + "save_analysis_json", "identities_to_plan_input", "identities_to_analysis_input", ] +def load_json_file(path: Path) -> dict: + """Load a JSON object from disk.""" + with open(path, "r", encoding="utf-8") as f: + return json.load(f) + + +def save_json_file(data: dict, path: Path) -> None: + """Save a JSON object to disk.""" + path.parent.mkdir(parents=True, exist_ok=True) + with open(path, "w", encoding="utf-8") as f: + json.dump(data, f, indent=2, ensure_ascii=False) + + def load_analysis_json(path: Path) -> dict: """Load analysis result from JSON file (metadata, completeness, duplicates). Caller should check file existence and handle missing/invalid keys. """ - with open(path, "r", encoding="utf-8") as f: - return json.load(f) + return load_json_file(path) def load_identities_json(path: Path) -> dict: """Load identities from JSON file.""" - with open(path, "r", encoding="utf-8") as f: - return json.load(f) + return load_json_file(path) def save_identities_json(data: dict, path: Path) -> None: """Save identities dict to JSON file.""" - path.parent.mkdir(parents=True, exist_ok=True) - with open(path, "w", encoding="utf-8") as f: - json.dump(data, f, indent=2, ensure_ascii=False) + save_json_file(data, path) + + +def save_analysis_json( + *, + completeness: list[dict], + duplicates: list[dict], + source_identities: Path, + total_movies: int, + total_series: int, + output: Path, +) -> None: + """Save analysis result JSON using the canonical schema.""" + generation_timestamp = utc_now().strftime("%Y-%m-%dT%H:%M:%S") + analysis_data = { + "vlm_schema_version": "1.0", + "metadata": { + "generated": generation_timestamp, + "source_identities": str(source_identities), + "total_movies": total_movies, + "total_series": total_series, + }, + "completeness": completeness, + "duplicates": duplicates, + } + save_json_file(analysis_data, output) def _video_file_from_record(record: dict) -> VideoFile: diff --git a/src/vlm/models.py b/src/vlm/models.py index 516b945..877e9d6 100644 --- a/src/vlm/models.py +++ b/src/vlm/models.py @@ -113,9 +113,9 @@ class SeriesIdentity: @dataclass class FileOperation: """Represents a single file operation in an execution plan. - + Attributes: - operation_type: Type of operation ("move", "rename", "quarantine", "no-op") + operation_type: Type of operation ("move", "rename", "quarantine", "no-op", "preserve-directory") source_path: Source file path destination_path: Destination file path (None for no-op operations) reason: Human-readable reason for the operation diff --git a/src/vlm/parser.py b/src/vlm/parser.py index de71802..8e829e0 100644 --- a/src/vlm/parser.py +++ b/src/vlm/parser.py @@ -7,14 +7,9 @@ logical identities such as movie titles/years and series titles/seasons/episodes import re from typing import Optional +from vlm.config import DEFAULT_VIDEO_EXTENSIONS from vlm.models import MovieIdentity, SeriesIdentity -# Default extensions used when extensions param is not provided (matches config default) -DEFAULT_VIDEO_EXTENSIONS = [ - ".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v" -] - - # Quality tags to remove from titles QUALITY_TAGS = [ r'\b1080p\b', r'\b720p\b', r'\b480p\b', r'\b2160p\b', diff --git a/src/vlm/planner.py b/src/vlm/planner.py index a9feaf7..8978a01 100644 --- a/src/vlm/planner.py +++ b/src/vlm/planner.py @@ -40,6 +40,42 @@ def _is_sample_path(path: Path) -> bool: return bool(re.search(r"(^|[\s._-])sample($|[\s._-])", stem)) +def _analyze_directory_impact(operations: list[FileOperation]) -> dict: + """Analyze which directories will be emptied by the plan.""" + # Get all source directories that have files being moved/renamed + source_dirs = set() + moved_files = set() + for operation in operations: + if operation.operation_type in ("move", "rename"): + source_dirs.add(operation.source_path.parent) + moved_files.add(operation.source_path) + + emptied_dirs = [] + for dir_path in source_dirs: + # Only analyze directories that actually exist + if not dir_path.exists(): + continue + + # Count files that will remain in this directory after operations + remaining_count = 0 + try: + for item in dir_path.iterdir(): + if item.is_file() and item not in moved_files: + remaining_count += 1 + except (OSError, PermissionError): + # If we can't read the directory, skip analysis + continue + + # If no files will remain, this directory will be emptied + if remaining_count == 0: + emptied_dirs.append(dir_path) + + return { + "emptied_directories": emptied_dirs, + "warning_required": len(emptied_dirs) > 0 + } + + def generate_plan( identities: list[tuple[VideoFile, Union[MovieIdentity, SeriesIdentity, None]]], config: Config, @@ -120,10 +156,29 @@ def generate_plan( conflict_reason=None, ) + # Analyze directory impact and add preservation operations + directory_analysis = _analyze_directory_impact(operations) + if directory_analysis["warning_required"]: + # Add directory preservation operations for emptied directories + for emptied_dir in directory_analysis["emptied_directories"]: + operations.append(FileOperation( + operation_type="preserve-directory", + source_path=emptied_dir, + destination_path=None, + reason=f"Preserve empty source directory: {emptied_dir.name}", + has_conflict=False, + conflict_reason=None + )) + summary = _generate_summary(operations) summary_by_reason = _generate_summary_by_reason(operations) human_summary = _generate_human_summary(operations, summary, summary_by_reason, metadata) + # Add directory warnings to metadata + if directory_analysis["warning_required"]: + metadata["emptied_directories"] = [str(d) for d in directory_analysis["emptied_directories"]] + metadata["directory_warning"] = True + return ExecutionPlan( plan_id=str(uuid.uuid4()), created_at=utc_now(), @@ -445,9 +500,10 @@ def _generate_summary(operations: list[FileOperation]) -> dict: "move": 0, "rename": 0, "quarantine": 0, - "no-op": 0 + "no-op": 0, + "preserve-directory": 0 } - + for operation in operations: op_type = operation.operation_type if op_type in summary: diff --git a/tests/test_analysis.py b/tests/test_analysis.py index 5e991a2..2b91312 100644 --- a/tests/test_analysis.py +++ b/tests/test_analysis.py @@ -5,7 +5,7 @@ Tests series completeness analysis, duplicate detection, and quality comparison. import pytest from pathlib import Path -from datetime import datetime +from datetime import datetime, timezone from vlm.models import SeriesIdentity, SeasonCompleteness, MovieIdentity, VideoFile, DuplicateGroup from vlm.analysis import analyze_series_completeness, detect_duplicates, compare_quality @@ -193,7 +193,7 @@ class TestDuplicateDetection: Path("/movies/The.Matrix.1999.1080p.mkv"), "The.Matrix.1999.1080p.mkv", 2000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1920x1080", codec="h264" @@ -202,7 +202,7 @@ class TestDuplicateDetection: Path("/movies/The.Matrix.1999.720p.mkv"), "The.Matrix.1999.720p.mkv", 1000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1280x720", codec="h264" @@ -211,7 +211,7 @@ class TestDuplicateDetection: Path("/movies/Inception.2010.mkv"), "Inception.2010.mkv", 1500000000, - datetime.now(), + datetime.now(timezone.utc), "movie" ), ] @@ -239,7 +239,7 @@ class TestDuplicateDetection: Path("/series/Breaking.Bad.S01E01.1080p.mkv"), "Breaking.Bad.S01E01.1080p.mkv", 1500000000, - datetime.now(), + datetime.now(timezone.utc), "series", resolution="1920x1080" ), @@ -247,7 +247,7 @@ class TestDuplicateDetection: Path("/series/Breaking.Bad.S01E01.720p.mkv"), "Breaking.Bad.S01E01.720p.mkv", 800000000, - datetime.now(), + datetime.now(timezone.utc), "series", resolution="1280x720" ), @@ -255,7 +255,7 @@ class TestDuplicateDetection: Path("/series/Breaking.Bad.S01E02.mkv"), "Breaking.Bad.S01E02.mkv", 1200000000, - datetime.now(), + datetime.now(timezone.utc), "series" ), ] @@ -278,8 +278,8 @@ class TestDuplicateDetection: ] files = [ - VideoFile(Path("/movies/Movie.A.2020.mkv"), "Movie.A.2020.mkv", 1000000000, datetime.now(), "movie"), - VideoFile(Path("/movies/Movie.B.2021.mkv"), "Movie.B.2021.mkv", 1000000000, datetime.now(), "movie"), + VideoFile(Path("/movies/Movie.A.2020.mkv"), "Movie.A.2020.mkv", 1000000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/Movie.B.2021.mkv"), "Movie.B.2021.mkv", 1000000000, datetime.now(timezone.utc), "movie"), ] result = detect_duplicates(list(zip(identities, files))) @@ -294,8 +294,8 @@ class TestDuplicateDetection: ] files = [ - VideoFile(Path("/movies/Unknown.Movie.mkv"), "Unknown.Movie.mkv", 1000000000, datetime.now(), "movie"), - VideoFile(Path("/movies/Unknown.Movie.2.mkv"), "Unknown.Movie.2.mkv", 1000000000, datetime.now(), "movie"), + VideoFile(Path("/movies/Unknown.Movie.mkv"), "Unknown.Movie.mkv", 1000000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/Unknown.Movie.2.mkv"), "Unknown.Movie.2.mkv", 1000000000, datetime.now(timezone.utc), "movie"), ] result = detect_duplicates(list(zip(identities, files))) @@ -311,8 +311,8 @@ class TestDuplicateDetection: ] files = [ - VideoFile(Path("/series/Unknown.Show.E01.mkv"), "Unknown.Show.E01.mkv", 1000000000, datetime.now(), "series"), - VideoFile(Path("/series/Unknown.Show.Episode.1.mkv"), "Unknown.Show.Episode.1.mkv", 1000000000, datetime.now(), "series"), + VideoFile(Path("/series/Unknown.Show.E01.mkv"), "Unknown.Show.E01.mkv", 1000000000, datetime.now(timezone.utc), "series"), + VideoFile(Path("/series/Unknown.Show.Episode.1.mkv"), "Unknown.Show.Episode.1.mkv", 1000000000, datetime.now(timezone.utc), "series"), ] result = detect_duplicates(list(zip(identities, files))) @@ -327,8 +327,8 @@ class TestDuplicateDetection: ] files = [ - VideoFile(Path("/series/Show.Name.S01.mkv"), "Show.Name.S01.mkv", 1000000000, datetime.now(), "series"), - VideoFile(Path("/series/Show.Name.Season.1.mkv"), "Show.Name.Season.1.mkv", 1000000000, datetime.now(), "series"), + VideoFile(Path("/series/Show.Name.S01.mkv"), "Show.Name.S01.mkv", 1000000000, datetime.now(timezone.utc), "series"), + VideoFile(Path("/series/Show.Name.Season.1.mkv"), "Show.Name.Season.1.mkv", 1000000000, datetime.now(timezone.utc), "series"), ] result = detect_duplicates(list(zip(identities, files))) @@ -347,7 +347,7 @@ class TestDuplicateDetection: Path("/movies/Test.Movie.2020.1080p.mkv"), "Test.Movie.2020.1080p.mkv", 2000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1920x1080", codec="h264", @@ -358,7 +358,7 @@ class TestDuplicateDetection: Path("/movies/Test.Movie.2020.720p.mkv"), "Test.Movie.2020.720p.mkv", 1000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1280x720", codec="h264", @@ -395,9 +395,9 @@ class TestDuplicateDetection: ] files = [ - VideoFile(Path("/series/Show.S01E01-E02.mkv"), "Show.S01E01-E02.mkv", 2000000000, datetime.now(), "series"), - VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"), - VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"), + VideoFile(Path("/series/Show.S01E01-E02.mkv"), "Show.S01E01-E02.mkv", 2000000000, datetime.now(timezone.utc), "series"), + VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"), + VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(timezone.utc), "series"), ] result = detect_duplicates(list(zip(identities, files))) @@ -413,8 +413,8 @@ class TestDuplicateDetection: ] files = [ - VideoFile(Path("/movies/The.Thing.1982.mkv"), "The.Thing.1982.mkv", 1000000000, datetime.now(), "movie"), - VideoFile(Path("/movies/The.Thing.2011.mkv"), "The.Thing.2011.mkv", 1000000000, datetime.now(), "movie"), + VideoFile(Path("/movies/The.Thing.1982.mkv"), "The.Thing.1982.mkv", 1000000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/The.Thing.2011.mkv"), "The.Thing.2011.mkv", 1000000000, datetime.now(timezone.utc), "movie"), ] result = detect_duplicates(list(zip(identities, files))) @@ -429,8 +429,8 @@ class TestDuplicateDetection: ] files = [ - VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"), - VideoFile(Path("/series/Show.S02E01.mkv"), "Show.S02E01.mkv", 1000000000, datetime.now(), "series"), + VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"), + VideoFile(Path("/series/Show.S02E01.mkv"), "Show.S02E01.mkv", 1000000000, datetime.now(timezone.utc), "series"), ] result = detect_duplicates(list(zip(identities, files))) @@ -448,7 +448,7 @@ class TestQualityComparison: Path("/test/file1.mkv"), "file1.mkv", 2000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1920x1080", codec="h264", @@ -459,7 +459,7 @@ class TestQualityComparison: Path("/test/file2.mkv"), "file2.mkv", 1000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1280x720", codec="h265", @@ -490,7 +490,7 @@ class TestQualityComparison: Path("/test/file1.mkv"), "file1.mkv", 2000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1920x1080" # codec, duration, bitrate not available @@ -499,7 +499,7 @@ class TestQualityComparison: Path("/test/file2.mkv"), "file2.mkv", 1000000000, - datetime.now(), + datetime.now(timezone.utc), "movie" # No optional metadata ), @@ -532,7 +532,7 @@ class TestQualityComparison: Path("/test/file.mkv"), "file.mkv", 1500000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1920x1080", codec="h264" diff --git a/tests/test_analysis_properties.py b/tests/test_analysis_properties.py index 9399328..edccede 100644 --- a/tests/test_analysis_properties.py +++ b/tests/test_analysis_properties.py @@ -6,7 +6,7 @@ Each test validates a specific property from the design document. import pytest from pathlib import Path -from datetime import datetime +from datetime import datetime, timezone from hypothesis import given, strategies as st, settings from vlm.models import ( SeriesIdentity, SeasonCompleteness, MovieIdentity, VideoFile, DuplicateGroup @@ -74,7 +74,7 @@ def video_file_strategy(draw, filename=None, category="movie"): path = Path(f"/{category}/{filename}") size_bytes = draw(st.integers(min_value=1000000, max_value=10000000000)) - modified_timestamp = datetime.now() + modified_timestamp = datetime.now(timezone.utc) # Optional metadata has_metadata = draw(st.booleans()) @@ -226,7 +226,7 @@ def test_property_12_duplicate_detection_movies(title, year, duplicate_count): Path(f"/movies/{filename}"), filename, 1000000000 + i * 100000000, - datetime.now(), + datetime.now(timezone.utc), "movie" )) @@ -272,7 +272,7 @@ def test_property_13_duplicate_detection_series(title, season, episode, duplicat Path(f"/series/{filename}"), filename, 1000000000 + i * 100000000, - datetime.now(), + datetime.now(timezone.utc), "series" )) @@ -321,7 +321,7 @@ def test_property_14_duplicate_quality_comparison(title, year, file_count): Path(f"/movies/{filename}"), filename, 1000000000 + i * 100000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1920x1080", codec="h264", @@ -333,7 +333,7 @@ def test_property_14_duplicate_quality_comparison(title, year, file_count): Path(f"/movies/{filename}"), filename, 1000000000 + i * 100000000, - datetime.now(), + datetime.now(timezone.utc), "movie" )) @@ -429,7 +429,7 @@ def test_property_43_duplicate_report_grouping(duplicate_count, format): Path(f"/movies/{filename}"), filename, 1000000000 + j * 500000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1920x1080" if j == 0 else "1280x720", codec="h264" @@ -492,7 +492,7 @@ def test_property_44_summary_report_accuracy(file_count, categories): Path(f"/{category}/{filename}"), filename, size, - datetime.now(), + datetime.now(timezone.utc), category )) diff --git a/tests/test_cli_rollback.py b/tests/test_cli_rollback.py index 6875ce8..7f4f946 100644 --- a/tests/test_cli_rollback.py +++ b/tests/test_cli_rollback.py @@ -2,7 +2,7 @@ import json from pathlib import Path -from datetime import datetime +from datetime import datetime, timezone import pytest from click.testing import CliRunner @@ -46,7 +46,7 @@ def sample_rollback_log(tmp_path): ), success=True, error_message=None, - executed_at=datetime.now() + executed_at=datetime.now(timezone.utc) ), OperationResult( operation=FileOperation( @@ -59,14 +59,14 @@ def sample_rollback_log(tmp_path): ), success=True, error_message=None, - executed_at=datetime.now() + executed_at=datetime.now(timezone.utc) ) ] rollback_log = RollbackLog( log_id="test-log-id", execution_plan_id="test-plan-id", - executed_at=datetime.now(), + executed_at=datetime.now(timezone.utc), operations=operations ) diff --git a/tests/test_executor.py b/tests/test_executor.py index d744307..759c9d3 100644 --- a/tests/test_executor.py +++ b/tests/test_executor.py @@ -4,7 +4,7 @@ Tests execution mode handling, dry-run simulation, and actual file operations. """ import logging -from datetime import datetime +from datetime import datetime, timezone from pathlib import Path from uuid import uuid4 @@ -73,7 +73,7 @@ def sample_plan(temp_test_dir): return ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=operations, summary={"move": 1, "rename": 1} ) @@ -208,7 +208,7 @@ class TestDryRunSimulation: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=[no_op_operation], summary={"no-op": 1} ) @@ -233,7 +233,7 @@ class TestDryRunSimulation: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=[conflicted_operation], summary={"move": 1} ) @@ -285,7 +285,7 @@ class TestExecuteMode: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=[operation], summary={"move": 1} ) @@ -319,7 +319,7 @@ class TestExecuteMode: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=[operation], summary={"move": 1} ) @@ -352,7 +352,7 @@ class TestExecuteMode: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=[conflicted_operation], summary={"move": 1} ) @@ -427,7 +427,7 @@ class TestRollbackLog: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=operations, summary={"move": 2} ) @@ -480,7 +480,7 @@ class TestExecutionSummary: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=operations, summary={"move": 2, "no-op": 1} ) @@ -573,7 +573,7 @@ class TestRollbackLogSaving: data = json.load(f) # Verify timestamps are in ISO format - from datetime import datetime + from datetime import datetime, timezone executed_at = datetime.fromisoformat(data["executed_at"]) assert executed_at is not None @@ -670,7 +670,7 @@ class TestRollbackExecution: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=operations, summary={"move": 2} ) @@ -759,7 +759,7 @@ class TestRollbackExecution: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=operations, summary={"move": 1, "no-op": 1} ) @@ -877,7 +877,7 @@ class TestRollbackExecution: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=operations, summary={"move": 2} ) @@ -920,7 +920,7 @@ class TestRollbackExecution: plan = ExecutionPlan( plan_id=str(uuid4()), - created_at=datetime.now(), + created_at=datetime.now(timezone.utc), operations=[ FileOperation( operation_type="quarantine", diff --git a/tests/test_planner.py b/tests/test_planner.py index 6187665..312110b 100644 --- a/tests/test_planner.py +++ b/tests/test_planner.py @@ -1,7 +1,7 @@ """Unit tests for plan generator.""" import pytest -from datetime import datetime +from datetime import datetime, timezone from pathlib import Path from vlm.config import Config @@ -36,7 +36,7 @@ def test_generate_plan_for_movie_with_year(config): path=Path("/mnt/nas/videos/movie/Some.Movie.2020.1080p.mkv"), filename="Some.Movie.2020.1080p.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -67,7 +67,7 @@ def test_generate_plan_for_movie_without_year(config): path=Path("/mnt/nas/videos/movie/random_movie.mkv"), filename="random_movie.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -94,7 +94,7 @@ def test_generate_plan_for_series_with_season_and_episode(config): path=Path("/mnt/nas/videos/series/Show.Name.S01E05.mkv"), filename="Show.Name.S01E05.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ) @@ -123,7 +123,7 @@ def test_generate_plan_blocks_series_with_high_season(config): path=Path("/mnt/nas/videos/series/Show.Name.S20E01.mkv"), filename="Show.Name.S20E01.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ) identity = SeriesIdentity( @@ -147,7 +147,7 @@ def test_generate_plan_blocks_series_with_high_episode(config): path=Path("/mnt/nas/videos/series/Show.Name.S01E120.mkv"), filename="Show.Name.S01E120.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ) identity = SeriesIdentity( @@ -171,7 +171,7 @@ def test_generate_plan_sample_is_noop_by_default(config): path=Path("/mnt/nas/videos/series/Show.Name.Sample.S01E01.mkv"), filename="Show.Name.Sample.S01E01.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ) identity = SeriesIdentity( @@ -196,7 +196,7 @@ def test_generate_plan_sample_can_be_included_via_config(config): path=Path("/mnt/nas/videos/series/Show.Name.Sample.S01E01.mkv"), filename="Show.Name.Sample.S01E01.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ) identity = SeriesIdentity( @@ -219,7 +219,7 @@ def test_generate_plan_for_series_without_season(config): path=Path("/mnt/nas/videos/series/ambiguous_show.mkv"), filename="ambiguous_show.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ) @@ -247,7 +247,7 @@ def test_generate_plan_for_anime_category(config): path=Path("/mnt/nas/videos/anime/Some.Anime.01.mkv"), filename="Some.Anime.01.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="anime" ) @@ -267,7 +267,7 @@ def test_generate_plan_for_other_category(config): path=Path("/mnt/nas/videos/other/random.mkv"), filename="random.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="other" ) @@ -286,7 +286,7 @@ def test_generate_plan_preserves_category_boundaries(config): path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"), filename="Movie.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -311,7 +311,7 @@ def test_generate_plan_for_multi_episode_file(config): path=Path("/mnt/nas/videos/series/Show.S01E01E02.mkv"), filename="Show.S01E01E02.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ) @@ -338,7 +338,7 @@ def test_generate_plan_file_already_at_target(config): path=Path("/mnt/nas/videos/movie/Some Movie (2020)/Some Movie (2020).mkv"), filename="Some Movie (2020).mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -364,7 +364,7 @@ def test_generate_plan_rename_vs_move(config): path=Path("/mnt/nas/videos/movie/Some Movie (2020)/old_name.mkv"), filename="old_name.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -385,7 +385,7 @@ def test_generate_plan_rename_vs_move(config): path=Path("/mnt/nas/videos/movie/wrong_dir/Some Movie (2020).mkv"), filename="Some Movie (2020).mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -403,7 +403,7 @@ def test_generate_plan_summary(config): path=Path("/mnt/nas/videos/movie/Movie1.2020.mkv"), filename="Movie1.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ), MovieIdentity( @@ -420,7 +420,7 @@ def test_generate_plan_summary(config): path=Path("/mnt/nas/videos/movie/Movie2.mkv"), filename="Movie2.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ), MovieIdentity( @@ -437,7 +437,7 @@ def test_generate_plan_summary(config): path=Path("/mnt/nas/videos/anime/Anime.01.mkv"), filename="Anime.01.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="anime" ), None @@ -448,7 +448,7 @@ def test_generate_plan_summary(config): path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"), filename="Show.S01E01.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ), SeriesIdentity( @@ -480,14 +480,14 @@ def test_generate_plan_with_analysis_by_quality(config): path=p_720, filename="Test.2020.720p.WEB-DL.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie", ) vf_1080 = VideoFile( path=p_1080, filename="Test.2020.1080p.BluRay.mkv", size_bytes=2000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie", ) identity = MovieIdentity( @@ -528,14 +528,14 @@ def test_generate_plan_with_analysis_by_reputation_quality_time_reason(config): path=p_old, filename="Test.2020.720p.WEB-DL.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie", ) vf_new = VideoFile( path=p_new, filename="Test.2020.1080p.BluRay.mkv", size_bytes=2000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie", ) identity = MovieIdentity( @@ -574,14 +574,14 @@ def test_generate_plan_with_analysis_by_reputation_missing_scores_uses_fallback_ path=p_web, filename="Test.2020.2160p.WEB-DL.mkv", size_bytes=3000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie", ) vf_bluray = VideoFile( path=p_bluray, filename="Test.2020.1080p.BluRay.mkv", size_bytes=2000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie", ) identity = MovieIdentity( @@ -622,14 +622,14 @@ def test_generate_plan_human_summary_marks_disc_files_as_high_risk(config): path=p_disc1, filename="The.Best.of.Youth.DISC1.mkv", size_bytes=2000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie", ) vf2 = VideoFile( path=p_disc2, filename="The.Best.of.Youth.DISC2.mkv", size_bytes=1800000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie", ) identity = MovieIdentity( @@ -669,7 +669,7 @@ def test_generate_plan_with_different_extensions(config): path=Path(f"/mnt/nas/videos/movie/Movie.2020{ext}"), filename=f"Movie.2020{ext}", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -704,7 +704,7 @@ def test_conflict_detection_for_movie(config, tmp_path): path=tmp_path / "movie" / "Some.Movie.2020.1080p.mkv", filename="Some.Movie.2020.1080p.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -741,7 +741,7 @@ def test_conflict_detection_for_series(config, tmp_path): path=tmp_path / "series" / "Show.Name.S01E05.1080p.mkv", filename="Show.Name.S01E05.1080p.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ) @@ -776,7 +776,7 @@ def test_no_conflict_when_destination_does_not_exist(config, tmp_path): path=source_dir / "Some.Movie.2020.mkv", filename="Some.Movie.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -814,7 +814,7 @@ def test_conflict_detection_with_multiple_files(config, tmp_path): path=tmp_path / "movie" / "Movie1.2020.mkv", filename="Movie1.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ), MovieIdentity( @@ -831,7 +831,7 @@ def test_conflict_detection_with_multiple_files(config, tmp_path): path=tmp_path / "movie" / "Movie2.2021.mkv", filename="Movie2.2021.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ), MovieIdentity( @@ -864,7 +864,7 @@ def test_save_plan_to_json(config, tmp_path): path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"), filename="Movie.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -909,7 +909,7 @@ def test_load_plan_from_json(config, tmp_path): path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"), filename="Movie.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -963,7 +963,7 @@ def test_save_and_load_plan_with_conflicts(config, tmp_path): path=tmp_path / "movie" / "Movie.2020.mkv", filename="Movie.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -1003,7 +1003,7 @@ def test_save_and_load_plan_with_no_op_operations(config, tmp_path): path=Path("/mnt/nas/videos/anime/Anime.01.mkv"), filename="Anime.01.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="anime" ), None @@ -1014,7 +1014,7 @@ def test_save_and_load_plan_with_no_op_operations(config, tmp_path): path=Path("/mnt/nas/videos/movie/Movie.mkv"), filename="Movie.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ), MovieIdentity( @@ -1048,7 +1048,7 @@ def test_save_plan_json_is_human_readable(config, tmp_path): path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"), filename="Movie.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -1089,7 +1089,7 @@ def test_save_plan_with_multiple_operations(config, tmp_path): path=Path("/mnt/nas/videos/movie/Movie1.2020.mkv"), filename="Movie1.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ), MovieIdentity( @@ -1106,7 +1106,7 @@ def test_save_plan_with_multiple_operations(config, tmp_path): path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"), filename="Show.S01E01.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ), SeriesIdentity( @@ -1124,7 +1124,7 @@ def test_save_plan_with_multiple_operations(config, tmp_path): path=Path("/mnt/nas/videos/anime/Anime.01.mkv"), filename="Anime.01.mkv", size_bytes=500000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="anime" ), None @@ -1176,7 +1176,7 @@ def test_plan_json_includes_all_required_fields(config, tmp_path): path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"), filename="Movie.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -1214,7 +1214,7 @@ def test_movie_rejected_by_review_generates_noop(config): path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"), filename="Movie.2020.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="movie" ) @@ -1238,7 +1238,7 @@ def test_series_rejected_by_review_generates_noop(config): path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"), filename="Show.S01E01.mkv", size_bytes=1000000, - modified_timestamp=datetime.now(), + modified_timestamp=datetime.now(timezone.utc), category="series" ) diff --git a/tests/test_quarantine.py b/tests/test_quarantine.py index 9d8c77f..b0d9694 100644 --- a/tests/test_quarantine.py +++ b/tests/test_quarantine.py @@ -3,7 +3,7 @@ import json import pytest from pathlib import Path -from datetime import datetime +from datetime import datetime, timezone from vlm.quarantine import QuarantineManager from vlm.config import Config diff --git a/tests/test_reports.py b/tests/test_reports.py index ea75ee0..d95f55e 100644 --- a/tests/test_reports.py +++ b/tests/test_reports.py @@ -185,7 +185,7 @@ class TestInventoryReport: Path("/test.mkv"), "test.mkv", 1000, - datetime.now(), + datetime.now(timezone.utc), "movie" ) ] @@ -377,7 +377,7 @@ class TestDuplicateReport: Path("/movies/The.Matrix.1999.1080p.mkv"), "The.Matrix.1999.1080p.mkv", 2000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1920x1080", codec="h264", @@ -388,7 +388,7 @@ class TestDuplicateReport: Path("/movies/The.Matrix.1999.720p.mkv"), "The.Matrix.1999.720p.mkv", 1000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1280x720", codec="h264" @@ -448,14 +448,14 @@ class TestDuplicateReport: Path("/movies/Inception.2010.1080p.mkv"), "Inception.2010.1080p.mkv", 2000000000, - datetime.now(), + datetime.now(timezone.utc), "movie" ), VideoFile( Path("/movies/Inception.2010.720p.mkv"), "Inception.2010.720p.mkv", 1000000000, - datetime.now(), + datetime.now(timezone.utc), "movie" ), ] @@ -501,14 +501,14 @@ class TestDuplicateReport: Path("/series/Breaking.Bad.S01E01.1080p.mkv"), "Breaking.Bad.S01E01.1080p.mkv", 1500000000, - datetime.now(), + datetime.now(timezone.utc), "series" ), VideoFile( Path("/series/Breaking.Bad.S01E01.720p.mkv"), "Breaking.Bad.S01E01.720p.mkv", 800000000, - datetime.now(), + datetime.now(timezone.utc), "series" ), ] @@ -564,8 +564,8 @@ class TestDuplicateReport: # Create two duplicate groups with different sizes identity1 = MovieIdentity("Small Movie", 2020, 0.9, False, "Small.Movie.mkv") files1 = [ - VideoFile(Path("/movies/Small.Movie.1.mkv"), "Small.Movie.1.mkv", 500000000, datetime.now(), "movie"), - VideoFile(Path("/movies/Small.Movie.2.mkv"), "Small.Movie.2.mkv", 600000000, datetime.now(), "movie"), + VideoFile(Path("/movies/Small.Movie.1.mkv"), "Small.Movie.1.mkv", 500000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/Small.Movie.2.mkv"), "Small.Movie.2.mkv", 600000000, datetime.now(timezone.utc), "movie"), ] quality1 = [ {'filename': 'Small.Movie.1.mkv', 'path': '/movies/Small.Movie.1.mkv', 'size_bytes': 500000000}, @@ -574,8 +574,8 @@ class TestDuplicateReport: identity2 = MovieIdentity("Large Movie", 2021, 0.9, False, "Large.Movie.mkv") files2 = [ - VideoFile(Path("/movies/Large.Movie.1.mkv"), "Large.Movie.1.mkv", 2000000000, datetime.now(), "movie"), - VideoFile(Path("/movies/Large.Movie.2.mkv"), "Large.Movie.2.mkv", 1800000000, datetime.now(), "movie"), + VideoFile(Path("/movies/Large.Movie.1.mkv"), "Large.Movie.1.mkv", 2000000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/Large.Movie.2.mkv"), "Large.Movie.2.mkv", 1800000000, datetime.now(timezone.utc), "movie"), ] quality2 = [ {'filename': 'Large.Movie.1.mkv', 'path': '/movies/Large.Movie.1.mkv', 'size_bytes': 2000000000}, @@ -600,8 +600,8 @@ class TestDuplicateReport: """Sort order should use quality_comparison sizes when VideoFile sizes are zero.""" identity1 = MovieIdentity("Tiny", 2020, 0.9, False, "Tiny.mkv") files1 = [ - VideoFile(Path("/movies/Tiny.1.mkv"), "Tiny.1.mkv", 0, datetime.now(), "movie"), - VideoFile(Path("/movies/Tiny.2.mkv"), "Tiny.2.mkv", 0, datetime.now(), "movie"), + VideoFile(Path("/movies/Tiny.1.mkv"), "Tiny.1.mkv", 0, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/Tiny.2.mkv"), "Tiny.2.mkv", 0, datetime.now(timezone.utc), "movie"), ] quality1 = [ {"filename": "Tiny.1.mkv", "path": "/movies/Tiny.1.mkv", "size_bytes": 600000000}, @@ -610,8 +610,8 @@ class TestDuplicateReport: identity2 = MovieIdentity("Huge", 2021, 0.9, False, "Huge.mkv") files2 = [ - VideoFile(Path("/movies/Huge.1.mkv"), "Huge.1.mkv", 0, datetime.now(), "movie"), - VideoFile(Path("/movies/Huge.2.mkv"), "Huge.2.mkv", 0, datetime.now(), "movie"), + VideoFile(Path("/movies/Huge.1.mkv"), "Huge.1.mkv", 0, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/Huge.2.mkv"), "Huge.2.mkv", 0, datetime.now(timezone.utc), "movie"), ] quality2 = [ {"filename": "Huge.1.mkv", "path": "/movies/Huge.1.mkv", "size_bytes": 3000000000}, @@ -632,12 +632,12 @@ class TestSummaryReport: def test_generate_summary_report(self): """Test generating summary report with various files.""" files = [ - VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(), "movie"), - VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(), "movie"), - VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"), - VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"), - VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(), "anime"), - VideoFile(Path("/other/Random.mkv"), "Random.mkv", 500000000, datetime.now(), "other"), + VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"), + VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(timezone.utc), "series"), + VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(timezone.utc), "anime"), + VideoFile(Path("/other/Random.mkv"), "Random.mkv", 500000000, datetime.now(timezone.utc), "other"), ] library_root = Path("/mnt/nas/videos") @@ -677,8 +677,8 @@ class TestSummaryReport: def test_generate_summary_report_single_category(self): """Test generating summary report with files in single category.""" files = [ - VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 1000000000, datetime.now(), "movie"), - VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 2000000000, datetime.now(), "movie"), + VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 1000000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 2000000000, datetime.now(timezone.utc), "movie"), ] library_root = Path("/mnt/nas/videos") diff --git a/tests/test_reports_integration.py b/tests/test_reports_integration.py index 0caf967..d0fc97a 100644 --- a/tests/test_reports_integration.py +++ b/tests/test_reports_integration.py @@ -5,7 +5,7 @@ Tests the complete workflow from analysis to report generation. import json from pathlib import Path -from datetime import datetime +from datetime import datetime, timezone from vlm.models import SeriesIdentity, VideoFile, MovieIdentity from vlm.analysis import analyze_series_completeness, detect_duplicates from vlm.reports import generate_completeness_report, generate_duplicate_report, generate_summary_report @@ -59,7 +59,7 @@ class TestReportsIntegration: Path("/movies/The.Matrix.1999.1080p.mkv"), "The.Matrix.1999.1080p.mkv", 2000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1920x1080", codec="h264" @@ -68,7 +68,7 @@ class TestReportsIntegration: Path("/movies/The.Matrix.1999.720p.mkv"), "The.Matrix.1999.720p.mkv", 1000000000, - datetime.now(), + datetime.now(timezone.utc), "movie", resolution="1280x720", codec="h264" @@ -77,7 +77,7 @@ class TestReportsIntegration: Path("/movies/Inception.2010.mkv"), "Inception.2010.mkv", 1500000000, - datetime.now(), + datetime.now(timezone.utc), "movie" ), ] @@ -107,11 +107,11 @@ class TestReportsIntegration: """Test summary report generation with mixed file types.""" # Create test files files = [ - VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(), "movie"), - VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(), "movie"), - VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"), - VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"), - VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(), "anime"), + VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(timezone.utc), "movie"), + VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"), + VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(timezone.utc), "series"), + VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(timezone.utc), "anime"), ] # Generate summary report diff --git a/tests/test_state.py b/tests/test_state.py index 1f71bd6..2eb284f 100644 --- a/tests/test_state.py +++ b/tests/test_state.py @@ -92,7 +92,7 @@ class TestLoadSaveState: store = StateStore( states={}, version='1.0', - last_updated=datetime.now() + last_updated=datetime.now(timezone.utc) ) save_state(store, state_path)