refactor CLI command modules and synchronize docs
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
## Project Structure & Module Organization
|
||||
- Core package lives in `src/vlm/`.
|
||||
- CLI entrypoint is `src/vlm/cli.py` (`vlm` console script). Commands use `pass_context` and `CLIContext` from `context.py`; some command logic lives in `commands/` (e.g. `scan`, `analyze`, `plan`).
|
||||
- CLI entrypoint is `src/vlm/cli.py` (`vlm` console script). Commands use `pass_context` and `CLIContext` from `context.py`; command logic is modularized in `commands/` (e.g. `scan`, `parse`, `enrich`, `analyze`, `plan`, `execute`).
|
||||
- Functional modules by concern: scanning (`scanner.py`), parsing (`parser.py`), enrichment (`enrichment.py`, `cache.py`, `providers/`), analysis/planning/execution (`analysis.py`, `planner.py`, `executor.py`), I/O helpers (`io.py`), utilities (`utils.py`), state/reporting/logging (`state.py`, `reports.py`, `logging_config.py`), config (`config.py`), models (`models.py`).
|
||||
- Tests live in `tests/` and mirror feature areas (e.g. `tests/test_scanner.py`, `tests/test_cli_state.py`, `tests/test_enrichment.py`).
|
||||
- Project metadata and tool config are in `pyproject.toml`.
|
||||
@@ -36,3 +36,8 @@
|
||||
## Security & Configuration Tips
|
||||
- Do not commit local paths, personal media metadata, or generated state/log artifacts.
|
||||
- Validate config changes against `vlm --help` and at least one end-to-end CLI flow before merging.
|
||||
|
||||
|
||||
## Documentation baseline
|
||||
- Updated to reflect refactor results as of 2026-02-16.
|
||||
- Canonical release notes are tracked in `CHANGELOG.md`.
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# Video Library Manager - Architecture Review
|
||||
|
||||
**Review Date**: 2026-02-13
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# Post-Audit Fixes Implementation Plan (2026-02-13)
|
||||
|
||||
## Objective
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# Changelog
|
||||
|
||||
## 2026-02-16
|
||||
|
||||
### Refactor Results
|
||||
|
||||
- Modularized CLI command implementations:
|
||||
- Added `src/vlm/commands/parse.py`
|
||||
- Added `src/vlm/commands/enrich.py`
|
||||
- Added `src/vlm/commands/execute.py`
|
||||
- Refactored `src/vlm/cli.py` to delegate `parse`, `enrich`, `execute`, and `rollback` to command modules
|
||||
|
||||
- Unified I/O layer for JSON handling:
|
||||
- Added `load_json_file`, `save_json_file`, `save_analysis_json` in `src/vlm/io.py`
|
||||
- Updated analyze/enrich paths to use unified I/O entry points
|
||||
|
||||
- Parser/config consistency improvements:
|
||||
- Added shared `DEFAULT_VIDEO_EXTENSIONS` in `src/vlm/config.py`
|
||||
- Updated `src/vlm/parser.py` to use the shared config constant instead of local hardcoded defaults
|
||||
|
||||
- Execution reliability hardening:
|
||||
- Updated `src/vlm/executor.py` transaction logging to fail gracefully when `~/.vlm` is not writable (warn + continue)
|
||||
|
||||
- Test stability improvements:
|
||||
- Standardized test timestamps to timezone-aware UTC (`datetime.now(timezone.utc)`)
|
||||
- Added missing `timezone` imports where required
|
||||
|
||||
### Verification
|
||||
|
||||
- Full test suite passed after refactor:
|
||||
- `477 passed`
|
||||
|
||||
### Documentation Sync
|
||||
|
||||
- Updated repository documentation set to match the refactor baseline:
|
||||
- Core docs: `README.md`, `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`
|
||||
- Skill docs: `skills/vlm-library-workflow/SKILL.md` and all references under `skills/vlm-library-workflow/references/`
|
||||
- Legacy review/plan docs now explicitly marked as historical snapshots and redirected to `CHANGELOG.md` as the current source of truth
|
||||
@@ -1,5 +1,8 @@
|
||||
# CLAUDE.md
|
||||
|
||||
## Documentation Status
|
||||
- Synchronized with refactor baseline on 2026-02-16 (see `CHANGELOG.md`).
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
@@ -91,11 +94,11 @@ VLM follows a read-first, multi-stage pipeline:
|
||||
### Module Organization
|
||||
- `cli.py` - Click-based CLI interface, global options, command registration
|
||||
- `context.py` - CLIContext (config, paths) and pass_context for commands
|
||||
- `commands/` - Command implementations (scan, analyze, plan)
|
||||
- `commands/` - Command implementations (scan, parse, enrich, analyze, plan, execute/rollback)
|
||||
- `scanner.py` - File discovery using system `find` command, metadata extraction via ffprobe
|
||||
- `parser.py` - Filename parsing using regex patterns (movies: title + year, series: SxxExx)
|
||||
- `enrichment.py` - Enrichment pipeline; `cache.py` - SQLite cache; `providers/` - TMDB etc.
|
||||
- `io.py` - Load/save identities JSON/CSV, plan/analysis input helpers
|
||||
- `io.py` - Unified JSON/CSV I/O helpers (including analysis writer and data adapters)
|
||||
- `utils.py` - UTC time, format_size, shared helpers
|
||||
- `analysis.py` - Completeness checking (episode gaps) and duplicate detection
|
||||
- `duplicate_resolve.py` - Duplicate group resolution (by_quality, by_reputation, first_seen, manual)
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# VLM 代码改进清单
|
||||
|
||||
本文档记录对 Video Library Manager (VLM) 项目的代码审查发现的问题及对应解决方案。排除 AI/OpenAPI 相关问题。
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# Fix Plan (Verified Issues Only)
|
||||
|
||||
## Objective
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
# GEMINI.md
|
||||
|
||||
## Documentation Status
|
||||
- Synced with repository refactor baseline on 2026-02-16 (source of truth: `CHANGELOG.md`).
|
||||
|
||||
This document provides a comprehensive overview of the Video Library Manager (VLM) project, intended to be used as instructional context for Gemini.
|
||||
|
||||
## Project Overview
|
||||
@@ -48,19 +51,19 @@ The project uses `uv` for dependency management.
|
||||
|
||||
The main entry point is the `vlm` command.
|
||||
|
||||
* Initialize configuration: `vlm config init`
|
||||
* Scan the library: `vlm scan`
|
||||
* Parse filenames: `vlm parse`
|
||||
* Enrich metadata: `vlm enrich`
|
||||
* Analyze the library: `vlm analyze`
|
||||
* Generate a plan: `vlm plan`
|
||||
* Execute the plan (dry-run): `vlm execute`
|
||||
* Execute the plan (with confirmation): `vlm execute --confirm`
|
||||
* Rollback the last execution: `vlm rollback`
|
||||
* Initialize configuration: `uv run vlm config init`
|
||||
* Scan the library: `uv run vlm scan`
|
||||
* Parse filenames: `uv run vlm parse`
|
||||
* Enrich metadata: `uv run vlm enrich`
|
||||
* Analyze the library: `uv run vlm analyze`
|
||||
* Generate a plan: `uv run vlm plan`
|
||||
* Execute the plan (dry-run): `uv run vlm execute`
|
||||
* Execute the plan (with confirmation): `uv run vlm execute --confirm`
|
||||
* Rollback the last execution: `uv run vlm rollback`
|
||||
|
||||
**Running tests:**
|
||||
|
||||
* Run all tests: `pytest`
|
||||
* Run all tests: `uv run pytest`
|
||||
|
||||
## Development Conventions
|
||||
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# VLM 改进实施计划(2026-02-13)
|
||||
|
||||
## 目标
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# VLM 项目与 Skill 改进建议(2026-02-13)
|
||||
|
||||
## 1. 评估范围与依据
|
||||
|
||||
@@ -1,5 +1,12 @@
|
||||
# Video Library Manager
|
||||
|
||||
## Documentation Status
|
||||
|
||||
- Last synchronized: **2026-02-16**
|
||||
- CLI command refactor landed (`parse`, `enrich`, `execute`, `rollback` logic moved to `src/vlm/commands/`).
|
||||
- Unified JSON I/O interfaces are available in `src/vlm/io.py`.
|
||||
- Full test baseline after refactor: **477 passed**.
|
||||
|
||||
A Python-based CLI tool for managing personal video collections with a safety-first, human-in-the-loop approach.
|
||||
|
||||
## Features
|
||||
@@ -806,8 +813,11 @@ src/vlm/
|
||||
├── context.py # CLIContext and pass_context for commands
|
||||
├── commands/ # Command implementations
|
||||
│ ├── scan.py # Scan command
|
||||
│ ├── parse.py # Parse command
|
||||
│ ├── enrich.py # Enrich command
|
||||
│ ├── analyze.py # Analyze command
|
||||
│ └── plan.py # Plan command
|
||||
│ ├── plan.py # Plan command
|
||||
│ └── execute.py # Execute/Rollback commands
|
||||
├── scanner.py # File discovery and metadata extraction
|
||||
├── parser.py # Filename parsing (titles, years, episodes)
|
||||
├── enrichment.py # Title/reputation enrichment pipeline
|
||||
@@ -815,7 +825,7 @@ src/vlm/
|
||||
├── providers/ # External metadata providers (TMDB, etc.)
|
||||
│ ├── base.py # Provider interface
|
||||
│ └── tmdb.py # TMDB API client
|
||||
├── io.py # JSON/CSV load/save, load_analysis_json, plan/analysis input helpers
|
||||
├── io.py # Unified JSON/CSV I/O helpers (load/save JSON, analysis writer, plan/analysis adapters)
|
||||
├── utils.py # UTC time, format_size, etc.
|
||||
├── analysis.py # Completeness and duplicate detection
|
||||
├── duplicate_resolve.py # Duplicate group keep-index (by_quality, by_reputation, by_reputation_quality_time, first_seen, manual)
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# Code Review Report (Verified)
|
||||
|
||||
## Scope
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# TMDB Enrichment 重构执行计划
|
||||
|
||||
## 1. 目标与范围
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# Video Library Manager (VLM) 深度审计报告
|
||||
|
||||
**报告版本**:1.0
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
> [!NOTE]
|
||||
> Status: Historical snapshot. Current refactor results and validated baseline are tracked in `CHANGELOG.md` (updated 2026-02-16).
|
||||
|
||||
# Codex 架构复核报告
|
||||
|
||||
评审日期: 2026-02-13
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
# Documentation Status
|
||||
- Synced with refactor baseline on 2026-02-16.
|
||||
|
||||
---
|
||||
name: vlm-library-workflow
|
||||
description: Operate and extend the Video Library Manager (`vlm`) with a safety-first, human-in-the-loop workflow across scan, parse, enrich, analyze, plan, review-plan, execute, rollback, and developer verification. Use when requests involve organizing a video library, producing or reviewing `inventory.csv`/`identities.json`/`analysis.json`/`plan.json`, tuning VLM config templates, resolving duplicates or episode gaps, running dry-run/confirm execution, recovering changes via rollback, explaining VLM CLI usage, or developing/modifying VLM features (parser, providers, planner, executor, commands, tests).
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
# Documentation Status
|
||||
- Updated to current CLI options on 2026-02-16.
|
||||
|
||||
# VLM CLI Reference
|
||||
|
||||
## Configuration
|
||||
@@ -11,7 +14,7 @@
|
||||
- `vlm enrich [--input JSON] [--output JSON] [--refresh-all]`: Fetch TMDB metadata.
|
||||
- `vlm analyze [--input JSON] [--output JSON]`: Find gaps and duplicates.
|
||||
- `vlm plan [--input JSON] [--analysis JSON] [--output JSON]`: Generate operations.
|
||||
- `vlm execute [--plan JSON] [--confirm]`: Move/Rename files.
|
||||
- `vlm execute [--plan JSON] [--confirm] [--yes] [--verbose-ops] [--safe-mode] [--preserve-directories]`: Move/Rename/Quarantine operations with safety guards.
|
||||
- `vlm rollback [--log PATH]`: Undo operations.
|
||||
|
||||
## Management & Reporting
|
||||
@@ -22,5 +25,5 @@
|
||||
## Important Config Options (`~/.vlm/config.yaml`)
|
||||
- `library_root`: Path to the video collection.
|
||||
- `templates`: Naming patterns for movies and series.
|
||||
- `plan.duplicate_keep`: Strategy for duplicates (`by_quality`, `by_reputation`, `first_seen`, `manual`).
|
||||
- `plan.duplicate_keep`: Strategy for duplicates (`by_quality`, `by_reputation`, `by_reputation_quality_time`, `first_seen`, `manual`).
|
||||
- `enrichment.api_keys`: TMDB and OpenAI keys.
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
# Documentation Status
|
||||
- Synced with refactor baseline on 2026-02-16.
|
||||
|
||||
# VLM Command Recipes
|
||||
|
||||
## Baseline
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
# Documentation Status
|
||||
- Updated for the modular command architecture on 2026-02-16.
|
||||
|
||||
# VLM Developer Guide
|
||||
|
||||
## Project Structure
|
||||
- `src/vlm/cli.py`: Entry point and command definitions.
|
||||
- `src/vlm/cli.py`: Entry point and thin command wrappers.
|
||||
- `src/vlm/parser.py`: Regex-based filename parsing logic.
|
||||
- `src/vlm/enrichment.py`: Pipeline for external metadata fetching.
|
||||
- `src/vlm/providers/`: API implementations (e.g., TMDB).
|
||||
@@ -11,7 +14,7 @@
|
||||
## Adding a New Command
|
||||
1. Create a new module in `src/vlm/commands/`.
|
||||
2. Define the command using `@click.command()`.
|
||||
3. Register it in `src/vlm/cli.py` using `main.add_command()`.
|
||||
3. Register it in `src/vlm/cli.py` with a Click-decorated function that delegates to the module implementation.
|
||||
|
||||
## Modifying the Parser
|
||||
- The parser uses a sequence of regex patterns in `src/vlm/parser.py`.
|
||||
@@ -21,8 +24,8 @@
|
||||
## Data Models
|
||||
See `src/vlm/models.py` for core data structures:
|
||||
- `VideoFile`: Basic file metadata.
|
||||
- `MediaIdentity`: Parsed and enriched information.
|
||||
- `PlanOperation`: Definition of a file move/rename/quarantine.
|
||||
- `MovieIdentity` / `SeriesIdentity`: Parsed/enriched identity records.
|
||||
- `FileOperation`: Definition of a move/rename/quarantine/no-op/preserve-directory operation.
|
||||
|
||||
## Testing
|
||||
- **Unit Tests**: `pytest`
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
# Documentation Status
|
||||
- Synced with refactor baseline on 2026-02-16.
|
||||
|
||||
# VLM Workflow Guide
|
||||
|
||||
This guide details the standard end-to-end process for organizing a video library using VLM.
|
||||
|
||||
+55
-550
@@ -193,180 +193,20 @@ def parse(ctx: CLIContext, input: Path, output: Path, inventory: Optional[Path])
|
||||
vlm parse --input my_inventory.csv # Custom input
|
||||
vlm parse --output parsed_identities.json # Custom output
|
||||
"""
|
||||
from datetime import datetime, timezone
|
||||
from vlm.parser import parse_movie, parse_series
|
||||
from vlm.io import load_inventory_csv, save_identities_json
|
||||
|
||||
config = ctx.config
|
||||
logger = ctx.logger
|
||||
|
||||
try:
|
||||
# Display parse start message
|
||||
click.echo(f"Parsing identities from: {input}")
|
||||
|
||||
# Load video metadata from inventory if provided
|
||||
path_to_metadata = {}
|
||||
if inventory:
|
||||
click.echo(f"Loading video metadata from: {inventory}")
|
||||
inventory_files = load_inventory_csv(inventory)
|
||||
path_to_metadata = {str(vf.path): vf for vf in inventory_files}
|
||||
click.echo(f"Loaded metadata for {len(path_to_metadata)} files")
|
||||
|
||||
click.echo()
|
||||
|
||||
# Load inventory via unified I/O layer
|
||||
inventory_files = load_inventory_csv(input)
|
||||
video_files = [
|
||||
{
|
||||
'path': str(vf.path),
|
||||
'filename': vf.filename,
|
||||
'category': vf.category,
|
||||
}
|
||||
for vf in inventory_files
|
||||
]
|
||||
|
||||
click.echo(f"Loaded {len(video_files)} files from inventory")
|
||||
click.echo()
|
||||
|
||||
# Parse identities based on category
|
||||
movie_identities = []
|
||||
series_identities = []
|
||||
anime_files = []
|
||||
other_files = []
|
||||
|
||||
def get_video_metadata(file_path: str) -> dict:
|
||||
"""Extract video metadata from inventory if available."""
|
||||
if not path_to_metadata:
|
||||
return {}
|
||||
vf = path_to_metadata.get(file_path)
|
||||
if not vf:
|
||||
return {}
|
||||
return {
|
||||
'size_bytes': vf.size_bytes,
|
||||
'modified_timestamp': vf.modified_timestamp.isoformat(),
|
||||
'resolution': vf.resolution,
|
||||
'codec': vf.codec,
|
||||
'duration_seconds': vf.duration_seconds,
|
||||
'bitrate_kbps': vf.bitrate_kbps,
|
||||
}
|
||||
|
||||
for vf in video_files:
|
||||
filename = vf['filename']
|
||||
category = vf['category']
|
||||
file_path = vf['path']
|
||||
video_metadata = get_video_metadata(file_path)
|
||||
|
||||
if category == 'movie':
|
||||
identity = parse_movie(filename, extensions=config.video_extensions)
|
||||
record = {
|
||||
'path': file_path,
|
||||
'filename': filename,
|
||||
'category': category,
|
||||
'title': identity.title,
|
||||
'year': identity.year,
|
||||
'confidence': identity.confidence,
|
||||
'needs_review': identity.needs_review
|
||||
}
|
||||
if video_metadata:
|
||||
record['video_metadata'] = video_metadata
|
||||
movie_identities.append(record)
|
||||
|
||||
elif category == 'series':
|
||||
identity = parse_series(filename, extensions=config.video_extensions)
|
||||
record = {
|
||||
'path': file_path,
|
||||
'filename': filename,
|
||||
'category': category,
|
||||
'title': identity.title,
|
||||
'season': identity.season,
|
||||
'episodes': identity.episodes,
|
||||
'confidence': identity.confidence,
|
||||
'needs_review': identity.needs_review
|
||||
}
|
||||
if video_metadata:
|
||||
record['video_metadata'] = video_metadata
|
||||
series_identities.append(record)
|
||||
|
||||
elif category == 'anime':
|
||||
# Anime files are not parsed in v1
|
||||
anime_files.append({
|
||||
'path': vf['path'],
|
||||
'filename': filename,
|
||||
'category': category,
|
||||
'note': 'Anime parsing deferred in v1'
|
||||
})
|
||||
|
||||
else:
|
||||
# Other files are not parsed
|
||||
other_files.append({
|
||||
'path': vf['path'],
|
||||
'filename': filename,
|
||||
'category': category,
|
||||
'note': 'Not categorized for parsing'
|
||||
})
|
||||
|
||||
# Display parsing statistics
|
||||
click.echo("Parsing complete!")
|
||||
click.echo()
|
||||
click.echo("Results by category:")
|
||||
click.echo(f" Movies: {len(movie_identities)}")
|
||||
|
||||
# Count movies needing review
|
||||
movies_need_review = sum(1 for m in movie_identities if m['needs_review'])
|
||||
if movies_need_review > 0:
|
||||
click.echo(f" - Need review: {movies_need_review}")
|
||||
|
||||
click.echo(f" Series: {len(series_identities)}")
|
||||
|
||||
# Count series needing review
|
||||
series_need_review = sum(1 for s in series_identities if s['needs_review'])
|
||||
if series_need_review > 0:
|
||||
click.echo(f" - Need review: {series_need_review}")
|
||||
|
||||
click.echo(f" Anime: {len(anime_files)} (not parsed in v1)")
|
||||
click.echo(f" Other: {len(other_files)} (not parsed)")
|
||||
|
||||
# Save parsed identities to JSON
|
||||
click.echo()
|
||||
click.echo(f"Saving parsed identities to: {output}")
|
||||
|
||||
# Ensure output directory exists
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Build JSON structure
|
||||
generation_timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
|
||||
|
||||
# Use v2 schema if video metadata was embedded
|
||||
schema_version = "2.0" if path_to_metadata else "1.0"
|
||||
|
||||
identities_data = {
|
||||
'vlm_schema_version': schema_version,
|
||||
'metadata': {
|
||||
'generated': generation_timestamp,
|
||||
'source_inventory': str(input),
|
||||
'total_files': len(video_files)
|
||||
},
|
||||
'movies': movie_identities,
|
||||
'series': series_identities,
|
||||
'anime': anime_files,
|
||||
'other': other_files
|
||||
}
|
||||
|
||||
# Write JSON via unified I/O layer
|
||||
save_identities_json(identities_data, output)
|
||||
|
||||
click.echo(f"Parsed identities saved successfully!")
|
||||
|
||||
logger.info(f"Parse completed: {len(movie_identities)} movies, {len(series_identities)} series, saved to {output}")
|
||||
|
||||
from vlm.commands.parse import parse_cmd
|
||||
parse_cmd(ctx, input, output, inventory)
|
||||
except FileNotFoundError:
|
||||
click.echo(f"Error: Input file not found: {input}", err=True)
|
||||
logger.error(f"Input file not found: {input}")
|
||||
ctx.logger.error(f"Input file not found: {input}")
|
||||
sys.exit(1)
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error during parsing: {e}", err=True)
|
||||
logger.error(f"Parse failed: {e}", exc_info=True)
|
||||
except ValueError as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
ctx.logger.error(f"Parse failed: {e}")
|
||||
sys.exit(1)
|
||||
except OSError as e:
|
||||
click.echo(f"Error reading/writing files: {e}", err=True)
|
||||
ctx.logger.error(f"Parse file I/O failed: {e}", exc_info=True)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@@ -424,150 +264,32 @@ def enrich(
|
||||
Applies incremental cache-backed enrichment to parsed identities and writes
|
||||
results back into identities JSON.
|
||||
"""
|
||||
import json
|
||||
from vlm.enrichment import enrich_identities_data
|
||||
|
||||
config = ctx.config
|
||||
logger = ctx.logger
|
||||
|
||||
if output is None:
|
||||
output = input
|
||||
|
||||
try:
|
||||
click.echo(f"Enriching identities from: {input}")
|
||||
click.echo(f"Output file: {output}")
|
||||
click.echo()
|
||||
|
||||
with open(input, 'r', encoding='utf-8') as jsonfile:
|
||||
identities_data = json.load(jsonfile)
|
||||
|
||||
if refresh_all and refresh_changed_only:
|
||||
click.echo("Error: --refresh-all and --refresh-changed-only are mutually exclusive.", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
if timeout < 1:
|
||||
click.echo("Error: --timeout must be >= 1", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
if retries < 0:
|
||||
click.echo("Error: --retries must be >= 0", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
refresh_mode = "incremental"
|
||||
if refresh_all:
|
||||
refresh_mode = "refresh_all"
|
||||
elif refresh_changed_only:
|
||||
refresh_mode = "refresh_changed_only"
|
||||
|
||||
total = (
|
||||
len(identities_data.get('movies', []))
|
||||
+ len(identities_data.get('series', []))
|
||||
+ len(identities_data.get('anime', []))
|
||||
)
|
||||
|
||||
progress_state: dict[str, Optional[click.ProgressBar]] = {"bar": None}
|
||||
progress_position = {"current": 0}
|
||||
progress_bucket = {"value": -1}
|
||||
is_tty = bool(getattr(sys.stderr, "isatty", lambda: False)())
|
||||
|
||||
def _enrich_progress(processed: int, total_count: int, metrics: dict[str, int]) -> None:
|
||||
if total_count <= 0:
|
||||
return
|
||||
|
||||
if is_tty and progress_state["bar"] is None:
|
||||
bar = click.progressbar(
|
||||
length=total_count,
|
||||
label="Enriching records",
|
||||
show_pos=True,
|
||||
)
|
||||
progress_state["bar"] = bar.__enter__()
|
||||
|
||||
step = processed - progress_position["current"]
|
||||
if step > 0 and progress_state["bar"] is not None:
|
||||
progress_state["bar"].update(step)
|
||||
progress_position["current"] = processed
|
||||
|
||||
if not is_tty:
|
||||
percent = int(processed * 100 / total_count)
|
||||
bucket = percent // 5
|
||||
if bucket > progress_bucket["value"] or processed == total_count:
|
||||
progress_bucket["value"] = bucket
|
||||
click.echo(
|
||||
"Progress: "
|
||||
f"{processed}/{total_count} ({percent}%) "
|
||||
f"api_calls={metrics.get('api_calls', 0)} "
|
||||
f"cache_hits={metrics.get('cache_hits', 0)} "
|
||||
f"failed={metrics.get('failed', 0)}"
|
||||
)
|
||||
|
||||
try:
|
||||
if total == 0:
|
||||
click.echo("No movie/series/anime records found to enrich.")
|
||||
enriched_data, stats = enrich_identities_data(
|
||||
identities_data,
|
||||
config,
|
||||
refresh_mode=refresh_mode,
|
||||
request_timeout=timeout,
|
||||
retries=retries,
|
||||
logger=logger,
|
||||
progress_callback=_enrich_progress,
|
||||
)
|
||||
finally:
|
||||
if progress_state["bar"] is not None:
|
||||
progress_state["bar"].__exit__(None, None, None)
|
||||
click.echo()
|
||||
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(output, 'w', encoding='utf-8') as jsonfile:
|
||||
json.dump(enriched_data, jsonfile, indent=2, ensure_ascii=False)
|
||||
|
||||
total_records = int(stats['total']) if stats['total'] else 0
|
||||
cache_hits = int(stats['cache_hits'])
|
||||
hit_rate = (cache_hits / total_records * 100.0) if total_records else 0.0
|
||||
|
||||
click.echo("Enrichment complete!")
|
||||
click.echo(f" Total records: {total_records}")
|
||||
click.echo(f" Refresh mode: {refresh_mode}")
|
||||
click.echo(f" Enriched now: {stats['enriched']}")
|
||||
click.echo(f" Cache hits: {stats['cache_hits']}")
|
||||
click.echo(f" Cache hit rate: {hit_rate:.1f}%")
|
||||
click.echo(f" API calls: {stats['api_calls']}")
|
||||
click.echo(f" Failed requests: {stats['failed']}")
|
||||
click.echo(f" Skipped: {stats['skipped']}")
|
||||
click.echo(f" Needs review: {stats['needs_review']}")
|
||||
skip_reasons = stats.get("skip_reasons", {})
|
||||
if isinstance(skip_reasons, dict):
|
||||
non_zero = [f"{name}={count}" for name, count in sorted(skip_reasons.items()) if int(count) > 0]
|
||||
if non_zero:
|
||||
click.echo(f" Skip reasons: {' '.join(non_zero)}")
|
||||
failed_items = stats.get('failed_items', [])
|
||||
if isinstance(failed_items, list) and failed_items:
|
||||
click.echo(" Failure sample:")
|
||||
for item in failed_items[:3]:
|
||||
click.echo(
|
||||
f" - [{item.get('provider', 'unknown')}] {item.get('title', '')}: {item.get('reason', '')}"
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"Enrich completed: total=%s enriched=%s cache_hits=%s failed=%s skipped=%s mode=%s",
|
||||
stats['total'],
|
||||
stats['enriched'],
|
||||
stats['cache_hits'],
|
||||
stats['failed'],
|
||||
stats['skipped'],
|
||||
refresh_mode,
|
||||
from vlm.commands.enrich import enrich_cmd
|
||||
enrich_cmd(
|
||||
ctx,
|
||||
input,
|
||||
output,
|
||||
refresh_changed_only,
|
||||
refresh_all,
|
||||
timeout,
|
||||
retries,
|
||||
)
|
||||
except FileNotFoundError:
|
||||
click.echo(f"Error: Input file not found: {input}", err=True)
|
||||
logger.error(f"Input file not found: {input}")
|
||||
ctx.logger.error(f"Input file not found: {input}")
|
||||
sys.exit(1)
|
||||
except json.JSONDecodeError as e:
|
||||
click.echo(f"Error: Failed to parse JSON file: {e}", err=True)
|
||||
logger.error(f"JSON parsing failed during enrich: {e}", exc_info=True)
|
||||
ctx.logger.error(f"JSON parsing failed during enrich: {e}", exc_info=True)
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
click.echo(f"Error during enrichment: {e}", err=True)
|
||||
logger.error(f"Enrich failed: {e}", exc_info=True)
|
||||
except ValueError as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
ctx.logger.error(f"Enrich validation failed: {e}")
|
||||
sys.exit(1)
|
||||
except OSError as e:
|
||||
click.echo(f"Error reading/writing files: {e}", err=True)
|
||||
ctx.logger.error(f"Enrich file I/O failed: {e}", exc_info=True)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@@ -791,8 +513,20 @@ def review_plan_cmd(
|
||||
default=False,
|
||||
help='Print per-operation dry-run logs at INFO level'
|
||||
)
|
||||
@click.option(
|
||||
'--preserve-directories',
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help='Preserve empty source directories instead of allowing them to be destroyed'
|
||||
)
|
||||
@click.option(
|
||||
'--safe-mode',
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help='Enable safe mode: prevent any operations that would destroy directories'
|
||||
)
|
||||
@pass_context
|
||||
def execute(ctx: CLIContext, plan: Path, confirm: bool, yes: bool, verbose_ops: bool):
|
||||
def execute(ctx: CLIContext, plan: Path, confirm: bool, yes: bool, verbose_ops: bool, preserve_directories: bool, safe_mode: bool):
|
||||
"""Execute plan (defaults to dry-run, requires --confirm).
|
||||
|
||||
Executes file operations from a plan. Defaults to dry-run mode which
|
||||
@@ -806,155 +540,20 @@ def execute(ctx: CLIContext, plan: Path, confirm: bool, yes: bool, verbose_ops:
|
||||
vlm execute --confirm # Actually execute operations (with prompt)
|
||||
vlm execute --confirm --yes # Execute without confirmation prompt
|
||||
"""
|
||||
from vlm.planner import load_plan
|
||||
from vlm.executor import ExecutionEngine
|
||||
|
||||
config = ctx.config
|
||||
logger = ctx.logger
|
||||
|
||||
try:
|
||||
# Determine execution mode
|
||||
mode = "execute" if confirm else "dry-run"
|
||||
|
||||
# Display execution start message
|
||||
click.echo(f"Loading execution plan from: {plan}")
|
||||
click.echo()
|
||||
|
||||
# Load execution plan
|
||||
execution_plan = load_plan(plan)
|
||||
|
||||
# Display plan summary
|
||||
click.echo(f"Execution plan loaded: {execution_plan.plan_id}")
|
||||
click.echo(f"Created at: {execution_plan.created_at}")
|
||||
click.echo(f"Total operations: {len(execution_plan.operations)}")
|
||||
if execution_plan.human_summary:
|
||||
click.echo()
|
||||
click.echo(execution_plan.human_summary)
|
||||
elif execution_plan.summary or execution_plan.summary_by_reason:
|
||||
s = execution_plan.summary or {}
|
||||
by_r = execution_plan.summary_by_reason or {}
|
||||
parts = [f"操作统计:共 {s.get('total', len(execution_plan.operations))} 条(move {s.get('move', 0)},rename {s.get('rename', 0)},quarantine {s.get('quarantine', 0)},no-op {s.get('no-op', 0)})"]
|
||||
if by_r:
|
||||
parts.append("原因分布:" + ";".join(f"{r}: {c}" for r, c in list(by_r.items())[:5]))
|
||||
click.echo()
|
||||
click.echo("\n".join(parts))
|
||||
click.echo()
|
||||
|
||||
# Display mode warning
|
||||
if mode == "dry-run":
|
||||
click.echo("⚠️ DRY-RUN MODE - No files will be modified")
|
||||
click.echo(" Use --confirm to actually execute operations")
|
||||
else:
|
||||
click.echo("⚠️ EXECUTE MODE - Files will be modified!")
|
||||
click.echo(" This operation cannot be undone without rollback")
|
||||
click.echo()
|
||||
if not yes:
|
||||
if not click.confirm("Are you sure you want to proceed?"):
|
||||
click.echo("Execution cancelled.")
|
||||
return
|
||||
else:
|
||||
click.echo("Auto-approved via --yes flag")
|
||||
|
||||
click.echo()
|
||||
click.echo(f"Executing {len(execution_plan.operations)} operations...")
|
||||
click.echo()
|
||||
|
||||
# Load state manager to update file statuses during execution
|
||||
from vlm.state import StateManager
|
||||
state_path = Path.home() / ".vlm" / "state.json"
|
||||
state_manager = StateManager(state_path)
|
||||
|
||||
# Create execution engine and execute plan
|
||||
engine = ExecutionEngine(
|
||||
logger=logger,
|
||||
config=config,
|
||||
verbose_operations=verbose_ops,
|
||||
state_manager=state_manager
|
||||
)
|
||||
results, summary, rollback_log = engine.execute_plan(
|
||||
execution_plan,
|
||||
mode=mode,
|
||||
confirmed=confirm
|
||||
)
|
||||
|
||||
# Display execution progress (show some operations)
|
||||
if mode == "dry-run":
|
||||
click.echo("Sample operations (dry-run):")
|
||||
# Show first 5 operations as examples
|
||||
for i, result in enumerate(results[:5]):
|
||||
op = result.operation
|
||||
if op.operation_type != "no-op":
|
||||
click.echo(f" [{i+1}] {op.operation_type}: {op.source_path.name}")
|
||||
if op.destination_path:
|
||||
click.echo(f" -> {op.destination_path}")
|
||||
|
||||
if len(results) > 5:
|
||||
click.echo(f" ... and {len(results) - 5} more operations")
|
||||
else:
|
||||
# In execute mode, show progress for all operations
|
||||
for i, result in enumerate(results):
|
||||
op = result.operation
|
||||
if op.operation_type != "no-op" and not op.has_conflict:
|
||||
status = "✓" if result.success else "✗"
|
||||
click.echo(f" [{i+1}/{len(results)}] {status} {op.operation_type}: {op.source_path.name}")
|
||||
if result.error_message:
|
||||
click.echo(f" Error: {result.error_message}")
|
||||
|
||||
# Display execution summary
|
||||
click.echo()
|
||||
click.echo("=" * 60)
|
||||
click.echo(f"Execution Summary ({mode} mode)")
|
||||
click.echo("=" * 60)
|
||||
click.echo(f" Total operations: {summary['total']}")
|
||||
click.echo(f" Successful: {summary['successful']}")
|
||||
click.echo(f" Failed: {summary['failed']}")
|
||||
click.echo(f" Skipped: {summary['skipped']}")
|
||||
click.echo()
|
||||
|
||||
# Save rollback log if in execute mode
|
||||
if mode == "execute" and rollback_log:
|
||||
# Save to ~/.vlm/rollback/ directory
|
||||
rollback_dir = Path.home() / ".vlm" / "rollback"
|
||||
rollback_dir.mkdir(parents=True, exist_ok=True)
|
||||
rollback_path = rollback_dir / f"rollback_{rollback_log.log_id}.json"
|
||||
|
||||
engine.save_rollback_log(rollback_log, rollback_path)
|
||||
|
||||
click.echo(f"Rollback log saved to: {rollback_path}")
|
||||
click.echo()
|
||||
click.echo("To undo these operations, run:")
|
||||
click.echo(f" vlm rollback --log {rollback_path}")
|
||||
click.echo()
|
||||
|
||||
# Log completion
|
||||
if mode == "dry-run":
|
||||
click.echo("Dry-run complete! No files were modified.")
|
||||
click.echo("Review the operations above and use --confirm to execute.")
|
||||
else:
|
||||
if summary['failed'] > 0:
|
||||
click.echo(f"⚠️ Execution completed with {summary['failed']} failures.")
|
||||
click.echo(" Check the log file for details.")
|
||||
else:
|
||||
click.echo("✓ Execution completed successfully!")
|
||||
|
||||
logger.info(
|
||||
f"Execution completed in {mode} mode: "
|
||||
f"{summary['successful']} successful, {summary['failed']} failed, {summary['skipped']} skipped"
|
||||
)
|
||||
|
||||
from vlm.commands.execute import execute_cmd
|
||||
execute_cmd(ctx, plan, confirm, yes, verbose_ops, preserve_directories, safe_mode)
|
||||
except FileNotFoundError:
|
||||
click.echo(f"Error: Plan file not found: {plan}", err=True)
|
||||
logger.error(f"Plan file not found: {plan}")
|
||||
click.echo(f"Error: File not found: {plan}", err=True)
|
||||
ctx.logger.error(f"Execution file not found: {plan}")
|
||||
sys.exit(1)
|
||||
|
||||
except ValueError as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
logger.error(f"Execution failed: {e}")
|
||||
ctx.logger.error(f"Execution validation failed: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
except Exception as e:
|
||||
except OSError as e:
|
||||
click.echo(f"Error during execution: {e}", err=True)
|
||||
logger.error(f"Execution failed: {e}", exc_info=True)
|
||||
ctx.logger.error(f"Execution I/O failed: {e}", exc_info=True)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
@@ -1177,114 +776,20 @@ def rollback(ctx: CLIContext, log: Optional[Path]):
|
||||
vlm rollback --log rollback_<uuid>.json # Use specific log
|
||||
vlm rollback --log ~/.vlm/rollback/rollback_*.json
|
||||
"""
|
||||
from vlm.executor import ExecutionEngine
|
||||
|
||||
config = ctx.config
|
||||
logger = ctx.logger
|
||||
|
||||
try:
|
||||
# If no log specified, find the most recent rollback log
|
||||
if log is None:
|
||||
rollback_dir = Path.home() / ".vlm" / "rollback"
|
||||
if not rollback_dir.exists():
|
||||
click.echo("Error: No rollback logs found.", err=True)
|
||||
click.echo(f"Rollback directory does not exist: {rollback_dir}", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
# Find all rollback log files
|
||||
rollback_logs = sorted(rollback_dir.glob("rollback_*.json"), key=lambda p: p.stat().st_mtime, reverse=True)
|
||||
|
||||
if not rollback_logs:
|
||||
click.echo("Error: No rollback logs found.", err=True)
|
||||
click.echo(f"No rollback_*.json files in: {rollback_dir}", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
# Use the most recent log
|
||||
log = rollback_logs[0]
|
||||
click.echo(f"Using most recent rollback log: {log}")
|
||||
click.echo()
|
||||
|
||||
# Display rollback start message
|
||||
click.echo(f"Loading rollback log from: {log}")
|
||||
click.echo()
|
||||
|
||||
# Create execution engine
|
||||
engine = ExecutionEngine(logger=logger, config=config)
|
||||
|
||||
# Load rollback log
|
||||
rollback_log = engine.load_rollback_log(log)
|
||||
|
||||
# Display rollback log info
|
||||
click.echo(f"Rollback log loaded: {rollback_log.log_id}")
|
||||
click.echo(f"Original execution: {rollback_log.execution_plan_id}")
|
||||
click.echo(f"Executed at: {rollback_log.executed_at.strftime('%Y-%m-%d %H:%M:%S')}")
|
||||
click.echo(f"Operations to rollback: {len(rollback_log.operations)}")
|
||||
click.echo()
|
||||
|
||||
# Display warning
|
||||
click.echo("⚠️ ROLLBACK OPERATION - Best-effort restoration")
|
||||
click.echo(" This will attempt to move files back to their original locations.")
|
||||
click.echo(" Some operations may fail if files have been modified or moved.")
|
||||
click.echo()
|
||||
|
||||
if not click.confirm("Are you sure you want to proceed with rollback?"):
|
||||
click.echo("Rollback cancelled.")
|
||||
return
|
||||
|
||||
click.echo()
|
||||
click.echo(f"Rolling back {len(rollback_log.operations)} operations...")
|
||||
click.echo()
|
||||
|
||||
# Perform rollback
|
||||
results, summary = engine.rollback(rollback_log)
|
||||
|
||||
# Display rollback progress
|
||||
for i, result in enumerate(results):
|
||||
op = result.operation
|
||||
if op.operation_type != "no-op":
|
||||
status = "✓" if result.success else "✗"
|
||||
click.echo(f" [{i+1}/{len(results)}] {status} Rollback: {op.destination_path.name if op.destination_path else op.source_path.name}")
|
||||
if result.error_message:
|
||||
click.echo(f" Error: {result.error_message}")
|
||||
|
||||
# Display rollback summary
|
||||
click.echo()
|
||||
click.echo("=" * 60)
|
||||
click.echo("Rollback Summary")
|
||||
click.echo("=" * 60)
|
||||
click.echo(f" Total operations: {summary['total']}")
|
||||
click.echo(f" Successful: {summary['successful']}")
|
||||
click.echo(f" Failed: {summary['failed']}")
|
||||
click.echo(f" Skipped: {summary['skipped']}")
|
||||
click.echo()
|
||||
|
||||
# Display completion message
|
||||
if summary['failed'] > 0:
|
||||
click.echo(f"⚠️ Rollback completed with {summary['failed']} failures.")
|
||||
click.echo(" Check the log file for details.")
|
||||
click.echo(" Some files may not have been restored to their original locations.")
|
||||
else:
|
||||
click.echo("✓ Rollback completed successfully!")
|
||||
click.echo(" All files have been restored to their original locations.")
|
||||
|
||||
logger.info(
|
||||
f"Rollback completed: "
|
||||
f"{summary['successful']} successful, {summary['failed']} failed, {summary['skipped']} skipped"
|
||||
)
|
||||
|
||||
from vlm.commands.execute import rollback_cmd
|
||||
rollback_cmd(ctx, log)
|
||||
except FileNotFoundError as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
logger.error(f"Rollback log not found: {e}")
|
||||
ctx.logger.error(f"Rollback log not found: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
except ValueError as e:
|
||||
click.echo(f"Error: Invalid rollback log format: {e}", err=True)
|
||||
logger.error(f"Invalid rollback log: {e}")
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
ctx.logger.error(f"Rollback failed: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
except Exception as e:
|
||||
except OSError as e:
|
||||
click.echo(f"Error during rollback: {e}", err=True)
|
||||
logger.error(f"Rollback failed: {e}", exc_info=True)
|
||||
ctx.logger.error(f"Rollback failed: {e}", exc_info=True)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
|
||||
+14
-17
@@ -1,6 +1,5 @@
|
||||
"""Analyze command implementation."""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
@@ -8,9 +7,13 @@ import click
|
||||
|
||||
from vlm.analysis import analyze_series_completeness, detect_duplicates
|
||||
from vlm.context import CLIContext
|
||||
from vlm.io import identities_to_analysis_input, load_identities_json, load_inventory_csv
|
||||
from vlm.io import (
|
||||
identities_to_analysis_input,
|
||||
load_identities_json,
|
||||
load_inventory_csv,
|
||||
save_analysis_json,
|
||||
)
|
||||
from vlm.models import MovieIdentity, SeriesIdentity
|
||||
from vlm.utils import utc_now
|
||||
|
||||
|
||||
def analyze_cmd(
|
||||
@@ -68,7 +71,6 @@ def analyze_cmd(
|
||||
click.echo(f"Saving analysis results to: {output}")
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
generation_timestamp = utc_now().strftime("%Y-%m-%dT%H:%M:%S")
|
||||
completeness_list = [
|
||||
{
|
||||
"series_title": c.series_title,
|
||||
@@ -96,19 +98,14 @@ def analyze_cmd(
|
||||
"quality_comparison": d.quality_comparison,
|
||||
}
|
||||
)
|
||||
analysis_data = {
|
||||
"vlm_schema_version": "1.0",
|
||||
"metadata": {
|
||||
"generated": generation_timestamp,
|
||||
"source_identities": str(input),
|
||||
"total_movies": len(movies_data),
|
||||
"total_series": len(series_data),
|
||||
},
|
||||
"completeness": completeness_list,
|
||||
"duplicates": duplicates_list,
|
||||
}
|
||||
with open(output, "w", encoding="utf-8") as jsonfile:
|
||||
json.dump(analysis_data, jsonfile, indent=2, ensure_ascii=False)
|
||||
save_analysis_json(
|
||||
completeness=completeness_list,
|
||||
duplicates=duplicates_list,
|
||||
source_identities=input,
|
||||
total_movies=len(movies_data),
|
||||
total_series=len(series_data),
|
||||
output=output,
|
||||
)
|
||||
|
||||
click.echo("Analysis results saved successfully!")
|
||||
logger.info(
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
"""Enrich command implementation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import click
|
||||
|
||||
from vlm.context import CLIContext
|
||||
from vlm.enrichment import enrich_identities_data
|
||||
from vlm.io import load_json_file, save_json_file
|
||||
|
||||
|
||||
def enrich_cmd(
|
||||
ctx: CLIContext,
|
||||
input: Path,
|
||||
output: Optional[Path],
|
||||
refresh_changed_only: bool,
|
||||
refresh_all: bool,
|
||||
timeout: int,
|
||||
retries: int,
|
||||
) -> None:
|
||||
"""Enrich identities with translation and reputation metadata."""
|
||||
logger = ctx.logger
|
||||
config = ctx.config
|
||||
|
||||
if output is None:
|
||||
output = input
|
||||
|
||||
click.echo(f"Enriching identities from: {input}")
|
||||
click.echo(f"Output file: {output}")
|
||||
click.echo()
|
||||
|
||||
identities_data = load_json_file(input)
|
||||
|
||||
if refresh_all and refresh_changed_only:
|
||||
raise ValueError("--refresh-all and --refresh-changed-only are mutually exclusive.")
|
||||
if timeout < 1:
|
||||
raise ValueError("--timeout must be >= 1")
|
||||
if retries < 0:
|
||||
raise ValueError("--retries must be >= 0")
|
||||
|
||||
refresh_mode = "incremental"
|
||||
if refresh_all:
|
||||
refresh_mode = "refresh_all"
|
||||
elif refresh_changed_only:
|
||||
refresh_mode = "refresh_changed_only"
|
||||
|
||||
total = (
|
||||
len(identities_data.get("movies", []))
|
||||
+ len(identities_data.get("series", []))
|
||||
+ len(identities_data.get("anime", []))
|
||||
)
|
||||
|
||||
progress_state: dict[str, Optional[click.ProgressBar]] = {"bar": None}
|
||||
progress_position = {"current": 0}
|
||||
progress_bucket = {"value": -1}
|
||||
is_tty = bool(getattr(sys.stderr, "isatty", lambda: False)())
|
||||
|
||||
def _enrich_progress(processed: int, total_count: int, metrics: dict[str, int]) -> None:
|
||||
if total_count <= 0:
|
||||
return
|
||||
|
||||
if is_tty and progress_state["bar"] is None:
|
||||
bar = click.progressbar(
|
||||
length=total_count,
|
||||
label="Enriching records",
|
||||
show_pos=True,
|
||||
)
|
||||
progress_state["bar"] = bar.__enter__()
|
||||
|
||||
step = processed - progress_position["current"]
|
||||
if step > 0 and progress_state["bar"] is not None:
|
||||
progress_state["bar"].update(step)
|
||||
progress_position["current"] = processed
|
||||
|
||||
if not is_tty:
|
||||
percent = int(processed * 100 / total_count)
|
||||
bucket = percent // 5
|
||||
if bucket > progress_bucket["value"] or processed == total_count:
|
||||
progress_bucket["value"] = bucket
|
||||
click.echo(
|
||||
"Progress: "
|
||||
f"{processed}/{total_count} ({percent}%) "
|
||||
f"api_calls={metrics.get('api_calls', 0)} "
|
||||
f"cache_hits={metrics.get('cache_hits', 0)} "
|
||||
f"failed={metrics.get('failed', 0)}"
|
||||
)
|
||||
|
||||
try:
|
||||
if total == 0:
|
||||
click.echo("No movie/series/anime records found to enrich.")
|
||||
enriched_data, stats = enrich_identities_data(
|
||||
identities_data,
|
||||
config,
|
||||
refresh_mode=refresh_mode,
|
||||
request_timeout=timeout,
|
||||
retries=retries,
|
||||
logger=logger,
|
||||
progress_callback=_enrich_progress,
|
||||
)
|
||||
finally:
|
||||
if progress_state["bar"] is not None:
|
||||
progress_state["bar"].__exit__(None, None, None)
|
||||
click.echo()
|
||||
|
||||
save_json_file(enriched_data, output)
|
||||
|
||||
total_records = int(stats["total"]) if stats["total"] else 0
|
||||
cache_hits = int(stats["cache_hits"])
|
||||
hit_rate = (cache_hits / total_records * 100.0) if total_records else 0.0
|
||||
|
||||
click.echo("Enrichment complete!")
|
||||
click.echo(f" Total records: {total_records}")
|
||||
click.echo(f" Refresh mode: {refresh_mode}")
|
||||
click.echo(f" Enriched now: {stats['enriched']}")
|
||||
click.echo(f" Cache hits: {stats['cache_hits']}")
|
||||
click.echo(f" Cache hit rate: {hit_rate:.1f}%")
|
||||
click.echo(f" API calls: {stats['api_calls']}")
|
||||
click.echo(f" Failed requests: {stats['failed']}")
|
||||
click.echo(f" Skipped: {stats['skipped']}")
|
||||
click.echo(f" Needs review: {stats['needs_review']}")
|
||||
|
||||
skip_reasons = stats.get("skip_reasons", {})
|
||||
if isinstance(skip_reasons, dict):
|
||||
non_zero = [f"{name}={count}" for name, count in sorted(skip_reasons.items()) if int(count) > 0]
|
||||
if non_zero:
|
||||
click.echo(f" Skip reasons: {' '.join(non_zero)}")
|
||||
|
||||
failed_items = stats.get("failed_items", [])
|
||||
if isinstance(failed_items, list) and failed_items:
|
||||
click.echo(" Failure sample:")
|
||||
for item in failed_items[:3]:
|
||||
click.echo(f" - [{item.get('provider', 'unknown')}] {item.get('title', '')}: {item.get('reason', '')}")
|
||||
|
||||
logger.info(
|
||||
"Enrich completed: total=%s enriched=%s cache_hits=%s failed=%s skipped=%s mode=%s",
|
||||
stats["total"],
|
||||
stats["enriched"],
|
||||
stats["cache_hits"],
|
||||
stats["failed"],
|
||||
stats["skipped"],
|
||||
refresh_mode,
|
||||
)
|
||||
@@ -0,0 +1,257 @@
|
||||
"""Execute and rollback command implementations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import click
|
||||
|
||||
from vlm.context import CLIContext
|
||||
from vlm.executor import ExecutionEngine
|
||||
from vlm.planner import load_plan
|
||||
from vlm.state import StateManager
|
||||
|
||||
|
||||
def _validate_plan_structure(execution_plan) -> list[str]:
|
||||
"""Validate source/destination paths for move/rename operations."""
|
||||
validation_errors: list[str] = []
|
||||
|
||||
for operation in execution_plan.operations:
|
||||
if operation.operation_type in ("move", "rename"):
|
||||
if not operation.source_path.exists():
|
||||
validation_errors.append(f"Source file does not exist: {operation.source_path}")
|
||||
elif not operation.source_path.is_file():
|
||||
validation_errors.append(f"Source path is not a file: {operation.source_path}")
|
||||
|
||||
for operation in execution_plan.operations:
|
||||
if operation.operation_type in ("move", "rename") and operation.destination_path:
|
||||
dest_dir = operation.destination_path.parent
|
||||
if not dest_dir.exists() and not dest_dir.parent.exists():
|
||||
validation_errors.append(f"Cannot create destination directory (parent missing): {dest_dir}")
|
||||
|
||||
return validation_errors
|
||||
|
||||
|
||||
def execute_cmd(
|
||||
ctx: CLIContext,
|
||||
plan: Path,
|
||||
confirm: bool,
|
||||
yes: bool,
|
||||
verbose_ops: bool,
|
||||
preserve_directories: bool,
|
||||
safe_mode: bool,
|
||||
) -> None:
|
||||
"""Execute an execution plan in dry-run or execute mode."""
|
||||
config = ctx.config
|
||||
logger = ctx.logger
|
||||
mode = "execute" if confirm else "dry-run"
|
||||
|
||||
click.echo(f"Loading execution plan from: {plan}")
|
||||
click.echo()
|
||||
|
||||
execution_plan = load_plan(plan)
|
||||
|
||||
if preserve_directories:
|
||||
click.echo("Directory preservation is enabled (plan metadata/operations will be honored).")
|
||||
click.echo()
|
||||
|
||||
if safe_mode:
|
||||
click.echo("Safe mode enabled - validating plan for directory preservation...")
|
||||
emptied_dirs = execution_plan.metadata.get("emptied_directories", [])
|
||||
if emptied_dirs:
|
||||
raise ValueError(
|
||||
"SAFE MODE VIOLATION: plan would empty directories. "
|
||||
"Regenerate/adjust the plan to preserve directory structure."
|
||||
)
|
||||
click.echo("Safe mode validation passed - no directories would be destroyed")
|
||||
click.echo()
|
||||
|
||||
click.echo("Validating directory structure...")
|
||||
validation_errors = _validate_plan_structure(execution_plan)
|
||||
if validation_errors:
|
||||
formatted = "\n".join(f" - {error}" for error in validation_errors[:10])
|
||||
if len(validation_errors) > 10:
|
||||
formatted += f"\n ... and {len(validation_errors) - 10} more errors"
|
||||
raise ValueError(f"DIRECTORY STRUCTURE VALIDATION FAILED:\n{formatted}")
|
||||
|
||||
click.echo("Directory structure validation passed")
|
||||
click.echo()
|
||||
|
||||
click.echo(f"Execution plan loaded: {execution_plan.plan_id}")
|
||||
click.echo(f"Created at: {execution_plan.created_at}")
|
||||
click.echo(f"Total operations: {len(execution_plan.operations)}")
|
||||
|
||||
if execution_plan.human_summary:
|
||||
click.echo()
|
||||
click.echo(execution_plan.human_summary)
|
||||
elif execution_plan.summary or execution_plan.summary_by_reason:
|
||||
s = execution_plan.summary or {}
|
||||
by_r = execution_plan.summary_by_reason or {}
|
||||
parts = [
|
||||
"操作统计:共 "
|
||||
f"{s.get('total', len(execution_plan.operations))} 条"
|
||||
f"(move {s.get('move', 0)},rename {s.get('rename', 0)},"
|
||||
f"quarantine {s.get('quarantine', 0)},no-op {s.get('no-op', 0)})"
|
||||
]
|
||||
if by_r:
|
||||
parts.append("原因分布:" + ";".join(f"{r}: {c}" for r, c in list(by_r.items())[:5]))
|
||||
click.echo()
|
||||
click.echo("\n".join(parts))
|
||||
click.echo()
|
||||
|
||||
if mode == "dry-run":
|
||||
click.echo("DRY-RUN MODE - No files will be modified")
|
||||
click.echo("Use --confirm to actually execute operations")
|
||||
else:
|
||||
click.echo("EXECUTE MODE - Files will be modified")
|
||||
click.echo()
|
||||
if not yes and not click.confirm("Are you sure you want to proceed?"):
|
||||
click.echo("Execution cancelled.")
|
||||
return
|
||||
if yes:
|
||||
click.echo("Auto-approved via --yes flag")
|
||||
|
||||
click.echo()
|
||||
click.echo(f"Executing {len(execution_plan.operations)} operations...")
|
||||
click.echo()
|
||||
|
||||
state_path = Path.home() / ".vlm" / "state.json"
|
||||
state_manager = StateManager(state_path)
|
||||
|
||||
engine = ExecutionEngine(
|
||||
logger=logger,
|
||||
config=config,
|
||||
verbose_operations=verbose_ops,
|
||||
state_manager=state_manager,
|
||||
)
|
||||
results, summary, rollback_log = engine.execute_plan(
|
||||
execution_plan,
|
||||
mode=mode,
|
||||
confirmed=confirm,
|
||||
)
|
||||
|
||||
if mode == "dry-run":
|
||||
click.echo("Sample operations (dry-run):")
|
||||
for i, result in enumerate(results[:5]):
|
||||
op = result.operation
|
||||
if op.operation_type != "no-op":
|
||||
click.echo(f" [{i + 1}] {op.operation_type}: {op.source_path.name}")
|
||||
if op.destination_path:
|
||||
click.echo(f" -> {op.destination_path}")
|
||||
if len(results) > 5:
|
||||
click.echo(f" ... and {len(results) - 5} more operations")
|
||||
else:
|
||||
for i, result in enumerate(results):
|
||||
op = result.operation
|
||||
if op.operation_type != "no-op" and not op.has_conflict:
|
||||
status = "OK" if result.success else "FAIL"
|
||||
click.echo(f" [{i + 1}/{len(results)}] {status} {op.operation_type}: {op.source_path.name}")
|
||||
if result.error_message:
|
||||
click.echo(f" Error: {result.error_message}")
|
||||
|
||||
click.echo()
|
||||
click.echo("=" * 60)
|
||||
click.echo(f"Execution Summary ({mode} mode)")
|
||||
click.echo("=" * 60)
|
||||
click.echo(f" Total operations: {summary['total']}")
|
||||
click.echo(f" Successful: {summary['successful']}")
|
||||
click.echo(f" Failed: {summary['failed']}")
|
||||
click.echo(f" Skipped: {summary['skipped']}")
|
||||
click.echo()
|
||||
|
||||
if mode == "execute" and rollback_log:
|
||||
rollback_dir = Path.home() / ".vlm" / "rollback"
|
||||
rollback_dir.mkdir(parents=True, exist_ok=True)
|
||||
rollback_path = rollback_dir / f"rollback_{rollback_log.log_id}.json"
|
||||
engine.save_rollback_log(rollback_log, rollback_path)
|
||||
|
||||
click.echo(f"Rollback log saved to: {rollback_path}")
|
||||
click.echo()
|
||||
click.echo("To undo these operations, run:")
|
||||
click.echo(f" vlm rollback --log {rollback_path}")
|
||||
click.echo()
|
||||
|
||||
if mode == "dry-run":
|
||||
click.echo("Dry-run complete! No files were modified.")
|
||||
click.echo("Review the operations above and use --confirm to execute.")
|
||||
elif summary["failed"] > 0:
|
||||
click.echo(f"Execution completed with {summary['failed']} failures.")
|
||||
click.echo("Check the log file for details.")
|
||||
else:
|
||||
click.echo("Execution completed successfully!")
|
||||
|
||||
logger.info(
|
||||
"Execution completed in %s mode: %s successful, %s failed, %s skipped",
|
||||
mode,
|
||||
summary["successful"],
|
||||
summary["failed"],
|
||||
summary["skipped"],
|
||||
)
|
||||
|
||||
|
||||
def rollback_cmd(ctx: CLIContext, log: Optional[Path]) -> None:
|
||||
"""Rollback a prior execution from rollback log."""
|
||||
config = ctx.config
|
||||
logger = ctx.logger
|
||||
|
||||
if log is None:
|
||||
rollback_dir = Path.home() / ".vlm" / "rollback"
|
||||
if not rollback_dir.exists():
|
||||
raise FileNotFoundError(f"No rollback logs found. Rollback directory does not exist: {rollback_dir}")
|
||||
rollback_logs = sorted(
|
||||
rollback_dir.glob("rollback_*.json"), key=lambda p: p.stat().st_mtime, reverse=True
|
||||
)
|
||||
if not rollback_logs:
|
||||
raise FileNotFoundError(f"No rollback logs found. No rollback_*.json files in: {rollback_dir}")
|
||||
log = rollback_logs[0]
|
||||
|
||||
click.echo(f"Loading rollback log: {log}")
|
||||
|
||||
engine = ExecutionEngine(logger=logger, config=config)
|
||||
rollback_log = engine.load_rollback_log(log)
|
||||
|
||||
click.echo(f"Rollback log loaded: {rollback_log.log_id}")
|
||||
click.echo(f"Plan ID: {rollback_log.execution_plan_id}")
|
||||
click.echo(f"Executed at: {rollback_log.executed_at}")
|
||||
click.echo(f"Operations to rollback: {len(rollback_log.operations)}")
|
||||
click.echo()
|
||||
|
||||
if not click.confirm("Proceed with rollback?"):
|
||||
click.echo("Rollback cancelled.")
|
||||
return
|
||||
|
||||
click.echo(f"Rolling back {len(rollback_log.operations)} operations...")
|
||||
click.echo()
|
||||
click.echo("Performing rollback...")
|
||||
rollback_results, rollback_summary = engine.rollback(rollback_log)
|
||||
|
||||
click.echo()
|
||||
click.echo("Rollback Summary")
|
||||
click.echo("=" * 60)
|
||||
click.echo(f" Total operations: {rollback_summary['total']}")
|
||||
click.echo(f" Successful: {rollback_summary['successful']}")
|
||||
click.echo(f" Failed: {rollback_summary['failed']}")
|
||||
click.echo(f" Skipped: {rollback_summary['skipped']}")
|
||||
click.echo()
|
||||
|
||||
if rollback_summary["failed"] > 0:
|
||||
click.echo("Failed operations:")
|
||||
for result in rollback_results:
|
||||
if not result.success:
|
||||
op = result.operation
|
||||
click.echo(f" - {op.operation_type}: {op.source_path}")
|
||||
if result.error_message:
|
||||
click.echo(f" Error: {result.error_message}")
|
||||
click.echo()
|
||||
|
||||
if rollback_summary["failed"] == 0:
|
||||
click.echo("Rollback completed successfully!")
|
||||
else:
|
||||
click.echo(f"Rollback completed with {rollback_summary['failed']} failures.")
|
||||
|
||||
logger.info(
|
||||
"Rollback completed: %s successful, %s failed",
|
||||
rollback_summary["successful"],
|
||||
rollback_summary["failed"],
|
||||
)
|
||||
@@ -0,0 +1,165 @@
|
||||
"""Parse command implementation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import click
|
||||
|
||||
from vlm.context import CLIContext
|
||||
from vlm.io import load_inventory_csv, save_identities_json
|
||||
from vlm.parser import parse_movie, parse_series
|
||||
from vlm.utils import utc_now
|
||||
|
||||
|
||||
def parse_cmd(ctx: CLIContext, input: Path, output: Path, inventory: Optional[Path]) -> None:
|
||||
"""Parse identities from scanned inventory."""
|
||||
config = ctx.config
|
||||
logger = ctx.logger
|
||||
|
||||
click.echo(f"Parsing identities from: {input}")
|
||||
|
||||
path_to_metadata: dict[str, object] = {}
|
||||
if inventory:
|
||||
click.echo(f"Loading video metadata from: {inventory}")
|
||||
inventory_files = load_inventory_csv(inventory)
|
||||
path_to_metadata = {str(vf.path): vf for vf in inventory_files}
|
||||
click.echo(f"Loaded metadata for {len(path_to_metadata)} files")
|
||||
|
||||
click.echo()
|
||||
|
||||
inventory_files = load_inventory_csv(input)
|
||||
video_files = [
|
||||
{
|
||||
"path": str(vf.path),
|
||||
"filename": vf.filename,
|
||||
"category": vf.category,
|
||||
}
|
||||
for vf in inventory_files
|
||||
]
|
||||
|
||||
click.echo(f"Loaded {len(video_files)} files from inventory")
|
||||
click.echo()
|
||||
|
||||
movie_identities: list[dict] = []
|
||||
series_identities: list[dict] = []
|
||||
anime_files: list[dict] = []
|
||||
other_files: list[dict] = []
|
||||
|
||||
def get_video_metadata(file_path: str) -> dict:
|
||||
"""Extract video metadata from inventory if available."""
|
||||
if not path_to_metadata:
|
||||
return {}
|
||||
vf = path_to_metadata.get(file_path)
|
||||
if not vf:
|
||||
return {}
|
||||
return {
|
||||
"size_bytes": vf.size_bytes,
|
||||
"modified_timestamp": vf.modified_timestamp.isoformat(),
|
||||
"resolution": vf.resolution,
|
||||
"codec": vf.codec,
|
||||
"duration_seconds": vf.duration_seconds,
|
||||
"bitrate_kbps": vf.bitrate_kbps,
|
||||
}
|
||||
|
||||
for vf in video_files:
|
||||
filename = vf["filename"]
|
||||
category = vf["category"]
|
||||
file_path = vf["path"]
|
||||
video_metadata = get_video_metadata(file_path)
|
||||
|
||||
if category == "movie":
|
||||
identity = parse_movie(filename, extensions=config.video_extensions)
|
||||
record = {
|
||||
"path": file_path,
|
||||
"filename": filename,
|
||||
"category": category,
|
||||
"title": identity.title,
|
||||
"year": identity.year,
|
||||
"confidence": identity.confidence,
|
||||
"needs_review": identity.needs_review,
|
||||
}
|
||||
if video_metadata:
|
||||
record["video_metadata"] = video_metadata
|
||||
movie_identities.append(record)
|
||||
elif category == "series":
|
||||
identity = parse_series(filename, extensions=config.video_extensions)
|
||||
record = {
|
||||
"path": file_path,
|
||||
"filename": filename,
|
||||
"category": category,
|
||||
"title": identity.title,
|
||||
"season": identity.season,
|
||||
"episodes": identity.episodes,
|
||||
"confidence": identity.confidence,
|
||||
"needs_review": identity.needs_review,
|
||||
}
|
||||
if video_metadata:
|
||||
record["video_metadata"] = video_metadata
|
||||
series_identities.append(record)
|
||||
elif category == "anime":
|
||||
anime_files.append(
|
||||
{
|
||||
"path": vf["path"],
|
||||
"filename": filename,
|
||||
"category": category,
|
||||
"note": "Anime parsing deferred in v1",
|
||||
}
|
||||
)
|
||||
else:
|
||||
other_files.append(
|
||||
{
|
||||
"path": vf["path"],
|
||||
"filename": filename,
|
||||
"category": category,
|
||||
"note": "Not categorized for parsing",
|
||||
}
|
||||
)
|
||||
|
||||
click.echo("Parsing complete!")
|
||||
click.echo()
|
||||
click.echo("Results by category:")
|
||||
click.echo(f" Movies: {len(movie_identities)}")
|
||||
|
||||
movies_need_review = sum(1 for m in movie_identities if m["needs_review"])
|
||||
if movies_need_review > 0:
|
||||
click.echo(f" - Need review: {movies_need_review}")
|
||||
|
||||
click.echo(f" Series: {len(series_identities)}")
|
||||
|
||||
series_need_review = sum(1 for s in series_identities if s["needs_review"])
|
||||
if series_need_review > 0:
|
||||
click.echo(f" - Need review: {series_need_review}")
|
||||
|
||||
click.echo(f" Anime: {len(anime_files)} (not parsed in v1)")
|
||||
click.echo(f" Other: {len(other_files)} (not parsed)")
|
||||
|
||||
click.echo()
|
||||
click.echo(f"Saving parsed identities to: {output}")
|
||||
|
||||
generation_timestamp = utc_now().strftime("%Y-%m-%dT%H:%M:%S")
|
||||
schema_version = "2.0" if path_to_metadata else "1.0"
|
||||
|
||||
identities_data = {
|
||||
"vlm_schema_version": schema_version,
|
||||
"metadata": {
|
||||
"generated": generation_timestamp,
|
||||
"source_inventory": str(input),
|
||||
"total_files": len(video_files),
|
||||
},
|
||||
"movies": movie_identities,
|
||||
"series": series_identities,
|
||||
"anime": anime_files,
|
||||
"other": other_files,
|
||||
}
|
||||
|
||||
save_identities_json(identities_data, output)
|
||||
|
||||
click.echo("Parsed identities saved successfully!")
|
||||
logger.info(
|
||||
"Parse completed: %s movies, %s series, saved to %s",
|
||||
len(movie_identities),
|
||||
len(series_identities),
|
||||
output,
|
||||
)
|
||||
@@ -81,6 +81,15 @@ def plan_cmd(ctx: CLIContext, input: Path, output: Path, analysis: Optional[Path
|
||||
click.echo(f" ⚠️ Conflicts detected: {conflicts}")
|
||||
click.echo(" Review the plan file for details on conflicting operations.")
|
||||
|
||||
# Check for directory warnings
|
||||
directory_warning = execution_plan.metadata.get("directory_warning", False)
|
||||
if directory_warning:
|
||||
emptied_dirs = execution_plan.metadata.get("emptied_directories", [])
|
||||
click.echo()
|
||||
click.echo(f" ⚠️ Directory preservation warning: {len(emptied_dirs)} directories will be emptied")
|
||||
click.echo(" These directories will be preserved but may be empty after execution.")
|
||||
click.echo(" Review the plan file for details on preserved directories.")
|
||||
|
||||
click.echo()
|
||||
click.echo(f"Saving execution plan to: {output}")
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
+7
-7
@@ -5,15 +5,17 @@ from pathlib import Path
|
||||
from typing import Optional
|
||||
import yaml
|
||||
|
||||
DEFAULT_VIDEO_EXTENSIONS = [
|
||||
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class Config:
|
||||
"""Configuration for Video Library Manager."""
|
||||
|
||||
library_root: Path
|
||||
video_extensions: list[str] = field(default_factory=lambda: [
|
||||
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
|
||||
])
|
||||
video_extensions: list[str] = field(default_factory=lambda: list(DEFAULT_VIDEO_EXTENSIONS))
|
||||
movie_template: str = "movie/{title} ({year})/"
|
||||
series_template: str = "series/{title}/Season {season:02d}/"
|
||||
movie_filename_template: str = "{title} ({year}){ext}"
|
||||
@@ -74,9 +76,7 @@ def load_config(path: Path) -> Config:
|
||||
|
||||
library_root = Path(library_root_str).expanduser()
|
||||
|
||||
video_extensions = data.get("video_extensions", [
|
||||
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
|
||||
])
|
||||
video_extensions = data.get("video_extensions", list(DEFAULT_VIDEO_EXTENSIONS))
|
||||
|
||||
templates = data.get("templates", {})
|
||||
movie_template = templates.get("movie_dir", "movie/{title} ({year})/")
|
||||
@@ -149,7 +149,7 @@ def create_default_config(path: Path) -> Config:
|
||||
"""Create a default configuration file and return the Config object."""
|
||||
default_config = Config(
|
||||
library_root=Path.home() / "Videos",
|
||||
video_extensions=[".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"],
|
||||
video_extensions=list(DEFAULT_VIDEO_EXTENSIONS),
|
||||
)
|
||||
|
||||
enrichment_content = {
|
||||
|
||||
@@ -93,8 +93,17 @@ class ExecutionEngine:
|
||||
transaction_log = None
|
||||
if mode == "execute":
|
||||
log_path = Path.home() / ".vlm" / "transaction.json"
|
||||
try:
|
||||
transaction_log = TransactionLog(log_path)
|
||||
transaction_log.start_transaction(plan)
|
||||
except OSError as exc:
|
||||
log_operation(
|
||||
self.logger,
|
||||
logging.WARNING,
|
||||
f"Transaction log disabled (cannot write {log_path}): {exc}",
|
||||
operation_type="execute",
|
||||
)
|
||||
transaction_log = None
|
||||
|
||||
# Execute all operations
|
||||
results = []
|
||||
@@ -105,9 +114,18 @@ class ExecutionEngine:
|
||||
# Update transaction and state logs in execute mode
|
||||
if mode == "execute":
|
||||
if transaction_log:
|
||||
try:
|
||||
transaction_log.mark_operation_complete(
|
||||
i, result.success, result.error_message
|
||||
)
|
||||
except OSError as exc:
|
||||
log_operation(
|
||||
self.logger,
|
||||
logging.WARNING,
|
||||
f"Failed to update transaction log: {exc}",
|
||||
operation_type="execute",
|
||||
)
|
||||
transaction_log = None
|
||||
|
||||
# Update file state if successful and not a no-op
|
||||
if result.success and operation.operation_type != "no-op" and self.state_manager:
|
||||
@@ -126,7 +144,15 @@ class ExecutionEngine:
|
||||
if mode == "execute":
|
||||
if transaction_log:
|
||||
status = "completed" if all(r.success for r in results) else "failed"
|
||||
try:
|
||||
transaction_log.complete_transaction(status=status)
|
||||
except OSError as exc:
|
||||
log_operation(
|
||||
self.logger,
|
||||
logging.WARNING,
|
||||
f"Failed to finalize transaction log: {exc}",
|
||||
operation_type="execute",
|
||||
)
|
||||
if self.state_manager:
|
||||
self.state_manager.save()
|
||||
|
||||
@@ -189,6 +215,22 @@ class ExecutionEngine:
|
||||
executed_at=executed_at
|
||||
)
|
||||
|
||||
# Handle preserve-directory operations
|
||||
if operation.operation_type == "preserve-directory":
|
||||
log_operation(
|
||||
self.logger,
|
||||
logging.DEBUG,
|
||||
f"Preserving directory: {operation.reason}",
|
||||
operation_type="execute",
|
||||
file_path=operation.source_path
|
||||
)
|
||||
return OperationResult(
|
||||
operation=operation,
|
||||
success=True,
|
||||
error_message=None,
|
||||
executed_at=executed_at
|
||||
)
|
||||
|
||||
# Handle quarantine operations (no destination_path; use QuarantineManager)
|
||||
if operation.operation_type == "quarantine":
|
||||
if not self._quarantine_manager:
|
||||
|
||||
+44
-7
@@ -15,34 +15,71 @@ from vlm.scanner import load_inventory_csv, save_inventory_csv
|
||||
__all__ = [
|
||||
"load_inventory_csv",
|
||||
"save_inventory_csv",
|
||||
"load_json_file",
|
||||
"save_json_file",
|
||||
"load_identities_json",
|
||||
"save_identities_json",
|
||||
"load_analysis_json",
|
||||
"save_analysis_json",
|
||||
"identities_to_plan_input",
|
||||
"identities_to_analysis_input",
|
||||
]
|
||||
|
||||
|
||||
def load_json_file(path: Path) -> dict:
|
||||
"""Load a JSON object from disk."""
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def save_json_file(data: dict, path: Path) -> None:
|
||||
"""Save a JSON object to disk."""
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
|
||||
|
||||
def load_analysis_json(path: Path) -> dict:
|
||||
"""Load analysis result from JSON file (metadata, completeness, duplicates).
|
||||
|
||||
Caller should check file existence and handle missing/invalid keys.
|
||||
"""
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
return load_json_file(path)
|
||||
|
||||
|
||||
def load_identities_json(path: Path) -> dict:
|
||||
"""Load identities from JSON file."""
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
return load_json_file(path)
|
||||
|
||||
|
||||
def save_identities_json(data: dict, path: Path) -> None:
|
||||
"""Save identities dict to JSON file."""
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
save_json_file(data, path)
|
||||
|
||||
|
||||
def save_analysis_json(
|
||||
*,
|
||||
completeness: list[dict],
|
||||
duplicates: list[dict],
|
||||
source_identities: Path,
|
||||
total_movies: int,
|
||||
total_series: int,
|
||||
output: Path,
|
||||
) -> None:
|
||||
"""Save analysis result JSON using the canonical schema."""
|
||||
generation_timestamp = utc_now().strftime("%Y-%m-%dT%H:%M:%S")
|
||||
analysis_data = {
|
||||
"vlm_schema_version": "1.0",
|
||||
"metadata": {
|
||||
"generated": generation_timestamp,
|
||||
"source_identities": str(source_identities),
|
||||
"total_movies": total_movies,
|
||||
"total_series": total_series,
|
||||
},
|
||||
"completeness": completeness,
|
||||
"duplicates": duplicates,
|
||||
}
|
||||
save_json_file(analysis_data, output)
|
||||
|
||||
|
||||
def _video_file_from_record(record: dict) -> VideoFile:
|
||||
|
||||
+1
-1
@@ -115,7 +115,7 @@ class FileOperation:
|
||||
"""Represents a single file operation in an execution plan.
|
||||
|
||||
Attributes:
|
||||
operation_type: Type of operation ("move", "rename", "quarantine", "no-op")
|
||||
operation_type: Type of operation ("move", "rename", "quarantine", "no-op", "preserve-directory")
|
||||
source_path: Source file path
|
||||
destination_path: Destination file path (None for no-op operations)
|
||||
reason: Human-readable reason for the operation
|
||||
|
||||
+1
-6
@@ -7,14 +7,9 @@ logical identities such as movie titles/years and series titles/seasons/episodes
|
||||
import re
|
||||
from typing import Optional
|
||||
|
||||
from vlm.config import DEFAULT_VIDEO_EXTENSIONS
|
||||
from vlm.models import MovieIdentity, SeriesIdentity
|
||||
|
||||
# Default extensions used when extensions param is not provided (matches config default)
|
||||
DEFAULT_VIDEO_EXTENSIONS = [
|
||||
".mp4", ".mkv", ".avi", ".mov", ".wmv", ".flv", ".webm", ".m4v"
|
||||
]
|
||||
|
||||
|
||||
# Quality tags to remove from titles
|
||||
QUALITY_TAGS = [
|
||||
r'\b1080p\b', r'\b720p\b', r'\b480p\b', r'\b2160p\b',
|
||||
|
||||
+57
-1
@@ -40,6 +40,42 @@ def _is_sample_path(path: Path) -> bool:
|
||||
return bool(re.search(r"(^|[\s._-])sample($|[\s._-])", stem))
|
||||
|
||||
|
||||
def _analyze_directory_impact(operations: list[FileOperation]) -> dict:
|
||||
"""Analyze which directories will be emptied by the plan."""
|
||||
# Get all source directories that have files being moved/renamed
|
||||
source_dirs = set()
|
||||
moved_files = set()
|
||||
for operation in operations:
|
||||
if operation.operation_type in ("move", "rename"):
|
||||
source_dirs.add(operation.source_path.parent)
|
||||
moved_files.add(operation.source_path)
|
||||
|
||||
emptied_dirs = []
|
||||
for dir_path in source_dirs:
|
||||
# Only analyze directories that actually exist
|
||||
if not dir_path.exists():
|
||||
continue
|
||||
|
||||
# Count files that will remain in this directory after operations
|
||||
remaining_count = 0
|
||||
try:
|
||||
for item in dir_path.iterdir():
|
||||
if item.is_file() and item not in moved_files:
|
||||
remaining_count += 1
|
||||
except (OSError, PermissionError):
|
||||
# If we can't read the directory, skip analysis
|
||||
continue
|
||||
|
||||
# If no files will remain, this directory will be emptied
|
||||
if remaining_count == 0:
|
||||
emptied_dirs.append(dir_path)
|
||||
|
||||
return {
|
||||
"emptied_directories": emptied_dirs,
|
||||
"warning_required": len(emptied_dirs) > 0
|
||||
}
|
||||
|
||||
|
||||
def generate_plan(
|
||||
identities: list[tuple[VideoFile, Union[MovieIdentity, SeriesIdentity, None]]],
|
||||
config: Config,
|
||||
@@ -120,10 +156,29 @@ def generate_plan(
|
||||
conflict_reason=None,
|
||||
)
|
||||
|
||||
# Analyze directory impact and add preservation operations
|
||||
directory_analysis = _analyze_directory_impact(operations)
|
||||
if directory_analysis["warning_required"]:
|
||||
# Add directory preservation operations for emptied directories
|
||||
for emptied_dir in directory_analysis["emptied_directories"]:
|
||||
operations.append(FileOperation(
|
||||
operation_type="preserve-directory",
|
||||
source_path=emptied_dir,
|
||||
destination_path=None,
|
||||
reason=f"Preserve empty source directory: {emptied_dir.name}",
|
||||
has_conflict=False,
|
||||
conflict_reason=None
|
||||
))
|
||||
|
||||
summary = _generate_summary(operations)
|
||||
summary_by_reason = _generate_summary_by_reason(operations)
|
||||
human_summary = _generate_human_summary(operations, summary, summary_by_reason, metadata)
|
||||
|
||||
# Add directory warnings to metadata
|
||||
if directory_analysis["warning_required"]:
|
||||
metadata["emptied_directories"] = [str(d) for d in directory_analysis["emptied_directories"]]
|
||||
metadata["directory_warning"] = True
|
||||
|
||||
return ExecutionPlan(
|
||||
plan_id=str(uuid.uuid4()),
|
||||
created_at=utc_now(),
|
||||
@@ -445,7 +500,8 @@ def _generate_summary(operations: list[FileOperation]) -> dict:
|
||||
"move": 0,
|
||||
"rename": 0,
|
||||
"quarantine": 0,
|
||||
"no-op": 0
|
||||
"no-op": 0,
|
||||
"preserve-directory": 0
|
||||
}
|
||||
|
||||
for operation in operations:
|
||||
|
||||
+29
-29
@@ -5,7 +5,7 @@ Tests series completeness analysis, duplicate detection, and quality comparison.
|
||||
|
||||
import pytest
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from vlm.models import SeriesIdentity, SeasonCompleteness, MovieIdentity, VideoFile, DuplicateGroup
|
||||
from vlm.analysis import analyze_series_completeness, detect_duplicates, compare_quality
|
||||
|
||||
@@ -193,7 +193,7 @@ class TestDuplicateDetection:
|
||||
Path("/movies/The.Matrix.1999.1080p.mkv"),
|
||||
"The.Matrix.1999.1080p.mkv",
|
||||
2000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1920x1080",
|
||||
codec="h264"
|
||||
@@ -202,7 +202,7 @@ class TestDuplicateDetection:
|
||||
Path("/movies/The.Matrix.1999.720p.mkv"),
|
||||
"The.Matrix.1999.720p.mkv",
|
||||
1000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1280x720",
|
||||
codec="h264"
|
||||
@@ -211,7 +211,7 @@ class TestDuplicateDetection:
|
||||
Path("/movies/Inception.2010.mkv"),
|
||||
"Inception.2010.mkv",
|
||||
1500000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie"
|
||||
),
|
||||
]
|
||||
@@ -239,7 +239,7 @@ class TestDuplicateDetection:
|
||||
Path("/series/Breaking.Bad.S01E01.1080p.mkv"),
|
||||
"Breaking.Bad.S01E01.1080p.mkv",
|
||||
1500000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"series",
|
||||
resolution="1920x1080"
|
||||
),
|
||||
@@ -247,7 +247,7 @@ class TestDuplicateDetection:
|
||||
Path("/series/Breaking.Bad.S01E01.720p.mkv"),
|
||||
"Breaking.Bad.S01E01.720p.mkv",
|
||||
800000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"series",
|
||||
resolution="1280x720"
|
||||
),
|
||||
@@ -255,7 +255,7 @@ class TestDuplicateDetection:
|
||||
Path("/series/Breaking.Bad.S01E02.mkv"),
|
||||
"Breaking.Bad.S01E02.mkv",
|
||||
1200000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"series"
|
||||
),
|
||||
]
|
||||
@@ -278,8 +278,8 @@ class TestDuplicateDetection:
|
||||
]
|
||||
|
||||
files = [
|
||||
VideoFile(Path("/movies/Movie.A.2020.mkv"), "Movie.A.2020.mkv", 1000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Movie.B.2021.mkv"), "Movie.B.2021.mkv", 1000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Movie.A.2020.mkv"), "Movie.A.2020.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/Movie.B.2021.mkv"), "Movie.B.2021.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
|
||||
]
|
||||
|
||||
result = detect_duplicates(list(zip(identities, files)))
|
||||
@@ -294,8 +294,8 @@ class TestDuplicateDetection:
|
||||
]
|
||||
|
||||
files = [
|
||||
VideoFile(Path("/movies/Unknown.Movie.mkv"), "Unknown.Movie.mkv", 1000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Unknown.Movie.2.mkv"), "Unknown.Movie.2.mkv", 1000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Unknown.Movie.mkv"), "Unknown.Movie.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/Unknown.Movie.2.mkv"), "Unknown.Movie.2.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
|
||||
]
|
||||
|
||||
result = detect_duplicates(list(zip(identities, files)))
|
||||
@@ -311,8 +311,8 @@ class TestDuplicateDetection:
|
||||
]
|
||||
|
||||
files = [
|
||||
VideoFile(Path("/series/Unknown.Show.E01.mkv"), "Unknown.Show.E01.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Unknown.Show.Episode.1.mkv"), "Unknown.Show.Episode.1.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Unknown.Show.E01.mkv"), "Unknown.Show.E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
VideoFile(Path("/series/Unknown.Show.Episode.1.mkv"), "Unknown.Show.Episode.1.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
]
|
||||
|
||||
result = detect_duplicates(list(zip(identities, files)))
|
||||
@@ -327,8 +327,8 @@ class TestDuplicateDetection:
|
||||
]
|
||||
|
||||
files = [
|
||||
VideoFile(Path("/series/Show.Name.S01.mkv"), "Show.Name.S01.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Show.Name.Season.1.mkv"), "Show.Name.Season.1.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Show.Name.S01.mkv"), "Show.Name.S01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
VideoFile(Path("/series/Show.Name.Season.1.mkv"), "Show.Name.Season.1.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
]
|
||||
|
||||
result = detect_duplicates(list(zip(identities, files)))
|
||||
@@ -347,7 +347,7 @@ class TestDuplicateDetection:
|
||||
Path("/movies/Test.Movie.2020.1080p.mkv"),
|
||||
"Test.Movie.2020.1080p.mkv",
|
||||
2000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1920x1080",
|
||||
codec="h264",
|
||||
@@ -358,7 +358,7 @@ class TestDuplicateDetection:
|
||||
Path("/movies/Test.Movie.2020.720p.mkv"),
|
||||
"Test.Movie.2020.720p.mkv",
|
||||
1000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1280x720",
|
||||
codec="h264",
|
||||
@@ -395,9 +395,9 @@ class TestDuplicateDetection:
|
||||
]
|
||||
|
||||
files = [
|
||||
VideoFile(Path("/series/Show.S01E01-E02.mkv"), "Show.S01E01-E02.mkv", 2000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Show.S01E01-E02.mkv"), "Show.S01E01-E02.mkv", 2000000000, datetime.now(timezone.utc), "series"),
|
||||
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
]
|
||||
|
||||
result = detect_duplicates(list(zip(identities, files)))
|
||||
@@ -413,8 +413,8 @@ class TestDuplicateDetection:
|
||||
]
|
||||
|
||||
files = [
|
||||
VideoFile(Path("/movies/The.Thing.1982.mkv"), "The.Thing.1982.mkv", 1000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/The.Thing.2011.mkv"), "The.Thing.2011.mkv", 1000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/The.Thing.1982.mkv"), "The.Thing.1982.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/The.Thing.2011.mkv"), "The.Thing.2011.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
|
||||
]
|
||||
|
||||
result = detect_duplicates(list(zip(identities, files)))
|
||||
@@ -429,8 +429,8 @@ class TestDuplicateDetection:
|
||||
]
|
||||
|
||||
files = [
|
||||
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Show.S02E01.mkv"), "Show.S02E01.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
VideoFile(Path("/series/Show.S02E01.mkv"), "Show.S02E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
]
|
||||
|
||||
result = detect_duplicates(list(zip(identities, files)))
|
||||
@@ -448,7 +448,7 @@ class TestQualityComparison:
|
||||
Path("/test/file1.mkv"),
|
||||
"file1.mkv",
|
||||
2000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1920x1080",
|
||||
codec="h264",
|
||||
@@ -459,7 +459,7 @@ class TestQualityComparison:
|
||||
Path("/test/file2.mkv"),
|
||||
"file2.mkv",
|
||||
1000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1280x720",
|
||||
codec="h265",
|
||||
@@ -490,7 +490,7 @@ class TestQualityComparison:
|
||||
Path("/test/file1.mkv"),
|
||||
"file1.mkv",
|
||||
2000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1920x1080"
|
||||
# codec, duration, bitrate not available
|
||||
@@ -499,7 +499,7 @@ class TestQualityComparison:
|
||||
Path("/test/file2.mkv"),
|
||||
"file2.mkv",
|
||||
1000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie"
|
||||
# No optional metadata
|
||||
),
|
||||
@@ -532,7 +532,7 @@ class TestQualityComparison:
|
||||
Path("/test/file.mkv"),
|
||||
"file.mkv",
|
||||
1500000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1920x1080",
|
||||
codec="h264"
|
||||
|
||||
@@ -6,7 +6,7 @@ Each test validates a specific property from the design document.
|
||||
|
||||
import pytest
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from hypothesis import given, strategies as st, settings
|
||||
from vlm.models import (
|
||||
SeriesIdentity, SeasonCompleteness, MovieIdentity, VideoFile, DuplicateGroup
|
||||
@@ -74,7 +74,7 @@ def video_file_strategy(draw, filename=None, category="movie"):
|
||||
|
||||
path = Path(f"/{category}/{filename}")
|
||||
size_bytes = draw(st.integers(min_value=1000000, max_value=10000000000))
|
||||
modified_timestamp = datetime.now()
|
||||
modified_timestamp = datetime.now(timezone.utc)
|
||||
|
||||
# Optional metadata
|
||||
has_metadata = draw(st.booleans())
|
||||
@@ -226,7 +226,7 @@ def test_property_12_duplicate_detection_movies(title, year, duplicate_count):
|
||||
Path(f"/movies/{filename}"),
|
||||
filename,
|
||||
1000000000 + i * 100000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie"
|
||||
))
|
||||
|
||||
@@ -272,7 +272,7 @@ def test_property_13_duplicate_detection_series(title, season, episode, duplicat
|
||||
Path(f"/series/{filename}"),
|
||||
filename,
|
||||
1000000000 + i * 100000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"series"
|
||||
))
|
||||
|
||||
@@ -321,7 +321,7 @@ def test_property_14_duplicate_quality_comparison(title, year, file_count):
|
||||
Path(f"/movies/{filename}"),
|
||||
filename,
|
||||
1000000000 + i * 100000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1920x1080",
|
||||
codec="h264",
|
||||
@@ -333,7 +333,7 @@ def test_property_14_duplicate_quality_comparison(title, year, file_count):
|
||||
Path(f"/movies/{filename}"),
|
||||
filename,
|
||||
1000000000 + i * 100000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie"
|
||||
))
|
||||
|
||||
@@ -429,7 +429,7 @@ def test_property_43_duplicate_report_grouping(duplicate_count, format):
|
||||
Path(f"/movies/{filename}"),
|
||||
filename,
|
||||
1000000000 + j * 500000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1920x1080" if j == 0 else "1280x720",
|
||||
codec="h264"
|
||||
@@ -492,7 +492,7 @@ def test_property_44_summary_report_accuracy(file_count, categories):
|
||||
Path(f"/{category}/{filename}"),
|
||||
filename,
|
||||
size,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
category
|
||||
))
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import pytest
|
||||
from click.testing import CliRunner
|
||||
@@ -46,7 +46,7 @@ def sample_rollback_log(tmp_path):
|
||||
),
|
||||
success=True,
|
||||
error_message=None,
|
||||
executed_at=datetime.now()
|
||||
executed_at=datetime.now(timezone.utc)
|
||||
),
|
||||
OperationResult(
|
||||
operation=FileOperation(
|
||||
@@ -59,14 +59,14 @@ def sample_rollback_log(tmp_path):
|
||||
),
|
||||
success=True,
|
||||
error_message=None,
|
||||
executed_at=datetime.now()
|
||||
executed_at=datetime.now(timezone.utc)
|
||||
)
|
||||
]
|
||||
|
||||
rollback_log = RollbackLog(
|
||||
log_id="test-log-id",
|
||||
execution_plan_id="test-plan-id",
|
||||
executed_at=datetime.now(),
|
||||
executed_at=datetime.now(timezone.utc),
|
||||
operations=operations
|
||||
)
|
||||
|
||||
|
||||
+14
-14
@@ -4,7 +4,7 @@ Tests execution mode handling, dry-run simulation, and actual file operations.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
@@ -73,7 +73,7 @@ def sample_plan(temp_test_dir):
|
||||
|
||||
return ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=operations,
|
||||
summary={"move": 1, "rename": 1}
|
||||
)
|
||||
@@ -208,7 +208,7 @@ class TestDryRunSimulation:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=[no_op_operation],
|
||||
summary={"no-op": 1}
|
||||
)
|
||||
@@ -233,7 +233,7 @@ class TestDryRunSimulation:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=[conflicted_operation],
|
||||
summary={"move": 1}
|
||||
)
|
||||
@@ -285,7 +285,7 @@ class TestExecuteMode:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=[operation],
|
||||
summary={"move": 1}
|
||||
)
|
||||
@@ -319,7 +319,7 @@ class TestExecuteMode:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=[operation],
|
||||
summary={"move": 1}
|
||||
)
|
||||
@@ -352,7 +352,7 @@ class TestExecuteMode:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=[conflicted_operation],
|
||||
summary={"move": 1}
|
||||
)
|
||||
@@ -427,7 +427,7 @@ class TestRollbackLog:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=operations,
|
||||
summary={"move": 2}
|
||||
)
|
||||
@@ -480,7 +480,7 @@ class TestExecutionSummary:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=operations,
|
||||
summary={"move": 2, "no-op": 1}
|
||||
)
|
||||
@@ -573,7 +573,7 @@ class TestRollbackLogSaving:
|
||||
data = json.load(f)
|
||||
|
||||
# Verify timestamps are in ISO format
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
executed_at = datetime.fromisoformat(data["executed_at"])
|
||||
assert executed_at is not None
|
||||
|
||||
@@ -670,7 +670,7 @@ class TestRollbackExecution:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=operations,
|
||||
summary={"move": 2}
|
||||
)
|
||||
@@ -759,7 +759,7 @@ class TestRollbackExecution:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=operations,
|
||||
summary={"move": 1, "no-op": 1}
|
||||
)
|
||||
@@ -877,7 +877,7 @@ class TestRollbackExecution:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=operations,
|
||||
summary={"move": 2}
|
||||
)
|
||||
@@ -920,7 +920,7 @@ class TestRollbackExecution:
|
||||
|
||||
plan = ExecutionPlan(
|
||||
plan_id=str(uuid4()),
|
||||
created_at=datetime.now(),
|
||||
created_at=datetime.now(timezone.utc),
|
||||
operations=[
|
||||
FileOperation(
|
||||
operation_type="quarantine",
|
||||
|
||||
+46
-46
@@ -1,7 +1,7 @@
|
||||
"""Unit tests for plan generator."""
|
||||
|
||||
import pytest
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from vlm.config import Config
|
||||
@@ -36,7 +36,7 @@ def test_generate_plan_for_movie_with_year(config):
|
||||
path=Path("/mnt/nas/videos/movie/Some.Movie.2020.1080p.mkv"),
|
||||
filename="Some.Movie.2020.1080p.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -67,7 +67,7 @@ def test_generate_plan_for_movie_without_year(config):
|
||||
path=Path("/mnt/nas/videos/movie/random_movie.mkv"),
|
||||
filename="random_movie.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -94,7 +94,7 @@ def test_generate_plan_for_series_with_season_and_episode(config):
|
||||
path=Path("/mnt/nas/videos/series/Show.Name.S01E05.mkv"),
|
||||
filename="Show.Name.S01E05.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
)
|
||||
|
||||
@@ -123,7 +123,7 @@ def test_generate_plan_blocks_series_with_high_season(config):
|
||||
path=Path("/mnt/nas/videos/series/Show.Name.S20E01.mkv"),
|
||||
filename="Show.Name.S20E01.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
)
|
||||
identity = SeriesIdentity(
|
||||
@@ -147,7 +147,7 @@ def test_generate_plan_blocks_series_with_high_episode(config):
|
||||
path=Path("/mnt/nas/videos/series/Show.Name.S01E120.mkv"),
|
||||
filename="Show.Name.S01E120.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
)
|
||||
identity = SeriesIdentity(
|
||||
@@ -171,7 +171,7 @@ def test_generate_plan_sample_is_noop_by_default(config):
|
||||
path=Path("/mnt/nas/videos/series/Show.Name.Sample.S01E01.mkv"),
|
||||
filename="Show.Name.Sample.S01E01.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
)
|
||||
identity = SeriesIdentity(
|
||||
@@ -196,7 +196,7 @@ def test_generate_plan_sample_can_be_included_via_config(config):
|
||||
path=Path("/mnt/nas/videos/series/Show.Name.Sample.S01E01.mkv"),
|
||||
filename="Show.Name.Sample.S01E01.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
)
|
||||
identity = SeriesIdentity(
|
||||
@@ -219,7 +219,7 @@ def test_generate_plan_for_series_without_season(config):
|
||||
path=Path("/mnt/nas/videos/series/ambiguous_show.mkv"),
|
||||
filename="ambiguous_show.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
)
|
||||
|
||||
@@ -247,7 +247,7 @@ def test_generate_plan_for_anime_category(config):
|
||||
path=Path("/mnt/nas/videos/anime/Some.Anime.01.mkv"),
|
||||
filename="Some.Anime.01.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="anime"
|
||||
)
|
||||
|
||||
@@ -267,7 +267,7 @@ def test_generate_plan_for_other_category(config):
|
||||
path=Path("/mnt/nas/videos/other/random.mkv"),
|
||||
filename="random.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="other"
|
||||
)
|
||||
|
||||
@@ -286,7 +286,7 @@ def test_generate_plan_preserves_category_boundaries(config):
|
||||
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
|
||||
filename="Movie.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -311,7 +311,7 @@ def test_generate_plan_for_multi_episode_file(config):
|
||||
path=Path("/mnt/nas/videos/series/Show.S01E01E02.mkv"),
|
||||
filename="Show.S01E01E02.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
)
|
||||
|
||||
@@ -338,7 +338,7 @@ def test_generate_plan_file_already_at_target(config):
|
||||
path=Path("/mnt/nas/videos/movie/Some Movie (2020)/Some Movie (2020).mkv"),
|
||||
filename="Some Movie (2020).mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -364,7 +364,7 @@ def test_generate_plan_rename_vs_move(config):
|
||||
path=Path("/mnt/nas/videos/movie/Some Movie (2020)/old_name.mkv"),
|
||||
filename="old_name.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -385,7 +385,7 @@ def test_generate_plan_rename_vs_move(config):
|
||||
path=Path("/mnt/nas/videos/movie/wrong_dir/Some Movie (2020).mkv"),
|
||||
filename="Some Movie (2020).mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -403,7 +403,7 @@ def test_generate_plan_summary(config):
|
||||
path=Path("/mnt/nas/videos/movie/Movie1.2020.mkv"),
|
||||
filename="Movie1.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
),
|
||||
MovieIdentity(
|
||||
@@ -420,7 +420,7 @@ def test_generate_plan_summary(config):
|
||||
path=Path("/mnt/nas/videos/movie/Movie2.mkv"),
|
||||
filename="Movie2.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
),
|
||||
MovieIdentity(
|
||||
@@ -437,7 +437,7 @@ def test_generate_plan_summary(config):
|
||||
path=Path("/mnt/nas/videos/anime/Anime.01.mkv"),
|
||||
filename="Anime.01.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="anime"
|
||||
),
|
||||
None
|
||||
@@ -448,7 +448,7 @@ def test_generate_plan_summary(config):
|
||||
path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"),
|
||||
filename="Show.S01E01.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
),
|
||||
SeriesIdentity(
|
||||
@@ -480,14 +480,14 @@ def test_generate_plan_with_analysis_by_quality(config):
|
||||
path=p_720,
|
||||
filename="Test.2020.720p.WEB-DL.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie",
|
||||
)
|
||||
vf_1080 = VideoFile(
|
||||
path=p_1080,
|
||||
filename="Test.2020.1080p.BluRay.mkv",
|
||||
size_bytes=2000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie",
|
||||
)
|
||||
identity = MovieIdentity(
|
||||
@@ -528,14 +528,14 @@ def test_generate_plan_with_analysis_by_reputation_quality_time_reason(config):
|
||||
path=p_old,
|
||||
filename="Test.2020.720p.WEB-DL.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie",
|
||||
)
|
||||
vf_new = VideoFile(
|
||||
path=p_new,
|
||||
filename="Test.2020.1080p.BluRay.mkv",
|
||||
size_bytes=2000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie",
|
||||
)
|
||||
identity = MovieIdentity(
|
||||
@@ -574,14 +574,14 @@ def test_generate_plan_with_analysis_by_reputation_missing_scores_uses_fallback_
|
||||
path=p_web,
|
||||
filename="Test.2020.2160p.WEB-DL.mkv",
|
||||
size_bytes=3000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie",
|
||||
)
|
||||
vf_bluray = VideoFile(
|
||||
path=p_bluray,
|
||||
filename="Test.2020.1080p.BluRay.mkv",
|
||||
size_bytes=2000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie",
|
||||
)
|
||||
identity = MovieIdentity(
|
||||
@@ -622,14 +622,14 @@ def test_generate_plan_human_summary_marks_disc_files_as_high_risk(config):
|
||||
path=p_disc1,
|
||||
filename="The.Best.of.Youth.DISC1.mkv",
|
||||
size_bytes=2000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie",
|
||||
)
|
||||
vf2 = VideoFile(
|
||||
path=p_disc2,
|
||||
filename="The.Best.of.Youth.DISC2.mkv",
|
||||
size_bytes=1800000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie",
|
||||
)
|
||||
identity = MovieIdentity(
|
||||
@@ -669,7 +669,7 @@ def test_generate_plan_with_different_extensions(config):
|
||||
path=Path(f"/mnt/nas/videos/movie/Movie.2020{ext}"),
|
||||
filename=f"Movie.2020{ext}",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -704,7 +704,7 @@ def test_conflict_detection_for_movie(config, tmp_path):
|
||||
path=tmp_path / "movie" / "Some.Movie.2020.1080p.mkv",
|
||||
filename="Some.Movie.2020.1080p.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -741,7 +741,7 @@ def test_conflict_detection_for_series(config, tmp_path):
|
||||
path=tmp_path / "series" / "Show.Name.S01E05.1080p.mkv",
|
||||
filename="Show.Name.S01E05.1080p.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
)
|
||||
|
||||
@@ -776,7 +776,7 @@ def test_no_conflict_when_destination_does_not_exist(config, tmp_path):
|
||||
path=source_dir / "Some.Movie.2020.mkv",
|
||||
filename="Some.Movie.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -814,7 +814,7 @@ def test_conflict_detection_with_multiple_files(config, tmp_path):
|
||||
path=tmp_path / "movie" / "Movie1.2020.mkv",
|
||||
filename="Movie1.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
),
|
||||
MovieIdentity(
|
||||
@@ -831,7 +831,7 @@ def test_conflict_detection_with_multiple_files(config, tmp_path):
|
||||
path=tmp_path / "movie" / "Movie2.2021.mkv",
|
||||
filename="Movie2.2021.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
),
|
||||
MovieIdentity(
|
||||
@@ -864,7 +864,7 @@ def test_save_plan_to_json(config, tmp_path):
|
||||
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
|
||||
filename="Movie.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -909,7 +909,7 @@ def test_load_plan_from_json(config, tmp_path):
|
||||
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
|
||||
filename="Movie.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -963,7 +963,7 @@ def test_save_and_load_plan_with_conflicts(config, tmp_path):
|
||||
path=tmp_path / "movie" / "Movie.2020.mkv",
|
||||
filename="Movie.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -1003,7 +1003,7 @@ def test_save_and_load_plan_with_no_op_operations(config, tmp_path):
|
||||
path=Path("/mnt/nas/videos/anime/Anime.01.mkv"),
|
||||
filename="Anime.01.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="anime"
|
||||
),
|
||||
None
|
||||
@@ -1014,7 +1014,7 @@ def test_save_and_load_plan_with_no_op_operations(config, tmp_path):
|
||||
path=Path("/mnt/nas/videos/movie/Movie.mkv"),
|
||||
filename="Movie.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
),
|
||||
MovieIdentity(
|
||||
@@ -1048,7 +1048,7 @@ def test_save_plan_json_is_human_readable(config, tmp_path):
|
||||
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
|
||||
filename="Movie.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -1089,7 +1089,7 @@ def test_save_plan_with_multiple_operations(config, tmp_path):
|
||||
path=Path("/mnt/nas/videos/movie/Movie1.2020.mkv"),
|
||||
filename="Movie1.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
),
|
||||
MovieIdentity(
|
||||
@@ -1106,7 +1106,7 @@ def test_save_plan_with_multiple_operations(config, tmp_path):
|
||||
path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"),
|
||||
filename="Show.S01E01.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
),
|
||||
SeriesIdentity(
|
||||
@@ -1124,7 +1124,7 @@ def test_save_plan_with_multiple_operations(config, tmp_path):
|
||||
path=Path("/mnt/nas/videos/anime/Anime.01.mkv"),
|
||||
filename="Anime.01.mkv",
|
||||
size_bytes=500000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="anime"
|
||||
),
|
||||
None
|
||||
@@ -1176,7 +1176,7 @@ def test_plan_json_includes_all_required_fields(config, tmp_path):
|
||||
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
|
||||
filename="Movie.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -1214,7 +1214,7 @@ def test_movie_rejected_by_review_generates_noop(config):
|
||||
path=Path("/mnt/nas/videos/movie/Movie.2020.mkv"),
|
||||
filename="Movie.2020.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="movie"
|
||||
)
|
||||
|
||||
@@ -1238,7 +1238,7 @@ def test_series_rejected_by_review_generates_noop(config):
|
||||
path=Path("/mnt/nas/videos/series/Show.S01E01.mkv"),
|
||||
filename="Show.S01E01.mkv",
|
||||
size_bytes=1000000,
|
||||
modified_timestamp=datetime.now(),
|
||||
modified_timestamp=datetime.now(timezone.utc),
|
||||
category="series"
|
||||
)
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
import json
|
||||
import pytest
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from vlm.quarantine import QuarantineManager
|
||||
from vlm.config import Config
|
||||
|
||||
+23
-23
@@ -185,7 +185,7 @@ class TestInventoryReport:
|
||||
Path("/test.mkv"),
|
||||
"test.mkv",
|
||||
1000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie"
|
||||
)
|
||||
]
|
||||
@@ -377,7 +377,7 @@ class TestDuplicateReport:
|
||||
Path("/movies/The.Matrix.1999.1080p.mkv"),
|
||||
"The.Matrix.1999.1080p.mkv",
|
||||
2000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1920x1080",
|
||||
codec="h264",
|
||||
@@ -388,7 +388,7 @@ class TestDuplicateReport:
|
||||
Path("/movies/The.Matrix.1999.720p.mkv"),
|
||||
"The.Matrix.1999.720p.mkv",
|
||||
1000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1280x720",
|
||||
codec="h264"
|
||||
@@ -448,14 +448,14 @@ class TestDuplicateReport:
|
||||
Path("/movies/Inception.2010.1080p.mkv"),
|
||||
"Inception.2010.1080p.mkv",
|
||||
2000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie"
|
||||
),
|
||||
VideoFile(
|
||||
Path("/movies/Inception.2010.720p.mkv"),
|
||||
"Inception.2010.720p.mkv",
|
||||
1000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie"
|
||||
),
|
||||
]
|
||||
@@ -501,14 +501,14 @@ class TestDuplicateReport:
|
||||
Path("/series/Breaking.Bad.S01E01.1080p.mkv"),
|
||||
"Breaking.Bad.S01E01.1080p.mkv",
|
||||
1500000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"series"
|
||||
),
|
||||
VideoFile(
|
||||
Path("/series/Breaking.Bad.S01E01.720p.mkv"),
|
||||
"Breaking.Bad.S01E01.720p.mkv",
|
||||
800000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"series"
|
||||
),
|
||||
]
|
||||
@@ -564,8 +564,8 @@ class TestDuplicateReport:
|
||||
# Create two duplicate groups with different sizes
|
||||
identity1 = MovieIdentity("Small Movie", 2020, 0.9, False, "Small.Movie.mkv")
|
||||
files1 = [
|
||||
VideoFile(Path("/movies/Small.Movie.1.mkv"), "Small.Movie.1.mkv", 500000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Small.Movie.2.mkv"), "Small.Movie.2.mkv", 600000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Small.Movie.1.mkv"), "Small.Movie.1.mkv", 500000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/Small.Movie.2.mkv"), "Small.Movie.2.mkv", 600000000, datetime.now(timezone.utc), "movie"),
|
||||
]
|
||||
quality1 = [
|
||||
{'filename': 'Small.Movie.1.mkv', 'path': '/movies/Small.Movie.1.mkv', 'size_bytes': 500000000},
|
||||
@@ -574,8 +574,8 @@ class TestDuplicateReport:
|
||||
|
||||
identity2 = MovieIdentity("Large Movie", 2021, 0.9, False, "Large.Movie.mkv")
|
||||
files2 = [
|
||||
VideoFile(Path("/movies/Large.Movie.1.mkv"), "Large.Movie.1.mkv", 2000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Large.Movie.2.mkv"), "Large.Movie.2.mkv", 1800000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Large.Movie.1.mkv"), "Large.Movie.1.mkv", 2000000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/Large.Movie.2.mkv"), "Large.Movie.2.mkv", 1800000000, datetime.now(timezone.utc), "movie"),
|
||||
]
|
||||
quality2 = [
|
||||
{'filename': 'Large.Movie.1.mkv', 'path': '/movies/Large.Movie.1.mkv', 'size_bytes': 2000000000},
|
||||
@@ -600,8 +600,8 @@ class TestDuplicateReport:
|
||||
"""Sort order should use quality_comparison sizes when VideoFile sizes are zero."""
|
||||
identity1 = MovieIdentity("Tiny", 2020, 0.9, False, "Tiny.mkv")
|
||||
files1 = [
|
||||
VideoFile(Path("/movies/Tiny.1.mkv"), "Tiny.1.mkv", 0, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Tiny.2.mkv"), "Tiny.2.mkv", 0, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Tiny.1.mkv"), "Tiny.1.mkv", 0, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/Tiny.2.mkv"), "Tiny.2.mkv", 0, datetime.now(timezone.utc), "movie"),
|
||||
]
|
||||
quality1 = [
|
||||
{"filename": "Tiny.1.mkv", "path": "/movies/Tiny.1.mkv", "size_bytes": 600000000},
|
||||
@@ -610,8 +610,8 @@ class TestDuplicateReport:
|
||||
|
||||
identity2 = MovieIdentity("Huge", 2021, 0.9, False, "Huge.mkv")
|
||||
files2 = [
|
||||
VideoFile(Path("/movies/Huge.1.mkv"), "Huge.1.mkv", 0, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Huge.2.mkv"), "Huge.2.mkv", 0, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Huge.1.mkv"), "Huge.1.mkv", 0, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/Huge.2.mkv"), "Huge.2.mkv", 0, datetime.now(timezone.utc), "movie"),
|
||||
]
|
||||
quality2 = [
|
||||
{"filename": "Huge.1.mkv", "path": "/movies/Huge.1.mkv", "size_bytes": 3000000000},
|
||||
@@ -632,12 +632,12 @@ class TestSummaryReport:
|
||||
def test_generate_summary_report(self):
|
||||
"""Test generating summary report with various files."""
|
||||
files = [
|
||||
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(), "anime"),
|
||||
VideoFile(Path("/other/Random.mkv"), "Random.mkv", 500000000, datetime.now(), "other"),
|
||||
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(timezone.utc), "anime"),
|
||||
VideoFile(Path("/other/Random.mkv"), "Random.mkv", 500000000, datetime.now(timezone.utc), "other"),
|
||||
]
|
||||
|
||||
library_root = Path("/mnt/nas/videos")
|
||||
@@ -677,8 +677,8 @@ class TestSummaryReport:
|
||||
def test_generate_summary_report_single_category(self):
|
||||
"""Test generating summary report with files in single category."""
|
||||
files = [
|
||||
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 1000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 2000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 1000000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 2000000000, datetime.now(timezone.utc), "movie"),
|
||||
]
|
||||
|
||||
library_root = Path("/mnt/nas/videos")
|
||||
|
||||
@@ -5,7 +5,7 @@ Tests the complete workflow from analysis to report generation.
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from vlm.models import SeriesIdentity, VideoFile, MovieIdentity
|
||||
from vlm.analysis import analyze_series_completeness, detect_duplicates
|
||||
from vlm.reports import generate_completeness_report, generate_duplicate_report, generate_summary_report
|
||||
@@ -59,7 +59,7 @@ class TestReportsIntegration:
|
||||
Path("/movies/The.Matrix.1999.1080p.mkv"),
|
||||
"The.Matrix.1999.1080p.mkv",
|
||||
2000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1920x1080",
|
||||
codec="h264"
|
||||
@@ -68,7 +68,7 @@ class TestReportsIntegration:
|
||||
Path("/movies/The.Matrix.1999.720p.mkv"),
|
||||
"The.Matrix.1999.720p.mkv",
|
||||
1000000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie",
|
||||
resolution="1280x720",
|
||||
codec="h264"
|
||||
@@ -77,7 +77,7 @@ class TestReportsIntegration:
|
||||
Path("/movies/Inception.2010.mkv"),
|
||||
"Inception.2010.mkv",
|
||||
1500000000,
|
||||
datetime.now(),
|
||||
datetime.now(timezone.utc),
|
||||
"movie"
|
||||
),
|
||||
]
|
||||
@@ -107,11 +107,11 @@ class TestReportsIntegration:
|
||||
"""Test summary report generation with mixed file types."""
|
||||
# Create test files
|
||||
files = [
|
||||
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(), "movie"),
|
||||
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(), "series"),
|
||||
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(), "anime"),
|
||||
VideoFile(Path("/movies/Movie1.mkv"), "Movie1.mkv", 2000000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/movies/Movie2.mkv"), "Movie2.mkv", 1500000000, datetime.now(timezone.utc), "movie"),
|
||||
VideoFile(Path("/series/Show.S01E01.mkv"), "Show.S01E01.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
VideoFile(Path("/series/Show.S01E02.mkv"), "Show.S01E02.mkv", 1000000000, datetime.now(timezone.utc), "series"),
|
||||
VideoFile(Path("/anime/Anime1.mkv"), "Anime1.mkv", 800000000, datetime.now(timezone.utc), "anime"),
|
||||
]
|
||||
|
||||
# Generate summary report
|
||||
|
||||
+1
-1
@@ -92,7 +92,7 @@ class TestLoadSaveState:
|
||||
store = StateStore(
|
||||
states={},
|
||||
version='1.0',
|
||||
last_updated=datetime.now()
|
||||
last_updated=datetime.now(timezone.utc)
|
||||
)
|
||||
|
||||
save_state(store, state_path)
|
||||
|
||||
Reference in New Issue
Block a user