Update analysis and plan files to enhance duplicate handling and reporting
- Updated `analysis.json` with a new generation timestamp. - Modified `plan.json` to include a new plan ID and created timestamp, and changed operation types from "no-op" to "quarantine" for specific files needing manual review. - Enhanced the README.md to document the new `--analysis` option for generating execution plans, which now includes a human-readable summary and duplicate handling strategies. - Introduced a new `duplicate_resolve.py` module to manage duplicate file resolution strategies. - Improved the execution engine to support quarantine operations and added rollback functionality for quarantined files. These changes improve the functionality of the Video Library Manager by providing better duplicate management and clearer reporting capabilities.
This commit is contained in:
windyboy
@@ -12,7 +12,8 @@ A Python-based CLI tool for managing personal video collections with a safety-fi
|
||||
- **Metadata Enrichment**: Add bilingual titles and reputation signals (TMDB + optional AI fallback)
|
||||
- **Incremental Performance**: SQLite-backed cache avoids repeated metadata lookups
|
||||
- **State Tracking**: Track file status throughout the workflow
|
||||
- **Detailed Reporting**: Generate inventory, completeness, and duplicate reports
|
||||
- **Detailed Reporting**: Generate inventory, completeness, and duplicate reports (reports can include plan content summary via `--plan`)
|
||||
- **Plan–Analysis Integration**: `vlm plan --analysis` applies duplicate resolution (keep by reputation, quarantine rest) and adds a Chinese human summary to the plan for quick review
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -120,9 +121,21 @@ Create a reviewable plan of file operations:
|
||||
vlm plan
|
||||
```
|
||||
|
||||
This creates `plan.json` with proposed operations (move, rename, quarantine).
|
||||
To let the plan automatically resolve duplicate groups (keep one file per group by reputation, quarantine the rest), pass the analysis file:
|
||||
|
||||
**Review the plan** by opening `plan.json` in your editor. You can edit it if needed.
|
||||
```bash
|
||||
vlm plan --analysis analysis.json
|
||||
```
|
||||
|
||||
This creates `plan.json` with:
|
||||
- Proposed operations (move, rename, quarantine, no-op)
|
||||
- **Summary**: counts by operation type and by reason
|
||||
- **Human summary** (中文): short narrative for quick review
|
||||
- **Metadata**: when using `--analysis`, duplicate groups considered and completeness gaps
|
||||
|
||||
Duplicate keep strategy is configurable in `~/.vlm/config.yaml` under `plan.duplicate_keep` (`by_reputation`, `first_seen`, or `manual`). Default is `by_reputation` (prefer external rating; fallback to first-seen).
|
||||
|
||||
**Review the plan** by opening `plan.json` in your editor, or read the human summary when you run `vlm execute`. You can edit the plan JSON if needed.
|
||||
|
||||
### 7. Execute (Dry-Run First)
|
||||
|
||||
@@ -132,7 +145,7 @@ Preview what will happen without making changes:
|
||||
vlm execute
|
||||
```
|
||||
|
||||
When ready to actually move/rename files:
|
||||
Before running, the CLI prints the plan’s **human summary** (or a short summary from counts) so you can confirm at a glance. When ready to actually move/rename/quarantine files:
|
||||
|
||||
```bash
|
||||
vlm execute --confirm
|
||||
@@ -175,9 +188,9 @@ vlm enrich
|
||||
vlm analyze
|
||||
# Output: analysis.json with 5 series with gaps, 12 duplicate groups
|
||||
|
||||
# 6. Generate execution plan
|
||||
vlm plan
|
||||
# Output: plan.json with 456 operations proposed
|
||||
# 6. Generate execution plan (optionally use analysis for duplicate handling)
|
||||
vlm plan --analysis analysis.json
|
||||
# Output: plan.json with operations, human summary, and duplicate quarantine decisions
|
||||
|
||||
# 7. Review the plan
|
||||
cat plan.json | less
|
||||
@@ -271,8 +284,12 @@ vlm analyze --input my_identities.json --output my_analysis.json
|
||||
# Generate plan with default files
|
||||
vlm plan
|
||||
|
||||
# Generate plan with custom files
|
||||
# Use analysis so duplicate groups become "keep one + quarantine rest" (by_reputation by default)
|
||||
vlm plan --analysis analysis.json
|
||||
|
||||
# Custom input/output
|
||||
vlm plan --input my_identities.json --output my_plan.json
|
||||
vlm plan --input my_identities.json --analysis my_analysis.json --output my_plan.json
|
||||
```
|
||||
|
||||
### Execution
|
||||
@@ -327,8 +344,12 @@ vlm report inventory --format json --output inventory_report.json
|
||||
# Generate completeness report (series with gaps)
|
||||
vlm report completeness
|
||||
|
||||
# Include plan content summary in the report (human_summary from plan.json)
|
||||
vlm report completeness --plan plan.json
|
||||
|
||||
# Generate duplicates report
|
||||
vlm report duplicates
|
||||
vlm report duplicates --plan plan.json
|
||||
|
||||
# Generate summary statistics
|
||||
vlm report summary
|
||||
@@ -383,6 +404,11 @@ quarantine_dir: ".quarantine"
|
||||
# Logging level (DEBUG, INFO, WARNING, ERROR)
|
||||
log_level: "INFO"
|
||||
|
||||
# Plan behavior (e.g. when using vlm plan --analysis)
|
||||
plan:
|
||||
# Duplicate keep strategy: "by_reputation" (default), "first_seen", or "manual"
|
||||
duplicate_keep: "by_reputation"
|
||||
|
||||
# Category mappings (directory name to category)
|
||||
categories:
|
||||
movie: [movie, movies, films]
|
||||
@@ -532,8 +558,9 @@ vlm enrich
|
||||
# 2. Analyze completeness
|
||||
vlm analyze
|
||||
|
||||
# 3. View report
|
||||
# 3. View report (optionally include plan summary if you have a plan)
|
||||
vlm report completeness
|
||||
vlm report completeness --plan plan.json
|
||||
```
|
||||
|
||||
### Scenario 3: Finding and Removing Duplicates
|
||||
@@ -550,12 +577,18 @@ vlm analyze
|
||||
# 3. View duplicates with quality comparison
|
||||
vlm report duplicates
|
||||
|
||||
# 4. Manually quarantine lower quality files
|
||||
# 4. Generate plan with analysis: VLM keeps one file per duplicate group (by reputation) and quarantines the rest
|
||||
vlm plan --analysis analysis.json
|
||||
|
||||
# 5. Review plan (human summary in plan.json and when you run execute)
|
||||
vlm execute
|
||||
vlm execute --confirm
|
||||
|
||||
# Alternatively: manual quarantine without plan
|
||||
vlm quarantine add /path/to/lower/quality/file.mkv --reason "duplicate - lower quality"
|
||||
|
||||
# 5. Or generate plan and let VLM suggest operations
|
||||
vlm plan
|
||||
vlm execute
|
||||
# Report with plan context
|
||||
vlm report duplicates --plan plan.json
|
||||
```
|
||||
|
||||
### Scenario 4: Reorganizing Your Library
|
||||
@@ -726,10 +759,11 @@ src/vlm/
|
||||
├── providers/ # External metadata providers (TMDB, etc.)
|
||||
│ ├── base.py # Provider interface
|
||||
│ └── tmdb.py # TMDB API client
|
||||
├── io.py # JSON/CSV load/save and plan/analysis input helpers
|
||||
├── io.py # JSON/CSV load/save, load_analysis_json, plan/analysis input helpers
|
||||
├── utils.py # UTC time, format_size, etc.
|
||||
├── analysis.py # Completeness and duplicate detection
|
||||
├── planner.py # Execution plan generation
|
||||
├── duplicate_resolve.py # Duplicate group keep-index (by_reputation, first_seen, manual)
|
||||
├── planner.py # Execution plan generation (optionally consumes analysis)
|
||||
├── executor.py # File operations and rollback
|
||||
├── quarantine.py # Quarantine management
|
||||
├── state.py # File state tracking
|
||||
|
||||
Reference in New Issue
Block a user