Fix identity reconstruction metadata loss with v2 schema
Implements identities.json v2 schema with embedded video metadata to fix duplicate resolution by quality, which previously failed due to VideoFile objects being reconstructed with hardcoded defaults (size_bytes=0, resolution=None, codec=None). Changes: - Add --inventory flag to vlm parse command to embed video metadata - Update _video_file_from_record() to extract embedded metadata if present - Add vlm_schema_version field to identities.json (v1.0 or v2.0) - Maintain backward compatibility with v1 files (no metadata) Schema v2 format: - Embeds video_metadata object in each record (movies/series) - Contains: size_bytes, modified_timestamp, resolution, codec, duration_seconds, bitrate_kbps - Enables accurate quality comparison during duplicate analysis Testing: - Added comprehensive unit tests for io.py functions - Added CLI integration tests for parse command - Added end-to-end tests for duplicate quality comparison - All 437 existing tests still pass (1 pre-existing failure in executor) Documentation: - Updated README.md with --inventory usage examples - Updated CLAUDE.md with schema versioning details - Added workflow examples showing metadata embedding This fix resolves the critical P0 issue where duplicate resolution by_quality strategy failed completely due to missing video metadata in reconstructed VideoFile objects. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 4.5
parent
1f55eab304
commit
065195b83b
@@ -68,12 +68,19 @@ Extract titles, years, seasons, and episodes from filenames:
|
||||
vlm parse
|
||||
```
|
||||
|
||||
This creates `identities.json` with parsed information.
|
||||
**For accurate duplicate resolution by quality, embed video metadata:**
|
||||
|
||||
```bash
|
||||
vlm parse --inventory inventory.csv
|
||||
```
|
||||
|
||||
This creates `identities.json` with parsed information. When using `--inventory`, video metadata (size, resolution, codec) is embedded, enabling accurate quality comparison during duplicate analysis.
|
||||
|
||||
**What it extracts:**
|
||||
- **Movies**: Title and year (e.g., "Inception (2010)")
|
||||
- **Series**: Title, season, and episode numbers (e.g., "Breaking Bad S01E01")
|
||||
- **Confidence scores**: Indicates parsing reliability
|
||||
- **Video metadata** (with `--inventory`): Size, resolution, codec, duration, bitrate
|
||||
|
||||
### 4. Enrich Titles and Reputation (Optional but Recommended)
|
||||
|
||||
@@ -180,9 +187,9 @@ vlm config init
|
||||
vlm scan
|
||||
# Output: inventory.csv with 1234 files discovered
|
||||
|
||||
# 3. Parse filenames
|
||||
vlm parse
|
||||
# Output: identities.json with parsed titles and episodes
|
||||
# 3. Parse filenames with metadata embedding (recommended for duplicate resolution)
|
||||
vlm parse --inventory inventory.csv
|
||||
# Output: identities.json with parsed titles, episodes, and embedded video metadata (v2 schema)
|
||||
|
||||
# 4. Enrich identities (translation + reputation)
|
||||
vlm enrich
|
||||
@@ -190,7 +197,7 @@ vlm enrich
|
||||
|
||||
# 5. Analyze for gaps and duplicates
|
||||
vlm analyze
|
||||
# Output: analysis.json with 5 series with gaps, 12 duplicate groups
|
||||
# Output: analysis.json with 5 series with gaps, 12 duplicate groups (accurate quality comparison)
|
||||
|
||||
# 6. Generate execution plan (optionally use analysis for duplicate handling)
|
||||
vlm plan --analysis analysis.json
|
||||
@@ -246,13 +253,23 @@ vlm scan --output my_library.csv
|
||||
### Parsing
|
||||
|
||||
```bash
|
||||
# Parse with default files
|
||||
# Parse with default files (v1 schema - no metadata embedding)
|
||||
vlm parse
|
||||
|
||||
# Parse with metadata embedding (v2 schema - enables quality comparison)
|
||||
vlm parse --inventory inventory.csv
|
||||
|
||||
# Parse with custom input/output
|
||||
vlm parse --input my_inventory.csv --output my_identities.json
|
||||
|
||||
# Parse with metadata from custom inventory
|
||||
vlm parse --input my_inventory.csv --output my_identities.json --inventory my_inventory.csv
|
||||
```
|
||||
|
||||
**Schema Versions:**
|
||||
- **v1** (without `--inventory`): Lightweight identities, suitable for basic organization
|
||||
- **v2** (with `--inventory`): Embeds video metadata, required for accurate duplicate resolution by quality
|
||||
|
||||
### Enrichment
|
||||
|
||||
```bash
|
||||
@@ -557,7 +574,7 @@ vlm report summary
|
||||
```bash
|
||||
# 1. Scan and parse
|
||||
vlm scan
|
||||
vlm parse
|
||||
vlm parse --inventory inventory.csv
|
||||
vlm enrich
|
||||
|
||||
# 2. Analyze completeness
|
||||
@@ -571,15 +588,15 @@ vlm report completeness --plan plan.json
|
||||
### Scenario 3: Finding and Removing Duplicates
|
||||
|
||||
```bash
|
||||
# 1. Scan and parse
|
||||
# 1. Scan and parse (with --inventory for accurate quality comparison)
|
||||
vlm scan
|
||||
vlm parse
|
||||
vlm parse --inventory inventory.csv
|
||||
vlm enrich
|
||||
|
||||
# 2. Analyze for duplicates
|
||||
vlm analyze
|
||||
|
||||
# 3. View duplicates with quality comparison
|
||||
# 3. View duplicates with quality comparison (now shows actual resolution/codec/size)
|
||||
vlm report duplicates
|
||||
|
||||
# 4. Generate plan with analysis: VLM keeps one file per duplicate group (by reputation) and quarantines the rest
|
||||
@@ -604,7 +621,7 @@ vlm report duplicates --plan plan.json
|
||||
|
||||
# 2. Scan and parse
|
||||
vlm scan
|
||||
vlm parse
|
||||
vlm parse --inventory inventory.csv
|
||||
vlm enrich
|
||||
|
||||
# 3. Generate plan
|
||||
|
||||
Reference in New Issue
Block a user