feat(tagging-system): Add comprehensive tagging system implementation

- Add tagging system Python package with modular architecture (config, core, impl) - Create core interfaces for file discovery, content analysis, and tag generation - Implement content analyzer with language detection and topic extraction - Implement file discovery engine with directory scanning and filtering - Implement tag generator with hierarchical tag creation and consolidation - Add configuration management with example config and validation - Create comprehensive design and requirements documentation in .kiro/specs - Add pytest test suite with unit tests for models, interfaces, and config - Add setup.py, requirements.txt, and pytest.ini for package management - Add README.md with project overview and usage instructions - Update workspace.json with new project structure - Add Excalidraw diagram for system architecture visualization - Establish foundation for automated vault tagging and metadata management
2025-12-31 08:37:16 +08:00
parent 6c7a6d0e3a
commit f99056a099
44 changed files with 3209 additions and 14 deletions
@@ -0,0 +1,270 @@
+# Design Document: Comprehensive Tagging System
+
+## Overview
+
+The comprehensive tagging system will analyze and tag all files in the Obsidian vault based on their location, content, and existing metadata. The system uses a multi-layered approach combining directory-based tags, content analysis, and hierarchical topic classification to create a consistent and discoverable knowledge base.
+
+## Architecture
+
+The system follows a pipeline architecture with these main components:
+
+```mermaid
+graph TD
+    A[File Discovery] --> B[Content Analysis]
+    B --> C[Tag Generation]
+    C --> D[Tag Validation]
+    D --> E[Frontmatter Update]
+    E --> F[Batch Processing]
+    
+    G[Tag Registry] --> C
+    G --> D
+    H[Directory Mapping] --> C
+    I[Language Detection] --> C
+    J[Content Classification] --> C
+```
+
+### Core Components
+
+1. **File Discovery Engine**: Recursively scans vault directories
+2. **Content Analyzer**: Extracts topics, language, and content type
+3. **Tag Generator**: Creates hierarchical tags based on analysis
+4. **Tag Validator**: Ensures consistency and prevents duplication
+5. **Frontmatter Manager**: Updates YAML metadata safely
+6. **Batch Processor**: Handles large-scale operations with progress tracking
+
+## Components and Interfaces
+
+### FileDiscovery Interface
+```typescript
+interface FileDiscovery {
+  scanDirectory(path: string): Promise<FileInfo[]>
+  filterByType(files: FileInfo[], types: string[]): FileInfo[]
+  excludeSensitive(files: FileInfo[]): FileInfo[]
+}
+```
+
+### ContentAnalyzer Interface
+```typescript
+interface ContentAnalyzer {
+  analyzeContent(content: string): ContentAnalysis
+  detectLanguage(content: string): LanguageInfo
+  extractTopics(content: string): string[]
+  classifyContentType(content: string, filename: string): ContentType
+}
+```
+
+### TagGenerator Interface
+```typescript
+interface TagGenerator {
+  generateDirectoryTags(filepath: string): string[]
+  generateContentTags(analysis: ContentAnalysis): string[]
+  generateHierarchicalTags(topics: string[]): string[]
+  consolidateTags(tags: string[]): string[]
+}
+```
+
+### FrontmatterManager Interface
+```typescript
+interface FrontmatterManager {
+  parseFrontmatter(content: string): FrontmatterData
+  updateFrontmatter(content: string, updates: FrontmatterData): string
+  validateFrontmatter(data: FrontmatterData): ValidationResult
+}
+```
+
+## Data Models
+
+### FileInfo Model
+```typescript
+interface FileInfo {
+  path: string
+  name: string
+  directory: string
+  extension: string
+  size: number
+  created: Date
+  modified: Date
+  content?: string
+}
+```
+
+### ContentAnalysis Model
+```typescript
+interface ContentAnalysis {
+  language: 'en' | 'zh' | 'mixed' | 'unknown'
+  contentType: ContentType
+  topics: string[]
+  mentions: {
+    tools: string[]
+    technologies: string[]
+    people: string[]
+    organizations: string[]
+  }
+  sentiment?: 'positive' | 'neutral' | 'negative'
+  complexity: 'basic' | 'intermediate' | 'advanced'
+}
+```
+
+### TagStructure Model
+```typescript
+interface TagStructure {
+  primary: string[]        // Main category tags
+  hierarchical: string[]   // Topic/subtopic/detail tags
+  content: string[]        // Content-derived tags
+  meta: string[]          // Metadata tags (language, type, etc.)
+  custom: string[]        // Manually added tags to preserve
+}
+```
+
+### FrontmatterData Model
+```typescript
+interface FrontmatterData {
+  title?: string
+  tags: string[]
+  created?: string
+  updated?: string
+  type?: string
+  lang?: string
+  source?: string
+  aliases?: string[]
+  description?: string
+}
+```
+
+## Tag Hierarchy Design
+
+### Directory-Based Tags
+- `project/` - Files in 100-project/
+  - `project/ai` - AI-related projects
+  - `project/infrastructure` - Infrastructure projects
+  - `project/work` - Work-related projects
+- `area/` - Files in 200-area/
+  - `area/productivity` - Productivity area
+  - `area/health` - Health and fitness
+  - `area/finance` - Financial management
+- `resource/` - Files in 300-resources/
+  - `resource/development` - Development resources
+  - `resource/cooking` - Cooking resources
+- `archive/` - Files in 400-archive/
+- `clipping/` - Web clippings and saved articles
+
+### Content-Based Tags
+- `tech/` - Technology-related content
+  - `tech/ai/llm` - Large Language Models
+  - `tech/infrastructure/docker` - Docker and containers
+  - `tech/development/python` - Python development
+- `personal/` - Personal content
+  - `personal/productivity/gtd` - Getting Things Done
+  - `personal/health/cycling` - Cycling and fitness
+- `work/` - Work-related content
+  - `work/government` - Government projects
+  - `work/enterprise` - Enterprise solutions
+
+### Meta Tags
+- `lang/en`, `lang/zh`, `lang/mixed` - Language tags
+- `type/hub`, `type/note`, `type/clipping`, `type/daily-note` - Content type
+- `status/draft`, `status/complete`, `status/archived` - Status tags
+- `sensitive/credentials`, `sensitive/personal` - Security tags
+
+## Correctness Properties
+
+*A property is a characteristic or behavior that should hold true across all valid executions of a system—essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
+
+Now I'll analyze the acceptance criteria to determine which ones can be tested as properties:
+
+### Converting EARS to Properties
+
+Based on the prework analysis, I'll create consolidated properties that eliminate redundancy while maintaining comprehensive coverage:
+
+Property 1: Frontmatter standardization
+*For any* processed file, the frontmatter should contain all required fields (title, tags, created, updated) in the correct YAML format, while preserving any existing data
+**Validates: Requirements 1.1, 1.2, 1.3, 1.5**
+
+Property 2: Directory-based tag mapping
+*For any* file in the vault, the tags should correctly reflect its directory location using the appropriate primary tag (project/area/resource/archive/clipping) and hierarchical subtags based on subdirectory structure
+**Validates: Requirements 2.1, 2.2, 2.3, 2.4, 2.5, 2.6**
+
+Property 3: Content type classification
+*For any* file, the content analysis should correctly identify and tag the content type (hub, clipping, daily-note, meeting, documentation, tutorial, reference, draft) based on content patterns and filename analysis
+**Validates: Requirements 3.1, 3.2, 3.3, 3.4, 3.5**
+
+Property 4: Hierarchical topic tagging
+*For any* file with identifiable topic content, the system should generate appropriate hierarchical tags using forward slash notation that correctly categorize the content into topic hierarchies (tech/, personal/, work/, home-automation/)
+**Validates: Requirements 4.1, 4.2, 4.3, 4.4, 4.5, 4.6**
+
+Property 5: Language detection and tagging
+*For any* file, the language detection should correctly identify the primary language (en/zh/mixed/unknown) based on character analysis and content patterns, and add the appropriate lang/ tag
+**Validates: Requirements 5.1, 5.2, 5.3, 5.4, 5.5**
+
+Property 6: Content analysis tag generation
+*For any* file, the content analysis should extract and tag relevant topics, technical terms, tools, frameworks, technologies, and methodologies mentioned in the content, while preserving any manually added tags
+**Validates: Requirements 6.1, 6.2, 6.3, 6.4, 6.5**
+
+Property 7: Tag consistency and validation
+*For any* generated tag set, all tags should follow kebab-case naming conventions, duplicate tags should be consolidated using canonical forms, and hierarchical relationships should be logically valid
+**Validates: Requirements 7.1, 7.2, 7.3, 7.4, 7.5**
+
+Property 8: Data preservation during processing
+*For any* file being processed, existing tags and metadata should be preserved, backups should be created before modifications, and batch processing should maintain data integrity with proper error handling
+**Validates: Requirements 8.1, 8.2, 8.3, 8.5**
+
+Property 9: Sensitive content detection and protection
+*For any* file containing sensitive information (credentials, personal data, financial/legal information), the system should add appropriate sensitive/ tags while ensuring the sensitive data itself is not exposed or logged during processing
+**Validates: Requirements 9.1, 9.2, 9.3, 9.5**
+
+Property 10: Obsidian compatibility
+*For any* generated tag structure, the tags should be compatible with Obsidian's tag pane, search functionality, Dataview queries, graph view, and common plugins, with hub files containing appropriate Dataview queries for tag-based content listing
+**Validates: Requirements 10.1, 10.2, 10.3, 10.4, 10.5**
+
+## Error Handling
+
+### File Processing Errors
+- **Invalid file formats**: Skip non-text files with logging
+- **Corrupted frontmatter**: Attempt repair or create new frontmatter
+- **Permission errors**: Log and continue with next file
+- **Large files**: Implement streaming analysis for files over 1MB
+
+### Content Analysis Errors
+- **Language detection failures**: Default to 'lang/unknown'
+- **Topic extraction failures**: Use directory-based fallback tags
+- **Encoding issues**: Attempt multiple encoding detection methods
+
+### Tag Generation Errors
+- **Invalid characters**: Sanitize and convert to kebab-case
+- **Circular hierarchies**: Detect and break circular references
+- **Tag conflicts**: Use canonical form resolution
+
+### Batch Processing Errors
+- **Interrupted processing**: Resume from last successful batch
+- **Memory constraints**: Process in smaller batches
+- **Concurrent access**: Implement file locking mechanism
+
+## Testing Strategy
+
+### Dual Testing Approach
+The system will use both unit tests and property-based tests for comprehensive coverage:
+
+**Unit Tests** will focus on:
+- Specific examples of frontmatter parsing and generation
+- Edge cases like empty files, malformed YAML, special characters
+- Integration points between components
+- Error conditions and recovery mechanisms
+
+**Property-Based Tests** will focus on:
+- Universal properties that hold across all file types and content
+- Comprehensive input coverage through randomized file generation
+- Tag consistency and validation across large datasets
+- Data preservation guarantees during batch processing
+
+### Property Test Configuration
+- Minimum 100 iterations per property test
+- Each property test references its design document property
+- Tag format: **Feature: comprehensive-tagging-system, Property {number}: {property_text}**
+
+### Test Data Generation
+- **File generators**: Create files with various directory structures, content types, and languages
+- **Content generators**: Generate text with different topics, technical terms, and sensitive information patterns
+- **Frontmatter generators**: Create valid and invalid YAML frontmatter variations
+- **Directory structure generators**: Create nested directory hierarchies matching vault patterns
+
+The testing strategy ensures that the tagging system maintains correctness across the diverse content and organizational patterns found in the actual vault structure.
@@ -0,0 +1,138 @@
+# Requirements Document
+
+## Introduction
+
+This specification defines a comprehensive tagging system for an Obsidian vault with mixed organizational patterns. The system will establish consistent tagging conventions, implement hierarchical tag structures based on actual content and directory structure, and ensure all content is properly categorized for improved discoverability and organization.
+
+## Glossary
+
+- **Tag_Hierarchy**: Multi-level tag structure using forward slashes (e.g., tech/ai/llm)
+- **Frontmatter**: YAML metadata block at the beginning of Markdown files
+- **Hub_File**: Index file that serves as entry point for a category or area
+- **Content_Type**: Classification of file purpose (note, project, resource, clipping, etc.)
+- **Language_Tag**: Identifier for content language (en, zh, mixed)
+- **Directory_Structure**: Existing folder organization (100-project, 200-area, 300-resources, 400-archive, etc.)
+
+## Requirements
+
+### Requirement 1: Standardized Frontmatter Structure
+
+**User Story:** As a vault user, I want consistent frontmatter across all files, so that I can rely on predictable metadata structure.
+
+#### Acceptance Criteria
+
+1. WHEN a file is processed, THE Tagging_System SHALL ensure it has standardized frontmatter with required fields
+2. THE Tagging_System SHALL use YAML array format for tags: `tags: [tag1, tag2, tag3]`
+3. THE Tagging_System SHALL include these core fields: title, tags, created, updated
+4. WHERE content has specific metadata needs, THE Tagging_System SHALL include additional fields like type, lang, source
+5. WHEN frontmatter exists but is incomplete, THE Tagging_System SHALL preserve existing data and add missing fields
+
+### Requirement 2: Directory-Based Tag Structure
+
+**User Story:** As a vault user, I want tags that reflect the actual directory organization, so that I can navigate content according to the existing structure.
+
+#### Acceptance Criteria
+
+1. WHEN a file is in 100-project directory, THE Tagging_System SHALL include 'project' tag and specific project subtags
+2. WHEN a file is in 200-area directory, THE Tagging_System SHALL include 'area' tag and area-specific subtags
+3. WHEN a file is in 300-resources directory, THE Tagging_System SHALL include 'resource' tag and topic-specific subtags
+4. WHEN a file is in 400-archive directory, THE Tagging_System SHALL include 'archive' tag
+5. WHEN a file is in Clippings or ReadItLater directories, THE Tagging_System SHALL include 'clipping' tag
+6. THE Tagging_System SHALL create hierarchical tags based on subdirectory structure (e.g., project/ai, area/productivity)
+
+### Requirement 3: Content-Type Classification
+
+**User Story:** As a content creator, I want files tagged by their content type, so that I can filter and find specific kinds of information.
+
+#### Acceptance Criteria
+
+1. THE Tagging_System SHALL identify and tag hub files with 'hub' type
+2. WHEN a file contains clipped web content, THE Tagging_System SHALL add 'clipping' tag
+3. WHEN a file is a daily note with date format, THE Tagging_System SHALL add 'daily-note' tag
+4. THE Tagging_System SHALL detect and tag meeting notes, documentation, tutorials, and reference materials
+5. WHEN file content indicates it's a draft or work-in-progress, THE Tagging_System SHALL add appropriate status tags
+
+### Requirement 4: Topic-Based Hierarchical Tags
+
+**User Story:** As a researcher, I want hierarchical topic tags based on actual content themes, so that I can explore related content at different levels of specificity.
+
+#### Acceptance Criteria
+
+1. THE Tagging_System SHALL implement hierarchical tags using forward slash notation (topic/subtopic/detail)
+2. WHEN content relates to AI/technology, THE Tagging_System SHALL use tech hierarchy (tech/ai, tech/infrastructure, tech/development)
+3. WHEN content relates to personal topics, THE Tagging_System SHALL use personal hierarchy (personal/health, personal/finance, personal/productivity)
+4. WHEN content relates to work projects, THE Tagging_System SHALL use work hierarchy (work/airport, work/ali, work/government)
+5. WHEN content relates to home automation, THE Tagging_System SHALL use home-automation hierarchy
+6. WHEN content spans multiple topics, THE Tagging_System SHALL include all relevant hierarchical tags
+
+### Requirement 5: Language and Localization Tags
+
+**User Story:** As a multilingual user, I want content tagged by language, so that I can filter content by language preference.
+
+#### Acceptance Criteria
+
+1. WHEN content is primarily in English, THE Tagging_System SHALL add 'lang/en' tag
+2. WHEN content is primarily in Chinese, THE Tagging_System SHALL add 'lang/zh' tag
+3. WHEN content mixes languages significantly, THE Tagging_System SHALL add 'lang/mixed' tag
+4. THE Tagging_System SHALL detect language based on character analysis and content patterns
+5. WHEN language cannot be determined, THE Tagging_System SHALL default to 'lang/unknown'
+
+### Requirement 6: Automated Tag Generation
+
+**User Story:** As a vault maintainer, I want automated tag generation based on content analysis, so that I don't have to manually tag every file.
+
+#### Acceptance Criteria
+
+1. THE Tagging_System SHALL analyze file content to suggest relevant topic tags
+2. WHEN file contains technical terms, THE Tagging_System SHALL add appropriate technology tags
+3. THE Tagging_System SHALL extract and tag mentioned tools, frameworks, and technologies
+4. WHEN content discusses specific methodologies or concepts, THE Tagging_System SHALL add conceptual tags
+5. THE Tagging_System SHALL preserve manually added tags while adding automated suggestions
+
+### Requirement 7: Tag Consistency and Validation
+
+**User Story:** As a vault organizer, I want consistent tag naming and validation, so that tags remain useful and don't fragment.
+
+#### Acceptance Criteria
+
+1. THE Tagging_System SHALL enforce kebab-case naming for all tags (lowercase with hyphens)
+2. WHEN duplicate or similar tags exist, THE Tagging_System SHALL consolidate them using canonical forms
+3. THE Tagging_System SHALL validate tag hierarchies to ensure logical parent-child relationships
+4. WHEN tags violate naming conventions, THE Tagging_System SHALL suggest corrections
+5. THE Tagging_System SHALL maintain a tag registry to prevent fragmentation
+
+### Requirement 8: Batch Processing and Migration
+
+**User Story:** As a vault administrator, I want to process all existing files in batches, so that I can systematically apply the tagging system to the entire vault.
+
+#### Acceptance Criteria
+
+1. THE Tagging_System SHALL process files in logical batches by directory structure
+2. WHEN processing existing files, THE Tagging_System SHALL preserve existing tags and metadata
+3. THE Tagging_System SHALL create backup copies before making changes to files
+4. WHEN conflicts arise between existing and generated tags, THE Tagging_System SHALL prompt for resolution
+5. THE Tagging_System SHALL provide progress reporting and error handling for batch operations
+
+### Requirement 9: Security and Sensitive Content Handling
+
+**User Story:** As a security-conscious user, I want sensitive files properly tagged and handled, so that I can manage access and visibility appropriately.
+
+#### Acceptance Criteria
+
+1. WHEN files contain credentials or API keys, THE Tagging_System SHALL add 'sensitive/credentials' tag
+2. WHEN files contain personal information, THE Tagging_System SHALL add 'sensitive/personal' tag
+3. THE Tagging_System SHALL detect and tag files with financial or legal information
+4. WHEN sensitive content is detected, THE Tagging_System SHALL suggest security best practices
+5. THE Tagging_System SHALL respect existing security classifications and not expose sensitive data
+
+### Requirement 10: Integration with Obsidian Features
+
+**User Story:** As an Obsidian user, I want tags that work seamlessly with Obsidian's built-in features, so that I can leverage the full power of the application.
+
+#### Acceptance Criteria
+
+1. THE Tagging_System SHALL generate tags compatible with Obsidian's tag pane and search
+2. THE Tagging_System SHALL support Dataview queries and tag-based filtering
+3. WHEN creating hub files, THE Tagging_System SHALL include appropriate Dataview queries for tag-based content listing
+4. THE Tagging_System SHALL ensure tags work with graph view and tag-based navigation
+5. THE Tagging_System SHALL maintain compatibility with Obsidian plugins that use tags
@@ -0,0 +1,153 @@
+# Implementation Plan: Comprehensive Tagging System
+
+## Overview
+
+This implementation plan creates a Python-based comprehensive tagging system for the Obsidian vault. The system will analyze files, generate appropriate tags based on directory structure and content analysis, and update frontmatter while preserving existing data. The implementation follows a modular architecture with clear separation of concerns.
+
+## Tasks
+
+- [x] 1. Set up project structure and core interfaces
+  - Create Python package structure with proper modules
+  - Define core data models and interfaces using dataclasses and protocols
+  - Set up testing framework with pytest and hypothesis for property-based testing
+  - Create configuration system for tag hierarchies and rules
+  - _Requirements: 1.1, 1.2, 1.3_
+
+- [x] 2. Implement file discovery and content analysis
+  - [x] 2.1 Create FileDiscovery class for vault scanning
+    - Implement recursive directory scanning with file filtering
+    - Add support for excluding sensitive directories and file types
+    - Handle file encoding detection and content reading
+    - _Requirements: 8.1, 9.5_
+
+  - [ ]* 2.2 Write property test for file discovery
+    - **Property 8: Data preservation during processing**
+    - **Validates: Requirements 8.1, 8.2, 8.3, 8.5**
+
+  - [x] 2.3 Implement ContentAnalyzer class
+    - Create language detection using character frequency analysis
+    - Implement content type classification based on patterns and keywords
+    - Add topic extraction using keyword analysis and NLP techniques
+    - Build entity extraction for tools, technologies, and frameworks
+    - _Requirements: 5.4, 3.4, 6.1, 6.2, 6.3_
+
+  - [ ]* 2.4 Write property tests for content analysis
+    - **Property 5: Language detection and tagging**
+    - **Property 3: Content type classification**
+    - **Property 6: Content analysis tag generation**
+    - **Validates: Requirements 5.1-5.5, 3.1-3.5, 6.1-6.5**
+
+- [x] 3. Implement tag generation system
+  - [x] 3.1 Create TagGenerator class
+    - Implement directory-based tag generation from file paths
+    - Build hierarchical tag creation using topic analysis
+    - Add tag consolidation and deduplication logic
+    - Create tag validation and naming convention enforcement
+    - _Requirements: 2.6, 4.1, 7.1, 7.2, 7.3_
+
+  - [ ]* 3.2 Write property tests for tag generation
+    - **Property 2: Directory-based tag mapping**
+    - **Property 4: Hierarchical topic tagging**
+    - **Property 7: Tag consistency and validation**
+    - **Validates: Requirements 2.1-2.6, 4.1-4.6, 7.1-7.5**
+
+  - [x] 3.3 Implement sensitive content detection
+    - Create pattern matching for credentials, API keys, and personal information
+    - Add financial and legal content detection
+    - Implement secure handling to avoid exposing sensitive data
+    - _Requirements: 9.1, 9.2, 9.3, 9.5_
+
+  - [ ]* 3.4 Write property test for sensitive content detection
+    - **Property 9: Sensitive content detection and protection**
+    - **Validates: Requirements 9.1, 9.2, 9.3, 9.5**
+
+- [ ] 4. Checkpoint - Core functionality validation
+  - Ensure all core components work together correctly
+  - Run property tests to validate tag generation logic
+  - Test with sample files from different vault directories
+  - Ask the user if questions arise about tag hierarchies or classification rules
+
+- [ ] 5. Implement frontmatter management
+  - [ ] 5.1 Create FrontmatterManager class
+    - Implement YAML frontmatter parsing with error handling
+    - Build frontmatter updating while preserving existing data
+    - Add validation for frontmatter structure and required fields
+    - Create backup functionality before file modifications
+    - _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 8.2, 8.3_
+
+  - [ ]* 5.2 Write property tests for frontmatter management
+    - **Property 1: Frontmatter standardization**
+    - **Validates: Requirements 1.1, 1.2, 1.3, 1.5**
+
+  - [ ] 5.3 Implement batch processing system
+    - Create BatchProcessor class for handling large-scale operations
+    - Add progress tracking and error handling
+    - Implement resume functionality for interrupted processing
+    - Build memory-efficient processing for large vaults
+    - _Requirements: 8.1, 8.2, 8.5_
+
+  - [ ]* 5.4 Write unit tests for batch processing
+    - Test error recovery and resume functionality
+    - Test memory management with large file sets
+    - Test progress reporting accuracy
+    - _Requirements: 8.1, 8.2, 8.5_
+
+- [ ] 6. Implement Obsidian compatibility features
+  - [ ] 6.1 Create ObsidianCompatibility class
+    - Ensure tag format compatibility with Obsidian tag system
+    - Generate Dataview queries for hub files
+    - Validate tags work with graph view and common plugins
+    - Test tag search and filtering functionality
+    - _Requirements: 10.1, 10.2, 10.3, 10.4, 10.5_
+
+  - [ ]* 6.2 Write property test for Obsidian compatibility
+    - **Property 10: Obsidian compatibility**
+    - **Validates: Requirements 10.1, 10.2, 10.3, 10.4, 10.5**
+
+- [ ] 7. Create command-line interface and configuration
+  - [ ] 7.1 Build CLI using argparse or click
+    - Add commands for full vault processing, single file processing, and validation
+    - Implement configuration file support for custom tag hierarchies
+    - Add dry-run mode for testing without file modifications
+    - Create verbose logging and progress reporting
+    - _Requirements: 8.1, 8.5_
+
+  - [ ]* 7.2 Write integration tests for CLI
+    - Test full vault processing workflow
+    - Test configuration file loading and validation
+    - Test dry-run mode accuracy
+    - _Requirements: 8.1, 8.5_
+
+- [ ] 8. Integration and comprehensive testing
+  - [ ] 8.1 Wire all components together in main application
+    - Create main TaggingSystem class that orchestrates all components
+    - Implement error handling and logging throughout the system
+    - Add configuration validation and setup
+    - _Requirements: All requirements_
+
+  - [ ]* 8.2 Write end-to-end property tests
+    - Test complete workflow from file discovery to frontmatter update
+    - Validate all properties work together in integrated system
+    - Test with realistic vault structure and content
+    - _Requirements: All requirements_
+
+  - [ ] 8.3 Create comprehensive test suite
+    - Add performance tests for large vault processing
+    - Create regression tests for edge cases
+    - Build test data generators for various content types
+    - _Requirements: All requirements_
+
+- [ ] 9. Final checkpoint and documentation
+  - Ensure all property tests pass with 100+ iterations each
+  - Validate system works with actual vault content
+  - Create usage documentation and configuration examples
+  - Ask the user if questions arise about final implementation
+
+## Notes
+
+- Tasks marked with `*` are optional and can be skipped for faster MVP
+- Each task references specific requirements for traceability
+- Property tests validate universal correctness properties with randomized inputs
+- Unit tests validate specific examples and edge cases
+- The system preserves existing data and creates backups before modifications
+- All components are designed to be modular and testable independently