mbrucedogs/KaraokeMerge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Signed-off-by: mbrucedogs <mbrucedogs@gmail.com>

Browse Source
This commit is contained in:
mbrucedogs 2025-07-26 18:57:11 -05:00
parent 54bce66584
commit 03a17c2422
4 changed files with 444 additions and 319 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

297
PRD.md
View File

@ -1,11 +1,11 @@
# Karaoke Song Library Cleanup Tool — PRD (v1 CLI)
# Karaoke Song Library Cleanup Tool — PRD (v2.0)
## 1. Project Summary
- **Goal:** Analyze, deduplicate, and suggest cleanup of a large karaoke song collection, outputting a JSON “skip list” (for future imports) and supporting flexible reporting and manual review.
- **Goal:** Analyze, deduplicate, and suggest cleanup of a large karaoke song collection, outputting a JSON "skip list" (for future imports) and supporting flexible reporting and manual review.
- **Primary User:** Admin (self, collection owner)
- **Initial Interface:** Command Line (CLI) with print/logging and JSON output
- **Future Expansion:** Optional web UI for filtering, review, and playback
- **Interfaces:** Command Line (CLI) with print/logging and JSON output, plus interactive Web UI for manual review
- **Current Status:** Fully functional CLI tool with comprehensive web UI for interactive review and priority management
---
@ -91,12 +91,7 @@ This documentation requirement is mandatory and ensures the project remains main
- **Docstring Format:** Use Google-style docstrings with parameter descriptions
- **Complex Logic:** Add inline comments explaining business logic and algorithms
#### **Testing Standards**
- **Unit Tests:** Write unit tests for all business logic functions
- **Test Coverage:** Aim for 80%+ code coverage
- **Test Organization:** Mirror the source code structure in test files
- **Test Data:** Use fixtures and factories for test data, never hardcode test values
- **Integration Tests:** Test complete workflows and API endpoints
#### **Configuration Management**
- **Environment Variables:** Use environment variables for sensitive data (API keys, paths)
@ -133,7 +128,6 @@ This documentation requirement is mandatory and ensures the project remains main
- [ ] Functions are appropriately sized and focused
- [ ] Error handling is comprehensive
- [ ] Type hints and docstrings are present
- [ ] Tests are included for new functionality
- [ ] Configuration is properly validated
- [ ] No hardcoded values or secrets
- [ ] Performance considerations addressed
@ -155,7 +149,7 @@ These standards ensure the codebase remains clean, maintainable, and accessible
- **Primary keys:** `artist` + `title`
- Fuzzy matching configurable (enabled/disabled with threshold)
- Multi-artist handling: parse delimiters (commas, “feat.”, etc.)
- Multi-artist handling: parse delimiters (commas, "feat.", etc.)
- **File type detection:** Use file extension from `path` (`.mp3`, `.cdg`, `.mp4`)
### 3.3 Channel Priority (for MP4s)
@ -176,7 +170,7 @@ These standards ensure the codebase remains clean, maintainable, and accessible
- **Format:** JSON (`/data/skipSongs.json`)
- List of file paths to skip in future imports
- Optionally: “reason” field (e.g., `{"path": "...", "reason": "duplicate"}`)
- Optionally: "reason" field (e.g., `{"path": "...", "reason": "duplicate"}`)
### 4.2 CLI Reporting
@ -210,6 +204,7 @@ These standards ensure the codebase remains clean, maintainable, and accessible
## 6. Tech Stack & Organization
- **CLI Language:** Python
- **Web UI:** Flask + HTML/CSS/JavaScript (Bootstrap, Font Awesome, Sortable.js)
- **Config:** JSON (channel priorities, settings)
- **Current Folder Structure:**
```
@ -217,6 +212,9 @@ KaraokeMerge/
├── data/
│ ├── allSongs.json # Input: Your song library data
│ ├── skipSongs.json # Output: Generated skip list
│ ├── preferences/ # User priority preferences
│ │ ├── priority_preferences.json
│ │ └── priority_preferences_backup_*.json
│ └── reports/ # Detailed analysis reports
│ ├── analysis_data.json
│ ├── actionable_insights_report.txt
@ -231,13 +229,13 @@ KaraokeMerge/
│ ├── main.py # Main CLI application
│ ├── matching.py # Song matching logic
│ ├── report.py # Report generation
│ ├── preferences.py # Priority preferences management
│ └── utils.py # Utility functions
├── web/ # Web UI for manual review
│ ├── app.py # Flask web application
│ └── templates/
│ └── index.html # Web interface template
├── start_web_ui.py # Web UI startup script
├── test_tool.py # Validation and testing script
├── requirements.txt # Python dependencies
├── .gitignore # Git ignore rules
├── PRD.md # Product Requirements Document
@ -249,89 +247,144 @@ KaraokeMerge/
## 7. Web UI Implementation
### 7.1 Current Web UI Features
#### **Core Functionality**
- **Interactive Table View**: Sortable, filterable grid of duplicate songs
- **Bulk Selection**: Select multiple items for batch operations
- **Search & Filter**: Real-time search across artists, titles, and paths
- **Responsive Design**: Mobile-friendly interface
- **Easy Startup**: Automated dependency checking and browser launch
#### **Media Preview & Playback**
- **Video Playback**: Direct MP4 video playback in modal popup for previewing karaoke videos
- **Drag-and-Drop Priority Management**: Interactive reordering of file priorities with persistent preferences
- **File Path Normalization**: Automatic correction of malformed file paths (handles `://` corruption)
- **Video Modal**: Full-screen video player with controls for karaoke video preview
**IMPORTANT: Web Player Path Handling Fix**
- **Issue**: File paths with backslashes were being corrupted when passed from HTML to JavaScript due to improper string literal escaping in `onclick` attributes
- **Root Cause**: Backslashes in `version.path` were not properly escaped when inserted into JavaScript string literals in HTML `onclick` attributes, causing them to be interpreted as escape characters
- **Solution**: Added `.replace(/\\/g, '\\\\')` to escape backslashes before inserting into `onclick` attributes in `web/templates/index.html`
- **Impact**: Ensures paths displayed in UI match paths received by JavaScript functions, preventing 404 errors in video playback
- **Files Modified**: `web/templates/index.html` (line ~1010), `web/app.py` (normalize_path function simplified)
#### **Priority Management System**
- **Drag-and-Drop Priority Management**: Interactive reordering of file priorities using Sortable.js
- **Visual Priority Indicators**: Real-time visual feedback showing KEPT/SKIPPED status
- **Dynamic Visual Updates**: Automatic color coding and badge updates based on priority order
- **Priority Persistence**: Save/load user priority preferences to/from JSON files
- **Priority Preferences API**: RESTful endpoints for managing priority preferences
#### **User Interface Enhancements**
- **Visual Status Indicators**: Color-coded cards (green for kept, red for skipped)
- **File Type Badges**: Visual indicators for MP3, MP4, and CDG files
- **Channel Badges**: Display channel information for MP4 files
- **Progress Indicators**: Loading states and status messages
- **Error Handling**: Comprehensive error handling with user-friendly messages
- **Debug Tools**: Console logging and debugging functions for troubleshooting
#### **Data Management**
- **Priority Preferences Storage**: Automatic backup creation with timestamps
- **Reset Functionality**: Ability to reset all priority preferences to defaults
- **Change Tracking**: Real-time tracking of unsaved priority changes
- **Save Button Management**: Dynamic enable/disable based on unsaved changes
### 7.2 Web UI Architecture
- **Flask Backend**: Lightweight web server (`web/app.py`)
- **HTML Template**: Modern, responsive interface (`web/templates/index.html`)
- **Startup Script**: Dependency management and server startup (`start_web_ui.py`)
### 7.3 Future Web UI Enhancements
#### **Frontend Technologies**
- **HTML5**: Semantic markup with Bootstrap 5 for responsive design
- **CSS3**: Custom styling with Bootstrap components and Font Awesome icons
- **JavaScript (ES6+)**: Modern JavaScript with async/await for API calls
- **Sortable.js**: Drag-and-drop library for priority reordering
- **Bootstrap 5**: UI framework for responsive design and components
- **Font Awesome 6**: Icon library for visual elements
#### **Backend Technologies**
- **Flask**: Lightweight Python web framework
- **JSON APIs**: RESTful endpoints for data management
- **File System Integration**: Direct file operations for preferences and data
- **Error Handling**: Comprehensive error handling and logging
#### **Key Components**
- **`web/app.py`**: Flask application with API endpoints
- **`web/templates/index.html`**: Main web interface template
- **`start_web_ui.py`**: Startup script with dependency management
### 7.3 API Endpoints
#### **Data Endpoints**
- **`/api/duplicates`**: Get duplicate song data with pagination
- **`/api/stats`**: Get statistical analysis of the song collection
- **`/api/artists`**: Get list of artists for filtering
- **`/api/mp3-songs`**: Get MP3 songs that remain after cleanup
- **`/api/config`**: Get current configuration settings
#### **Priority Management Endpoints**
- **`/api/save-priority-preferences`**: Save user priority preferences to JSON
- **`/api/load-priority-preferences`**: Load saved priority preferences
- **`/api/reset-priority-preferences`**: Reset all priority preferences to defaults
#### **File Serving Endpoints**
- **`/api/video/<path>`**: Serve MP4 video files for preview
- **`/api/download/mp3-songs`**: Download MP3 song list as JSON
### 7.4 Future Web UI Enhancements
- Audio preview for MP3 files
- Real-time configuration editing
- Advanced filtering and sorting options
- Export capabilities for manual selections
- Batch video preview functionality
- Video thumbnail generation
- Real-time collaboration features
- Advanced analytics dashboard
---
## 8. Open Questions (for future refinement)
## 8. Priority Management System
### 8.1 Overview
The priority management system allows users to manually override the default priority algorithm through an interactive web interface. User decisions are persisted and used by the CLI tool for future processing runs.
### 8.2 Features
- **Interactive Drag-and-Drop**: Reorder song versions using intuitive drag-and-drop interface
- **Visual Feedback**: Real-time visual indicators showing kept/skipped status
- **Persistent Storage**: User preferences saved to `data/preferences/priority_preferences.json`
- **Automatic Backups**: Timestamped backup files created on each save
- **Reset Capability**: Ability to reset all preferences to default algorithm
- **Change Tracking**: Real-time tracking of unsaved changes with visual indicators
### 8.3 Data Flow
1. **Web UI**: User makes priority changes via drag-and-drop
2. **Frontend**: Changes stored in `priorityChanges` JavaScript object
3. **Save Action**: Changes sent to backend via `/api/save-priority-preferences`
4. **Backend**: Preferences saved to JSON file with automatic backup
5. **CLI Integration**: CLI tool reads preferences file and applies user decisions
6. **Persistence**: Changes persist across web UI sessions and CLI runs
### 8.4 File Structure
```
data/preferences/
├── priority_preferences.json # Current user preferences
└── priority_preferences_backup_*.json # Timestamped backups
```
---
## 9. Open Questions (for future refinement)
- Fuzzy matching library/thresholds?
- Best parsing rules for multi-artist/feat. strings?
- Any alternate export formats needed?
- Temporary/partial skip support for "under review" songs?
- Integration with karaoke software APIs?
- Audio fingerprinting for better duplicate detection?
---
## 9. Implementation Status
## 10. Implementation Status
### ✅ Completed Features
- [x] Write initial CLI tool to parse allSongs.json, deduplicate, and output skipSongs.json
- [x] Print CLI summary reports (with verbosity control)
- [x] Implement config file support for channel priority
- [x] Organize folder/file structure for easy expansion
### 🎯 Current Implementation
The tool has been successfully implemented with the following components:
**Core Modules:**
- `cli/main.py` - Main CLI application with argument parsing
- `cli/matching.py` - Song matching and deduplication logic
- `cli/report.py` - Report generation and output formatting
- `cli/utils.py` - Utility functions for file operations and data processing
**Configuration:**
- `config/config.json` - Configurable settings for channel priorities, matching rules, and output options
**Features Implemented:**
- Multi-format support (MP3, CDG, MP4)
- **CDG/MP3 Pairing Logic**: Files with same base filename treated as single karaoke song units
- Channel priority system for MP4 files (based on folder names in path)
- Fuzzy matching support with configurable threshold
- Multi-artist parsing with various delimiters
- **Enhanced Analysis & Reporting**: Comprehensive statistical analysis with actionable insights
- Channel priority analysis and manual review identification
- Non-destructive operation (skip lists only)
- Verbose and dry-run modes
- Detailed duplicate analysis
- Skip list generation with metadata
- **Pattern Analysis**: Skip list pattern analysis and channel optimization suggestions
**File Type Priority System:**
1. **MP4 files** (with channel priority sorting)
2. **CDG/MP3 pairs** (treated as single units)
3. **Standalone MP3** files
4. **Standalone CDG** files
**Performance Results:**
- Successfully processed 37,015 songs
- Identified 12,424 duplicates (33.6% duplicate rate)
- Generated comprehensive skip list with metadata (10,998 unique files after deduplication)
- Optimized for large datasets with progress indicators
- **Enhanced Analysis**: Generated 7 detailed reports with actionable insights
- **Bug Fix**: Resolved duplicate entries in skip list (removed 1,426 duplicate entries)
### 📋 Next Steps Checklist
#### ✅ **Completed**
#### **Core CLI Functionality**
- [x] Write initial CLI tool to parse allSongs.json, deduplicate, and output skipSongs.json
- [x] Print CLI summary reports (with verbosity control)
- [x] Implement config file support for channel priority
@ -341,18 +394,118 @@ The tool has been successfully implemented with the following components:
- [x] Optimize performance for large datasets (37,000+ songs)
- [x] Add progress indicators and error handling
- [x] Generate detailed analysis reports (`--save-reports` functionality)
- [x] Create web UI for manual review of ambiguous cases
- [x] Add test tool for validation and debugging
- [x] Create startup script for web UI with dependency checking
- [x] Add comprehensive .gitignore file
- [x] Update documentation with required data file information
- [x] Implement drag-and-drop priority management with persistent preferences
#### **Web UI Implementation**
- [x] Create web UI for manual review of ambiguous cases
- [x] Implement interactive table view with sorting and filtering
- [x] Add MP4 video playback functionality in web UI modal
- [x] Implement drag-and-drop priority management with persistent preferences
- [x] Add visual status indicators and dynamic updates
- [x] Create priority preferences API endpoints
- [x] Implement automatic backup system for preferences
- [x] Add comprehensive error handling and debugging tools
- [x] Create responsive design with Bootstrap 5
- [x] Add file path normalization for corrupted paths
- [x] Implement change tracking and save button management
#### **Advanced Features**
- [x] Multi-format support (MP3, CDG, MP4)
- [x] Channel priority system for MP4 files
- [x] Fuzzy matching support with configurable threshold
- [x] Multi-artist parsing with various delimiters
- [x] Enhanced analysis & reporting with actionable insights
- [x] Pattern analysis and channel optimization suggestions
- [x] Non-destructive operation (skip lists only)
- [x] Verbose and dry-run modes
### 🎯 Current Implementation
The tool has been successfully implemented with the following components:
**Core Modules:**
- `cli/main.py` - Main CLI application with argument parsing
- `cli/matching.py` - Song matching and deduplication logic
- `cli/report.py` - Report generation and output formatting
- `cli/preferences.py` - Priority preferences management
- `cli/utils.py` - Utility functions for file operations and data processing
**Web UI Components:**
- `web/app.py` - Flask web application with API endpoints
- `web/templates/index.html` - Modern, responsive web interface
- `start_web_ui.py` - Startup script with dependency management
**Configuration:**
- `config/config.json` - Configurable settings for channel priorities, matching rules, and output options
**Features Implemented:**
- **File Type Priority System:**
1. MP4 files (with channel priority sorting)
2. CDG/MP3 pairs (treated as single units)
3. Standalone MP3 files
4. Standalone CDG files
- **Priority Management System:**
- Interactive drag-and-drop reordering
- Visual feedback with color-coded indicators
- Persistent storage with automatic backups
- Reset functionality for default preferences
- Real-time change tracking
- **Web UI Features:**
- Responsive design with Bootstrap 5
- Video playback for MP4 files
- Real-time filtering and search
- Pagination for large datasets
- Comprehensive error handling
- Debug tools and console logging
**Performance Results:**
- Successfully processed 37,015 songs
- Identified 12,424 duplicates (33.6% duplicate rate)
- Generated comprehensive skip list with metadata (10,998 unique files after deduplication)
- Optimized for large datasets with progress indicators
- Enhanced analysis with 7 detailed reports and actionable insights
- Bug fix: Resolved duplicate entries in skip list (removed 1,426 duplicate entries)
### 📋 Next Steps Checklist
#### 🎯 **Next Priority Items**
- [ ] Analyze MP4 files without channel priorities to suggest new folder names
- [ ] Add support for additional file formats if needed
- [ ] Implement batch processing capabilities
- [ ] Create integration scripts for karaoke software
- [ ] Add unit tests for core functionality
- [ ] Implement audio fingerprinting for better duplicate detection
- [ ] Add audio preview for MP3 files in web UI
- [ ] Create advanced analytics dashboard
- [ ] Implement real-time collaboration features
- [ ] Add video thumbnail generation
#### 🔄 **Maintenance & Improvements**
- [ ] Regular dependency updates and security patches
- [ ] Performance optimization for larger datasets
- [ ] Enhanced error handling and user feedback
- [ ] Additional configuration options
- [ ] Extended documentation and tutorials
- [ ] Community feedback integration
---
## 11. Version History
### v2.0 (Current)
- **Major Web UI Enhancement**: Complete drag-and-drop priority management system
- **Priority Persistence**: User preferences saved and loaded automatically
- **Visual Improvements**: Dynamic visual indicators and real-time feedback
- **Enhanced Error Handling**: Comprehensive error handling with debugging tools
- **File Path Normalization**: Automatic correction of corrupted file paths
- **Backup System**: Automatic timestamped backups for user preferences
### v1.0 (Initial Release)
- **Core CLI Functionality**: Basic deduplication and skip list generation
- **Web UI Foundation**: Initial web interface for manual review
- **Configuration System**: JSON-based configuration management
- **Reporting System**: Comprehensive analysis and reporting capabilities

197
test_tool.py
View File

@ -1,70 +1,165 @@
#!/usr/bin/env python3
"""
Simple test script to validate the Karaoke Song Library Cleanup Tool.
Test and validation tool for Karaoke Song Library Cleanup Tool
Provides validation and debugging capabilities for the tool.
"""
import sys
import json
import os
import sys
from pathlib import Path
# Add the cli directory to the path
sys.path.append(os.path.join(os.path.dirname(__file__), 'cli'))
def load_json_file(file_path: str) -> dict:
"""Load JSON file safely."""
try:
with open(file_path, 'r', encoding='utf-8') as f:
return json.load(f)
except Exception as e:
print(f"Error loading {file_path}: {e}")
return {}
def test_basic_functionality():
"""Test basic functionality of the tool."""
print("Testing Karaoke Song Library Cleanup Tool...")
print("=" * 60)
def validate_data_files():
"""Validate that required data files exist and are properly formatted."""
print("=== Data File Validation ===")
# Check for required files
required_files = [
'data/allSongs.json',
'config/config.json'
]
for file_path in required_files:
if os.path.exists(file_path):
print(f"✅ {file_path} - Found")
try:
data = load_json_file(file_path)
if isinstance(data, (dict, list)):
print(f" 📄 Valid JSON format")
else:
print(f" ❌ Invalid JSON format")
except Exception as e:
print(f" ❌ Error reading JSON: {e}")
else:
print(f"❌ {file_path} - Missing")
# Check for optional files
optional_files = [
'data/skipSongs.json',
'data/preferences/priority_preferences.json'
]
print("\n=== Optional Files ===")
for file_path in optional_files:
if os.path.exists(file_path):
print(f"✅ {file_path} - Found")
else:
print(f"⚠️ {file_path} - Not found (will be created when needed)")
def analyze_song_data():
"""Analyze the song data structure and provide insights."""
print("\n=== Song Data Analysis ===")
all_songs_path = 'data/allSongs.json'
if not os.path.exists(all_songs_path):
print(f"❌ {all_songs_path} not found - cannot analyze song data")
return
try:
# Test imports
from utils import load_json_file, save_json_file
from matching import SongMatcher
from report import ReportGenerator
print("✅ All modules imported successfully")
songs = load_json_file(all_songs_path)
if not songs:
print("❌ No songs found in data file")
return
# Test config loading
config = load_json_file('config/config.json')
print("✅ Configuration loaded successfully")
print(f"📊 Total songs: {len(songs)}")
# Test song data loading (first few entries)
songs = load_json_file('data/allSongs.json')
print(f"✅ Song data loaded successfully ({len(songs):,} songs)")
# Analyze file types
file_types = {}
artists = set()
titles = set()
# Test with a small sample
sample_songs = songs[:1000] # Test with first 1000 songs
print(f"Testing with sample of {len(sample_songs)} songs...")
for song in songs:
# File type analysis
path = song.get('path', '').lower()
if path.endswith('.mp4'):
file_types['MP4'] = file_types.get('MP4', 0) + 1
elif path.endswith('.mp3'):
file_types['MP3'] = file_types.get('MP3', 0) + 1
elif path.endswith('.cdg'):
file_types['CDG'] = file_types.get('CDG', 0) + 1
else:
file_types['Other'] = file_types.get('Other', 0) + 1
# Initialize components
matcher = SongMatcher(config, data_dir)
reporter = ReportGenerator(config)
# Artist and title analysis
artist = song.get('artist', '').strip()
title = song.get('title', '').strip()
if artist:
artists.add(artist)
if title:
titles.add(title)
# Process sample
best_songs, skip_songs, stats = matcher.process_songs(sample_songs)
print(f"📁 File types: {file_types}")
print(f"🎤 Unique artists: {len(artists)}")
print(f"🎵 Unique titles: {len(titles)}")
print(f"✅ Processing completed successfully")
print(f" - Total songs: {stats['total_songs']}")
print(f" - Unique songs: {stats['unique_songs']}")
print(f" - Duplicates found: {stats['duplicates_found']}")
# Test report generation
summary_report = reporter.generate_summary_report(stats)
print("✅ Report generation working")
print("\n" + "=" * 60)
print("🎉 All tests passed! The tool is ready to use.")
print("\nTo run the full analysis:")
print(" python cli/main.py")
print("\nTo run with verbose output:")
print(" python cli/main.py --verbose")
print("\nTo run a dry run (no skip list generated):")
print(" python cli/main.py --dry-run")
# Sample data structure
if songs:
print(f"\n📋 Sample song structure:")
sample = songs[0]
for key, value in sample.items():
if isinstance(value, str) and len(value) > 50:
value = value[:50] + "..."
print(f" {key}: {value}")
except Exception as e:
print(f"❌ Test failed: {e}")
import traceback
traceback.print_exc()
return False
print(f"❌ Error analyzing song data: {e}")
return True
def validate_config():
"""Validate the configuration file."""
print("\n=== Configuration Validation ===")
config_path = 'config/config.json'
if not os.path.exists(config_path):
print(f"❌ {config_path} not found")
return
try:
config = load_json_file(config_path)
# Check for required config sections
required_sections = ['channel_priorities', 'matching_rules']
for section in required_sections:
if section in config:
print(f"✅ {section} - Found")
else:
print(f"❌ {section} - Missing")
# Display current configuration
print(f"\n📋 Current configuration:")
for key, value in config.items():
if isinstance(value, list):
print(f" {key}: {len(value)} items")
else:
print(f" {key}: {value}")
except Exception as e:
print(f"❌ Error validating config: {e}")
def main():
"""Main test function."""
print("🎵 Karaoke Song Library Cleanup Tool - Test & Validation")
print("=" * 60)
# Validate data files
validate_data_files()
# Analyze song data
analyze_song_data()
# Validate configuration
validate_config()
print("\n" + "=" * 60)
print("✅ Test completed")
if __name__ == "__main__":
success = test_basic_functionality()
sys.exit(0 if success else 1)
main()

117
web/app.py
View File

@ -160,97 +160,29 @@ def extract_channel(path: str) -> str:
def normalize_path(file_path: str) -> str:
"""Normalize malformed file paths that have been corrupted with ://."""
# Debug logging to track path transformation - show original path first
print(f"DEBUG: normalize_path ORIGINAL PATH: {file_path}")
# Fix malformed paths that have been corrupted with ://
# Example: z://MP4KaraFun KaraokeKaraoke I m Not In Love - 10CC.mp4
# Should be: z:\MP4\KaraFun Karaoke\Karaoke I'm Not In Love - 10CC.mp4
# Since we fixed the HTML string literal escaping, paths should now come in correctly
# Just handle the basic :// to :\ conversion
if '://' in file_path:
print(f"DEBUG: Detected malformed path, attempting to fix: {file_path}")
print(f"DEBUG: Detected malformed path with ://, attempting to fix: {file_path}")
# Extract drive letter and rest of path
# Clean up special characters in the path first - but be more conservative
import re
match = re.match(r'^([a-zA-Z])://(.+)$', file_path)
if match:
drive_letter = match.group(1)
rest_of_path = match.group(2)
# Only remove actual control characters, not legitimate path characters
file_path = re.sub(r'[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]', '', file_path) # Remove only control characters
file_path = re.sub(r'\s+', ' ', file_path) # Normalize multiple spaces to single space
file_path = file_path.strip() # Remove leading/trailing spaces
# Try to reconstruct the proper path structure
# Look for common patterns in the corrupted path
# Pattern 1: Handle the specific case where "Karaoke" appears twice
# Example: "MP4KaraFun KaraokeKaraoke I m Not In Love - 10CC.mp4"
# Should become: "MP4\KaraFun Karaoke\Karaoke I'm Not In Love - 10CC.mp4"
karaoke_double_match = re.match(r'^MP4([A-Za-z\s]+)KaraokeKaraoke(.+)$', rest_of_path)
if karaoke_double_match:
channel_name = karaoke_double_match.group(1) + "Karaoke"
file_name = "Karaoke" + karaoke_double_match.group(2)
fixed_path = f"{drive_letter}:\\MP4\\{channel_name}\\{file_name}"
print(f"DEBUG: Fixed path (pattern 1 - double karaoke): {fixed_path}")
# Simple fix: replace :// with :\
fixed_path = file_path.replace('://', ':\\')
print(f"DEBUG: Fixed path (simple :// to :\ conversion): {fixed_path}")
return fixed_path
# Pattern 2: MP4 followed by channel name (e.g., MP4KaraFun Karaoke)
mp4_match = re.match(r'^MP4([A-Za-z\s]+Karaoke)(.+)$', rest_of_path)
if mp4_match:
channel_name = mp4_match.group(1)
file_name = mp4_match.group(2)
fixed_path = f"{drive_letter}:\\MP4\\{channel_name}\\{file_name}"
print(f"DEBUG: Fixed path (pattern 2): {fixed_path}")
return fixed_path
# Pattern 3: Direct channel name followed by filename
channel_match = re.match(r'^([A-Za-z\s]+Karaoke)(.+)$', rest_of_path)
if channel_match:
channel_name = channel_match.group(1)
file_name = channel_match.group(2)
fixed_path = f"{drive_letter}:\\MP4\\{channel_name}\\{file_name}"
print(f"DEBUG: Fixed path (pattern 3): {fixed_path}")
return fixed_path
# Pattern 4: Look for any known channel names
known_channels = ['Sing King Karaoke', 'KaraFun Karaoke', 'Stingray Karaoke']
for channel in known_channels:
if channel.lower().replace(' ', '') in rest_of_path.lower().replace(' ', ''):
# Extract the part before and after the channel name
channel_lower = channel.lower().replace(' ', '')
rest_lower = rest_of_path.lower().replace(' ', '')
channel_index = rest_lower.find(channel_lower)
if channel_index >= 0:
# Reconstruct the path
before_channel = rest_of_path[:channel_index]
after_channel = rest_of_path[channel_index + len(channel):]
# If there's content before the channel, it might be a folder like "MP4"
if before_channel and before_channel.lower() in ['mp4']:
fixed_path = f"{drive_letter}:\\{before_channel}\\{channel}{after_channel}"
else:
fixed_path = f"{drive_letter}:\\MP4\\{channel}{after_channel}"
print(f"DEBUG: Fixed path (pattern 4): {fixed_path}")
return fixed_path
# Pattern 5: Try to split by common separators and reconstruct
# Look for patterns like "MP4KaraFunKaraoke" -> "MP4\KaraFun Karaoke"
if 'karaoke' in rest_of_path.lower():
# Try to find where "karaoke" appears and reconstruct
karaoke_index = rest_of_path.lower().find('karaoke')
if karaoke_index > 0:
before_karaoke = rest_of_path[:karaoke_index]
after_karaoke = rest_of_path[karaoke_index + 7:] # length of "karaoke"
# If before_karaoke starts with "MP4", extract the channel name
if before_karaoke.lower().startswith('mp4'):
channel_part = before_karaoke[3:] # Remove "MP4"
if channel_part:
fixed_path = f"{drive_letter}:\\MP4\\{channel_part} Karaoke{after_karaoke}"
print(f"DEBUG: Fixed path (pattern 5): {fixed_path}")
return fixed_path
# Fallback: just replace :// with :\ and hope for the best
fallback_path = file_path.replace('://', ':\\')
print(f"DEBUG: Fallback path fix: {fallback_path}")
return fallback_path
# If no :// corruption detected, return the path as-is
print(f"DEBUG: normalize_path output (no changes): {file_path}")
return file_path
@ -637,10 +569,11 @@ def serve_video(file_path):
# Normalize the path to fix any malformed paths
normalized_path = normalize_path(decoded_path)
# Debug logging
print(f"DEBUG: Video request for path: {decoded_path}")
# Debug logging - show original path first
print(f"DEBUG: serve_video ORIGINAL PATH: {decoded_path}")
print(f"DEBUG: Normalized path: {normalized_path}")
print(f"DEBUG: Current working directory: {os.getcwd()}")
print(f"DEBUG: Path transformation: '{decoded_path}' -> '{normalized_path}'")
# Security check: ensure the path is within allowed directories
# This prevents directory traversal attacks
@ -662,6 +595,16 @@ def serve_video(file_path):
# Check if file exists
if not os.path.exists(normalized_path):
print(f"DEBUG: File does not exist: {normalized_path}")
# Try to list the directory to see what's available
try:
directory = os.path.dirname(normalized_path)
if os.path.exists(directory):
files_in_dir = os.listdir(directory)
print(f"DEBUG: Files in directory {directory}: {files_in_dir[:10]}...") # Show first 10 files
else:
print(f"DEBUG: Directory does not exist: {directory}")
except Exception as e:
print(f"DEBUG: Error listing directory: {e}")
return jsonify({'error': 'Video file not found'}), 404
# Check if it's a video file and determine MIME type
@ -713,6 +656,8 @@ def serve_video(file_path):
except Exception as e:
print(f"DEBUG: Exception in serve_video: {str(e)}")
import traceback
print(f"DEBUG: Full traceback: {traceback.format_exc()}")
return jsonify({'error': f'Error serving video: {str(e)}'}), 500
if __name__ == '__main__':

136
web/templates/index.html
View File

@ -522,7 +522,7 @@
let sortableInstances = [];
// Load data on page load
document.addEventListener('DOMContentLoaded', function() {
document.addEventListener('DOMContentLoaded', async function() {
// Check if Sortable.js is loaded
if (typeof Sortable === 'undefined') {
console.error('Sortable.js library not loaded!');
@ -533,7 +533,8 @@
loadStats();
loadArtists();
loadPriorityPreferences();
// Load priority preferences first, then load duplicates
await loadPriorityPreferences();
loadDuplicates();
});
@ -769,12 +770,16 @@
async function loadPriorityPreferences() {
try {
console.log('Loading priority preferences...');
const response = await fetch('/api/load-priority-preferences');
const data = await response.json();
if (data.success) {
priorityChanges = data.preferences;
console.log('Loaded priority preferences:', priorityChanges);
updatePrioritySaveButton();
} else {
console.log('No priority preferences found or error loading them');
}
} catch (error) {
@ -932,6 +937,7 @@
// Get current priority order for this song
const songKey = duplicate.artist + ' - ' + duplicate.title;
const currentPriorities = priorityChanges[songKey] || [];
console.log('Creating song card for:', songKey, 'with priorities:', currentPriorities);
// Create all versions array (kept + skipped)
const allVersions = [
@ -954,6 +960,12 @@
if (bIndex === -1) return 0;
return aIndex - bIndex;
});
// Update is_kept property based on new order (first item is kept)
allVersions.forEach((version, index) => {
version.is_kept = (index === 0);
});
console.log('Applied priority order for:', songKey, 'kept version:', allVersions[0].path);
}
return '<div class="card duplicate-card">' +
@ -996,7 +1008,7 @@
'<span class="badge ' + (version.is_kept ? 'bg-info' : 'bg-warning') + ' channel-badge">' + version.channel + '</span>' +
(version.is_kept ? '<span class="badge bg-success ms-1 status-badge">KEPT</span>' : '<span class="badge bg-danger ms-1 status-badge">SKIPPED</span>') +
(version.file_type === 'MP4' ?
'<button class="play-button" onclick="openVideoPlayer(\'' + version.path.replace(/'/g, "\\'") + '\', \'' + duplicate.artist.replace(/'/g, "\\'") + '\', \'' + duplicate.title.replace(/'/g, "\\'") + '\')">' +
'<button class="play-button" onclick="openVideoPlayer(\'' + version.path.replace(/\\/g, '\\\\').replace(/'/g, "\\'") + '\', \'' + duplicate.artist.replace(/'/g, "\\'") + '\', \'' + duplicate.title.replace(/'/g, "\\'") + '\')">' +
'<i class="fas fa-play"></i> Play' +
'</button>' :
''
@ -1292,110 +1304,29 @@
// Video Player Functions
function normalizePath(filePath) {
// Debug logging to track path transformation - show original path first
console.log('DEBUG: normalizePath ORIGINAL PATH:', filePath);
// Fix malformed paths that have been corrupted with ://
// Example: z://MP4KaraFun KaraokeKaraoke I m Not In Love - 10CC.mp4
// Should be: z:\MP4\KaraFun Karaoke\Karaoke I'm Not In Love - 10CC.mp4
// Since we fixed the HTML string literal escaping, paths should now come in correctly
// Just handle the basic :// to :\ conversion
if (filePath.includes('://')) {
console.log('DEBUG: Detected malformed path, attempting to fix:', filePath);
console.log('DEBUG: Detected malformed path with ://, attempting to fix:', filePath);
// Extract drive letter and rest of path
const match = filePath.match(/^([a-zA-Z]):\/\/(.+)$/);
if (match) {
const driveLetter = match[1];
let restOfPath = match[2];
// Clean up special characters in the path first - but be more conservative
// Only remove actual control characters, not legitimate path characters
filePath = filePath.replace(/[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]/g, ''); // Remove only control characters
filePath = filePath.replace(/\s+/g, ' '); // Normalize multiple spaces to single space
filePath = filePath.trim(); // Remove leading/trailing spaces
// Try to reconstruct the proper path structure
// Look for common patterns in the corrupted path
// Pattern 1: Handle the specific case where "Karaoke" appears twice
// Example: "MP4KaraFun KaraokeKaraoke I m Not In Love - 10CC.mp4"
// Should become: "MP4\KaraFun Karaoke\Karaoke I'm Not In Love - 10CC.mp4"
const karaokeDoubleMatch = restOfPath.match(/^MP4([A-Za-z\s]+)KaraokeKaraoke(.+)$/);
if (karaokeDoubleMatch) {
const channelName = karaokeDoubleMatch[1] + "Karaoke";
const fileName = "Karaoke" + karaokeDoubleMatch[2];
const fixedPath = driveLetter + ":\\MP4\\" + channelName + "\\" + fileName;
console.log('DEBUG: Fixed path (pattern 1 - double karaoke):', fixedPath);
// Simple fix: replace :// with :\
const fixedPath = filePath.replace('://', ':\\');
console.log('DEBUG: Fixed path (simple :// to :\ conversion):', fixedPath);
return fixedPath;
}
// Pattern 2: MP4 followed by channel name (e.g., MP4KaraFun Karaoke)
const mp4Match = restOfPath.match(/^MP4([A-Za-z\s]+Karaoke)(.+)$/);
if (mp4Match) {
const channelName = mp4Match[1];
const fileName = mp4Match[2];
const fixedPath = driveLetter + ":\\MP4\\" + channelName + "\\" + fileName;
console.log('DEBUG: Fixed path (pattern 2):', fixedPath);
return fixedPath;
}
// Pattern 3: Direct channel name followed by filename
const channelMatch = restOfPath.match(/^([A-Za-z\s]+Karaoke)(.+)$/);
if (channelMatch) {
const channelName = channelMatch[1];
const fileName = channelMatch[2];
const fixedPath = driveLetter + ":\\MP4\\" + channelName + "\\" + fileName;
console.log('DEBUG: Fixed path (pattern 3):', fixedPath);
return fixedPath;
}
// Pattern 4: Look for any known channel names
const knownChannels = ['Sing King Karaoke', 'KaraFun Karaoke', 'Stingray Karaoke'];
for (const channel of knownChannels) {
if (channel.toLowerCase().replace(/\s/g, '') === restOfPath.toLowerCase().replace(/\s/g, '').substring(0, channel.toLowerCase().replace(/\s/g, '').length)) {
// Extract the part before and after the channel name
const channelLower = channel.toLowerCase().replace(/\s/g, '');
const restLower = restOfPath.toLowerCase().replace(/\s/g, '');
const channelIndex = restLower.indexOf(channelLower);
if (channelIndex >= 0) {
// Reconstruct the path
const beforeChannel = restOfPath.substring(0, channelIndex);
const afterChannel = restOfPath.substring(channelIndex + channel.length);
// If there's content before the channel, it might be a folder like "MP4"
let fixedPath;
if (beforeChannel && beforeChannel.toLowerCase() === 'mp4') {
fixedPath = driveLetter + ":\\" + beforeChannel + "\\" + channel + afterChannel;
} else {
fixedPath = driveLetter + ":\\MP4\\" + channel + afterChannel;
}
console.log('DEBUG: Fixed path (pattern 4):', fixedPath);
return fixedPath;
}
}
}
// Pattern 5: Try to split by common separators and reconstruct
// Look for patterns like "MP4KaraFunKaraoke" -> "MP4\KaraFun Karaoke"
if (restOfPath.toLowerCase().includes('karaoke')) {
// Try to find where "karaoke" appears and reconstruct
const karaokeIndex = restOfPath.toLowerCase().indexOf('karaoke');
if (karaokeIndex > 0) {
const beforeKaraoke = restOfPath.substring(0, karaokeIndex);
const afterKaraoke = restOfPath.substring(karaokeIndex + 7); // length of "karaoke"
// If beforeKaraoke starts with "MP4", extract the channel name
if (beforeKaraoke.toLowerCase().startsWith('mp4')) {
const channelPart = beforeKaraoke.substring(3); // Remove "MP4"
if (channelPart) {
const fixedPath = driveLetter + ":\\MP4\\" + channelPart + " Karaoke" + afterKaraoke;
console.log('DEBUG: Fixed path (pattern 5):', fixedPath);
return fixedPath;
}
}
}
}
// Fallback: just replace :// with :\ and hope for the best
const fallbackPath = filePath.replace('://', ':\\');
console.log('DEBUG: Fallback path fix:', fallbackPath);
return fallbackPath;
}
}
// If no :// corruption detected, return the path as-is
console.log('DEBUG: normalizePath output (no changes):', filePath);
return filePath;
}
@ -1413,10 +1344,11 @@
// Encode the file path for the URL
const encodedPath = encodeURIComponent(normalizedPath);
console.log('DEBUG: Opening video player for:', filePath);
console.log('DEBUG: openVideoPlayer ORIGINAL PATH:', filePath);
console.log('DEBUG: Normalized path:', normalizedPath);
console.log('DEBUG: Encoded path:', encodedPath);
console.log('DEBUG: Video URL:', '/api/video/' + encodedPath);
console.log('DEBUG: Path transformation:', filePath, '->', normalizedPath, '->', encodedPath);
// Clear any previous source
videoPlayer.src = '';
@ -1436,7 +1368,7 @@
// Add error handling
videoPlayer.onerror = function(e) {
console.error('Error loading video:', filePath);
console.error('Error loading video - ORIGINAL PATH:', filePath);
console.error('Normalized path:', normalizedPath);
console.error('Video error details:', e);
console.error('Video error code:', videoPlayer.error ? videoPlayer.error.code : 'unknown');