Signed-off-by: Matt Bruce <mbrucedogs@gmail.com>

This commit is contained in:
Matt Bruce 2025-08-11 09:45:12 -05:00
parent dd916a646a
commit 029b9492d2
12 changed files with 764 additions and 44 deletions

14
PRD.md
View File

@ -161,7 +161,7 @@ These standards ensure the codebase remains clean, maintainable, and accessible
### 3.1 Input ### 3.1 Input
- Reads from `/data/allSongs.json` - Reads from `/data/songs.json`
- Each song includes at least: - Each song includes at least:
- `artist`, `title`, `path`, (plus id3 tag info, `channel` for MP4s) - `artist`, `title`, `path`, (plus id3 tag info, `channel` for MP4s)
@ -230,7 +230,7 @@ These standards ensure the codebase remains clean, maintainable, and accessible
``` ```
KaraokeMerge/ KaraokeMerge/
├── data/ ├── data/
│ ├── allSongs.json # Input: Your song library data │ ├── songs.json # Input: Your song library data
│ ├── skipSongs.json # Output: Generated skip list │ ├── skipSongs.json # Output: Generated skip list
│ ├── preferences/ # User priority preferences │ ├── preferences/ # User priority preferences
│ │ ├── priority_preferences.json │ │ ├── priority_preferences.json
@ -297,6 +297,13 @@ KaraokeMerge/
- **Priority Persistence**: Save/load user priority preferences to/from JSON files - **Priority Persistence**: Save/load user priority preferences to/from JSON files
- **Priority Preferences API**: RESTful endpoints for managing priority preferences - **Priority Preferences API**: RESTful endpoints for managing priority preferences
#### **Reset & Regenerate System**
- **One-Click Reset**: Delete all generated files and regenerate everything with a single button click
- **Complete Cleanup**: Removes skipSongs.json, reports directory, and preferences directory
- **Automatic CLI Execution**: Runs the CLI tool automatically to regenerate all data
- **Progress Feedback**: Shows loading state and provides detailed feedback on completion
- **Safety Confirmation**: Requires user confirmation before performing destructive operations
#### **User Interface Enhancements** #### **User Interface Enhancements**
- **Visual Status Indicators**: Color-coded cards (green for kept, red for skipped) - **Visual Status Indicators**: Color-coded cards (green for kept, red for skipped)
- **File Type Badges**: Visual indicators for MP3, MP4, and CDG files - **File Type Badges**: Visual indicators for MP3, MP4, and CDG files
@ -410,7 +417,7 @@ data/preferences/
### ✅ Completed Features ### ✅ Completed Features
#### **Core CLI Functionality** #### **Core CLI Functionality**
- [x] Write initial CLI tool to parse allSongs.json, deduplicate, and output skipSongs.json - [x] Write initial CLI tool to parse songs.json, deduplicate, and output skipSongs.json
- [x] Print CLI summary reports (with verbosity control) - [x] Print CLI summary reports (with verbosity control)
- [x] Implement config file support for channel priority - [x] Implement config file support for channel priority
- [x] Organize folder/file structure for easy expansion - [x] Organize folder/file structure for easy expansion
@ -447,6 +454,7 @@ data/preferences/
- [x] Pattern analysis and channel optimization suggestions - [x] Pattern analysis and channel optimization suggestions
- [x] Non-destructive operation (skip lists only) - [x] Non-destructive operation (skip lists only)
- [x] Verbose and dry-run modes - [x] Verbose and dry-run modes
- [x] Reset & regenerate functionality with one-click cleanup
### 🎯 Current Implementation ### 🎯 Current Implementation

View File

@ -31,6 +31,12 @@ A comprehensive tool for analyzing, deduplicating, and cleaning up large karaoke
- **Priority Indicators**: Visual numbered indicators show the current priority order - **Priority Indicators**: Visual numbered indicators show the current priority order
- **Reset Functionality**: Easily reset to default priorities if needed - **Reset Functionality**: Easily reset to default priorities if needed
### 🔄 Reset & Regenerate Feature
- **One-Click Reset**: Delete all generated files and regenerate everything with a single button click
- **Complete Cleanup**: Removes skipSongs.json, reports directory, and preferences directory
- **Automatic CLI Execution**: Runs the CLI tool automatically to regenerate all data
- **Progress Feedback**: Shows loading state and provides detailed feedback on completion
## Installation ## Installation
### Prerequisites ### Prerequisites
@ -61,6 +67,19 @@ A comprehensive tool for analyzing, deduplicating, and cleaning up large karaoke
python -c "import flask, fuzzywuzzy; print('All dependencies installed successfully!')" python -c "import flask, fuzzywuzzy; print('All dependencies installed successfully!')"
``` ```
### Migration from Previous Versions
If you're upgrading from a previous version that used `allSongs.json`, run the migration script:
```bash
python3 migrate_to_songs_json.py
```
This script will:
- Rename `allSongs.json` to `songs.json`
- Add `data_directory` configuration to `config.json`
- Create backups of your original files
## Usage ## Usage
### CLI Tool ### CLI Tool
@ -125,7 +144,7 @@ Edit `config/config.json` to customize:
``` ```
KaraokeMerge/ KaraokeMerge/
├── data/ ├── data/
│ ├── allSongs.json # Input: Your song library data │ ├── songs.json # Input: Your song library data
│ ├── skipSongs.json # Output: Generated skip list │ ├── skipSongs.json # Output: Generated skip list
│ ├── preferences/ # User priority preferences │ ├── preferences/ # User priority preferences
│ │ └── priority_preferences.json │ │ └── priority_preferences.json
@ -151,7 +170,7 @@ KaraokeMerge/
## Data Requirements ## Data Requirements
Place your song library data in `data/allSongs.json` with the following format: Place your song library data in `data/songs.json` with the following format:
```json ```json
[ [
{ {

View File

@ -11,7 +11,7 @@ cd cli
python3 main.py python3 main.py
``` ```
Runs the tool with default settings: Runs the tool with default settings:
- Input: `data/allSongs.json` - Input: `data/songs.json`
- Config: `config/config.json` - Config: `config/config.json`
- Output: `data/skipSongs.json` - Output: `data/skipSongs.json`
- Reports: **Automatically generated** - Reports: **Automatically generated**
@ -71,7 +71,7 @@ Displays the current configuration settings and exits.
```bash ```bash
python3 main.py --input path/to/songs.json python3 main.py --input path/to/songs.json
``` ```
Specifies a custom input file instead of the default `data/allSongs.json`. Specifies a custom input file instead of the default `data/songs.json`.
#### Custom Output Directory #### Custom Output Directory
```bash ```bash
@ -130,7 +130,7 @@ Generated reports include:
```bash ```bash
python3 playlist_validator.py python3 playlist_validator.py
``` ```
Validates all playlists in `data/songLists.json` against the song library. Validates all playlists in `data/songList.json` against the song library.
#### Validate Specific Playlist #### Validate Specific Playlist
```bash ```bash
@ -323,7 +323,7 @@ python3 main.py --show-config
| Option | Description | Default | | Option | Description | Default |
|--------|-------------|---------| |--------|-------------|---------|
| `--config` | Configuration file path | `../config/config.json` | | `--config` | Configuration file path | `../config/config.json` |
| `--input` | Input songs file path | `../data/allSongs.json` | | `--input` | Input songs file path | `../data/songs.json` |
| `--output-dir` | Output directory | `../data` | | `--output-dir` | Output directory | `../data` |
| `--verbose, -v` | Enable verbose output | `False` | | `--verbose, -v` | Enable verbose output | `False` |
| `--dry-run` | Analyze without generating files | `False` | | `--dry-run` | Analyze without generating files | `False` |
@ -348,13 +348,13 @@ python3 main.py --show-config
## File Structure Requirements ## File Structure Requirements
### Required Files ### Required Files
- `data/allSongs.json` - Main song library - `data/songs.json` - Main song library
- `config/config.json` - Configuration settings - `config/config.json` - Configuration settings
### Optional Files ### Optional Files
- `data/favorites.json` - Favorites list (for processing) - `data/favorites.json` - Favorites list (for processing)
- `data/history.json` - History list (for processing) - `data/history.json` - History list (for processing)
- `data/songLists.json` - Playlists (for validation) - `data/songList.json` - Playlists (for validation)
### Generated Files ### Generated Files
- `data/skipSongs.json` - Skip list for future imports - `data/skipSongs.json` - Skip list for future imports
@ -391,7 +391,7 @@ The default configuration file (`config/config.json`) contains:
## Input File Formats ## Input File Formats
### Song Library Format (allSongs.json) ### Song Library Format (songs.json)
```json ```json
[ [
{ {
@ -402,7 +402,7 @@ The default configuration file (`config/config.json`) contains:
] ]
``` ```
### Playlist Format (songLists.json) ### Playlist Format (songList.json)
```json ```json
[ [
{ {

View File

@ -230,14 +230,14 @@ Examples:
parser.add_argument( parser.add_argument(
'--input', '--input',
default='../data/allSongs.json', default=None,
help='Path to input songs file (default: ../data/allSongs.json)' help='Path to input songs file (default: auto-detected from config)'
) )
parser.add_argument( parser.add_argument(
'--output-dir', '--output-dir',
default='../data', default=None,
help='Directory for output files (default: ../data)' help='Directory for output files (default: auto-detected from config)'
) )
parser.add_argument( parser.add_argument(
@ -335,14 +335,20 @@ def main():
reporter.print_report("config", config) reporter.print_report("config", config)
return return
# Determine data directory and input file from config or args
data_dir = args.output_dir or config.get('data_directory', '../data')
# Resolve relative paths from CLI directory
if not os.path.isabs(data_dir):
data_dir = os.path.join(os.path.dirname(__file__), '..', data_dir)
input_file = args.input or os.path.join(data_dir, 'songs.json')
# Load songs (only if needed for processing) # Load songs (only if needed for processing)
data_dir = args.output_dir
songs = None songs = None
matcher = None matcher = None
reporter = None reporter = None
if not args.merge_history: if not args.merge_history:
songs = load_songs(args.input) songs = load_songs(input_file)
matcher = SongMatcher(config, data_dir) matcher = SongMatcher(config, data_dir)
reporter = ReportGenerator(config) reporter = ReportGenerator(config)
@ -402,7 +408,7 @@ def main():
# Save skip list if not dry run # Save skip list if not dry run
if not args.dry_run and skip_songs: if not args.dry_run and skip_songs:
skip_list_path = os.path.join(args.output_dir, 'skipSongs.json') skip_list_path = os.path.join(data_dir, 'skipSongs.json')
# Create simplified skip list (just paths and reasons) with deduplication # Create simplified skip list (just paths and reasons) with deduplication
seen_paths = set() seen_paths = set()
@ -430,7 +436,7 @@ def main():
# Always generate detailed reports (not just when --save-reports is used) # Always generate detailed reports (not just when --save-reports is used)
if not args.dry_run: if not args.dry_run:
reports_dir = os.path.join(args.output_dir, 'reports') reports_dir = os.path.join(data_dir, 'reports')
os.makedirs(reports_dir, exist_ok=True) os.makedirs(reports_dir, exist_ok=True)
print(f"\n📊 Generating enhanced analysis reports...") print(f"\n📊 Generating enhanced analysis reports...")

View File

@ -46,8 +46,8 @@ class PlaylistValidator:
self._build_lookup_tables() self._build_lookup_tables()
def _load_all_songs(self) -> List[Dict[str, Any]]: def _load_all_songs(self) -> List[Dict[str, Any]]:
"""Load the song library from allSongs.json.""" """Load the song library from songs.json."""
all_songs_path = os.path.join(self.data_dir, 'allSongs.json') all_songs_path = os.path.join(self.data_dir, 'songs.json')
try: try:
with open(all_songs_path, 'r', encoding='utf-8') as f: with open(all_songs_path, 'r', encoding='utf-8') as f:
return json.load(f) return json.load(f)
@ -192,8 +192,8 @@ class PlaylistValidator:
return results return results
def validate_all_playlists(self, dry_run: bool = True) -> Dict[str, Any]: def validate_all_playlists(self, dry_run: bool = True) -> Dict[str, Any]:
"""Validate all playlists in songLists.json.""" """Validate all playlists in songList.json."""
playlists_path = os.path.join(self.data_dir, 'songLists.json') playlists_path = os.path.join(self.data_dir, 'songList.json')
try: try:
with open(playlists_path, 'r', encoding='utf-8') as f: with open(playlists_path, 'r', encoding='utf-8') as f:
@ -231,7 +231,7 @@ class PlaylistValidator:
def update_playlist_song(self, playlist_index: int, song_position: int, def update_playlist_song(self, playlist_index: int, song_position: int,
new_artist: str, new_title: str, dry_run: bool = True) -> bool: new_artist: str, new_title: str, dry_run: bool = True) -> bool:
"""Update a playlist song with corrected artist/title.""" """Update a playlist song with corrected artist/title."""
playlists_path = os.path.join(self.data_dir, 'songLists.json') playlists_path = os.path.join(self.data_dir, 'songList.json')
try: try:
with open(playlists_path, 'r', encoding='utf-8') as f: with open(playlists_path, 'r', encoding='utf-8') as f:
@ -305,7 +305,7 @@ def main():
if args.playlist_index is not None: if args.playlist_index is not None:
# Validate specific playlist # Validate specific playlist
playlists_path = os.path.join(args.data_dir, 'songLists.json') playlists_path = os.path.join(args.data_dir, 'songList.json')
try: try:
with open(playlists_path, 'r', encoding='utf-8') as f: with open(playlists_path, 'r', encoding='utf-8') as f:
playlists = json.load(f) playlists = json.load(f)

View File

@ -510,7 +510,16 @@ class ReportGenerator:
def save_report_to_file(self, report_content: str, file_path: str) -> None: def save_report_to_file(self, report_content: str, file_path: str) -> None:
"""Save a report to a text file.""" """Save a report to a text file."""
import os import os
os.makedirs(os.path.dirname(file_path), exist_ok=True)
# Validate file_path
if not file_path or file_path is None:
print("Warning: Invalid file path provided, skipping report save")
return
# Get directory and create it if needed
directory = os.path.dirname(file_path)
if directory: # Only create directory if there is one
os.makedirs(directory, exist_ok=True)
with open(file_path, 'w', encoding='utf-8') as f: with open(file_path, 'w', encoding='utf-8') as f:
f.write(report_content) f.write(report_content)

View File

@ -1,4 +1,5 @@
{ {
"data_directory": "data",
"channel_priorities": [ "channel_priorities": [
"Sing King Karaoke", "Sing King Karaoke",
"KaraFun Karaoke", "KaraFun Karaoke",

144
migrate_to_songs_json.py Normal file
View File

@ -0,0 +1,144 @@
#!/usr/bin/env python3
"""
Migration script to help users move from allSongs.json to songs.json
and update their configuration to use the new dynamic data directory.
"""
import os
import json
import shutil
from pathlib import Path
def load_json_file(file_path: str):
"""Load JSON file safely."""
try:
with open(file_path, 'r', encoding='utf-8') as f:
return json.load(f)
except Exception as e:
print(f"Error loading {file_path}: {e}")
return None
def save_json_file(file_path: str, data):
"""Save JSON file safely."""
try:
with open(file_path, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2, ensure_ascii=False)
return True
except Exception as e:
print(f"Error saving {file_path}: {e}")
return False
def migrate_songs_file():
"""Migrate allSongs.json to songs.json if it exists."""
old_file = 'data/allSongs.json'
new_file = 'data/songs.json'
if not os.path.exists(old_file):
print(f"⚠️ {old_file} not found - no migration needed")
return True
if os.path.exists(new_file):
print(f"⚠️ {new_file} already exists - skipping migration")
return True
print(f"🔄 Migrating {old_file} to {new_file}...")
# Load the old file
songs_data = load_json_file(old_file)
if not songs_data:
print(f"❌ Failed to load {old_file}")
return False
# Save to new file
if save_json_file(new_file, songs_data):
print(f"✅ Successfully migrated to {new_file}")
# Create backup of old file
backup_file = 'data/allSongs.json.backup'
shutil.copy2(old_file, backup_file)
print(f"📦 Created backup at {backup_file}")
return True
else:
print(f"❌ Failed to save {new_file}")
return False
def update_config():
"""Update config.json to include data_directory if not present."""
config_file = 'config/config.json'
if not os.path.exists(config_file):
print(f"{config_file} not found")
return False
print(f"🔄 Updating {config_file}...")
# Load current config
config = load_json_file(config_file)
if not config:
print(f"❌ Failed to load {config_file}")
return False
# Check if data_directory already exists
if 'data_directory' in config:
print(f"✅ data_directory already configured: {config['data_directory']}")
return True
# Add data_directory
config['data_directory'] = 'data'
# Create backup
backup_file = 'config/config.json.backup'
shutil.copy2(config_file, backup_file)
print(f"📦 Created backup at {backup_file}")
# Save updated config
if save_json_file(config_file, config):
print(f"✅ Successfully added data_directory to {config_file}")
return True
else:
print(f"❌ Failed to save {config_file}")
return False
def main():
"""Main migration function."""
print("🎤 KaraokeMerge Migration Script")
print("=" * 40)
print("This script will help you migrate to the new configuration:")
print("- Rename allSongs.json to songs.json")
print("- Add data_directory to config.json")
print()
# Check if we're in the right directory
if not os.path.exists('config') or not os.path.exists('data'):
print("❌ Please run this script from the KaraokeMerge root directory")
return False
success = True
# Migrate songs file
if not migrate_songs_file():
success = False
# Update config
if not update_config():
success = False
print()
if success:
print("✅ Migration completed successfully!")
print()
print("Next steps:")
print("1. Test the CLI tool: python cli/main.py --show-config")
print("2. Test the web UI: python start_web_ui.py")
print("3. If everything works, you can delete the backup files")
else:
print("❌ Migration failed - please check the errors above")
return False
return True
if __name__ == "__main__":
success = main()
if not success:
exit(1)

View File

@ -88,7 +88,7 @@ def start_web_ui():
# Start Flask app # Start Flask app
try: try:
print("🌐 Web UI will be available at: http://localhost:5000") print("🌐 Web UI will be available at: http://localhost:5002")
print("📱 You can open this URL in your web browser") print("📱 You can open this URL in your web browser")
print("\n⏳ Starting server... (Press Ctrl+C to stop)") print("\n⏳ Starting server... (Press Ctrl+C to stop)")
print("-" * 60) print("-" * 60)
@ -96,7 +96,7 @@ def start_web_ui():
# Open browser after a short delay # Open browser after a short delay
def open_browser(): def open_browser():
sleep(2) sleep(2)
webbrowser.open("http://localhost:5000") webbrowser.open("http://localhost:5002")
import threading import threading
browser_thread = threading.Thread(target=open_browser) browser_thread = threading.Thread(target=open_browser)

View File

@ -24,7 +24,7 @@ def validate_data_files():
# Check for required files # Check for required files
required_files = [ required_files = [
'data/allSongs.json', 'data/songs.json',
'config/config.json' 'config/config.json'
] ]
@ -59,7 +59,7 @@ def analyze_song_data():
"""Analyze the song data structure and provide insights.""" """Analyze the song data structure and provide insights."""
print("\n=== Song Data Analysis ===") print("\n=== Song Data Analysis ===")
all_songs_path = 'data/allSongs.json' all_songs_path = 'data/songs.json'
if not os.path.exists(all_songs_path): if not os.path.exists(all_songs_path):
print(f"{all_songs_path} not found - cannot analyze song data") print(f"{all_songs_path} not found - cannot analyze song data")
return return

View File

@ -7,6 +7,7 @@ Provides interactive interface for reviewing duplicates and making decisions.
from flask import Flask, render_template, jsonify, request, send_from_directory from flask import Flask, render_template, jsonify, request, send_from_directory
import json import json
import os import os
import time
from typing import Dict, List, Any from typing import Dict, List, Any
from datetime import datetime from datetime import datetime
@ -18,10 +19,28 @@ from playlist_validator import PlaylistValidator
app = Flask(__name__) app = Flask(__name__)
# Configuration # Configuration
DATA_DIR = '../data'
REPORTS_DIR = os.path.join(DATA_DIR, 'reports')
CONFIG_FILE = '../config/config.json' CONFIG_FILE = '../config/config.json'
# Global variable to store progress
progress_data = {
'status': 'idle',
'message': '',
'progress': 0,
'current_step': '',
'cli_output': []
}
def reset_progress():
"""Reset progress data to idle state."""
global progress_data
progress_data = {
'status': 'idle',
'message': '',
'progress': 0,
'current_step': '',
'cli_output': []
}
def load_json_file(file_path: str) -> Any: def load_json_file(file_path: str) -> Any:
"""Load JSON file safely.""" """Load JSON file safely."""
try: try:
@ -31,6 +50,20 @@ def load_json_file(file_path: str) -> Any:
print(f"Error loading {file_path}: {e}") print(f"Error loading {file_path}: {e}")
return None return None
def get_data_directory():
"""Get data directory from config."""
config = load_json_file(CONFIG_FILE)
if config and 'data_directory' in config:
# When running from web/ directory, we need to go up one level
data_dir = config['data_directory']
if not os.path.isabs(data_dir):
return os.path.join('..', data_dir)
return data_dir
return '../data'
DATA_DIR = get_data_directory()
REPORTS_DIR = os.path.join(DATA_DIR, 'reports')
def get_duplicate_groups(skip_songs: List[Dict[str, Any]]) -> List[Dict[str, Any]]: def get_duplicate_groups(skip_songs: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""Group skip songs by artist/title to show duplicates together.""" """Group skip songs by artist/title to show duplicates together."""
duplicate_groups = {} duplicate_groups = {}
@ -329,7 +362,7 @@ def get_stats():
return jsonify({'error': 'No skip songs data found'}), 404 return jsonify({'error': 'No skip songs data found'}), 404
# Load original all songs data to get total counts # Load original all songs data to get total counts
all_songs = load_json_file(os.path.join(DATA_DIR, 'allSongs.json')) all_songs = load_json_file(os.path.join(DATA_DIR, 'songs.json'))
if not all_songs: if not all_songs:
all_songs = [] all_songs = []
@ -477,7 +510,7 @@ def get_artists():
def get_mp3_songs(): def get_mp3_songs():
"""API endpoint to get MP3 songs that remain after cleanup.""" """API endpoint to get MP3 songs that remain after cleanup."""
# Load all songs and skip songs # Load all songs and skip songs
all_songs = load_json_file(os.path.join(DATA_DIR, 'allSongs.json')) all_songs = load_json_file(os.path.join(DATA_DIR, 'songs.json'))
skip_songs = load_json_file(os.path.join(DATA_DIR, 'reports', 'skip_songs_detailed.json')) skip_songs = load_json_file(os.path.join(DATA_DIR, 'reports', 'skip_songs_detailed.json'))
if not all_songs: if not all_songs:
@ -495,7 +528,7 @@ def get_mp3_songs():
def get_remaining_songs(): def get_remaining_songs():
"""Get all remaining songs (MP4 and MP3) after cleanup with pagination.""" """Get all remaining songs (MP4 and MP3) after cleanup with pagination."""
try: try:
all_songs = load_json_file(os.path.join(DATA_DIR, 'allSongs.json')) all_songs = load_json_file(os.path.join(DATA_DIR, 'songs.json'))
skip_songs = load_json_file(os.path.join(DATA_DIR, 'reports', 'skip_songs_detailed.json')) skip_songs = load_json_file(os.path.join(DATA_DIR, 'reports', 'skip_songs_detailed.json'))
if not all_songs: if not all_songs:
@ -577,7 +610,7 @@ def get_remaining_songs():
def download_mp3_songs(): def download_mp3_songs():
"""Download MP3 songs list as JSON file.""" """Download MP3 songs list as JSON file."""
# Load all songs and skip songs # Load all songs and skip songs
all_songs = load_json_file(os.path.join(DATA_DIR, 'allSongs.json')) all_songs = load_json_file(os.path.join(DATA_DIR, 'songs.json'))
skip_songs = load_json_file(os.path.join(DATA_DIR, 'reports', 'skip_songs_detailed.json')) skip_songs = load_json_file(os.path.join(DATA_DIR, 'reports', 'skip_songs_detailed.json'))
if not all_songs: if not all_songs:
@ -696,7 +729,7 @@ def load_priority_preferences():
return jsonify({'error': f'Error loading priority preferences: {str(e)}'}), 500 return jsonify({'error': f'Error loading priority preferences: {str(e)}'}), 500
def find_matching_songs(item: Dict[str, Any], all_songs: List[Dict[str, Any]]) -> List[Dict[str, Any]]: def find_matching_songs(item: Dict[str, Any], all_songs: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""Find all songs from allSongs.json that match a given item (favorite/history).""" """Find all songs from songs.json that match a given item (favorite/history)."""
matching_songs = [] matching_songs = []
# Validate input # Validate input
@ -749,7 +782,7 @@ def get_favorites():
try: try:
# Load data files # Load data files
favorites_file = os.path.join(DATA_DIR, 'favorites.json') favorites_file = os.path.join(DATA_DIR, 'favorites.json')
all_songs_file = os.path.join(DATA_DIR, 'allSongs.json') all_songs_file = os.path.join(DATA_DIR, 'songs.json')
favorites = load_json_file(favorites_file) favorites = load_json_file(favorites_file)
all_songs = load_json_file(all_songs_file) all_songs = load_json_file(all_songs_file)
@ -806,7 +839,7 @@ def get_history():
try: try:
# Load data files # Load data files
history_file = os.path.join(DATA_DIR, 'history.json') history_file = os.path.join(DATA_DIR, 'history.json')
all_songs_file = os.path.join(DATA_DIR, 'allSongs.json') all_songs_file = os.path.join(DATA_DIR, 'songs.json')
history = load_json_file(history_file) history = load_json_file(history_file)
all_songs = load_json_file(all_songs_file) all_songs = load_json_file(all_songs_file)
@ -1295,7 +1328,7 @@ def playlist_validation():
def get_playlists(): def get_playlists():
"""Get list of all playlists.""" """Get list of all playlists."""
try: try:
playlists_path = os.path.join(DATA_DIR, 'songLists.json') playlists_path = os.path.join(DATA_DIR, 'songList.json')
with open(playlists_path, 'r', encoding='utf-8') as f: with open(playlists_path, 'r', encoding='utf-8') as f:
playlists = json.load(f) playlists = json.load(f)
@ -1326,7 +1359,7 @@ def validate_playlist(playlist_index):
validator = PlaylistValidator(config, DATA_DIR) validator = PlaylistValidator(config, DATA_DIR)
# Load playlists # Load playlists
playlists_path = os.path.join(DATA_DIR, 'songLists.json') playlists_path = os.path.join(DATA_DIR, 'songList.json')
with open(playlists_path, 'r', encoding='utf-8') as f: with open(playlists_path, 'r', encoding='utf-8') as f:
playlists = json.load(f) playlists = json.load(f)
@ -1448,5 +1481,246 @@ def apply_all_updates():
return jsonify({'error': f'Error applying updates: {str(e)}'}), 500 return jsonify({'error': f'Error applying updates: {str(e)}'}), 500
@app.route('/api/reset-and-regenerate', methods=['POST'])
def reset_and_regenerate():
"""Delete all generated files and run the CLI tool again."""
try:
import subprocess
import shutil
import os
import threading
# Reset progress data
reset_progress()
progress_data.update({
'status': 'starting',
'message': 'Initializing reset and regenerate process...',
'progress': 0,
'current_step': 'Initializing',
'cli_output': []
})
# Get the project root directory (parent of web directory)
project_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# Files and directories to delete
files_to_delete = [
os.path.join(DATA_DIR, 'skipSongs.json'),
os.path.join(DATA_DIR, 'reports'),
os.path.join(DATA_DIR, 'preferences')
]
deleted_items = []
# Update progress
progress_data.update({
'status': 'deleting',
'message': 'Deleting generated files...',
'progress': 10,
'current_step': 'Cleaning up old files'
})
# Delete files and directories
for item_path in files_to_delete:
if os.path.exists(item_path):
try:
if os.path.isfile(item_path):
os.remove(item_path)
deleted_items.append(f"File: {os.path.basename(item_path)}")
elif os.path.isdir(item_path):
shutil.rmtree(item_path)
deleted_items.append(f"Directory: {os.path.basename(item_path)}")
except Exception as e:
print(f"Warning: Could not delete {item_path}: {e}")
# Run the CLI tool
cli_dir = os.path.join(project_root, 'cli')
cli_script = os.path.join(cli_dir, 'main.py')
if not os.path.exists(cli_script):
progress_data.update({
'status': 'error',
'message': 'CLI script not found',
'progress': 0,
'current_step': 'Error'
})
return jsonify({'error': 'CLI script not found'}), 500
# Update progress
progress_data.update({
'status': 'running',
'message': 'Running CLI tool to analyze songs...',
'progress': 20,
'current_step': 'Running CLI Analysis'
})
# Change to CLI directory and run the tool
original_cwd = os.getcwd()
os.chdir(cli_dir)
def run_cli_with_progress():
global progress_data
try:
# Run the CLI tool with process-all flag and capture output in real-time
process = subprocess.Popen(
[sys.executable, 'main.py', '--process-all', '--verbose'],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
bufsize=1,
universal_newlines=True
)
# Read output line by line
line_count = 0
for line in iter(process.stdout.readline, ''):
if line:
line = line.strip()
progress_data['cli_output'].append(line)
line_count += 1
# Update progress based on output patterns
if 'Loading songs' in line or 'loading' in line.lower():
progress_data.update({
'progress': 30,
'current_step': 'Loading song library',
'message': f'Loading song library... ({line_count} lines processed)'
})
elif 'Finding duplicates' in line or 'duplicate' in line.lower():
progress_data.update({
'progress': 50,
'current_step': 'Finding duplicates',
'message': f'Finding duplicates... ({line_count} lines processed)'
})
elif 'Saving reports' in line or 'saving' in line.lower():
progress_data.update({
'progress': 80,
'current_step': 'Saving reports',
'message': f'Saving reports... ({line_count} lines processed)'
})
elif 'Complete' in line or 'Finished' in line or 'done' in line.lower():
progress_data.update({
'progress': 100,
'current_step': 'Complete',
'message': 'Process completed successfully!'
})
else:
# Update progress gradually based on line count
if line_count < 100:
progress_data.update({
'progress': min(20 + (line_count * 2), 30),
'current_step': 'Initializing',
'message': f'Initializing... ({line_count} lines processed)'
})
elif line_count < 500:
progress_data.update({
'progress': min(30 + ((line_count - 100) * 0.1), 50),
'current_step': 'Processing',
'message': f'Processing songs... ({line_count} lines processed)'
})
elif line_count < 1000:
progress_data.update({
'progress': min(50 + ((line_count - 500) * 0.05), 80),
'current_step': 'Analyzing',
'message': f'Analyzing duplicates... ({line_count} lines processed)'
})
else:
progress_data.update({
'progress': min(80 + ((line_count - 1000) * 0.02), 95),
'current_step': 'Finalizing',
'message': f'Finalizing... ({line_count} lines processed)'
})
process.stdout.close()
return_code = process.wait()
if return_code == 0:
progress_data.update({
'status': 'completed',
'message': f'✅ Reset and regeneration completed successfully!\n\nDeleted items:\n' + "\n".join(deleted_items),
'progress': 100,
'current_step': 'Complete'
})
else:
progress_data.update({
'status': 'error',
'message': f'CLI tool failed with return code {return_code}',
'progress': 0,
'current_step': 'Error'
})
except Exception as e:
progress_data.update({
'status': 'error',
'message': f'Error during CLI execution: {str(e)}',
'progress': 0,
'current_step': 'Error'
})
finally:
# Restore original working directory
os.chdir(original_cwd)
# Run CLI in a separate thread
cli_thread = threading.Thread(target=run_cli_with_progress)
cli_thread.daemon = True
cli_thread.start()
return jsonify({
'success': True,
'message': 'Reset and regenerate process started. Check progress endpoint for updates.',
'deleted_items': deleted_items
})
except Exception as e:
progress_data.update({
'status': 'error',
'message': f'Error during reset and regenerate: {str(e)}',
'progress': 0,
'current_step': 'Error'
})
return jsonify({'error': f'Error during reset and regenerate: {str(e)}'}), 500
@app.route('/api/progress')
def get_progress():
"""Get current progress of reset and regenerate process."""
global progress_data
return jsonify(progress_data)
@app.route('/api/progress/reset', methods=['POST'])
def reset_progress_endpoint():
"""Reset progress data to idle state."""
reset_progress()
return jsonify({'success': True, 'message': 'Progress reset to idle'})
@app.route('/api/progress/stream')
def progress_stream():
"""Server-Sent Events endpoint for real-time progress updates."""
def generate():
global progress_data
last_data = None
while True:
current_data = progress_data.copy()
# Only send if data has changed
if current_data != last_data:
yield f"data: {json.dumps(current_data)}\n\n"
last_data = current_data.copy()
# If process is complete or error, stop streaming
if current_data['status'] in ['completed', 'error']:
break
time.sleep(1) # Update every second
return app.response_class(
generate(),
mimetype='text/plain'
)
if __name__ == '__main__': if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0', port=5001) app.run(debug=True, host='0.0.0.0', port=5002)

View File

@ -254,6 +254,103 @@
color: #007bff; color: #007bff;
font-weight: bold; font-weight: bold;
} }
/* Reset & Regenerate Button Styles */
#reset-regenerate-btn {
background: linear-gradient(135deg, #ff6b6b 0%, #ee5a24 100%);
border: none;
color: white;
font-weight: bold;
box-shadow: 0 4px 15px rgba(255, 107, 107, 0.3);
transition: all 0.3s ease;
}
#reset-regenerate-btn:hover {
background: linear-gradient(135deg, #ee5a24 0%, #ff6b6b 100%);
transform: translateY(-2px);
box-shadow: 0 6px 20px rgba(255, 107, 107, 0.4);
}
#reset-regenerate-btn:disabled {
background: #6c757d;
transform: none;
box-shadow: none;
}
.action-buttons-section {
background: linear-gradient(135deg, #f8f9fa 0%, #e9ecef 100%);
border-radius: 10px;
padding: 1rem;
border: 1px solid #dee2e6;
}
/* Progress Modal Styles */
.progress-container {
margin: 20px 0;
}
.progress-step {
font-size: 1.1rem;
font-weight: bold;
color: #007bff;
margin-bottom: 10px;
}
.progress-bar-container {
display: flex;
align-items: center;
gap: 10px;
margin-bottom: 15px;
}
.progress-bar {
flex: 1;
height: 20px;
background-color: #e9ecef;
border-radius: 10px;
overflow: hidden;
}
.progress-bar-fill {
height: 100%;
background: linear-gradient(90deg, #007bff, #0056b3);
transition: width 0.3s ease;
}
.progress-message {
color: #6c757d;
font-style: italic;
}
.cli-output-container {
margin-top: 20px;
border-top: 1px solid #dee2e6;
padding-top: 15px;
}
.cli-output {
background-color: #f8f9fa;
border: 1px solid #dee2e6;
border-radius: 5px;
padding: 10px;
max-height: 300px;
overflow-y: auto;
font-family: 'Courier New', monospace;
font-size: 0.9rem;
white-space: pre-wrap;
}
.modal-close {
color: #aaa;
float: right;
font-size: 28px;
font-weight: bold;
cursor: pointer;
}
.modal-close:hover {
color: #000;
}
</style> </style>
</head> </head>
<body> <body>
@ -345,6 +442,20 @@
</div> </div>
</div> </div>
<!-- Action Buttons -->
<div class="row mb-4">
<div class="col-12">
<div class="action-buttons-section">
<div class="d-flex justify-content-end">
<button id="reset-regenerate-btn" class="btn btn-lg" onclick="resetAndRegenerate()"
title="Delete all generated files and run the CLI tool again to regenerate everything">
<i class="fas fa-sync-alt"></i> Reset & Regenerate
</button>
</div>
</div>
</div>
</div>
<!-- File Type Breakdown --> <!-- File Type Breakdown -->
<div class="row mb-4"> <div class="row mb-4">
<div class="col-md-4"> <div class="col-md-4">
@ -1351,6 +1462,124 @@
} }
} }
async function resetAndRegenerate() {
if (confirm('⚠️ WARNING: This will delete all generated files and run the CLI tool again.\n\nThis will:\n• Delete skipSongs.json\n• Delete all files in data/reports/\n• Delete all files in data/preferences/\n• Run the CLI tool to regenerate everything\n\nAre you sure you want to continue?')) {
try {
// Show progress modal
showProgressModal();
// Disable the button
const button = document.getElementById('reset-regenerate-btn');
button.disabled = true;
// Start the reset and regenerate process
const response = await fetch('/api/reset-and-regenerate', {
method: 'POST'
});
const result = await response.json();
if (result.success) {
// Start monitoring progress
startProgressMonitoring();
} else {
hideProgressModal();
alert('❌ Error: ' + result.error);
button.disabled = false;
}
} catch (error) {
console.error('Error during reset and regenerate:', error);
hideProgressModal();
alert('❌ Error during reset and regenerate: ' + error.message);
const button = document.getElementById('reset-regenerate-btn');
button.disabled = false;
}
}
}
function showProgressModal() {
const modal = document.getElementById('progressModal');
modal.style.display = 'block';
// Reset progress
document.getElementById('currentStep').textContent = 'Initializing...';
document.getElementById('progressBarFill').style.width = '0%';
document.getElementById('progressText').textContent = '0%';
document.getElementById('progressMessage').textContent = 'Starting process...';
document.getElementById('cliOutput').textContent = '';
}
function hideProgressModal() {
const modal = document.getElementById('progressModal');
modal.style.display = 'none';
}
function closeProgressModal() {
hideProgressModal();
// Re-enable the button
const button = document.getElementById('reset-regenerate-btn');
button.disabled = false;
}
function startProgressMonitoring() {
// Use polling for progress updates (more reliable than SSE)
const pollInterval = setInterval(async function() {
try {
const response = await fetch('/api/progress');
const data = await response.json();
updateProgress(data);
// If process is complete or error, stop polling
if (data.status === 'completed' || data.status === 'error') {
clearInterval(pollInterval);
if (data.status === 'completed') {
setTimeout(() => {
hideProgressModal();
alert('✅ Reset and regeneration completed successfully!\n\n' + data.message);
window.location.reload();
}, 2000);
} else {
setTimeout(() => {
hideProgressModal();
alert('❌ Error: ' + data.message);
const button = document.getElementById('reset-regenerate-btn');
button.disabled = false;
}, 2000);
}
}
} catch (error) {
console.error('Error polling progress:', error);
clearInterval(pollInterval);
hideProgressModal();
alert('❌ Error: Lost connection to progress updates');
const button = document.getElementById('reset-regenerate-btn');
button.disabled = false;
}
}, 1000); // Poll every second
}
function updateProgress(data) {
// Update progress bar
const progressBar = document.getElementById('progressBarFill');
const progressText = document.getElementById('progressText');
progressBar.style.width = data.progress + '%';
progressText.textContent = data.progress + '%';
// Update current step
document.getElementById('currentStep').textContent = data.current_step;
// Update message
document.getElementById('progressMessage').textContent = data.message;
// Update CLI output
const cliOutput = document.getElementById('cliOutput');
cliOutput.textContent = data.cli_output.join('\n');
cliOutput.scrollTop = cliOutput.scrollHeight; // Auto-scroll to bottom
}
// Video Player Functions // Video Player Functions
function normalizePath(filePath) { function normalizePath(filePath) {
// Debug logging to track path transformation - show original path first // Debug logging to track path transformation - show original path first
@ -1587,5 +1816,35 @@
</div> </div>
</div> </div>
</div> </div>
<!-- Progress Modal -->
<div id="progressModal" class="modal">
<div class="modal-content" style="max-width: 800px;">
<span class="modal-close" onclick="closeProgressModal()">&times;</span>
<h3><i class="fas fa-cog fa-spin"></i> Processing...</h3>
<div class="progress-container">
<div class="progress-step">
<span id="currentStep">Initializing...</span>
</div>
<div class="progress-bar-container">
<div class="progress-bar">
<div id="progressBarFill" class="progress-bar-fill" style="width: 0%"></div>
</div>
<span id="progressText">0%</span>
</div>
<div class="progress-message">
<span id="progressMessage">Starting process...</span>
</div>
</div>
<div class="cli-output-container">
<h4>CLI Output:</h4>
<div id="cliOutput" class="cli-output"></div>
</div>
</div>
</div>
</body> </body>
</html> </html>