Compare commits

...

1 Commits

Author SHA1 Message Date
bed46ff2d2 Signed-off-by: Matt Bruce <mbrucedogs@gmail.com> 2025-08-05 12:59:09 -05:00
13 changed files with 601 additions and 68 deletions

24
PRD.md
View File

@ -1,8 +1,8 @@
# 🎤 Karaoke Video Downloader PRD (v3.3)
# 🎤 Karaoke Video Downloader PRD (v3.5)
## ✅ Overview
A Python-based Windows CLI tool to download karaoke videos from YouTube channels/playlists using `yt-dlp.exe`, with advanced tracking, songlist prioritization, and flexible configuration. The codebase has been comprehensively refactored into a modular architecture with centralized utilities for improved maintainability, error handling, and code reuse.
A Python-based cross-platform CLI tool to download karaoke videos from YouTube channels/playlists using `yt-dlp`, with advanced tracking, songlist prioritization, and flexible configuration. Supports Windows, macOS, and Linux with automatic platform detection and optimized caching. The codebase has been comprehensively refactored into a modular architecture with centralized utilities for improved maintainability, error handling, and code reuse.
---
@ -63,9 +63,9 @@ The codebase has been refactored into focused modules with centralized utilities
---
## ⚙️ Platform & Stack
- **Platform:** Windows
- **Platform:** Windows, macOS, Linux
- **Interface:** Command-line (CLI)
- **Tech Stack:** Python 3.7+, yt-dlp.exe, mutagen (for ID3 tagging)
- **Tech Stack:** Python 3.7+, yt-dlp (platform-specific binary), mutagen (for ID3 tagging)
---
@ -159,7 +159,9 @@ KaroakeVideoDownloader/
├── downloads/ # All video output
│ └── [ChannelName]/ # Per-channel folders
├── logs/ # Download logs
├── downloader/yt-dlp.exe # yt-dlp binary
├── downloader/yt-dlp.exe # yt-dlp binary (Windows)
├── downloader/yt-dlp_macos # yt-dlp binary (macOS)
├── downloader/yt-dlp # yt-dlp binary (Linux)
├── tests/ # Diagnostic and test scripts
│ └── test_installation.py
├── download_karaoke.py # Main entry point (thin wrapper)
@ -260,6 +262,17 @@ The codebase has been comprehensively refactored to improve maintainability and
- **Performance improvements:** Significantly faster downloads for large batches (3-5x speedup with 3-5 workers)
- **Integrated with all modes:** Works with both songlist-across-channels and latest-per-channel download modes
### **Cross-Platform Support (v3.5)**
- **Platform detection:** Automatic detection of Windows, macOS, and Linux systems
- **Flexible yt-dlp integration:** Supports both binary files and pip-installed yt-dlp modules
- **Platform-specific configuration:** Automatic selection of appropriate yt-dlp binary/command for each platform
- **Setup automation:** `setup_platform.py` script for easy platform-specific setup
- **Command parsing:** Intelligent parsing of yt-dlp commands (file paths vs. module commands)
- **Enhanced documentation:** Platform-specific setup instructions and troubleshooting
- **Backward compatibility:** Maintains full compatibility with existing Windows installations
- **FFmpeg integration:** Automatic FFmpeg installation and configuration for optimal video processing
- **Optimized caching:** Enhanced channel video caching with format compatibility and instant video list loading
---
## 🚀 Future Enhancements
@ -268,6 +281,7 @@ The codebase has been comprehensively refactored to improve maintainability and
- [ ] Download scheduling and retry logic
- [ ] More granular status reporting
- [x] **Parallel downloads for improved speed** ✅ **COMPLETED**
- [x] **Cross-platform support (Windows, macOS, Linux)** ✅ **COMPLETED**
- [ ] Unit tests for all modules
- [ ] Integration tests for end-to-end workflows
- [ ] Plugin system for custom file operations

View File

@ -1,6 +1,6 @@
# 🎤 Karaoke Video Downloader
A Python-based Windows CLI tool to download karaoke videos from YouTube channels/playlists using `yt-dlp.exe`, with advanced tracking, songlist prioritization, and flexible configuration.
A Python-based cross-platform CLI tool to download karaoke videos from YouTube channels/playlists using `yt-dlp`, with advanced tracking, songlist prioritization, and flexible configuration. Supports Windows, macOS, and Linux with automatic platform detection, optimized caching, and FFmpeg integration.
## ✨ Features
- 🎵 **Channel & Playlist Downloads**: Download all videos from a YouTube channel or playlist
@ -21,6 +21,9 @@ A Python-based Windows CLI tool to download karaoke videos from YouTube channels
- ⚡ **Optimized Scanning**: High-performance channel scanning with O(n×m) complexity, pre-processed lookups, and early termination for faster matching
- 🏷️ **Server Duplicates Tracking**: Automatically checks against local songs.json file and marks duplicates for future skipping, preventing re-downloads of songs already on the server
- ⚡ **Parallel Downloads**: Enable concurrent downloads with `--parallel --workers N` for significantly faster batch downloads (3-5x speedup)
- 🌐 **Cross-Platform Support**: Automatic platform detection and yt-dlp integration for Windows, macOS, and Linux
- 🚀 **Optimized Caching**: Enhanced channel video caching with instant video list loading
- 🎬 **FFmpeg Integration**: Automatic FFmpeg installation and configuration for optimal video processing
## 🏗️ Architecture
The codebase has been comprehensively refactored into a modular architecture with centralized utilities for improved maintainability, error handling, and code reuse:
@ -80,13 +83,57 @@ The codebase has been comprehensively refactored into a modular architecture wit
- **Type Safety**: Comprehensive type hints across all new modules
## 📋 Requirements
- **Windows 10/11**
- **Windows 10/11, macOS 10.14+, or Linux**
- **Python 3.7+**
- **yt-dlp.exe** (in `downloader/`)
- **yt-dlp binary** (platform-specific, see setup instructions below)
- **mutagen** (for ID3 tagging, optional)
- **ffmpeg/ffprobe** (for video validation, optional but recommended)
- **rapidfuzz** (for fuzzy matching, optional, falls back to difflib)
## 🖥️ Platform Setup
### Automatic Setup (Recommended)
Run the platform setup script to automatically set up yt-dlp for your system:
```bash
python setup_platform.py
```
This script will:
- Detect your platform (Windows, macOS, or Linux)
- Offer two installation options:
1. **Download binary file** (recommended for most users)
2. **Install via pip** (alternative method)
- Make binaries executable (on Unix-like systems)
- Install FFmpeg (for optimal video processing)
- Test the installation
### Manual Setup
If you prefer to set up manually:
#### Option 1: Download Binary Files
1. **Windows**: Download `yt-dlp.exe` from [yt-dlp releases](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.exe)
2. **macOS**: Download `yt-dlp_macos` from [yt-dlp releases](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_macos)
3. **Linux**: Download `yt-dlp` from [yt-dlp releases](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp)
Place the downloaded file in the `downloader/` directory and make it executable on Unix-like systems:
```bash
chmod +x downloader/yt-dlp_macos # macOS
chmod +x downloader/yt-dlp # Linux
```
#### Option 2: Install via pip
```bash
pip install yt-dlp
```
The tool will automatically detect and use the pip-installed version on macOS.
**Note**: FFmpeg is also required for optimal video processing. The setup script will attempt to install it automatically, or you can install it manually:
- **macOS**: `brew install ffmpeg`
- **Linux**: `sudo apt install ffmpeg` (Ubuntu/Debian) or `sudo yum install ffmpeg` (CentOS/RHEL)
- **Windows**: Download from [ffmpeg.org](https://ffmpeg.org/download.html)
## 🚀 Quick Start
> **💡 Pro Tip**: For a complete list of all available commands, see `commands.txt` - you can copy/paste any command directly into your terminal!
@ -230,7 +277,11 @@ KaroakeVideoDownloader/
├── downloads/ # All video output
│ └── [ChannelName]/ # Per-channel folders
├── logs/ # Download logs
├── downloader/yt-dlp.exe # yt-dlp binary
├── downloader/yt-dlp.exe # yt-dlp binary (Windows)
├── downloader/yt-dlp_macos # yt-dlp binary (macOS)
├── downloader/yt-dlp # yt-dlp binary (Linux)
├── setup_platform.py # Platform setup script
├── test_platform.py # Platform test script
├── tests/ # Diagnostic and test scripts
│ └── test_installation.py
├── download_karaoke.py # Main entry point (thin wrapper)
@ -311,7 +362,7 @@ python download_karaoke.py --clear-server-duplicates
> **🔄 Maintenance Note**: The `commands.txt` file should be kept up to date with any CLI changes. When adding new command-line options or modifying existing ones, update this file to reflect all available commands and their usage.
## 🔧 Refactoring Improvements (v3.3)
## 🔧 Refactoring Improvements (v3.5)
The codebase has been comprehensively refactored to improve maintainability and reduce code duplication. Recent improvements have enhanced reliability, performance, and code organization:
### **New Utility Modules (v3.3)**
@ -346,6 +397,17 @@ The codebase has been comprehensively refactored to improve maintainability and
- **Improved Testability**: Modular components can be tested independently
- **Better Developer Experience**: Clear function signatures and comprehensive documentation
### **Cross-Platform Support (v3.5)**
- **Platform detection:** Automatic detection of Windows, macOS, and Linux systems
- **Flexible yt-dlp integration:** Supports both binary files and pip-installed yt-dlp modules
- **Platform-specific configuration:** Automatic selection of appropriate yt-dlp binary/command for each platform
- **Setup automation:** `setup_platform.py` script for easy platform-specific setup
- **Command parsing:** Intelligent parsing of yt-dlp commands (file paths vs. module commands)
- **Enhanced documentation:** Platform-specific setup instructions and troubleshooting
- **Backward compatibility:** Maintains full compatibility with existing Windows installations
- **FFmpeg integration:** Automatic FFmpeg installation and configuration for optimal video processing
- **Optimized caching:** Enhanced channel video caching with format compatibility and instant video list loading
### **New Parallel Download System (v3.4)**
- **Parallel downloader module:** `parallel_downloader.py` provides thread-safe concurrent download management
- **Configurable concurrency:** Use `--parallel --workers N` to enable parallel downloads with N workers (1-10)
@ -372,7 +434,11 @@ The codebase has been comprehensively refactored to improve maintainability and
- **Robust download plan execution:** Fixed index management in download plan execution to prevent errors during interrupted downloads.
## 🐞 Troubleshooting
- Ensure `yt-dlp.exe` is in the `downloader/` folder
- **Platform-specific yt-dlp setup**:
- **Windows**: Ensure `yt-dlp.exe` is in the `downloader/` folder
- **macOS**: Either ensure `yt-dlp_macos` is in the `downloader/` folder (make executable with `chmod +x`) OR install via pip (`pip install yt-dlp`)
- **Linux**: Ensure `yt-dlp` is in the `downloader/` folder (make executable with `chmod +x`)
- Run `python setup_platform.py` to automatically set up yt-dlp for your platform
- Check `logs/` for error details
- Use `python -m karaoke_downloader.check_resolution` to verify video quality
- If you see errors about ffmpeg/ffprobe, install [ffmpeg](https://ffmpeg.org/download.html) and ensure it is in your PATH

View File

@ -1,6 +1,6 @@
# 🎤 Karaoke Video Downloader - CLI Commands Reference
# Copy and paste these commands into your terminal
# Updated: v3.4 (includes parallel downloads and all refactoring improvements)
# Updated: v3.5 (includes cross-platform support, optimized caching, and all refactoring improvements)
## 📥 BASIC DOWNLOADS
@ -197,11 +197,27 @@ python download_karaoke.py --reset-channel SingKingKaraoke --reset-songlist
python download_karaoke.py --status
python download_karaoke.py --clear-cache all
## 🌐 PLATFORM SETUP COMMANDS (v3.5)
# Automatic platform setup (detects OS and installs yt-dlp + FFmpeg)
python setup_platform.py
# Test platform detection and yt-dlp integration
python test_platform.py
# Manual platform-specific setup
# Windows: Download yt-dlp.exe to downloader/ folder
# macOS: brew install ffmpeg && pip install yt-dlp
# Linux: sudo apt install ffmpeg && download yt-dlp to downloader/ folder
## 🔧 TROUBLESHOOTING COMMANDS
# Check if everything is working
python download_karaoke.py --version
# Test platform setup
python test_platform.py
# Force refresh everything
python download_karaoke.py --force-download-plan --refresh --clear-cache all
@ -233,6 +249,8 @@ python download_karaoke.py --clear-server-duplicates
# - Use --fuzzy-match for better song discovery
# - Use --refresh sparingly (forces re-scan)
# - Clear cache if you encounter issues
# - Channel caching provides instant video list loading (no YouTube API calls)
# - FFmpeg integration ensures optimal video processing and merging
# Parallel download tips:
# - Start with --workers 3 for conservative approach

View File

@ -34,5 +34,13 @@
"include_console": true,
"include_file": true
},
"platform_settings": {
"auto_detect_platform": true,
"yt_dlp_paths": {
"windows": "downloader/yt-dlp.exe",
"macos": "python3 -m yt_dlp",
"linux": "downloader/yt-dlp"
}
},
"yt_dlp_path": "downloader/yt-dlp.exe"
}

BIN
downloader/yt-dlp Normal file

Binary file not shown.

BIN
downloader/yt-dlp_macos Executable file

Binary file not shown.

View File

@ -177,11 +177,37 @@ Examples:
print("❌ Error: --workers must be between 1 and 10")
sys.exit(1)
yt_dlp_path = Path("downloader/yt-dlp.exe")
if not yt_dlp_path.exists():
print("❌ Error: yt-dlp.exe not found in downloader/ directory")
print("Please ensure yt-dlp.exe is present in the downloader/ folder")
sys.exit(1)
# Load configuration to get platform-aware yt-dlp path
from karaoke_downloader.config_manager import load_config
config = load_config()
yt_dlp_path = config.yt_dlp_path
# Check if it's a command string (like "python3 -m yt_dlp") or a file path
if yt_dlp_path.startswith(('python', 'python3')):
# It's a command string, test if it works
try:
import subprocess
from karaoke_downloader.youtube_utils import _parse_yt_dlp_command
cmd = _parse_yt_dlp_command(yt_dlp_path) + ["--version"]
result = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
if result.returncode != 0:
raise Exception(f"Command failed: {result.stderr}")
except Exception as e:
platform_name = "macOS" if sys.platform == "darwin" else "Windows" if sys.platform == "win32" else "Linux"
print(f"❌ Error: yt-dlp command failed: {yt_dlp_path}")
print(f"Please ensure yt-dlp is properly installed for {platform_name}")
print(f"Error: {e}")
sys.exit(1)
else:
# It's a file path, check if it exists
yt_dlp_file = Path(yt_dlp_path)
if not yt_dlp_file.exists():
platform_name = "macOS" if sys.platform == "darwin" else "Windows" if sys.platform == "win32" else "Linux"
binary_name = yt_dlp_file.name
print(f"❌ Error: {binary_name} not found in downloader/ directory")
print(f"Please ensure {binary_name} is present in the downloader/ folder for {platform_name}")
print(f"Expected path: {yt_dlp_file}")
sys.exit(1)
downloader = KaraokeDownloader()

View File

@ -4,6 +4,8 @@ Provides centralized configuration loading, validation, and management.
"""
import json
import platform
import sys
from dataclasses import dataclass, field
from datetime import datetime
from pathlib import Path
@ -42,6 +44,14 @@ DEFAULT_CONFIG = {
"include_console": True,
"include_file": True,
},
"platform_settings": {
"auto_detect_platform": True,
"yt_dlp_paths": {
"windows": "downloader/yt-dlp.exe",
"macos": "downloader/yt-dlp_macos",
"linux": "downloader/yt-dlp",
},
},
"yt_dlp_path": "downloader/yt-dlp.exe",
}
@ -55,6 +65,26 @@ RESOLUTION_MAP = {
}
def detect_platform() -> str:
"""Detect the current platform and return the appropriate platform key."""
system = platform.system().lower()
if system == "windows":
return "windows"
elif system == "darwin":
return "macos"
elif system == "linux":
return "linux"
else:
# Default to windows for unknown platforms
return "windows"
def get_platform_yt_dlp_path(platform_paths: Dict[str, str]) -> str:
"""Get the appropriate yt-dlp path for the current platform."""
platform_key = detect_platform()
return platform_paths.get(platform_key, platform_paths.get("windows", "downloader/yt-dlp.exe"))
@dataclass
class DownloadSettings:
"""Configuration for download settings."""
@ -234,11 +264,21 @@ class ConfigManager:
folder_structure = FolderStructure(**config_data.get("folder_structure", {}))
logging_config = LoggingConfig(**config_data.get("logging", {}))
# Handle platform-specific yt-dlp path
yt_dlp_path = config_data.get("yt_dlp_path", "downloader/yt-dlp.exe")
# Check if platform auto-detection is enabled
platform_settings = config_data.get("platform_settings", {})
if platform_settings.get("auto_detect_platform", True):
platform_paths = platform_settings.get("yt_dlp_paths", {})
if platform_paths:
yt_dlp_path = get_platform_yt_dlp_path(platform_paths)
return AppConfig(
download_settings=download_settings,
folder_structure=folder_structure,
logging=logging_config,
yt_dlp_path=config_data.get("yt_dlp_path", "downloader/yt-dlp.exe"),
yt_dlp_path=yt_dlp_path,
_config_file=self.config_file,
)

View File

@ -78,7 +78,7 @@ class KaraokeDownloader:
self.config = self.config_manager.load_config()
# Initialize paths
self.yt_dlp_path = Path(self.config.yt_dlp_path)
self.yt_dlp_path = self.config.yt_dlp_path # Keep as string for command parsing
self.downloads_dir = Path(self.config.folder_structure.downloads_dir)
self.logs_dir = Path(self.config.folder_structure.logs_dir)
@ -191,25 +191,11 @@ class KaraokeDownloader:
server_duplicates_tracking = load_server_duplicates_tracking()
limit = getattr(self.config, "limit", 1)
cmd = [
str(self.yt_dlp_path),
"--flat-playlist",
"--print",
"%(title)s|%(id)s|%(url)s",
url,
]
try:
result = subprocess.run(cmd, capture_output=True, text=True, check=True)
lines = result.stdout.strip().splitlines()
except subprocess.CalledProcessError as e:
print(f"❌ yt-dlp failed to fetch playlist: {e}")
return False
available_videos = []
for line in lines:
parts = line.split("|")
if len(parts) >= 2:
title, video_id = parts[0].strip(), parts[1].strip()
available_videos.append({"title": title, "id": video_id})
# Use tracking manager's cache instead of calling yt-dlp directly
available_videos = self.tracker.get_channel_video_list(
url, yt_dlp_path=self.yt_dlp_path, force_refresh=force_refresh
)
# Normalize songlist for matching
normalized_songlist = {
create_song_key(s["artist"], s["title"]): s for s in songlist
@ -282,7 +268,7 @@ class KaraokeDownloader:
return True
# Download only the first N matches using the new pipeline
pipeline = DownloadPipeline(
yt_dlp_path=str(self.yt_dlp_path),
yt_dlp_path=self.yt_dlp_path,
config=self.config,
downloads_dir=self.downloads_dir,
songlist_tracking=self.songlist_tracking,
@ -538,7 +524,7 @@ class KaraokeDownloader:
# Create parallel downloader
parallel_downloader = create_parallel_downloader(
yt_dlp_path=str(self.yt_dlp_path),
yt_dlp_path=self.yt_dlp_path,
config=self.config,
downloads_dir=self.downloads_dir,
max_workers=self.parallel_workers,
@ -620,7 +606,7 @@ class KaraokeDownloader:
# Create parallel downloader
parallel_downloader = create_parallel_downloader(
yt_dlp_path=str(self.yt_dlp_path),
yt_dlp_path=self.yt_dlp_path,
config=self.config,
downloads_dir=self.downloads_dir,
max_workers=self.parallel_workers,
@ -769,7 +755,7 @@ class KaraokeDownloader:
# Use the new pipeline for consistent processing
pipeline = DownloadPipeline(
yt_dlp_path=str(self.yt_dlp_path),
yt_dlp_path=self.yt_dlp_path,
config=self.config,
downloads_dir=self.downloads_dir,
songlist_tracking=self.songlist_tracking,
@ -874,7 +860,7 @@ class KaraokeDownloader:
channel_name, channel_id = get_channel_info(channel_url)
print(f"\n🚦 Starting channel: {channel_name} ({channel_url})")
available_videos = self.tracker.get_channel_video_list(
channel_url, yt_dlp_path=str(self.yt_dlp_path), force_refresh=False
channel_url, yt_dlp_path=self.yt_dlp_path, force_refresh=False
)
print(
f" → Found {len(available_videos)} total videos for this channel."

View File

@ -56,6 +56,14 @@ def update_resolution(resolution):
"include_console": True,
"include_file": True,
},
"platform_settings": {
"auto_detect_platform": True,
"yt_dlp_paths": {
"windows": "downloader/yt-dlp.exe",
"macos": "downloader/yt-dlp_macos",
"linux": "downloader/yt-dlp",
},
},
"yt_dlp_path": "downloader/yt-dlp.exe",
}

View File

@ -283,28 +283,54 @@ class TrackingManager:
self._save()
def get_channel_video_list(
self, channel_url, yt_dlp_path="downloader/yt-dlp.exe", force_refresh=False
self, channel_url, yt_dlp_path=None, force_refresh=False
):
"""
Return a list of videos (dicts with 'title' and 'id') for the channel, using cache if available unless force_refresh is True.
"""
# Use platform-aware path if none provided
if yt_dlp_path is None:
from karaoke_downloader.config_manager import load_config
config = load_config()
yt_dlp_path = config.yt_dlp_path
channel_name, channel_id = None, None
from karaoke_downloader.youtube_utils import get_channel_info
channel_name, channel_id = get_channel_info(channel_url)
# Try multiple possible cache keys
possible_keys = [
channel_id, # The extracted channel ID
channel_url, # The full URL
channel_name, # The extracted channel name
]
# Check if cache has the old flat structure or new nested structure
cache_data = None
cache_key = None
for key in possible_keys:
if key and key in self.cache:
cache_key = key
break
# Try nested structure first (new format)
if "channels" in self.cache:
# Try multiple possible cache keys in nested structure
possible_keys = [
channel_id, # The extracted channel ID
channel_url, # The full URL
channel_name, # The extracted channel name
]
for key in possible_keys:
if key and key in self.cache["channels"]:
cache_data = self.cache["channels"][key]["videos"]
cache_key = key
break
# Try flat structure (old format) as fallback
if cache_data is None:
possible_keys = [
channel_id, # The extracted channel ID
channel_url, # The full URL
channel_name, # The extracted channel name
]
for key in possible_keys:
if key and key in self.cache:
cache_data = self.cache[key]
cache_key = key
break
if not cache_key:
cache_key = channel_id or channel_url # Use as fallback for new entries
@ -312,19 +338,31 @@ class TrackingManager:
print(f" 🔍 Trying cache keys: {possible_keys}")
print(f" 🔍 Selected cache key: '{cache_key}'")
if not force_refresh and cache_key in self.cache:
if not force_refresh and cache_data is not None:
print(
f" 📋 Using cached video list ({len(self.cache[cache_key])} videos)"
f" 📋 Using cached video list ({len(cache_data)} videos)"
)
return self.cache[cache_key]
# Convert old cache format to new format if needed
converted_videos = []
for video in cache_data:
if "video_id" in video and "id" not in video:
# Convert old format to new format
converted_videos.append({
"title": video["title"],
"id": video["video_id"]
})
else:
# Already in new format
converted_videos.append(video)
return converted_videos
else:
print(f" ❌ Cache miss for all keys")
# Fetch with yt-dlp
print(f" 🌐 Fetching video list from YouTube (this may take a while)...")
import subprocess
from karaoke_downloader.youtube_utils import _parse_yt_dlp_command
cmd = [
yt_dlp_path,
cmd = _parse_yt_dlp_command(yt_dlp_path) + [
"--flat-playlist",
"--print",
"%(title)s|%(id)s|%(url)s",
@ -339,7 +377,18 @@ class TrackingManager:
if len(parts) >= 2:
title, video_id = parts[0].strip(), parts[1].strip()
videos.append({"title": title, "id": video_id})
self.cache[cache_key] = videos
# Save in nested structure format
if "channels" not in self.cache:
self.cache["channels"] = {}
self.cache["channels"][cache_key] = {
"videos": videos,
"last_updated": datetime.now().isoformat(),
"channel_name": channel_name,
"channel_id": channel_id
}
self.save_cache()
return videos
except subprocess.CalledProcessError as e:

View File

@ -9,10 +9,29 @@ from typing import Any, Dict, List, Optional, Union
from karaoke_downloader.config_manager import AppConfig
def _parse_yt_dlp_command(yt_dlp_path: str) -> List[str]:
"""
Parse yt-dlp path/command into a list of command arguments.
Handles both file paths and command strings like 'python3 -m yt_dlp'.
"""
if yt_dlp_path.startswith(('python', 'python3')):
# It's a Python module command
return yt_dlp_path.split()
else:
# It's a file path
return [yt_dlp_path]
def get_channel_info(
channel_url: str, yt_dlp_path: str = "downloader/yt-dlp.exe"
channel_url: str, yt_dlp_path: str = None
) -> tuple[str, str]:
"""Get channel information using yt-dlp. Returns (channel_name, channel_id)."""
# Use platform-aware path if none provided
if yt_dlp_path is None:
from karaoke_downloader.config_manager import load_config
config = load_config()
yt_dlp_path = config.yt_dlp_path
try:
# Extract channel name from URL for now (faster than calling yt-dlp)
if "/@" in channel_url:
@ -39,11 +58,17 @@ def get_channel_info(
def get_playlist_info(
playlist_url: str, yt_dlp_path: str = "downloader/yt-dlp.exe"
playlist_url: str, yt_dlp_path: str = None
) -> List[Dict[str, Any]]:
"""Get playlist information using yt-dlp."""
# Use platform-aware path if none provided
if yt_dlp_path is None:
from karaoke_downloader.config_manager import load_config
config = load_config()
yt_dlp_path = config.yt_dlp_path
try:
cmd = [yt_dlp_path, "--dump-json", "--flat-playlist", playlist_url]
cmd = _parse_yt_dlp_command(yt_dlp_path) + ["--dump-json", "--flat-playlist", playlist_url]
result = subprocess.run(cmd, capture_output=True, text=True, check=True)
videos = []
for line in result.stdout.strip().split("\n"):
@ -75,8 +100,7 @@ def build_yt_dlp_command(
Returns:
List of command arguments for subprocess.run
"""
cmd = [
str(yt_dlp_path),
cmd = _parse_yt_dlp_command(str(yt_dlp_path)) + [
"--no-check-certificates",
"--ignore-errors",
"--no-warnings",
@ -117,7 +141,7 @@ def execute_yt_dlp_command(
def show_available_formats(
video_url: str, yt_dlp_path: str = "downloader/yt-dlp.exe", timeout: int = 30
video_url: str, yt_dlp_path: str = None, timeout: int = 30
) -> None:
"""
Show available formats for a video (debugging utility).
@ -127,8 +151,14 @@ def show_available_formats(
yt_dlp_path: Path to yt-dlp executable
timeout: Timeout in seconds
"""
# Use platform-aware path if none provided
if yt_dlp_path is None:
from karaoke_downloader.config_manager import load_config
config = load_config()
yt_dlp_path = config.yt_dlp_path
print(f"🔍 Checking available formats for: {video_url}")
format_cmd = [str(yt_dlp_path), "--list-formats", video_url]
format_cmd = _parse_yt_dlp_command(str(yt_dlp_path)) + ["--list-formats", video_url]
try:
format_result = subprocess.run(
format_cmd, capture_output=True, text=True, timeout=timeout

288
setup_platform.py Normal file
View File

@ -0,0 +1,288 @@
#!/usr/bin/env python3
"""
Platform setup script for Karaoke Video Downloader.
This script helps users download the correct yt-dlp binary for their platform.
"""
import os
import platform
import sys
import urllib.request
import zipfile
import tarfile
from pathlib import Path
def detect_platform():
"""Detect the current platform and return platform info."""
system = platform.system().lower()
machine = platform.machine().lower()
if system == "windows":
return "windows", "yt-dlp.exe"
elif system == "darwin":
return "macos", "yt-dlp_macos"
elif system == "linux":
return "linux", "yt-dlp"
else:
return "unknown", "yt-dlp"
def get_download_url(platform_name):
"""Get the download URL for yt-dlp based on platform."""
base_url = "https://github.com/yt-dlp/yt-dlp/releases/latest/download"
if platform_name == "windows":
return f"{base_url}/yt-dlp.exe"
elif platform_name == "macos":
return f"{base_url}/yt-dlp_macos"
elif platform_name == "linux":
return f"{base_url}/yt-dlp"
else:
raise ValueError(f"Unsupported platform: {platform_name}")
def install_via_pip():
"""Install yt-dlp via pip."""
print("📦 Installing yt-dlp via pip...")
try:
import subprocess
result = subprocess.run([sys.executable, "-m", "pip", "install", "yt-dlp"],
capture_output=True, text=True, check=True)
print("✅ yt-dlp installed successfully via pip!")
return True
except subprocess.CalledProcessError as e:
print(f"❌ Failed to install yt-dlp via pip: {e}")
return False
def check_ffmpeg():
"""Check if FFmpeg is installed and available."""
try:
import subprocess
result = subprocess.run(["ffmpeg", "-version"], capture_output=True, text=True, timeout=10)
return result.returncode == 0
except (subprocess.TimeoutExpired, FileNotFoundError):
return False
def install_ffmpeg():
"""Install FFmpeg based on platform."""
import subprocess
platform_name, _ = detect_platform()
print("🎬 Installing FFmpeg...")
if platform_name == "macos":
# Try using Homebrew first
try:
print("🍺 Attempting to install FFmpeg via Homebrew...")
result = subprocess.run(["brew", "install", "ffmpeg"],
capture_output=True, text=True, check=True)
print("✅ FFmpeg installed successfully via Homebrew!")
return True
except (subprocess.CalledProcessError, FileNotFoundError):
print("⚠️ Homebrew not found or failed. Trying alternative methods...")
# Try using MacPorts
try:
print("🍎 Attempting to install FFmpeg via MacPorts...")
result = subprocess.run(["sudo", "port", "install", "ffmpeg"],
capture_output=True, text=True, check=True)
print("✅ FFmpeg installed successfully via MacPorts!")
return True
except (subprocess.CalledProcessError, FileNotFoundError):
print("❌ Could not install FFmpeg automatically.")
print("Please install FFmpeg manually:")
print("1. Install Homebrew: https://brew.sh/")
print("2. Run: brew install ffmpeg")
print("3. Or download from: https://ffmpeg.org/download.html")
return False
elif platform_name == "linux":
try:
print("🐧 Attempting to install FFmpeg via package manager...")
# Try apt (Ubuntu/Debian)
try:
result = subprocess.run(["sudo", "apt", "update"], capture_output=True, text=True, check=True)
result = subprocess.run(["sudo", "apt", "install", "-y", "ffmpeg"],
capture_output=True, text=True, check=True)
print("✅ FFmpeg installed successfully via apt!")
return True
except subprocess.CalledProcessError:
# Try yum (CentOS/RHEL)
try:
result = subprocess.run(["sudo", "yum", "install", "-y", "ffmpeg"],
capture_output=True, text=True, check=True)
print("✅ FFmpeg installed successfully via yum!")
return True
except subprocess.CalledProcessError:
print("❌ Could not install FFmpeg automatically.")
print("Please install FFmpeg manually for your Linux distribution.")
return False
except FileNotFoundError:
print("❌ Could not install FFmpeg automatically.")
print("Please install FFmpeg manually for your Linux distribution.")
return False
elif platform_name == "windows":
print("❌ FFmpeg installation not automated for Windows.")
print("Please install FFmpeg manually:")
print("1. Download from: https://ffmpeg.org/download.html")
print("2. Extract to a folder and add to PATH")
print("3. Or use Chocolatey: choco install ffmpeg")
return False
return False
def download_file(url, destination):
"""Download a file from URL to destination."""
print(f"📥 Downloading from: {url}")
print(f"📁 Saving to: {destination}")
try:
urllib.request.urlretrieve(url, destination)
print("✅ Download completed successfully!")
return True
except Exception as e:
print(f"❌ Download failed: {e}")
return False
def make_executable(file_path):
"""Make a file executable (for Unix-like systems)."""
try:
os.chmod(file_path, 0o755)
print(f"🔧 Made {file_path} executable")
except Exception as e:
print(f"⚠️ Could not make file executable: {e}")
def main():
print("🎤 Karaoke Video Downloader - Platform Setup")
print("=" * 50)
# Detect platform
platform_name, binary_name = detect_platform()
print(f"🖥️ Detected platform: {platform_name}")
print(f"📦 Binary name: {binary_name}")
# Create downloader directory if it doesn't exist
downloader_dir = Path("downloader")
downloader_dir.mkdir(exist_ok=True)
# Check if binary already exists
binary_path = downloader_dir / binary_name
if binary_path.exists():
print(f"{binary_name} already exists in downloader/ directory")
response = input("Do you want to re-download it? (y/N): ").strip().lower()
if response != 'y':
print("Setup completed!")
return
# Offer installation options
print(f"\n🔧 Installation options for {platform_name}:")
print("1. Download binary file (recommended for most users)")
print("2. Install via pip (alternative method)")
choice = input("Choose installation method (1 or 2): ").strip()
if choice == "2":
# Install via pip
if install_via_pip():
print(f"\n✅ yt-dlp installed successfully!")
# Test the installation
print(f"\n🧪 Testing yt-dlp installation...")
try:
import subprocess
result = subprocess.run([sys.executable, "-m", "yt_dlp", "--version"],
capture_output=True, text=True, timeout=10)
if result.returncode == 0:
version = result.stdout.strip()
print(f"✅ yt-dlp is working! Version: {version}")
else:
print(f"⚠️ yt-dlp test failed: {result.stderr}")
except Exception as e:
print(f"⚠️ Could not test yt-dlp: {e}")
# Check and install FFmpeg
print(f"\n🎬 Checking FFmpeg installation...")
if check_ffmpeg():
print(f"✅ FFmpeg is already installed and working!")
else:
print(f"⚠️ FFmpeg not found. Installing...")
if install_ffmpeg():
print(f"✅ FFmpeg installed successfully!")
else:
print(f"⚠️ FFmpeg installation failed. The tool will still work but may be slower.")
print(f"\n🎉 Setup completed successfully!")
print(f"📦 yt-dlp installed via pip")
print(f"🖥️ Platform: {platform_name}")
print(f"\n🎉 You're ready to use the Karaoke Video Downloader!")
print(f"Run: python download_karaoke.py --help")
return
else:
print("❌ Pip installation failed. Trying binary download...")
# Download binary file
try:
download_url = get_download_url(platform_name)
except ValueError as e:
print(f"{e}")
print("Please manually download yt-dlp for your platform from:")
print("https://github.com/yt-dlp/yt-dlp/releases/latest")
return
# Download the binary
print(f"\n🚀 Downloading yt-dlp for {platform_name}...")
if download_file(download_url, binary_path):
# Make executable on Unix-like systems
if platform_name in ["macos", "linux"]:
make_executable(binary_path)
print(f"\n✅ yt-dlp binary downloaded successfully!")
print(f"📁 yt-dlp binary location: {binary_path}")
print(f"🖥️ Platform: {platform_name}")
# Test the binary
print(f"\n🧪 Testing yt-dlp installation...")
try:
import subprocess
result = subprocess.run([str(binary_path), "--version"],
capture_output=True, text=True, timeout=10)
if result.returncode == 0:
version = result.stdout.strip()
print(f"✅ yt-dlp is working! Version: {version}")
else:
print(f"⚠️ yt-dlp test failed: {result.stderr}")
except Exception as e:
print(f"⚠️ Could not test yt-dlp: {e}")
# Check and install FFmpeg
print(f"\n🎬 Checking FFmpeg installation...")
if check_ffmpeg():
print(f"✅ FFmpeg is already installed and working!")
else:
print(f"⚠️ FFmpeg not found. Installing...")
if install_ffmpeg():
print(f"✅ FFmpeg installed successfully!")
else:
print(f"⚠️ FFmpeg installation failed. The tool will still work but may be slower.")
print(f"\n🎉 Setup completed successfully!")
print(f"📁 yt-dlp binary location: {binary_path}")
print(f"🖥️ Platform: {platform_name}")
print(f"\n🎉 You're ready to use the Karaoke Video Downloader!")
print(f"Run: python download_karaoke.py --help")
else:
print(f"\n❌ Setup failed. Please manually download yt-dlp for {platform_name}")
print(f"Download URL: {download_url}")
print(f"Save to: {binary_path}")
if __name__ == "__main__":
main()