diff --git a/PODCAST_ARCHITECTURE.md b/PODCAST_ARCHITECTURE.md new file mode 100644 index 0000000..dc5bc7d --- /dev/null +++ b/PODCAST_ARCHITECTURE.md @@ -0,0 +1,97 @@ +# Podcast Architecture & Implementation Plan + +## Overview +Convert Daily Digest blog posts into a podcast format with automated TTS generation, RSS feed for distribution, and Supabase Storage for audio file hosting. + +## Architecture + +### 1. Database Schema Updates +Add `audio_url` and `audio_duration` fields to the `blog_messages` table. + +### 2. TTS Generation +**Option A: Piper TTS (Recommended - Free)** +- Local execution, no API costs +- High quality neural voices +- Fast processing +- No rate limits + +**Option B: OpenAI TTS (Paid)** +- Premium quality voices +- Simple API integration +- ~$2-4/month for daily 5-min content + +### 3. Audio Storage +- **Provider**: Supabase Storage +- **Bucket**: `podcast-audio` +- **Cost**: Free tier includes 1GB storage +- **Access**: Public read via signed URLs + +### 4. RSS Feed Generation +- **Endpoint**: `/api/podcast/rss` +- **Format**: RSS 2.0 with iTunes extensions +- **Compatible with**: Apple Podcasts, Spotify, Google Podcasts +- **Auto-updates**: Pulls from blog_messages table + +### 5. Integration Points +1. **Daily Digest Workflow** (`/api/digest` POST): + - After saving post, trigger async TTS generation + - Upload audio to Supabase Storage + - Update database with audio_url + +2. **RSS Feed** (`/api/podcast/rss`): + - Returns XML RSS feed + - Includes all posts with audio_url + +3. **Podcast Page** (`/podcast`): + - Web player for each episode + - Subscribe links + - Episode list + +## Implementation Steps + +### Phase 1: Database & Storage Setup +1. Create `podcast-audio` bucket in Supabase +2. Add columns to blog_messages table + +### Phase 2: TTS Service +1. Create `src/lib/tts.ts` - TTS abstraction +2. Create `src/lib/storage.ts` - Supabase storage helpers +3. Create `src/scripts/generate-tts.ts` - TTS generation script + +### Phase 3: API Endpoints +1. Create `src/app/api/podcast/rss/route.ts` - RSS feed +2. Update `src/app/api/digest/route.ts` - Add TTS trigger + +### Phase 4: UI +1. Create `src/app/podcast/page.tsx` - Podcast page +2. Update post display to show audio player + +### Phase 5: Documentation +1. Create `PODCAST_SETUP.md` - Setup instructions +2. Update README with podcast features + +## Cost Estimate +- **Piper TTS**: $0 (local processing) +- **OpenAI TTS**: ~$2-4/month +- **Supabase Storage**: $0 (within free tier 1GB) +- **RSS Hosting**: $0 (generated by Next.js API) +- **Total**: $0 (Piper) or $2-4/month (OpenAI) + +## File Structure +``` +src/ +├── app/ +│ ├── api/ +│ │ ├── digest/route.ts (updated) +│ │ └── podcast/ +│ │ └── rss/route.ts (new) +│ ├── podcast/ +│ │ └── page.tsx (new) +│ └── page.tsx (updated - add audio player) +├── lib/ +│ ├── tts.ts (new) +│ ├── storage.ts (new) +│ └── podcast.ts (new) +└── scripts/ + └── generate-tts.ts (new) +``` diff --git a/PODCAST_SETUP.md b/PODCAST_SETUP.md new file mode 100644 index 0000000..2442308 --- /dev/null +++ b/PODCAST_SETUP.md @@ -0,0 +1,318 @@ +# Podcast Setup Guide + +This guide covers setting up and using the podcast feature for the Daily Digest blog. + +## Overview + +The podcast feature automatically converts Daily Digest blog posts into audio format using Text-to-Speech (TTS) and provides: + +- 🎧 **Web Player** - Listen directly on the blog +- 📱 **RSS Feed** - Subscribe in any podcast app (Apple Podcasts, Spotify, etc.) +- 🔄 **Auto-generation** - TTS runs automatically when new posts are created +- 💾 **Audio Storage** - Files stored in Supabase Storage (free tier) + +## Architecture + +``` +┌─────────────┐ ┌─────────────┐ ┌─────────────┐ +│ Daily │────▶│ TTS API │────▶│ Supabase │ +│ Digest │ │ (OpenAI/ │ │ Storage │ +│ Post │ │ Piper) │ │ │ +└─────────────┘ └─────────────┘ └──────┬──────┘ + │ + ▼ + ┌─────────────┐ + │ RSS Feed │ + │ (/api/ │ + │ podcast/ │ + │ rss) │ + └──────┬──────┘ + │ + ▼ + ┌─────────────┐ + │ Podcast │ + │ Apps │ + │ (Apple, │ + │ Spotify) │ + └─────────────┘ +``` + +## Quick Start + +### 1. Configure Environment Variables + +Add to your `.env.local` and `.env.production`: + +```bash +# Enable TTS generation +ENABLE_TTS=true + +# Choose TTS Provider: "openai" (paid, best quality) or "macsay" (free, macOS only) +TTS_PROVIDER=openai + +# For OpenAI TTS (recommended) +OPENAI_API_KEY=sk-your-key-here +TTS_VOICE=alloy # Options: alloy, echo, fable, onyx, nova, shimmer +``` + +### 2. Update Database Schema + +Run this SQL in your Supabase SQL Editor to add audio columns: + +```sql +-- Add audio URL column +ALTER TABLE blog_messages +ADD COLUMN IF NOT EXISTS audio_url TEXT; + +-- Add audio duration column +ALTER TABLE blog_messages +ADD COLUMN IF NOT EXISTS audio_duration INTEGER; + +-- Create index for faster RSS feed queries +CREATE INDEX IF NOT EXISTS idx_blog_messages_audio +ON blog_messages(audio_url) +WHERE audio_url IS NOT NULL; +``` + +### 3. Create Supabase Storage Bucket + +The app will automatically create the `podcast-audio` bucket on first use, or you can create it manually: + +1. Go to Supabase Dashboard → Storage +2. Click "New Bucket" +3. Name: `podcast-audio` +4. Check "Public bucket" +5. Click "Create" + +### 4. Deploy + +```bash +npm run build +vercel --prod +``` + +### 5. Subscribe to the Podcast + +The RSS feed is available at: +``` +https://blog-backup-two.vercel.app/api/podcast/rss +``` + +**Apple Podcasts:** +1. Open Podcasts app +2. Tap Library → Edit → Add a Show by URL +3. Paste the RSS URL + +**Spotify:** +1. Go to Spotify for Podcasters +2. Submit RSS feed + +**Other Apps:** +Just paste the RSS URL into any podcast app. + +## TTS Providers + +### Option 1: OpenAI TTS (Recommended) + +**Cost:** ~$2-4/month for daily 5-minute episodes + +**Pros:** +- Excellent voice quality +- Multiple voices available +- Simple API integration +- Fast processing + +**Cons:** +- Paid service +- Requires API key + +**Setup:** +```bash +TTS_PROVIDER=openai +OPENAI_API_KEY=sk-your-key-here +TTS_VOICE=alloy # or echo, fable, onyx, nova, shimmer +``` + +### Option 2: macOS `say` Command (Free) + +**Cost:** $0 + +**Pros:** +- Free, built into macOS +- No API key needed +- Works offline + +**Cons:** +- Lower voice quality +- macOS only +- Limited to macOS deployment + +**Setup:** +```bash +TTS_PROVIDER=macsay +TTS_VOICE=Samantha # or Alex, Victoria, etc. +``` + +### Option 3: Piper TTS (Free, Local) + +**Cost:** $0 + +**Pros:** +- Free and open source +- High quality neural voices +- Runs locally (privacy) +- No rate limits + +**Cons:** +- Requires downloading voice models (~100MB) +- More complex setup +- Requires local execution + +**Setup:** +1. Install Piper: + ```bash + brew install piper-tts # or download from GitHub + ``` + +2. Download voice model: + ```bash + mkdir -p models + cd models + wget https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/lessac/medium/en_US-lessac-medium.onnx + wget https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_US/lessac/medium/en_US-lessac-medium.onnx.json + ``` + +3. Configure: + ```bash + TTS_PROVIDER=piper + PIPER_MODEL_PATH=./models/en_US-lessac-medium.onnx + ``` + +## Usage + +### Automatic Generation + +When `ENABLE_TTS=true`, audio is automatically generated when a new digest is posted via the `/api/digest` endpoint. + +### Manual Generation + +Generate audio for a specific post: +```bash +npm run generate-tts -- +``` + +Generate audio for all posts missing audio: +```bash +npm run generate-tts:all +``` + +Force regeneration (overwrite existing): +```bash +npm run generate-tts -- --force +``` + +## API Endpoints + +### GET /api/podcast/rss + +Returns the podcast RSS feed in XML format with iTunes extensions. + +**Headers:** +- `Accept: application/xml` + +**Response:** RSS 2.0 XML feed + +### POST /api/digest (Updated) + +Now accepts an optional `generateAudio` parameter: + +```json +{ + "date": "2026-02-23", + "content": "# Daily Digest\n\nToday's news...", + "tags": ["daily-digest", "ai"], + "generateAudio": true // Optional, defaults to true +} +``` + +## Database Schema + +The `blog_messages` table now includes: + +| Column | Type | Description | +|--------|------|-------------| +| `audio_url` | TEXT | Public URL to the audio file in Supabase Storage | +| `audio_duration` | INTEGER | Estimated duration in seconds | + +## File Structure + +``` +src/ +├── app/ +│ ├── api/ +│ │ ├── digest/route.ts # Updated with TTS trigger +│ │ └── podcast/ +│ │ └── rss/route.ts # RSS feed endpoint +│ ├── podcast/ +│ │ └── page.tsx # Podcast web player page +│ └── page.tsx # Updated with audio player +├── lib/ +│ ├── tts.ts # TTS service abstraction +│ ├── storage.ts # Supabase storage helpers +│ └── podcast.ts # RSS generation utilities +└── scripts/ + └── generate-tts.ts # Manual TTS generation script +``` + +## Cost Analysis + +### OpenAI TTS (Daily Digest ~5 min) +- Characters per day: ~4,000 +- Cost: $0.015 per 1,000 chars (tts-1) +- Monthly cost: ~$1.80 +- HD voice (tts-1-hd): ~$3.60/month + +### Supabase Storage +- Free tier: 1 GB storage +- Audio files: ~5 MB per episode +- Monthly storage: ~150 MB (30 episodes) +- Well within free tier + +### Total Monthly Cost +- **OpenAI TTS:** ~$2-4/month +- **Supabase Storage:** $0 (free tier) +- **RSS Hosting:** $0 (Next.js API route) +- **Total:** ~$2-4/month + +## Troubleshooting + +### TTS not generating +1. Check `ENABLE_TTS=true` in environment variables +2. Check `TTS_PROVIDER` is set correctly +3. For OpenAI: Verify `OPENAI_API_KEY` is valid +4. Check Vercel logs for errors + +### Audio not playing +1. Check Supabase Storage bucket is public +2. Verify `audio_url` in database is not null +3. Check browser console for CORS errors + +### RSS feed not updating +1. RSS is cached for 5 minutes (`max-age=300`) +2. Check that posts have `audio_url` set +3. Verify RSS URL is accessible: `/api/podcast/rss` + +## Future Enhancements + +- [ ] Background job queue for TTS generation (using Inngest/Upstash) +- [ ] Voice selection per post +- [ ] Chapter markers in audio +- [ ] Transcript generation +- [ ] Podcast analytics + +## Resources + +- [OpenAI TTS Documentation](https://platform.openai.com/docs/guides/text-to-speech) +- [Piper TTS GitHub](https://github.com/rhasspy/piper) +- [Apple Podcasts RSS Requirements](https://help.apple.com/itc/podcasts_connect/#/itcb54333f1) +- [Podcast RSS 2.0 Spec](https://cyber.harvard.edu/rss/rss.html) diff --git a/PODCAST_SUMMARY.md b/PODCAST_SUMMARY.md new file mode 100644 index 0000000..2848fe6 --- /dev/null +++ b/PODCAST_SUMMARY.md @@ -0,0 +1,126 @@ +# Podcast Implementation Summary + +## What Was Built + +A complete podcast solution for the Daily Digest blog that automatically converts blog posts to audio using Text-to-Speech (TTS) and distributes them via RSS feed. + +## Features Delivered + +### 1. TTS Service (`src/lib/tts.ts`) +- **Multi-provider support**: OpenAI (paid), Piper (free), macOS say (free) +- **Text preprocessing**: Automatically strips markdown, URLs, and code blocks +- **Async generation**: Non-blocking to not delay post creation +- **Error handling**: Graceful fallback if TTS fails + +### 2. Audio Storage (`src/lib/storage.ts`) +- **Supabase Storage integration**: Uses existing Supabase project +- **Automatic bucket creation**: Creates `podcast-audio` bucket if needed +- **Public access**: Audio files accessible for podcast apps + +### 3. RSS Feed (`src/app/api/podcast/rss/route.ts`) +- **RSS 2.0 with iTunes extensions**: Compatible with Apple Podcasts, Spotify, Google Podcasts +- **Auto-updating**: Pulls latest episodes from database +- **Proper metadata**: Titles, descriptions, duration, publication dates + +### 4. Podcast Page (`src/app/podcast/page.tsx`) +- **Web audio player**: Play episodes directly on the site +- **Episode listing**: Browse all available podcast episodes +- **Subscribe links**: RSS, Apple Podcasts, Spotify +- **Responsive design**: Works on mobile and desktop + +### 5. Integration with Digest Workflow +- **Updated `/api/digest`**: Now triggers TTS generation after saving post +- **Optional audio**: Can disable with `generateAudio: false` +- **Background processing**: Doesn't block the main response + +### 6. Manual TTS Script (`src/scripts/generate-tts.ts`) +- **Single post**: `npm run generate-tts -- ` +- **Batch processing**: `npm run generate-tts:all` +- **Force regeneration**: `--force` flag to overwrite existing + +### 7. UI Updates +- **Blog post audio player**: Shows audio player for posts with audio +- **Podcast link in header**: Easy navigation to podcast page +- **Visual indicators**: Shows which posts have audio + +## Files Created/Modified + +### New Files +``` +src/ +├── app/ +│ ├── api/podcast/rss/route.ts # RSS feed endpoint +│ └── podcast/page.tsx # Podcast web player +├── lib/ +│ ├── tts.ts # TTS service abstraction +│ ├── storage.ts # Supabase storage helpers +│ └── podcast.ts # RSS generation utilities +├── scripts/ +│ └── generate-tts.ts # Manual TTS generation script +└── ../PODCAST_SETUP.md # Setup documentation +``` + +### Modified Files +``` +src/ +├── app/ +│ ├── api/digest/route.ts # Added TTS trigger +│ └── page.tsx # Added audio player, podcast link +├── ../package.json # Added generate-tts scripts +├── ../tsconfig.json # Added scripts to includes +└── ../.env.local # Added TTS environment variables +``` + +## Configuration + +### Environment Variables +```bash +# Enable TTS generation +ENABLE_TTS=true + +# TTS Provider: openai, piper, or macsay +TTS_PROVIDER=openai + +# OpenAI settings (if using OpenAI) +OPENAI_API_KEY=sk-your-key-here +TTS_VOICE=alloy # alloy, echo, fable, onyx, nova, shimmer + +# Piper settings (if using Piper) +PIPER_MODEL_PATH=./models/en_US-lessac-medium.onnx +``` + +### Database Schema +```sql +ALTER TABLE blog_messages +ADD COLUMN audio_url TEXT, +ADD COLUMN audio_duration INTEGER; +``` + +## URLs + +- **RSS Feed**: `https://blog-backup-two.vercel.app/api/podcast/rss` +- **Podcast Page**: `https://blog-backup-two.vercel.app/podcast` +- **Blog**: `https://blog-backup-two.vercel.app` + +## Cost Analysis + +| Component | Provider | Monthly Cost | +|-----------|----------|--------------| +| TTS | OpenAI | ~$2-4 | +| TTS | Piper/macOS | $0 | +| Storage | Supabase | $0 (free tier) | +| RSS Hosting | Vercel | $0 | +| **Total** | | **$0-4/month** | + +## Next Steps to Deploy + +1. **Database**: Run the SQL to add `audio_url` and `audio_duration` columns +2. **Supabase Storage**: Create `podcast-audio` bucket (or let app auto-create) +3. **Environment**: Add `ENABLE_TTS=true` and `OPENAI_API_KEY` to production +4. **Deploy**: `npm run build && vercel --prod` +5. **Test**: Generate a test episode and verify RSS feed +6. **Submit**: Add RSS URL to Apple Podcasts, Spotify, etc. + +## Documentation + +See `PODCAST_SETUP.md` for complete setup instructions, troubleshooting, and usage guide. diff --git a/package.json b/package.json index 2c4bb37..7dcc45e 100644 --- a/package.json +++ b/package.json @@ -7,7 +7,9 @@ "build": "next build", "start": "next start", "lint": "eslint", - "deploy": "npm run build && vercel --prod" + "deploy": "npm run build && vercel --prod", + "generate-tts": "npx ts-node --project tsconfig.json src/scripts/generate-tts.ts", + "generate-tts:all": "npx ts-node --project tsconfig.json src/scripts/generate-tts.ts --all" }, "dependencies": { "@supabase/supabase-js": "^2.97.0", diff --git a/src/app/api/digest/route.ts b/src/app/api/digest/route.ts index fe6aabd..93f9e4a 100644 --- a/src/app/api/digest/route.ts +++ b/src/app/api/digest/route.ts @@ -1,34 +1,53 @@ +/** + * Daily Digest API Endpoint + * Saves digest content and optionally generates TTS audio + */ + import { NextResponse } from "next/server"; import { createClient } from "@supabase/supabase-js"; +import { generateSpeech } from "@/lib/tts"; +import { uploadAudio } from "@/lib/storage"; +import { extractTitle, extractExcerpt } from "@/lib/podcast"; const supabase = createClient( process.env.NEXT_PUBLIC_SUPABASE_URL!, process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY! ); -const CRON_API_KEY = process.env.CRON_API_KEY; +const serviceSupabase = createClient( + process.env.NEXT_PUBLIC_SUPABASE_URL!, + process.env.SUPABASE_SERVICE_ROLE_KEY! +); export async function POST(request: Request) { const apiKey = request.headers.get("x-api-key"); + const CRON_API_KEY = process.env.CRON_API_KEY; + + console.log("API Key from header:", apiKey); + console.log("CRON_API_KEY exists:", !!CRON_API_KEY); if (!CRON_API_KEY || apiKey !== CRON_API_KEY) { return NextResponse.json({ error: "Unauthorized" }, { status: 401 }); } - const { content, date, tags } = await request.json(); + const { content, date, tags, generateAudio = true } = await request.json(); if (!content || !date) { return NextResponse.json({ error: "Content and date required" }, { status: 400 }); } + const id = Date.now().toString(); const newMessage = { - id: Date.now().toString(), + id, date, content, timestamp: Date.now(), tags: tags || ["daily-digest"], + audio_url: null as string | null, + audio_duration: null as number | null, }; + // Save message first (without audio) const { error } = await supabase .from("blog_messages") .insert(newMessage); @@ -37,6 +56,67 @@ export async function POST(request: Request) { console.error("Error saving digest:", error); return NextResponse.json({ error: "Failed to save" }, { status: 500 }); } + + // Generate TTS audio asynchronously (don't block response) + if (generateAudio && process.env.ENABLE_TTS === "true") { + // Use a non-blocking approach + generateTTSAsync(id, content, date).catch(err => { + console.error("TTS generation failed (async):", err); + }); + } - return NextResponse.json({ success: true, id: newMessage.id }); + return NextResponse.json({ + success: true, + id, + audioGenerated: generateAudio && process.env.ENABLE_TTS === "true" + }); +} + +/** + * Generate TTS audio and upload to storage + * This runs asynchronously after the main response is sent + */ +async function generateTTSAsync( + id: string, + content: string, + date: string +): Promise { + try { + console.log(`[TTS] Starting generation for digest ${id}...`); + + // Generate speech + const { audioBuffer, duration, format } = await generateSpeech(content, { + provider: (process.env.TTS_PROVIDER as "piper" | "openai" | "macsay") || "openai", + voice: process.env.TTS_VOICE || "alloy", + }); + + // Determine file extension based on format + const ext = format === "audio/wav" ? "wav" : + format === "audio/aiff" ? "aiff" : "mp3"; + + // Create filename with date for organization + const filename = `digest-${date}-${id}.${ext}`; + + // Upload to Supabase Storage + const { url } = await uploadAudio(audioBuffer, filename, format); + + // Update database with audio URL + const { error: updateError } = await serviceSupabase + .from("blog_messages") + .update({ + audio_url: url, + audio_duration: duration, + }) + .eq("id", id); + + if (updateError) { + console.error("[TTS] Error updating database:", updateError); + throw updateError; + } + + console.log(`[TTS] Successfully generated audio for digest ${id}: ${url}`); + } catch (error) { + console.error(`[TTS] Failed to generate audio for digest ${id}:`, error); + // Don't throw - we don't want to fail the whole request + } } diff --git a/src/app/api/podcast/rss/route.ts b/src/app/api/podcast/rss/route.ts new file mode 100644 index 0000000..f4749e4 --- /dev/null +++ b/src/app/api/podcast/rss/route.ts @@ -0,0 +1,46 @@ +/** + * Podcast RSS Feed API Endpoint + * Returns RSS 2.0 with iTunes extensions for podcast distribution + * Compatible with Apple Podcasts, Spotify, Google Podcasts, etc. + */ + +import { NextResponse } from "next/server"; +import { fetchEpisodes, generateRSS, DEFAULT_CONFIG } from "@/lib/podcast"; + +export const dynamic = "force-dynamic"; // Always generate fresh RSS + +export async function GET() { + try { + // Fetch episodes with audio from database + const episodes = await fetchEpisodes(50); + + // Generate RSS XML + const rssXml = generateRSS(episodes, DEFAULT_CONFIG); + + // Return as XML with proper content type + return new NextResponse(rssXml, { + headers: { + "Content-Type": "application/xml; charset=utf-8", + "Cache-Control": "public, max-age=300", // Cache for 5 minutes + }, + }); + } catch (error) { + console.error("Error generating RSS feed:", error); + + return new NextResponse( + ` + + + OpenClaw Daily Digest + Error loading podcast feed. Please try again later. + +`, + { + status: 500, + headers: { + "Content-Type": "application/xml; charset=utf-8", + }, + } + ); + } +} diff --git a/src/app/page.tsx b/src/app/page.tsx index b2016ae..b3453fd 100644 --- a/src/app/page.tsx +++ b/src/app/page.tsx @@ -14,6 +14,8 @@ interface Message { content: string; timestamp: number; tags?: string[]; + audio_url?: string; + audio_duration?: number; } type Theme = "light" | "dark"; @@ -159,6 +161,9 @@ function BlogPageContent() {