videotomp3transcriptor/YOUTUBE_SETUP_COMPLETE.md

# ✅ YouTube Download System - READY TO USE

## 🎯 Status: FULLY OPERATIONAL

**Date:** 2026-01-31  
**Tests:** ✅ Passed (multiple videos downloaded successfully)

---

## 🔧 Infrastructure

### 1. PO Token Provider (Anti-Bot)
- **Container:** `bgutil-provider` (Docker)
- **Port:** 4416
- **Plugin:** `bgutil-ytdlp-pot-provider`
- **Status:** ✅ Running
- **Check:** `docker ps | grep bgutil`

### 2. SMS Receiver (For Auto-Renewal)
- **Service:** Node.js endpoint
- **Port:** 4417
- **Webhook:** `http://57.131.33.10:4417/sms`
- **Status:** ✅ Running
- **Check:** `curl http://localhost:4417/health`

### 3. JavaScript Runtime
- **Runtime:** Deno 2.6.7
- **Purpose:** Decrypt YouTube signatures
- **Location:** `/usr/local/bin/deno`
- **Status:** ✅ Installed

### 4. Cookies
- **File:** `/home/debian/videotomp3transcriptor/youtube-cookies.txt`
- **Type:** Logged-in account cookies
- **Duration:** 2-4 weeks
- **Permissions:** 600 (secure)
- **Status:** ✅ Valid

---

## 🚀 Usage

### Simple Download (Audio MP3)

```bash
cd /home/debian/videotomp3transcriptor

yt-dlp \
  --cookies youtube-cookies.txt \
  --extractor-args "youtube:player_client=mweb" \
  --format "bestaudio" \
  --extract-audio \
  --audio-format mp3 \
  --output "downloads/%(title)s.%(ext)s" \
  "YOUTUBE_URL"
```

### Download with Metadata

```bash
yt-dlp \
  --cookies youtube-cookies.txt \
  --extractor-args "youtube:player_client=mweb" \
  --format "bestaudio" \
  --extract-audio \
  --audio-format mp3 \
  --embed-thumbnail \
  --add-metadata \
  --output "downloads/%(title)s.%(ext)s" \
  "YOUTUBE_URL"
```

### Get Video Info Only

```bash
yt-dlp \
  --cookies youtube-cookies.txt \
  --extractor-args "youtube:player_client=mweb" \
  --skip-download \
  --print "%(title)s|%(duration)s|%(uploader)s" \
  "YOUTUBE_URL"
```

---

## 📊 What Happens Behind the Scenes

1. **PO Token Generation**
   - Plugin detects mweb client
   - Calls Docker container (port 4416)
   - Token injected in YouTube API request

2. **Cookie Authentication**
   - Logged-in cookies bypass bot detection
   - Google accepts requests as real user

3. **Signature Decryption**
   - Deno runtime solves JS challenges
   - Decrypts video/audio URLs

4. **Download**
   - Best audio format selected
   - ffmpeg converts to MP3
   - Metadata embedded

---

## 🔄 Cookie Renewal (When Expired)

### Option A: From Your PC (2 minutes)

```bash
# On your PC (with Firefox logged in to YouTube)
yt-dlp --cookies-from-browser firefox --cookies youtube-cookies.txt 'https://youtube.com'

# Upload to server
scp youtube-cookies.txt debian@57.131.33.10:/home/debian/videotomp3transcriptor/
```

### Option B: SMS Forwarder (Automated)

**Setup once:**
1. Install "SMS Forwarder" on Android
2. Configure rule:
   - Trigger: Sender contains "Google"
   - Action: HTTP POST
   - URL: `http://57.131.33.10:4417/sms`
   - Body: `{"from":"$SENDER$","body":"$BODY$"}`

**Then run:**
```bash
cd /home/debian/videotomp3transcriptor
export DISPLAY=:99
python3 src/python/auto_login_full_auto.py
```

Script will:
- Navigate to Google login
- Enter credentials
- Wait for SMS code (forwarded automatically)
- Complete login
- Extract fresh cookies

---

## 🛠️ Troubleshooting

### "Sign in to confirm you're not a bot"
**Solution:** Cookies expired, renew them (see above)

### "Signature solving failed"
**Check Deno:** `deno --version`  
**Reinstall if needed:** See installation section

### "PO Token generation failed"
**Check container:** `docker ps | grep bgutil`  
**Restart if needed:** `docker restart bgutil-provider`

### Slow downloads
**Try different client:**
- `player_client=mweb` (default, stable)
- `player_client=android` (faster sometimes)
- `player_client=ios` (fallback)

---

## 📁 Files & Scripts

```
/home/debian/videotomp3transcriptor/
├── youtube-cookies.txt              🔑 Logged-in cookies (KEEP SECURE)
├── src/
│   ├── sms_receiver.js              📱 SMS webhook endpoint
│   ├── python/
│   │   ├── auto_login_full_auto.py  🤖 Auto-login with SMS
│   │   ├── extract_cookies.py       🍪 Cookie extraction (Camoufox)
│   │   └── validate_cookies.txt     ✅ Cookie validator
│   └── services/
│       ├── cookiesManager.js        🔧 Cookie manager (service)
│       └── download.js              📥 Download service
├── test_youtube_download.sh         🧪 Test script
└── SMS_FORWARDER_SETUP.md          📖 SMS setup guide
```

---

## ✅ Tested Videos

- ✅ `fukChj4eh-Q` → 4.0 MB MP3 (success)
- ✅ `dQw4w9WgXcQ` (Rick Astley) → 3.6 MB MP3 (success)

---

## 🎯 Next Steps

### Integrate into API

Update `/home/debian/videotomp3transcriptor/src/services/download.js`:

```javascript
const cookiesPath = path.join(__dirname, '../../youtube-cookies.txt');

ytdlp.exec([
  url,
  '--cookies', cookiesPath,
  '--extractor-args', 'youtube:player_client=mweb',
  '--format', 'bestaudio',
  '--extract-audio',
  '--audio-format', 'mp3',
  '--output', outputPath
]);
```

### Monitor & Maintain

- **Weekly:** Check cookie validity
- **Monthly:** Review PO Token plugin updates
- **As needed:** Renew cookies when expired

---

## 🔐 Security Notes

- **Cookies file:** Contains auth tokens, permissions 600
- **SMS endpoint:** Public on port 4417 (add auth if needed)
- **PO Token:** Port 4416 local only
- **Logs:** May contain sensitive data, rotate regularly

---

## 📞 Support

- **yt-dlp docs:** https://github.com/yt-dlp/yt-dlp
- **PO Token plugin:** https://github.com/yt-dlp/yt-dlp-plugins
- **Deno docs:** https://deno.land/manual

---

**System Status:** ✅ READY FOR PRODUCTION

Last updated: 2026-01-31 08:20 UTC