Add YouTube cookies support to bypass bot detection

- Add cookies support in youtube.js service
- Create extract-cookies.sh script for automatic cookie extraction
- Add YOUTUBE_COOKIES_PATH environment variable
- Update .env.example with cookies configuration
- Add comprehensive documentation (YOUTUBE_COOKIES.md, COOKIES_QUICK_START.md)
- Update .gitignore to exclude cookies files for security
- Pass cookiesPath option through all download functions

This fixes Sign in to confirm you're not a bot errors from YouTube.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Debian User 2025-12-05 13:32:35 +00:00
parent 1f349834e6
commit ea0caa9349
7 changed files with 500 additions and 13 deletions

View File

@ -9,3 +9,8 @@ PORT=3000
# Output directory (optional, default: ./output)
OUTPUT_DIR=./output
# YouTube cookies file path (optional, helps bypass bot detection)
# Run: bash scripts/extract-cookies.sh
# Then set the path to your cookies file:
YOUTUBE_COOKIES_PATH=./youtube-cookies.txt

4
.gitignore vendored
View File

@ -24,6 +24,10 @@ output/
# Text/transcription files
*.txt
# YouTube cookies (contains sensitive authentication data)
*cookies*.txt
cookies.txt
# Logs
*.log
npm-debug.log*

109
COOKIES_QUICK_START.md Normal file
View File

@ -0,0 +1,109 @@
# Quick Start: Fix YouTube "Sign in to confirm you're not a bot"
## The Problem
YouTube blocks automated downloads with this error:
```
Sign in to confirm you're not a bot
```
## The Solution
Use cookies from your browser to authenticate!
## Step-by-Step Instructions
### 1. Extract Cookies (Choose ONE method)
#### Method A: Automatic (Chrome/Firefox on same machine)
```bash
cd /home/debian/videotomp3transcriptor
bash scripts/extract-cookies.sh
```
Follow the prompts and choose your browser.
#### Method B: Manual (Any browser, any machine)
1. **Install browser extension:**
- Chrome/Edge: [Get cookies.txt LOCALLY](https://chrome.google.com/webstore/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpdbgdldlbecc)
- Firefox: [cookies.txt](https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/)
2. **Go to YouTube and log in:**
- Visit https://www.youtube.com
- Make sure you're logged into your account
3. **Export cookies:**
- Click the extension icon
- Click "Export" or "Download"
- Save the file as `youtube-cookies.txt`
4. **Upload to server:**
```bash
# On your local machine
scp youtube-cookies.txt debian@toMP3.etheryale.com:/home/debian/videotomp3transcriptor/
```
### 2. Configure the API
Edit `.env` file:
```bash
nano /home/debian/videotomp3transcriptor/.env
```
Add or uncomment this line:
```bash
YOUTUBE_COOKIES_PATH=/home/debian/videotomp3transcriptor/youtube-cookies.txt
```
Save and exit (Ctrl+X, then Y, then Enter)
### 3. Restart the API
```bash
pm2 restart toMP3-api
```
### 4. Test It
```bash
curl -X POST http://localhost:3001/download \
-H "Content-Type: application/json" \
-d '{"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"}'
```
If it works, you'll see download progress! 🎉
## Common Issues
### "Permission denied" when reading cookies
```bash
chmod 600 /home/debian/videotomp3transcriptor/youtube-cookies.txt
```
### Cookies expired
Re-export fresh cookies from your browser. Cookies typically last a few weeks.
### Still getting "not a bot" error
1. Make sure you logged into YouTube before exporting cookies
2. Try a different browser
3. Check the path in `.env` is correct
4. Verify the file exists: `ls -la /home/debian/videotomp3transcriptor/youtube-cookies.txt`
## Security Warning ⚠️
**Cookies = Your YouTube Session**
- Keep cookies file secure (like a password)
- Never commit to git (already in .gitignore)
- Don't share publicly
- Re-export periodically when they expire
## Need More Help?
See full documentation: [docs/YOUTUBE_COOKIES.md](docs/YOUTUBE_COOKIES.md)

94
FILE_SHARING.md Normal file
View File

@ -0,0 +1,94 @@
# Serveur de partage de fichiers temporaire
## Démarrage rapide
Pour héberger des fichiers temporairement et les partager via un lien :
```bash
# Créer le dossier et démarrer le serveur
mkdir -p /tmp/share && cd /tmp/share && python3 -m http.server 8000
```
Ou en arrière-plan :
```bash
mkdir -p /tmp/share
cd /tmp/share
python3 -m http.server 8000 &
```
## Utilisation
### 1. Ajouter des fichiers à partager
```bash
cp votre-fichier.ext /tmp/share/
```
### 2. Partager le lien
Le fichier sera accessible à :
```
http://localhost:8000/votre-fichier.ext
```
Pour un accès externe (si le serveur est exposé) :
```
http://VOTRE_IP:8000/votre-fichier.ext
```
### 3. Lister les fichiers disponibles
Accédez simplement à :
```
http://localhost:8000/
```
### 4. Supprimer un fichier après téléchargement
```bash
rm /tmp/share/nom-du-fichier.ext
```
## Arrêter le serveur
Si lancé en arrière-plan, trouver le processus et l'arrêter :
```bash
# Trouver le processus
ps aux | grep "http.server"
# Tuer le processus (remplacer PID par le numéro du processus)
kill PID
```
Ou plus simplement :
```bash
pkill -f "http.server"
```
## Nettoyer complètement
```bash
# Arrêter le serveur
pkill -f "http.server"
# Supprimer tous les fichiers
rm -rf /tmp/share
```
## Sécurité
⚠️ **Attention** : Ce serveur est ultra simple et ne doit être utilisé que pour du partage temporaire :
- Pas d'authentification
- Pas de HTTPS
- Accessible à tous ceux qui ont l'URL
- À utiliser uniquement sur des réseaux de confiance
- Supprimer les fichiers après téléchargement
## Alternative : Utiliser le serveur du projet
Le serveur principal du projet peut aussi servir des fichiers :
- Endpoint : `GET /files/<nom-fichier>`
- Port : 3001 (configuré dans .env)
- Les fichiers doivent être dans le dossier `./output/`
Exemple :
```bash
cp fichier.mp3 ./output/
# Accessible à : http://localhost:3001/files/fichier.mp3
```

132
docs/YOUTUBE_COOKIES.md Normal file
View File

@ -0,0 +1,132 @@
# YouTube Cookies Setup Guide
## Why Do I Need Cookies?
YouTube has anti-bot protections that may block yt-dlp requests. Using cookies from your browser allows yt-dlp to authenticate as if you're logged in, bypassing these restrictions.
## Quick Start
### Option 1: Automatic Extraction (Recommended)
Run the helper script:
```bash
bash scripts/extract-cookies.sh
```
Follow the prompts to extract cookies from Chrome or Firefox.
### Option 2: Using yt-dlp Directly
```bash
# For Chrome/Chromium
yt-dlp --cookies-from-browser chrome --cookies youtube-cookies.txt 'https://www.youtube.com'
# For Firefox
yt-dlp --cookies-from-browser firefox --cookies youtube-cookies.txt 'https://www.youtube.com'
# For Edge
yt-dlp --cookies-from-browser edge --cookies youtube-cookies.txt 'https://www.youtube.com'
```
### Option 3: Browser Extension
1. Install a cookies export extension:
- **Chrome/Edge**: [Get cookies.txt LOCALLY](https://chrome.google.com/webstore/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpdbgdldlbecc)
- **Firefox**: [cookies.txt](https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/)
2. Go to [youtube.com](https://www.youtube.com) and log in
3. Click the extension icon and export cookies
4. Save the file as `youtube-cookies.txt` in your project directory
## Configuration
After extracting cookies, update your `.env` file:
```bash
YOUTUBE_COOKIES_PATH=/home/debian/videotomp3transcriptor/youtube-cookies.txt
```
Or use a relative path:
```bash
YOUTUBE_COOKIES_PATH=./youtube-cookies.txt
```
## Verifying It Works
Test with a video:
```bash
curl -X POST http://localhost:3001/download \
-H "Content-Type: application/json" \
-d '{"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"}'
```
If it works without cookies errors, you're good to go!
## Security Notes
⚠️ **IMPORTANT**:
1. **Never commit cookies to git**: The `.gitignore` file should already exclude `youtube-cookies.txt`
2. **Keep cookies secure**: They provide access to your YouTube account
3. **Cookies expire**: You may need to re-export them periodically (typically every few weeks/months)
4. **Don't share cookies**: Treat them like passwords
## Troubleshooting
### "Sign in to confirm you're not a bot"
This usually means:
- Cookies are not being used
- Cookies have expired
- Cookies file path is incorrect
**Solutions**:
1. Check the path in `.env` is correct and absolute
2. Re-export fresh cookies
3. Verify the cookies file exists: `ls -la youtube-cookies.txt`
4. Check logs: `pm2 logs toMP3-api`
### "HTTP Error 403: Forbidden"
YouTube is blocking your IP or the video is region-restricted.
**Solutions**:
1. Try with fresh cookies
2. Use a VPN if region-restricted
3. Wait a bit if rate-limited
### Cookies Not Working
1. Make sure you're logged into YouTube in the browser before extracting
2. Try extracting from a different browser
3. Verify the cookies file format (should be Netscape format)
4. Check file permissions: `chmod 600 youtube-cookies.txt`
## Cookie File Format
The cookies file should be in Netscape format and look like this:
```
# Netscape HTTP Cookie File
.youtube.com TRUE / TRUE 1234567890 CONSENT YES+
.youtube.com TRUE / FALSE 1234567890 VISITOR_INFO1_LIVE xxxxx
```
## Without Cookies
The API will still work for many videos without cookies, but you may encounter:
- "Sign in to confirm you're not a bot" errors
- Rate limiting
- Blocked downloads for certain videos
For best results, always use cookies!
## Additional Resources
- [yt-dlp Cookie Documentation](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#how-do-i-pass-cookies-to-yt-dlp)
- [Browser Cookie Extraction](https://github.com/yt-dlp/yt-dlp#:~:text=You%20can%20use%20cookies%20from%20your%20browser)

124
scripts/extract-cookies.sh Executable file
View File

@ -0,0 +1,124 @@
#!/bin/bash
# YouTube Cookies Extractor Script
# This script helps you extract cookies from your browser for yt-dlp
echo "=========================================="
echo "YouTube Cookies Extractor"
echo "=========================================="
echo ""
echo "This script will help you extract cookies from your browser"
echo "to bypass YouTube's bot detection."
echo ""
COOKIES_FILE="./youtube-cookies.txt"
echo "Options:"
echo ""
echo "1. Extract from Chrome/Chromium"
echo "2. Extract from Firefox"
echo "3. Manual instructions (export from browser extension)"
echo ""
read -p "Choose an option (1-3): " choice
case $choice in
1)
echo ""
echo "Extracting cookies from Chrome/Chromium..."
echo ""
# Check if yt-dlp can extract cookies
if command -v yt-dlp &> /dev/null; then
echo "Using yt-dlp to extract cookies from Chrome..."
yt-dlp --cookies-from-browser chrome --cookies "$COOKIES_FILE" --print-to-file webpage_url - "https://www.youtube.com" 2>/dev/null
if [ -f "$COOKIES_FILE" ]; then
echo "✓ Cookies extracted successfully to: $COOKIES_FILE"
echo ""
echo "Add this to your .env file:"
echo "YOUTUBE_COOKIES_PATH=$(realpath $COOKIES_FILE)"
else
echo "✗ Failed to extract cookies automatically."
echo "You may need to export cookies manually (see option 3)"
fi
else
echo "✗ yt-dlp not found. Please install it first:"
echo " pip install yt-dlp"
fi
;;
2)
echo ""
echo "Extracting cookies from Firefox..."
echo ""
if command -v yt-dlp &> /dev/null; then
echo "Using yt-dlp to extract cookies from Firefox..."
yt-dlp --cookies-from-browser firefox --cookies "$COOKIES_FILE" --print-to-file webpage_url - "https://www.youtube.com" 2>/dev/null
if [ -f "$COOKIES_FILE" ]; then
echo "✓ Cookies extracted successfully to: $COOKIES_FILE"
echo ""
echo "Add this to your .env file:"
echo "YOUTUBE_COOKIES_PATH=$(realpath $COOKIES_FILE)"
else
echo "✗ Failed to extract cookies automatically."
echo "You may need to export cookies manually (see option 3)"
fi
else
echo "✗ yt-dlp not found. Please install it first:"
echo " pip install yt-dlp"
fi
;;
3)
echo ""
echo "=========================================="
echo "Manual Cookie Export Instructions"
echo "=========================================="
echo ""
echo "Method 1: Using a browser extension (recommended)"
echo " 1. Install one of these browser extensions:"
echo " - Chrome: 'Get cookies.txt LOCALLY' or 'cookies.txt'"
echo " - Firefox: 'cookies.txt'"
echo ""
echo " 2. Go to https://www.youtube.com and log in"
echo ""
echo " 3. Click on the extension icon and export cookies as 'youtube-cookies.txt'"
echo ""
echo " 4. Save the file to: $(realpath .)/youtube-cookies.txt"
echo ""
echo "Method 2: Using yt-dlp directly"
echo " Run this command:"
echo " yt-dlp --cookies-from-browser chrome --cookies youtube-cookies.txt 'https://www.youtube.com'"
echo ""
echo " (Replace 'chrome' with 'firefox', 'edge', 'safari', etc. if needed)"
echo ""
echo "Method 3: Using browser DevTools (advanced)"
echo " 1. Go to https://www.youtube.com and log in"
echo " 2. Open DevTools (F12)"
echo " 3. Go to Application/Storage tab"
echo " 4. Find Cookies > https://www.youtube.com"
echo " 5. Export all cookies manually to Netscape format"
echo ""
echo "After creating the file, add this to your .env:"
echo "YOUTUBE_COOKIES_PATH=$(realpath .)/youtube-cookies.txt"
echo ""
;;
*)
echo "Invalid option"
exit 1
;;
esac
echo ""
echo "=========================================="
echo ""
echo "IMPORTANT: Keep your cookies file secure!"
echo "Do NOT share it or commit it to git."
echo "The cookies give access to your YouTube account."
echo ""
echo "The cookies will expire eventually, so you may need"
echo "to re-export them periodically."
echo ""

View File

@ -6,12 +6,28 @@ import { spawn } from 'child_process';
// Use system yt-dlp binary (check common paths)
const YTDLP_PATH = process.env.YTDLP_PATH || 'yt-dlp';
// Path to cookies file (optional)
const COOKIES_PATH = process.env.YOUTUBE_COOKIES_PATH || null;
/**
* Add cookies argument if cookies file exists
*/
function addCookiesArg(args, cookiesPath = null) {
const cookies = cookiesPath || COOKIES_PATH;
if (cookies && fs.existsSync(cookies)) {
return ['--cookies', cookies, ...args];
}
return args;
}
/**
* Execute yt-dlp command and return parsed JSON
*/
async function ytdlp(url, args = []) {
async function ytdlp(url, args = [], options = {}) {
const { cookiesPath } = options;
const finalArgs = addCookiesArg(args, cookiesPath);
return new Promise((resolve, reject) => {
const proc = spawn(YTDLP_PATH, [...args, url]);
const proc = spawn(YTDLP_PATH, [...finalArgs, url]);
let stdout = '';
let stderr = '';
@ -35,9 +51,11 @@ async function ytdlp(url, args = []) {
/**
* Execute yt-dlp command with progress callback
*/
function ytdlpExec(url, args = [], onProgress) {
function ytdlpExec(url, args = [], onProgress, options = {}) {
const { cookiesPath } = options;
const finalArgs = addCookiesArg(args, cookiesPath);
return new Promise((resolve, reject) => {
const proc = spawn(YTDLP_PATH, [...args, url]);
const proc = spawn(YTDLP_PATH, [...finalArgs, url]);
let stderr = '';
proc.stdout.on('data', (data) => {
@ -108,7 +126,7 @@ function extractPlaylistUrl(url) {
/**
* Get video/playlist info without downloading
*/
export async function getInfo(url, forcePlaylist = false) {
export async function getInfo(url, forcePlaylist = false, options = {}) {
try {
// If URL contains a playlist ID and we want to force playlist mode
const playlistUrl = extractPlaylistUrl(url);
@ -119,7 +137,7 @@ export async function getInfo(url, forcePlaylist = false) {
'--no-download',
'--no-warnings',
'--flat-playlist',
]);
], options);
return info;
} catch (error) {
throw new Error(`Failed to get info: ${error.message}`);
@ -138,7 +156,7 @@ export async function isPlaylist(url) {
* Download a single video as MP3
*/
export async function downloadVideo(url, options = {}) {
const { outputDir = OUTPUT_DIR, onProgress, onDownloadProgress } = options;
const { outputDir = OUTPUT_DIR, onProgress, onDownloadProgress, cookiesPath } = options;
// Ensure output directory exists
if (!fs.existsSync(outputDir)) {
@ -151,7 +169,7 @@ export async function downloadVideo(url, options = {}) {
'--dump-single-json',
'--no-download',
'--no-warnings',
]);
], { cookiesPath });
const title = sanitizeFilename(info.title);
const outputPath = path.join(outputDir, `${title}.mp3`);
@ -171,7 +189,7 @@ export async function downloadVideo(url, options = {}) {
title: info.title,
});
}
});
}, { cookiesPath });
return {
success: true,
@ -189,7 +207,7 @@ export async function downloadVideo(url, options = {}) {
* Download all videos from a playlist as MP3
*/
export async function downloadPlaylist(url, options = {}) {
const { outputDir = OUTPUT_DIR, onProgress, onVideoComplete, onDownloadProgress, forcePlaylist = false } = options;
const { outputDir = OUTPUT_DIR, onProgress, onVideoComplete, onDownloadProgress, forcePlaylist = false, cookiesPath } = options;
// Ensure output directory exists
if (!fs.existsSync(outputDir)) {
@ -198,7 +216,7 @@ export async function downloadPlaylist(url, options = {}) {
try {
// Get playlist info (force playlist mode if URL has list= param)
const info = await getInfo(url, forcePlaylist || hasPlaylistParam(url));
const info = await getInfo(url, forcePlaylist || hasPlaylistParam(url), { cookiesPath });
if (info._type !== 'playlist') {
// Single video, redirect to downloadVideo
@ -237,7 +255,7 @@ export async function downloadPlaylist(url, options = {}) {
});
} : undefined;
const result = await downloadVideo(videoUrl, { outputDir, onDownloadProgress: wrappedProgress });
const result = await downloadVideo(videoUrl, { outputDir, onDownloadProgress: wrappedProgress, cookiesPath });
results.push(result);
if (onVideoComplete) {
@ -271,9 +289,10 @@ export async function downloadPlaylist(url, options = {}) {
* Smart download - detects if URL is video or playlist
*/
export async function download(url, options = {}) {
const { cookiesPath } = options;
// If URL contains list= parameter, treat it as a playlist
const isPlaylistUrl = hasPlaylistParam(url);
const info = await getInfo(url, isPlaylistUrl);
const info = await getInfo(url, isPlaylistUrl, { cookiesPath });
if (info._type === 'playlist') {
return downloadPlaylist(url, { ...options, forcePlaylist: true });