YouTube Transcripts for RAG and LLMs: No API Key Required

YouTube is one of the largest knowledge repositories on the internet — tutorials, lectures, conference talks, product demos, earnings calls. Almost none of it is indexed by traditional RAG pipelines because it lives in video. Transcripts change that: once you have the text, a YouTube video becomes a document you can chunk, embed, and retrieve like any other.

The YouTube Data API v3 gives you transcript access, but it comes with a 10,000-unit daily quota, rate limits, and a Google Cloud project to maintain. The YouTube Transcript Scraper fetches captions directly from YouTube’s internal timedtext endpoint — no API key, no quota, no Google account. $0.003 per transcript, nothing charged on failed pulls.

How It Works

YouTube embeds caption track metadata in every watch page inside a ytInitialPlayerResponse JSON object. The actor extracts the caption track list from that object, picks the best match for your preferred language, and fetches the timedtext XML — the same endpoint the YouTube player itself uses for subtitles.

The result is a clean list of {start, dur, text} segments plus a fullText string that joins all segments for easy LLM ingestion.

Fetch Transcripts in Python

import os
from apify_client import ApifyClient

apify = ApifyClient(os.environ["APIFY_TOKEN"])

videos = [
    "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
    "https://youtu.be/VMj-3S1tku0",   # short link form
    "BUU0x6UCHLk",                     # bare video ID also works
]

run = apify.actor("themineworks/youtube-transcript-scraper").call(run_input={
    "videoUrls": videos,
    "language": "en",
    "includeTimestamps": True,
})

for item in apify.dataset(run["defaultDatasetId"]).iterate_items():
    if item.get("status") == "summary":
        continue
    if item.get("status") == "ok":
        print(f"\n{item['title']} ({item['videoId']})")
        print(f"  {item['segmentCount']} segments, {item['charCount']} chars")
        print(f"  language: {item['language']}  auto-generated: {item['isAutoGenerated']}")
        print(f"  preview: {item['fullText'][:120]}...")
    else:
        print(f"\n{item.get('url')} → {item.get('status')}: {item.get('error', '')}")

Build a YouTube RAG Pipeline

The fullText field maps directly onto any chunking and embedding workflow:

import chromadb
from openai import OpenAI

openai = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
chroma = chromadb.PersistentClient(path="./yt_chroma")
collection = chroma.get_or_create_collection("youtube")

def index_transcripts(items: list[dict]):
    docs, ids, metas = [], [], []
    for item in items:
        if item.get("status") != "ok":
            continue
        text = item["fullText"]
        if len(text) < 200:
            continue
        # Chunk at ~1000 chars if needed; here we index the full transcript
        docs.append(f"{item['title']}\n\n{text}")
        ids.append(item["videoId"])
        metas.append({
            "title": item["title"],
            "url": item["url"],
            "language": item["language"],
            "chars": item["charCount"],
        })
    if not docs:
        return
    embeddings = [e.embedding for e in openai.embeddings.create(
        model="text-embedding-3-small", input=docs).data]
    collection.add(documents=docs, embeddings=embeddings, ids=ids, metadatas=metas)

For long transcripts (lectures, earnings calls) split the fullText into smaller chunks before embedding. The segments array with per-segment timestamps makes this straightforward: accumulate segments until you hit your token limit, then start a new chunk. This preserves the ability to return a timestamp alongside each retrieved chunk — useful for surfacing the exact moment in the video that contains the answer.

Use It as an MCP Tool in Claude

{
  "mcpServers": {
    "youtube-transcripts": {
      "command": "npx",
      "args": ["-y", "@apify/mcp-client"],
      "env": {
        "APIFY_TOKEN": "your-token-here",
        "MCP_ACTOR_URL": "https://mcp.apify.com/?tools=themineworks/youtube-transcript-scraper"
      }
    }
  }
}

Once connected, Claude can fetch transcripts inline during a conversation. Paste a YouTube URL and ask Claude to summarise, extract action items, translate, or compare it to a document you have open. Claude calls the actor, receives the fullText, and reasons over it directly — no copy-paste required.

What Happens Without Captions

The actor returns status: "no-captions" for videos that have no caption track at all, and this does not incur a charge. Videos that are private, age-restricted, or unavailable return status: "error" with a reason field — also not charged.

Most videos from established creators, educational institutions, and corporate channels have either manual captions or YouTube’s auto-generated ASR captions. The actor prefers manual captions when available and falls back to ASR. The isAutoGenerated field in each result tells you which was used.

Pairing With RAG Crawler

YouTube Transcript Scraper and RAG Crawler are complementary. RAG Crawler handles written web content — articles, documentation, blog posts — while Transcript Scraper handles spoken video content. Together they give you a unified text corpus covering the full range of published knowledge: written and spoken, static pages and video platforms.

A research agent that can ingest both a company’s documentation site and their YouTube channel of product demos has considerably more context than one limited to either source alone.

Cost

Each transcript that successfully delivers costs $0.003 — roughly a fraction of a cent per video. A library of 10,000 videos costs $30 to transcript. That is orders of magnitude cheaper than manual transcription services, and the output is already structured JSON rather than raw text that needs parsing.