How to Chunk Long Transcripts for LLMs and RAG: A Developer's Guide

A one-hour talk produces roughly 9,000 spoken words. A full podcast season, a semester of lectures or a year of customer calls quickly runs into millions of tokens. You can't paste that into a prompt, and even when you can, you often shouldn't.

Chunking is how you break long transcripts into pieces an LLM or a retrieval system can work with. Done well, your app finds the exact moment someone said something and links to it. Done badly, it retrieves half a sentence of filler and the model makes up the rest.

This guide covers the whole job: the token math, the main chunking strategies and when to use each, overlap, metadata for citations, a working Python example and how to measure whether it's working.

Why transcripts are harder to chunk than documents

Most chunking advice is written for documentation and PDFs, which have headings, paragraphs and punctuation. Transcripts usually have none of that.

• No structure. There are no headings or paragraphs to split on. Auto-generated captions may not even have sentence punctuation.
• Low information density. Speech is full of filler ("so, yeah, I mean"), repetition and false starts. A 200-token chunk of speech carries far less meaning than 200 tokens of written text.
• Multiple speakers. A question and its answer belong together, but they come from different people.
• Time matters. Users want to jump to the moment in the recording. If you lose timestamps during chunking, you lose citations.
• Caption artifacts. Caption files often repeat lines across cues or break sentences mid-phrase to fit on screen.

So the first rule is: never chunk a flat string if you can chunk timed segments. Keep the structure you have and build chunks from it.

Step 1: Normalize into timed segments

Whatever your source (an SRT or VTT file, a speech-to-text API response, a JSON transcript), convert it into one shape:

[
  {"start": 12.4, "end": 17.9, "speaker": "Host", "text": "So what made you start the company?"},
  {"start": 18.1, "end": 31.0, "speaker": "Guest", "text": "Honestly, we kept hitting the same problem at work."}
]

While you're at it, clean the text:

• Remove duplicated lines that caption formats repeat across overlapping cues.
• Strip non-speech tags like [Music] or [Applause] unless they matter to your use case.
• Light filler removal ("um", "uh") is fine. Don't rewrite content, or your quotes stop matching the recording.
• If the transcript has no punctuation, run a punctuation restoration model or ask a small LLM to add punctuation only. Sentence boundaries make every later step better.

If your captions are in SRT or VTT, our guide to SRT vs VTT subtitle formats shows how to parse the timing lines.

Step 2: Do the token math

Before choosing a strategy, find out how big the problem is. A rough rule for English is that one token is about three quarters of a word, so 1,000 words is about 1,300 tokens. Speech runs at roughly 130 to 160 words per minute.

Use the real tokenizer for your model when it matters (tiktoken for OpenAI models, the tokenizer shipped with open-source models, or your provider's token counting endpoint). Other languages can use noticeably more tokens per word than English.

Then decide whether you need retrieval at all:

• One transcript, fits easily in context, a few questions: skip chunking. Send the whole transcript and ask. It's simpler and the model sees everything.
• One long transcript, many questions, or cost matters: chunk and retrieve.
• Many transcripts: you need retrieval. No context window holds a whole library.

Step 3: Choose a chunking strategy

There are four common approaches. Most production systems combine two of them.

Fixed-size chunking

Take N tokens, then the next N, with some overlap. It's easy and predictable, and it's a fine baseline. The problem is that it ignores meaning, so a chunk can start in the middle of an answer and end halfway through the next question.

Sentence-based chunking

Split the transcript into sentences, then pack sentences into a chunk until you hit your size target. Chunks always start and end on a complete thought. This is the best default for single-speaker content like lectures and tutorials.

Speaker-turn chunking

Use speaker changes as the primary boundary. Merge consecutive short turns ("Right." "Exactly.") so you don't create tiny chunks, and split long turns by sentence. This keeps questions and answers together, which matters a lot for interviews. If your transcript has no speaker labels, a speech-to-text API with diarization can add them (see our speech-to-text API guide).

Semantic chunking

Embed each sentence, compare neighbors, and start a new chunk where similarity drops sharply, which usually signals a topic change. It produces the most coherent chunks for long, rambling conversations, but it costs an embedding call per sentence and needs a tuned threshold. Try it after a sentence or speaker-turn baseline, not before.

Step 4: Set chunk size and overlap

Chunk size is a trade-off. Small chunks are precise but lose context. Large chunks keep context but dilute the embedding, so a short relevant passage gets buried in an average of everything else.

Good starting points for transcripts:

• Target size: 300 to 500 tokens. Spoken text is less dense, so go a little larger than you would for documentation.
• Overlap: 10 to 20 percent, or one to two sentences.
• Hard ceiling: stay well under your embedding model's maximum input. Many popular embedding models accept 512 tokens, and some accept 8,000 or more, but very long inputs usually embed less sharply even when they're allowed.

Check the input limit in your embedding model's documentation. If a chunk exceeds it, many APIs silently truncate, and the end of your chunk never gets embedded.

Step 5: Attach metadata to every chunk

Metadata is what turns retrieval into citations. For every chunk, store:

• source_id and title of the recording
• start and end time in seconds
• speakers who appear in the chunk
• chunk_index so you can fetch neighbors for extra context
• Optional: topic or chapter label, language, date

A powerful trick is to prepend a short context header to the text you embed, for example "Podcast: Founders Weekly. Episode: Pricing mistakes. Speaker: Guest." The header adds meaning the raw speech doesn't carry, which improves retrieval noticeably for vague questions. Keep the original text separately for display and quoting.

A working Python example

This example packs timed segments into sentence-aware chunks with overlap and keeps timestamps and speakers. It uses tiktoken for counting, which you can swap for your model's tokenizer.

import tiktoken

enc = tiktoken.get_encoding("cl100k_base")

def count_tokens(text):
    return len(enc.encode(text))

def chunk_segments(segments, target=400, overlap=60, source_id="ep-001"):
    chunks = []
    current = []
    current_tokens = 0

    def flush():
        if not current:
            return
        chunks.append({
            "source_id": source_id,
            "chunk_index": len(chunks),
            "start": current[0]["start"],
            "end": current[-1]["end"],
            "speakers": sorted({s["speaker"] for s in current}),
            "text": " ".join(s["speaker"] + ": " + s["text"] for s in current),
        })

    for seg in segments:
        seg_tokens = count_tokens(seg["text"])
        if current and current_tokens + seg_tokens > target:
            flush()
            # carry the tail of the previous chunk forward as overlap
            tail, tail_tokens = [], 0
            for prev in reversed(current):
                t = count_tokens(prev["text"])
                if tail_tokens + t > overlap:
                    break
                tail.insert(0, prev)
                tail_tokens += t
            current, current_tokens = tail, tail_tokens
        current.append(seg)
        current_tokens += seg_tokens

    flush()
    return chunks

A few notes on this code:

• It splits on segment boundaries, so if your segments are sentences or speaker turns, chunks never cut a thought in half.
• Overlap is whole segments, not raw tokens, so it never starts mid-sentence.
• Speaker names are written into the text, which helps both retrieval and the LLM's answer.
• A single segment longer than the target becomes its own chunk. If your source has very long segments, split them by sentence first.

To build a link that jumps to the right moment, use the chunk's start time. For many players, that's just a time parameter in seconds on the URL.

Step 6: Retrieve, then answer with citations

At question time, embed the question, fetch the top results, and pass them to the model with their timestamps. Two upgrades matter more than anything else:

1. Hybrid search. Combine vector search with keyword search (BM25). Transcripts are full of names, product terms and numbers that pure embeddings handle poorly.
2. Reranking. Retrieve 20 to 50 candidates, then use a reranker model to pick the best 5. This fixes a lot of "almost right" retrieval.

Then instruct the model to cite. A prompt like this works well:

Answer the question using only the transcript excerpts below. After each claim, cite the excerpt's timestamp in [mm:ss] format. If the excerpts don't contain the answer, say so instead of guessing.

Question: {question}

Excerpts:
{excerpts_with_timestamps}

Step 7: Evaluate before you tune

Don't tune chunk size by feel. Build a small evaluation set:

1. Pick 30 to 50 real questions your users would ask.
2. For each, note where the answer lives (source and approximate timestamp).
3. Run retrieval and check whether a chunk covering that timestamp appears in the top 3 or top 5. That's your hit rate.
4. Change one variable at a time (size, overlap, strategy, context header, reranker) and re-run.

Once retrieval is good, check answer quality separately: is the answer correct, and do its citations point to the right moments? An LLM judge can score this at scale, but spot check it by hand too.

Common mistakes

• Chunking the flat text and losing timestamps. You can't add citations back later.
• Chunks that are too small. 100-token chunks of speech often contain no complete idea.
• Splitting a question from its answer. Use speaker turns as boundaries for conversations.
• Ignoring caption duplicates. Repeated lines skew embeddings and waste tokens.
• Silent truncation. Chunks longer than the embedding model's limit lose their endings.
• No keyword search. Names and numbers go missing with vectors alone.
• Never re-indexing. If you change the chunker, re-embed everything, or old and new chunks won't be comparable.

Getting clean transcripts in the first place

Chunking quality depends on input quality. Timed segments with punctuation and speaker labels make every strategy above work better. If your source is video, our API returns transcripts as timed JSON segments that drop straight into the chunker above, and the free SRT to text converter helps if you're starting from caption files.

From there, the same chunks can feed more than search. See our guide on summarizing long documents with LLMs for map-reduce summaries built on the same pieces.