How to Summarize Long Documents with LLMs: Map-Reduce, Refine and Beyond

Summarizing a two-page memo with an LLM is a solved problem. Summarizing a 300-page report, a year of meeting notes or a three-hour interview is not. The model might run out of context, skim the middle, merge two people's opinions or confidently invent a number.

This guide covers the strategies that work for long inputs, when each one is worth its cost, the prompts to use and how to check the result. The examples are provider-agnostic: they work with any chat model you can call from code.

Start with the job, not the technique

"Summarize this" is the most common prompt and the least useful one. Before choosing a strategy, answer three questions:

• Who reads it, and what do they do next? An executive deciding whether to fund a project needs decisions and risks. A student needs concepts and definitions. A support team needs problems and fixes.
• How long should it be? A one-line description, a five-bullet brief and a two-page digest are different tasks.
• What must it never get wrong? Numbers, names, dates, who said what. These become the things you check.

Write the answers into your prompt. A summary with a clear audience and purpose is easier for the model to write and far easier for you to evaluate.

When a long context window is enough

Modern models accept very long inputs, so the first question is whether you need any special strategy at all.

A single call is a good choice when:

• the document fits comfortably, leaving room for instructions and the answer
• you want one general summary rather than an exhaustive extraction
• the document is well structured, with headings the model can follow

It gets risky when the input is very long or dense. Research on long-context models, notably the 2023 paper Lost in the Middle by Liu and colleagues, found that models use information at the start and end of a long input more reliably than information in the middle. Newer models have improved, but the practical lesson still holds: the longer the input, the more likely a specific detail gets skipped. If missing a detail matters, use one of the chunked strategies below, or at least test the single call against a version you know well.

Cost matters too. You pay for every input token on every call, so if you will ask many questions about the same document, summarizing or indexing it once is cheaper than re-sending it each time.

Step 1: Clean and measure the source

Garbage in, garbage summarized. Before anything else:

• Strip boilerplate: headers, footers, page numbers, navigation text, legal disclaimers repeated on every page.
• Fix extraction damage: PDFs often break words across lines and scramble tables. Spot-check a few pages.
• For transcripts, remove filler words ("um", "you know"), false starts and repeated phrases, but keep speaker labels and timestamps.
• Count tokens with your provider's tokenizer or token-counting endpoint. A rough rule for English is about four characters per token, but measure rather than guess.

Step 2: Split along natural boundaries

If you need to chunk, split where the content already breaks: chapters, headings, paragraphs, slides or speaker turns. Fixed character counts cut sentences and ideas in half. Add a small overlap (a paragraph or two) so a point that spans a boundary isn't lost, and keep metadata on every chunk: its position, heading, page numbers or timestamps.

Our guide to chunking transcripts for LLMs goes deeper on chunk size and overlap.

The four strategies

Map-reduce

Summarize each chunk independently, then summarize the summaries.

• Strengths: chunks run in parallel, so it's fast. It scales to very long documents, and each chunk gets the model's full attention.
• Weaknesses: each chunk is summarized without knowing the rest, so the same point can appear five times and cross-chunk connections get lost. The reduce step has to deduplicate.

Refine

Summarize the first chunk, then pass the running summary plus the next chunk to the model and ask it to update the summary. Repeat to the end.

• Strengths: keeps order and narrative, good for stories, case studies and arguments that build.
• Weaknesses: strictly sequential, so slow. Early content tends to dominate and late details can get squeezed out as the summary hits its length limit.

Hierarchical

Map-reduce in layers. Summarize chunks, group the chunk summaries by section, summarize each section, then summarize the sections. This mirrors the document's own structure.

• Strengths: the best choice for books, long reports and multi-session meeting series. You also get useful intermediate outputs, like per-chapter summaries.
• Weaknesses: more calls, and errors at the bottom layer propagate upward.

Extract, then summarize

Instead of asking for a summary per chunk, ask for structured notes: key claims, decisions, numbers, open questions and notable quotes, each with its location. Then write the final summary from the merged notes.

• Strengths: the most faithful approach. Notes are easy to deduplicate, filter and check, and you can reuse them for other outputs.
• Weaknesses: a little more prompt work, and the final prose can feel mechanical unless you give the writing step a clear voice and audience.

In practice, extract-then-summarize combined with a map-reduce or hierarchical structure is the most robust default for long documents.

Prompts that work

A map step that extracts structured notes:

You are reading part {i} of {n} of a longer document. Extract notes from this part only. Do not use outside knowledge.

Return JSON with these keys:
- "claims": the main points made, each one sentence
- "facts": numbers, dates, names and decisions, copied exactly
- "quotes": up to 3 short verbatim quotes worth keeping
- "open_questions": anything raised but not resolved

Give every item a "source" field with the location label shown at the start of the text.

If this part is boilerplate or has nothing substantive, return empty lists.

TEXT:
{chunk}

A reduce step that writes for a specific reader:

Below are notes extracted from every part of a document, in order.

Write a summary for {audience} who needs to {goal}.

Rules:
- Length: {length}.
- Merge points that repeat across parts.
- Keep every number, name and date exactly as written in the notes.
- After each sentence, cite its sources in square brackets, like [p12] or [00:14:32].
- If the notes disagree, say so instead of picking one.
- Do not add anything that isn't in the notes.

NOTES:
{notes}

Asking for JSON in the map step pays off. If your provider supports structured outputs or a JSON schema, use it, so you don't have to parse free text.

A minimal Python implementation

This sketch uses a placeholder llm() function. Swap in your provider's SDK call.

import json
from concurrent.futures import ThreadPoolExecutor

# MAP_PROMPT and REDUCE_PROMPT are the two prompts above, as format strings

def llm(prompt: str) -> str:
    """Call your model of choice and return the text response."""
    raise NotImplementedError

def chunk(paragraphs: list[str], max_chars: int = 12_000, overlap: int = 1):
    chunks, current = [], []
    for p in paragraphs:
        if current and sum(len(x) for x in current) + len(p) > max_chars:
            chunks.append(current)
            current = current[-overlap:]  # carry a little context forward
        current.append(p)
    if current:
        chunks.append(current)
    return ["\n\n".join(c) for c in chunks]

def extract(i: int, n: int, text: str) -> dict:
    prompt = MAP_PROMPT.format(i=i + 1, n=n, chunk=text)
    return json.loads(llm(prompt))

def summarize(paragraphs, audience, goal, length="5 bullet points"):
    chunks = chunk(paragraphs)
    with ThreadPoolExecutor(max_workers=8) as pool:
        notes = list(pool.map(lambda a: extract(a[0], len(chunks), a[1]), enumerate(chunks)))
    merged = json.dumps(notes, ensure_ascii=False)
    # If the notes are still too long, group and reduce them in layers first
    return llm(REDUCE_PROMPT.format(audience=audience, goal=goal, length=length, notes=merged))

Things to add before production: retries with backoff for rate limits, validation that each map response parsed as JSON, a token count check on merged that triggers an extra reduce layer, and caching so re-running a summary doesn't re-pay for unchanged chunks.

Summarizing transcripts specifically

Spoken content behaves differently from written documents, and it's one of the most common long-input jobs: lectures, podcasts, interviews, webinars and meetings.

• Clean it, but not too much. Remove filler words and false starts, but keep hedges like "I think" or "probably". They change meaning, and a summary that turns "we might ship in March" into "we ship in March" is wrong.
• Keep speakers. "The CFO objected to the budget" and "someone objected" are very different summaries. Chunk on speaker turns and keep labels in every chunk.
• Keep timestamps. Put the start time at the top of each chunk and ask the model to cite it. Readers can then jump straight to the moment a point was made, and you can verify claims in seconds.
• Expect repetition. Speakers circle back. Tell the reduce step explicitly to merge repeated points and to keep the most complete version.
• Watch for transcription errors. Names and technical terms are where speech-to-text slips. Give the model a glossary of correct spellings, or fix them before summarizing.

If you're starting from video or audio, a clean transcript with timestamps does most of the work. Our speech-to-text API guide covers how to get one, and our free AI summarizer shows the timestamp-citation pattern in action.

How to check a summary is faithful

A fluent summary is not the same as a correct one. Build checking into the workflow rather than trusting the output.

1. Spot-check against citations. Pick five claims at random and look up their cited source. If the summary has no citations, that's the first thing to fix.
2. Check the things that must never be wrong. Compare every number, date and name in the summary with the source text. A simple script can flag numbers that don't appear in the source at all.
3. Use a second model as a judge. Give it one claim and its cited passage, and ask whether the passage supports the claim, contradicts it or doesn't mention it. Run this on every claim for high-stakes summaries. It's not perfect, but it catches a lot.
4. Check coverage, not just accuracy. Write three to five questions the summary should answer before you run it. Afterwards, see whether it does.
5. Look for patterns in the errors. If mistakes cluster in one section, the chunking or the cleanup is probably the cause, not the model.

Keep a small set of documents with summaries you've verified by hand. Whenever you change the prompt, chunk size or model, re-run them and compare. It's the cheapest way to avoid quietly getting worse.

Cost and latency in practice

A few rules of thumb hold across providers:

• Input tokens dominate. Every strategy reads the whole document at least once. Refine and hierarchical re-read summaries on top of that.
• Parallel beats sequential. Map-reduce and extract-then-summarize finish in roughly the time of the slowest chunk plus the reduce call. Refine takes the sum of every call.
• Use a cheaper model for the map step. Extraction is simpler than writing. A smaller, faster model often does it well, with a stronger model writing the final summary.
• Cache aggressively. Store chunk notes keyed by a hash of the chunk text. Re-summarizing for a different audience then costs one reduce call.
• Use prompt caching if your provider offers it and you send the same long instructions or document many times.

The default recipe

If you want one approach to start with: clean the text, split it on natural boundaries with light overlap, extract structured notes with locations from each chunk in parallel, reduce them into a summary for a named reader with citations, and spot-check the citations. Change it only when your evaluation set tells you to.