Every RAG Guide Says Use 10-20% Chunk Overlap. A 2026 Study Found It Does Nothing.

If you have read any guide to building a RAG system in the last two years, you have been told to overlap your chunks. Usually by 10 to 20 percent. The reasoning sounds airtight: if a sentence gets split across a chunk boundary, overlap means at least one chunk contains the whole thought.

In January 2026, Sofia Bennani and Charles Moslonka published a systematic analysis of chunking strategies that measured this directly. Their finding, stated plainly in the abstract: overlap provides no measurable benefit and increases indexing cost.

We build retrieval systems for small businesses, and we had been using overlap by default because everyone does. This paper made us go back and check. What follows is what it actually says, what we changed, and where we think it stops applying.

What the study measured

The paper is A Systematic Analysis of Chunking Strategies for Reliable Question Answering, submitted 20 January 2026. It evaluates token, sentence, semantic and code chunking methods using SPLADE retrieval with Mistral-8B generation on the Natural Questions dataset.

It reports four findings. Every one of them contradicts something you will read in a popular chunking tutorial.

Why the overlap result is the interesting one

The other three findings are refinements. The overlap result is a straight contradiction of near-universal practice.

Overlap is not free. If you overlap 500-token chunks by 20 percent, you are storing and embedding roughly 20 percent more text on every document and every re-index. At a small corpus that is a few pounds a month, so we are not going to pretend the money is the point. The point is that it is a setting nobody has justified. Defaults that go unmeasured are how systems accumulate configuration no one can explain, and this one turns out to have had no evidence behind it at all.

The reason the intuition fails is worth understanding. Overlap protects against a boundary splitting a fact in half. But retrieval systems do not return one chunk. They return the top k. If the relevant material sits at a boundary, both adjacent chunks tend to score highly on the query anyway, so both get retrieved. The overlap was insuring against a failure the ranking already handles.

The context cliff is the finding that will cost you money

The paper identifies a context cliff past roughly 2,500 tokens, beyond which answer quality drops.

This one matters commercially because the instinct when a RAG system gives a wrong answer is to feed it more. Retrieve ten chunks instead of five. Widen the window. Modern models advertise enormous context windows, so why not use them.

On this evidence, past a certain point that makes the system worse and the bill larger at the same time. You are paying more per query for a lower-quality answer. If you have a RAG system that got worse after someone increased the retrieval count, this is a plausible explanation and it is cheap to test by turning it back down.

Semantic chunking is often a purchase you do not need

Semantic chunking splits documents at meaning boundaries rather than fixed lengths. It requires running an embedding model across the document during indexing, which costs more and takes longer than splitting on sentences.

The paper found sentence chunking matched semantic chunking up to around 5k tokens. For a small business knowledge base built from policy documents, product pages, support articles and PDFs, a great deal of that content sits under that threshold.

These two numbers measure different things and are easy to conflate. The 5k figure is about how large a document the chunking methods were compared on. The 2.5k figure is about how much retrieved text you stuff into the prompt at query time. They are not in conflict, but reading them as one number will lead you somewhere strange.

We make a longer version of this argument about vector databases, where the threshold is measured rather than assumed.

What we changed, and what we did not

We turned overlap off on new builds and measured before and after on the client's own question set. We have not seen a case yet where turning it off hurt. We are not claiming that proves the paper right; our sample is small and our evaluation is less rigorous than theirs. It is consistent, which is enough to make overlap something we now justify rather than assume.

We did not change chunking method wholesale. Structure-aware chunking, where you split on the document's own headings and sections, still beats naive sentence splitting on the messy PDFs small businesses actually have, and that is not what this paper tested.

The limits of this result

One paper, one dataset, one retriever, one generator. Natural Questions is a Wikipedia-based benchmark of short factual questions. That is not the same as a support bot answering questions over a company's own inconsistent internal documentation.

Specifically, we would not assume the overlap finding transfers to documents with heavy cross-references, legal text where a clause qualifies a distant clause, or transcripts where a single answer sprawls across many turns. Those are exactly the cases where boundary effects should matter most, and the paper does not cover them.

The safe reading is not "overlap is useless." It is "overlap is a cost you should be able to justify with a measurement, and most teams have never measured it."

What to do with this

• Build a question set from real questions your users ask, with known correct answers. Thirty is enough to start.
• Measure your current system against it. Without this, every other step is guesswork.
• Turn overlap off and re-measure. Then try retrieving fewer chunks, especially if your total context runs past ~2.5k tokens.

None of these require rebuilding anything. They are configuration changes measured against a fixed question set, which is the only way to know whether a RAG system is improving or you are just changing it.

The question set is the part people skip, and it is the part that matters. Thirty real questions with known answers, written down before you change anything, will tell you more about your retrieval system than any amount of reading about chunking. That is where we start on AI and automation work, and you can do it yourself before you talk to us.