Chunking

Chunking is the first technical decision of RAG: the size, the overlap and the cutting strategy determine what retrieval can find later. Bad pieces become answers without context.

LangChainLlamaIndexUnstructuredspaCy

What it is

Chunking is the process of splitting a document into smaller pieces that will be indexed and retrieved. The size of the piece is not a detail: it defines what fits in the model's window, what the unit of search is, and what the granularity of the citation that appears in the answer is.

In almost every RAG architecture, chunking happens once at ingestion and never again. The standard pipeline reads the document, applies a splitter, generates one embedding per block, and moves on. If the cut was poorly thought out, the whole index carries the problem until someone redoes the indexing.

Size and overlap

The most common question is 'what is the right size?'. The honest answer is 'it depends'. For short questions and factual answers, smaller blocks work better: each block carries a single idea, so retrieval brings back the exact passage. For questions that require reasoning across paragraphs, larger blocks preserve context, and blocks that are too small break the argument in the middle.

Overlap is the second parameter. Without overlap, an idea that starts at the end of one block and continues at the start of the next is orphaned: neither block has the complete meaning. With an overlap of 10 to 20% of the block size, the start of the next block repeats the end of the previous one, so the boundary never cuts a sentence in half.

Typical parameterstext

  block size            200 to 1000 tokens
  overlap               10% to 20% of the block
  rule of thumb         block ~512 tokens, overlap ~50

  small block  -> precise citation, short context
  large block  -> long reasoning, noisy citation

Cutting strategies

Three families dominate practice. The first is cutting by characters or tokens, simple and predictable, and it works when the document is clean prose. The second is recursive cutting by separators, which tries to break first by paragraph, then by sentence, then by word, and by character only as a last resort. It preserves the structure of the text without needing a model. The third is semantic cutting, which groups sentences by similarity and only cuts when the next sentence has low relation to the previous one. It costs more and uses embeddings, but produces coherent blocks.

There is also cutting by structure: headings, sections, table cells, code paragraphs. In documents with clear markup (Markdown, HTML, converted docx), splitting by structure and keeping the hierarchy in the metadata gives more useful chunks than any generic splitter.

In practice

LangChain offers the recursive splitter ready to use. The example configures size and overlap and applies it to any text. For Portuguese text, it is worth splitting by paragraph, sentence and word, in that order.

chunking/splitter.pypython

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512,
    chunk_overlap=50,
    separators=[
        "\n\n",     # paragraph
        "\n",       # line
        "(?<=[.!?])\\s+",  # end of sentence in PT-BR (lookbehind)
        " ",        # word
        "",         # character (last resort)
    ],
    length_function=len,
    is_separator_regex=True,
)

with open("manual.txt", encoding="utf-8") as f:
    texto = f.read()

blocos = splitter.split_text(texto)
for i, bloco in enumerate(blocos):
    print(f"#{i:03d}  {len(bloco)} chars  {bloco[:60].replace(chr(10), ' ')}...")

Strengths

stability: deterministic chunking produces the same blocks for the same document, so the index stays reproducible
granularity control: adjusting size and overlap is a direct lever on retrieval quality
compatibility: almost every framework (LangChain, LlamaIndex, Haystack) ships a ready-made splitter, so the barrier to entry is low

Limitations

semantic blindness: a recursive splitter cuts by separator, not by meaning; an idea that crosses the boundary disappears
lost structure: splitting by character ignores headings, lists and tables; in mixed documents, a type-specific splitter is better
reindexing cost: changing the chunking strategy forces reindexing everything; on large bases, that becomes a migration

connections on the map

Chunkinggenerates vectorsEmbeddings

updated on 14/10/2026 · sources to go deeper · report an error

Study before

LLM - large language model

Unlocks

Retrieval

test what you learned

Loading questions…