RAG Chunking Strategies 2026: Fixed Size, Recursive and Semantic Splitting Compared。Retrieval quality is capped at the moment you split the document. A chunk serves two roles: the embedding inpu

📜 Table of Contents

• Five strategies
• Parameters
• Match chunks to query types
• Validate with data
• Practical notes
• FAQ
• Related

RAG Chunking Strategies in 2026: Fixed, Recursive, Semantic

Retrieval quality is capped at the moment you split the document. A chunk serves two roles: the embedding input and the context handed to the model. Too large and several topics average out into a vague vector; too small and pronouns lose their referents.

Five strategies

1. Fixed size: cut every N characters with an overlap window. Structure-blind, best for messy plain text. Overlap of 10-20% prevents mid-sentence cuts.
2. Recursive: split on paragraphs, then sentences, then characters. RecursiveCharacterTextSplitter is the workhorse. For Chinese, add punctuation to the separator list, and remember chunk_size counts characters, so copying 1000 from an English tutorial produces very long chunks.
3. Structure-aware: split Markdown by headers or HTML by tags, keeping hierarchy in metadata. Best for manuals and technical docs.
4. Semantic: embed sentences and cut where similarity drops. Good boundaries, but slow, costly and threshold-sensitive.
5. Parent-child: retrieve small child chunks for precision, return the larger parent for context. ParentDocumentRetriever implements this and usually beats tuning chunk size alone.

Parameters

No universal optimum exists. Public benchmarks favor a few hundred tokens for typical Q&A, but your corpus and question mix change the answer.

Match chunks to query types

Fact lookup wants small, single-fact chunks. Process questions want complete steps even if long. Comparison questions need one full argument per chunk. Summaries want section-level chunks with hierarchy. Mixed traffic often justifies two indexes.

Validate with data

Pull 50-100 real questions, label the source passage for each, build indexes with two or three strategies, then measure whether the label appears in top 3 or top 5. If semantic splitting beats recursive by only a few points, skip the cost. Inspect individual failures rather than averages, and watch the chunk-length distribution for tiny fragments that signal a bad separator.

Practical notes

• Chinese characters are not English tokens; do not reuse English parameters
• Split tables by row with the header repeated, or table retrieval fails
• Keep code blocks intact
• Attach file name, section and page metadata so answers can cite sources
• Rebuild incrementally; full re-chunking does not scale

FAQ

What chunk size? Start at recursive, 500-800 characters, 10% overlap, then measure.

Is overlap required? Barely, with good sentence separators. Character cuts benefit most, at the cost of index size and duplicate hits.

Is semantic chunking worth it? For long, topic-shifting documents with high quality demands, yes. For routine Q&A, recursive gets most of the gain.

Chunking or reranking first? Chunking. Reranking cannot rescue a candidate set that never contained the answer.

Related

• Document chunking strategy
• RAG evaluation metrics
• Open-source RAG framework comparison