Chunking Strategies for RAG: What Actually Works

Garbage Chunks In, Garbage Answers Out.

The Context Problem

You have a 50-page PDF. You cant feed the whole thing into the embedding model at once (usually). You have to cut it up into chunks.

Most developers use the default: CharacterTextSplitter(chunk_size=1000).

This is why their RAG apps fail.

Why Naive Chunking Fails ✂️

Imagine a sentence spans across the cut:

Chunk 1: ...the revenue for 2023 was
Chunk 2: $10 million, which was a disappointment...

If the user asks What was 2023 revenue?, the search engine might find Chunk 2, but without the context of revenue for 2023, its meaningless. Or it finds Chunk 1, which has no answer.

Better Strategies

1. Recursive Character Splitting (The Baseline)

Instead of cutting blindly at 1000 characters, try to cut at paragraphs (\n\n). If thats too big, cut at sentences (. ). If thats too big, cut at words.

Pros: Keeps semantic units together better than fixed-size.
Cons: Still loses context if a topic spans multiple paragraphs.

2. Parent-Child Chunking (The Pro Move) 👨‍👧

Child Chunks: Small (e.g., 200 tokens). These are what we embed and search against. They are precise.
Parent Chunks: Large (e.g., 2000 tokens). This is what we send to the LLM.
How it works: When a search matches a small child chunk, we fetch its parent. This gives the LLM the specific needle plus the surrounding haystack of context.

3. Semantic Chunking 🧠

Using an embedding model to scan the document sentence by sentence. When the topic changes (cosine similarity drops), we make a cut.

Pros: Creates naturally coherent chunks.
Cons: Expensive and slow to index.

The Overlap Rule

Always overlap your chunks (e.g., 10-20%). This ensures that sentences on the boundary arent sliced in half.

Conclusion

Chunking is an art. For contracts, we chunk by Clause. For code, we chunk by Function. For manuals, we chunk by Header.

Dont use the default. Know your data.

Next in the series: [Embeddings Deep Dive](/blog/embeddings-deep-dive-guide)

How This Article Was Made

This article is a live example of the AI-enabled content workflow we build for clients.

Stage	Who	What
Research	Claude Opus 4.5	Analyzed current industry data, studies, and expert sources
Curation	Tom Hundley	Directed focus, validated relevance, ensured strategic alignment
Drafting	Claude Opus 4.5	Synthesized research into structured narrative
Fact-Check	Human + AI	All statistics linked to original sources below
Editorial	Tom Hundley	Final review for accuracy, tone, and value

The result: Research-backed content in a fraction of the time, with full transparency and human accountability.

Why We Work This Way

Were an AI enablement company. It would be strange if we didnt use AI to create content. But more importantly, we believe the future of professional content isnt AI vs. Human—its AI amplifying human expertise.

Every article we publish demonstrates the same workflow we help clients implement: AI handles the heavy lifting of research and drafting, humans provide direction, judgment, and accountability.

Want to build this capability for your team? Lets talk about AI enablement →