Google Cloud RAG architecture
Primary guidance on preparing and retrieving external information for generative applications.
← Back to Compute College
Compute College
Chunking and document preparation
Prepare documents and retrieval chunks so relevant information stays together, searchable, and traceable.
Plain-English definition
Chunking divides a document into retrieval units that can be searched and placed into a model context. Good preparation preserves headings, document identity, page or section boundaries, effective dates, permissions, and the relationship between a claim and its surrounding explanation. Chunk size and overlap are workload choices, not universal constants.
Memory trick: A chunk is a retrieval unit, not a random slice. Keep the meaning, the metadata, and the boundary together.
Why it matters
A chunk that is too small can lose the definition or exception needed to interpret a sentence; a chunk that is too large can dilute relevance and consume context tokens. Poor extraction from tables, PDFs, or headings can make a correct document effectively unretrievable. Preparation quality often limits RAG before ranking or model choice does.
- Boundaries should preserve meaning, not merely split text every fixed number of characters.
- Metadata enables filtering by source, version, audience, region, and permission.
- Larger chunks can improve completeness while increasing irrelevant context and token cost.
Simple example
A refund policy should not be split so the sentence “except for digital goods” lands in a different chunk from the rule it qualifies. Store the policy name, section heading, effective date, and access scope with both chunks, then test queries about the exception rather than only the headline rule.
- Keep headings and nearby definitions with the content they govern.
- Preserve table headers when extracting rows.
- Test boundary questions that require a sentence before and after the matched phrase.
Example figures are illustrative calculations, not current quoted market prices.
Current example
Document retrieval reference
Google’s RAG guidance treats data preparation and retrieval as parts of the system rather than an afterthought. The best chunking choice depends on document structure, query shape, corpus rights, and the context budget available to the model.
Source discipline: the reference explains the concept or method; it is not a substitute for measuring a production workload under its actual provider, model, and data conditions.
Common mistake
One chunk size rarely fits every document type. A legal clause, a product FAQ, a table, and a code file have different boundaries and retrieval needs.
Practical takeaway
What you can do with this
Prepare a small corpus with source metadata and three chunk strategies. Test direct questions, boundary questions, and permission filters. Compare retrieval relevance, context tokens, answer grounding, and latency before scaling ingestion.
- Knowledge teams: preserve document version and effective-date metadata.
- Builders: make chunk boundaries and source IDs inspectable.
- Operators: monitor ingestion failures and re-embedding cost when documents change.
Decision check: can a reviewer reconstruct the source section and its governing metadata from the chunk that reached the model?
Compute College track
Prompt & Context Engineering
Step 13 of 18: Chunking and document preparation