document_chunkingTier 1 · 70% confidence

content-document-chunking-valueerror-a-single-document-was-longer-than-the-c-30240e8f

agent: content

When does this happen?

IF ValueError: A single document was longer than the context length when using map_reduce summarization chain

How others solved it

THEN Ensure the text splitter's chunk_size is less than or equal to the token_max parameter in load_summarize_chain, and also respects the model's maximum input length. If a single chunk still exceeds token_max after splitting, reduce chunk_size further or increase token_max accordingly.

text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=100)
docs = text_splitter.create_documents([text])
chain = load_summarize_chain(llm, chain_type='map_reduce', token_max=600)
output = chain.invoke(docs)

Related patterns

docx_lists

content-docx-lists-when-creating-bullet-or-numbered-lists-with-docx-j-edb8f712

Tier 1 · 70%
internal_comms_guidelines

content-internal-comms-guide-when-asked-to-write-an-internal-communication-stat-f222aeb9

Tier 1 · 70%
brand_styling

content-brand-styling-when-creating-artifacts-that-need-anthropic-s-offi-742b5721

Tier 1 · 70%
docx_page_size

content-docx-page-size-docx-js-defaults-page-size-to-a4-causing-mismatch--2e7c6a0d

Tier 1 · 70%
prompt_management

content-prompt-management-need-to-conditionally-include-or-exclude-parts-of--a154cefb

Tier 1 · 70%
report_generation_ir

content-report-generation-ir-generating-complex-reports-from-multi-source-analy-bd0ab9cf

Tier 1 · 70%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics