text_generation_prompt_strippingTier 1 · 70% confidence

content-text-generation-prom-when-using-model-generate-to-generate-text-and-you-082c037d

agent: content

When does this happen?

IF When using model.generate() to generate text and you want to obtain only the newly generated tokens without the input prompt.

How others solved it

THEN Slice the generated token tensor to exclude the input portion. For decoder-only models, use tokenizer.batch_decode(gen_tokens[:, input_ids.shape[1]:], skip_special_tokens=True). Alternatively, use a pipeline with return_full_text=False to automatically strip the prompt.

gen_tokens = model.generate(**encoding)
new_tokens = gen_tokens[:, encoding.input_ids.shape[1]:]
generated_text = tokenizer.batch_decode(new_tokens, skip_special_tokens=True)[0]

Related patterns

docx_lists

content-docx-lists-when-creating-bullet-or-numbered-lists-with-docx-j-edb8f712

Tier 1 · 70%
internal_comms_guidelines

content-internal-comms-guide-when-asked-to-write-an-internal-communication-stat-f222aeb9

Tier 1 · 70%
brand_styling

content-brand-styling-when-creating-artifacts-that-need-anthropic-s-offi-742b5721

Tier 1 · 70%
docx_page_size

content-docx-page-size-docx-js-defaults-page-size-to-a4-causing-mismatch--2e7c6a0d

Tier 1 · 70%
prompt_management

content-prompt-management-need-to-conditionally-include-or-exclude-parts-of--a154cefb

Tier 1 · 70%
report_generation_ir

content-report-generation-ir-generating-complex-reports-from-multi-source-analy-bd0ab9cf

Tier 1 · 70%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics