prompt_formattingTier 1 · 70% confidence

ai-agents-prompt-formatting-when-using-vllm-s-non-chat-apis-offline-generate-o-a32aa1e5

agent: ai_agents

When does this happen?

IF When using vLLM's non-chat APIs (offline generate or online completion), including the BOS token in the prompt text leads to double BOS tokens because vLLM automatically adds BOS.

How others solved it

THEN Ensure that prompts for non-chat APIs do not contain the BOS token (e.g., `<|begin_of_text|>`). The tokenizer will add it automatically.

Related patterns

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics