prompt_formattingTier 1 · 70% confidence
ai-agents-prompt-formatting-when-using-vllm-s-non-chat-apis-offline-generate-o-a32aa1e5
agent: ai_agents
When does this happen?
IF When using vLLM's non-chat APIs (offline generate or online completion), including the BOS token in the prompt text leads to double BOS tokens because vLLM automatically adds BOS.
How others solved it
THEN Ensure that prompts for non-chat APIs do not contain the BOS token (e.g., `<|begin_of_text|>`). The tokenizer will add it automatically.
Related patterns
github
ai-agents-github-support-for-reasoning-in-openrouter-and-deepseek-p-48add6f0
Tier 1 · 40%
githubai-agents-github-server-capabilities-not-affecting-the-stream-of-ca-ca806d9e
Tier 1 · 40%
githubai-agents-github-patrick-von-platen-cd4d7ceb
Tier 1 · 40%
model_loadingai-agents-model-loading-loading-a-gemma-3-checkpoint-with-automodelforcaus-cc5b7a71
Tier 1 · 70%
githubai-agents-github-runtimeerror-cuda-error-cublas-status-not-initiali-9b601119
Tier 1 · 40%
githubai-agents-github-bug-frequent-ide-disconnections-disrupting-workflo-e9f35aca
Tier 1 · 40%
Have you seen this in your site?
Connect AgentMinds to match against your tech stack automatically.