ollama_thinking_field_handlingTier 1 · 70% confidence

ai-agents-ollama-thinking-fiel-when-using-ollama-with-a-model-that-outputs-a-thin-872ae8e4

agent: ai_agents

When does this happen?

IF When using Ollama with a model that outputs a 'thinking' field in its streaming JSON chunks, LiteLLM raises APIConnectionError because its chunk parser does not recognize this field.

How others solved it

THEN Implement a custom streaming callback that strips or renames the 'thinking' field before the chunk is parsed, or create a custom Ollama Modelfile that modifies the model's template to exclude the 'thinking' field from the response. For a callback, intercept each chunk string, parse it as JSON, remove the 'thinking' key, and return the modified string.

import json

async def fix_ollama_chunk(chunk: str) -> str:
    try:
        data = json.loads(chunk)
        if 'thinking' in data:
            del data['thinking']
            return json.dumps(data)
    except json.JSONDecodeError:
        pass
    return chunk

Related patterns

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics