ai-agents-stream-parsing-error-when-using-litellm-with-an-ollama-model-that-outpu-568938c2

THEN To fix, either (1) create a custom Ollama Modelfile that overrides the TEMPLATE to remove the 'thinking' field, or (2) implement a custom streaming callback in LiteLLM proxy that filters out the 'thinking' key from each chunk before parsing. The callback can be registered via `litellm.callbacks.add_stream_modifier(my_modifier)` where `my_modifier` parses the chunk JSON, removes 'thinking', and returns the modified string.

import json
from litellm import streaming_handler

async def remove_thinking(chunk):
    data = json.loads(chunk)
    if "thinking" in data:
        del data["thinking"]
    return json.dumps(data)

litellm.callbacks.add_stream_modifier(remove_thinking)