ai-agents-tool-call-parsing-when-using-llama-3-1-with-langchain-and-vllm-s-ope-76b14c3a

THEN Add a custom parsing function to the LangChain chain that strips the `<python_tag|>` prefix from the AI message content, then attempts to parse the remaining JSON to extract tool name and parameters, and re-assigns them as ToolCall objects. This workaround intercepts the output and enables tool execution.

def parse_tool_call(x):
    if isinstance(x, AIMessage):
        if x.content.startswith('<|python_tag|>'):
            x.content = x.content.replace('<|python_tag|>', '')
        try:
            parsed = json.loads(x.content)
            if parsed.get('name') and parsed.get('parameters'):
                x.tool_calls = [ToolCall(name=parsed['name'], args=parsed['parameters'], id=x.id)]
        except json.JSONDecodeError:
            pass
    return x

chain = llm_with_tools | RunnableLambda(parse_tool_call)