ai-agents-tool-calling-workaro-unhandled-tool-invocations-with-llama-3-1-when-usi-c4fee768

THEN Implement a custom parser that strips the '<|python_tag|>' prefix from the assistant message content, attempts to parse the remainder as JSON containing 'name' and 'parameters', and constructs a ToolCall object to pass to the LangChain chain. This workaround bridges the gap until vLLM fully supports native tool calls for Llama 3.1.

def parse_tool_call(x):
    if isinstance(x, AIMessage):
        if x.content.startswith('<|python_tag|>'):
            x.content = x.content.replace('<|python_tag|>', '')
        try:
            parsed = json.loads(x.content)
            if 'name' in parsed and 'parameters' in parsed:
                x.tool_calls = [ToolCall(name=parsed['name'], args=parsed['parameters'], id=x.id)]
        except json.JSONDecodeError:
            pass
    return x

chain = llm_with_tools | RunnableLambda(parse_tool_call)