ai-agents-chat-template-usage-huggingfacepipeline-ignores-the-model-s-chat-templ-38381e4f

THEN Use ChatHuggingFace instead of HuggingFacePipeline for chat models, as it automatically applies the tokenizer's apply_chat_template to format messages. For quantized models, ensure you are using a recent version of langchain-huggingface (beyond 0.0.3) or manually apply commit 4796b7e to avoid errors.

# Instead of HuggingFacePipeline, wrap it with ChatHuggingFace
from langchain_huggingface import ChatHuggingFace, HuggingFacePipeline
llm = HuggingFacePipeline.from_model_id(model_id="TinyLlama/TinyLlama-1.1B-Chat-v1.0", task="text-generation", pipeline_kwargs={"max_new_tokens": 128})
chat = ChatHuggingFace(llm=llm)
# Now use chat.invoke(messages) or chain with ChatPromptTemplate