ai-agents-conversation-looping-conversationchain-with-huggingface-models-loops-or-92d99832

THEN Use the model's tokenizer.apply_chat_template() to format messages according to the model's expected chat template, or switch to LangChain's ChatHuggingFace wrapper which handles formatting automatically. Avoid manually constructing prompt strings without incorporating the model-specific chat structure.

# Instead of a custom PromptTemplate, use tokenizer.apply_chat_template:
from langchain.llms import HuggingFacePipeline
from transformers import pipeline, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("CohereForAI/aya-23-8B")
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
llm = HuggingFacePipeline(pipeline=pipe, model_kwargs={"tokenizer": tokenizer})
# Then use the chain without a manual prompt—LangChain will format via tokenizer.