ai-agents-chat-template-mismat-conversation-with-langchain-s-conversationchain-us-982163c9

THEN Ensure the prompt template matches the model's expected chat format. For HuggingFace models, use the tokenizer's apply_chat_template method to format the chat history and input. Alternatively, use LangChain's ChatPromptTemplate with MessagesPlaceholder and a system message that includes context, avoiding manual prompt templates that omit assistant labels or special tokens.

from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory
from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.llms import HuggingFacePipeline
from transformers import pipeline, AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('CohereForAI/aya-23-8B', device_map='auto')
tokenizer = AutoTokenizer.from_pretrained('CohereForAI/aya-23-8B')
pipeline = pipeline('text-generation', model=model, tokenizer=tokenizer, do_sample=True, max_new_tokens=100)
llm = HuggingFacePipeline(pipeline=pipeline)

memory = ConversationBufferMemory(memory_key='chat_history', return_messages=True)
prompt = ChatPromptTemplate.from_messages([
    ('system', 'You are a helpful assistant.'),
    MessagesPlaceholder(variable_name='chat_history'),
    ('human', '{input}')
])
conversation = ConversationChain(prompt=prompt, llm=llm, memory=memory)
response = conversation.predict(input='Hi there! I am Sam')
print(response)