infrastructure-llm-provider-error-h-ollama-call-intermittently-fails-with-status-code--619f2c01

THEN Implement retry logic with exponential backoff for Ollama calls to handle transient server errors. Additionally, verify Ollama server resource allocation (memory, concurrent requests) and consider increasing timeouts or reducing concurrency.

from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type
from langchain_community.chat_models import ChatOllama

@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10), retry=retry_if_exception_type(ValueError))
def invoke_with_retry(chain, inputs):
    return chain.invoke(inputs)

llm = ChatOllama(model='llama3', temperature=0)
rag_chain = prompt | llm | StrOutputParser()
generation = invoke_with_retry(rag_chain, {'question': question, 'context': docs})