ai-agents-structured-output-re-when-using-structured-llm-in-llama-index-the-llm-r-1d255352

THEN Implement a retry mechanism that catches Pydantic validation errors during structured output parsing. After receiving the LLM response, attempt to parse it into the Pydantic model. If parsing fails, re-prompt the LLM with an error message asking for valid output. Limit retries to a maximum number (e.g., 3) and raise a custom exception if all fail. This approach mirrors the ReflectionWorkflow pattern used in llama_index examples.

from pydantic import BaseModel, ValidationError

async def structured_chat_with_retry(llm, messages, output_cls, max_retries=3):
    for attempt in range(max_retries):
        response = await llm.chat(messages)
        try:
            return output_cls.model_validate_json(response.message.content)
        except ValidationError as e:
            messages.append({"role": "user", "content": f"Invalid output: {e}. Please respond with valid JSON for {output_cls.model_json_schema()}."})
    raise Exception("Failed to get valid structured output after retries")