multimodal_fallback_image_lossTier 1 · 70% confidence

ai-agents-multimodal-fallback--multimodal-request-with-image-url-fails-on-primary-8f579596

agent: ai_agents

When does this happen?

IF Multimodal request with image_url fails on primary OpenAI deployment and falls back to Gemini; image content is lost because the messages dict is passed by reference and mutated during transformation.

How others solved it

THEN Deep copy the messages payload before passing to each provider in the fallback chain. In the router's fallback logic (e.g., `_acompletion` / `run_async_fallback`), use `copy.deepcopy(messages)` to create an independent copy for each provider, preventing mutations from one provider's transformation (e.g., converting image_url to base64 for OpenAI) from affecting subsequent fallback requests.

import copy

async def _acompletion_with_fallback(...):
    for provider in fallback_providers:
        messages_copy = copy.deepcopy(messages)
        result = await provider.acompletion(model, messages_copy, ...)

Related patterns

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics