ai-agents-llava-multiple-image-when-using-llava-or-pixtral-models-with-multiple-i-29c052d3

THEN Downgrade transformers to version 4.45.2 or apply the fix from PR #34332. For LLaVa, ensure processor.patch_size and vision_feature_select_strategy are set as in the reproduction. Alternatively, use a single image per input to avoid the issue.

import torch
from transformers import LlavaForConditionalGeneration, LlavaProcessor

model = LlavaForConditionalGeneration.from_pretrained("llava-hf/llava-1.5-7b-hf")
processor = LlavaProcessor.from_pretrained("llava-hf/llava-1.5-7b-hf")
processor.patch_size = 14
processor.vision_feature_select_strategy = "default"

device = torch.device("cuda")
model = model.eval().to(device)

# This will fail in transformers >=4.46.0:
inputs = processor(
    text=["Sentence with two images 1. <image> 2. <image>", "Sentence with one image <image>"],
    images=torch.rand((3, 3, 336, 336), dtype=torch.float),
    return_tensors="pt",
    truncation=True,
    padding=True,
)
inputs = inputs.to(device)
with torch.no_grad():
    model(**inputs)
# Workaround: downgrade transformers to 4.45.2