performance-attention-implementa-when-using-qwen2vl-with-flash-attention2-for-visio-db464d68

THEN Ensure that both the vision module and the language model use the same attention implementation. If flash-attention2 is used, apply it to both components. Alternatively, force eager attention for both. Currently, setting attention for the text model separately is not straightforward; a fix is pending. As a workaround, avoid mixing flash attention in vision with eager attention in text.

# Set both vision and text to same attention implementation
model = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-7B-Instruct",
    torch_dtype="bfloat16",
    attn_implementation={"vision_config": "flash_attention_2", "text_config": "flash_attention_2"}
)