ai-agents-attention-mask-when-using-attn-implementation-sdpa-with-an-attent-7a52ec6c

THEN Modify `_prepare_4d_causal_attention_mask_with_cache_position` to accept an `is_causal` parameter and return a non-causal mask when `is_causal=False`. Alternatively, simplify the mask preparation by relying on PyTorch's SDPA built-in `is_causal` parameter and using a simpler mask when possible, reducing unnecessary allocations.

def _prepare_4d_causal_attention_mask_with_cache_position(
    attention_mask, sequence_length, target_length, dtype, device, cache_position, batch_size, is_causal=True, **kwargs
):
    if is_causal and attention_mask is None:
        # let SDPA handle causality
        return None
    # else compute mask manually respecting is_causal
    ...