ai-agents-sliding-window-off-b-when-using-flash-attention-with-a-sliding-window-t-f0cd6be0

THEN Change the window_size argument in the flash attention call from (sliding_window, sliding_window) to (sliding_window-1, sliding_window) when causal masking is applied. This ensures that the effective window size matches the expected behavior and other implementations.

# Instead of:
flash_kwargs = {"window_size": (sliding_window, sliding_window)} if use_sliding_windows else {}
# Use:
flash_kwargs = {"window_size": (sliding_window - 1, sliding_window)} if use_sliding_windows else {}