infrastructure-distributed-evaluati-evaluating-gpt-j-gpt-neox-or-codegen-models-with-m-8b11f515

THEN Ensure tensors are contiguous before all-gather operations in distributed evaluation. Add `.contiguous()` calls after operations like transpose or view in the model's forward pass, particularly in attention layers. If you cannot modify the library code, monkey-patch the model's forward method in your script to call `.contiguous()` on relevant tensors before they are gathered.

```python
# In the model's attention forward, after computing Q, K, V:
query = query.contiguous()
key = key.contiguous()
value = value.contiguous()
```
Or apply a wrapper: `model.transformer.h[0].attn._attn = lambda q,k,v: original_attn(q.contiguous(), k.contiguous(), v.contiguous())`