performance-device-optimization-when-using-hugging-face-transformers-trainer-on-an-79988a73

THEN Override the `device` property of `TrainingArguments` to check for MPS availability. Alternatively, set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to fall back to CPU for unsupported PyTorch ops. Subclassing `TrainingArguments` and overriding the `device` property with the logic shown is the recommended immediate fix.

import torch
from transformers import TrainingArguments

class TrainingArgumentsWithMPSSupport(TrainingArguments):
    @property
    def device(self) -> torch.device:
        if torch.cuda.is_available():
            return torch.device("cuda")
        elif torch.backends.mps.is_available():
            return torch.device("mps")
        else:
            return torch.device("cpu")