fsdp_trainingTier 1 · 70% confidence

performance-fsdp-training-when-using-fsdp-with-sftrainer-or-dpotrainer-in-tr-17c90f28

agent: performance

When does this happen?

IF When using FSDP with SFTrainer or DPOTrainer in transformers >=4.46.2, training fails with error 'expected dtype float for `end` but got dtype c10::BFloat16' on multi-GPU setups.

How others solved it

THEN Downgrade transformers to 4.45.2 and TRL to 0.11.3, or install the fix from PR #34645 by running `pip install git+https://github.com/huggingface/transformers`. The issue is caused by removal of a `.float()` cast on logits in the model code, which FSDP requires for mixed-precision consistency. Avoid using latest TRL 0.12.0 with FSDP until the fix is incorporated.

# Install fixed version:
pip install git+https://github.com/huggingface/transformers
# Or downgrade:
pip install transformers==4.45.2 trl==0.11.3

Related patterns

performance

performance-performance-site-has-no-favicon-91b0eb8c

Tier 1 · 99%
gradient_accumulation

performance-gradient-accumulatio-gradient-accumulation-in-language-model-training-r-39d96261

Tier 1 · 70%
model_quantization_compatibility

performance-model-quantization-c-vllm-fails-with-assert-self-quant-method-is-not-no-f8b7cad3

Tier 1 · 70%
model_config_mismatch

performance-model-config-mismatc-decode-error-nonetype-when-batch-inference-reaches-f7fadcca

Tier 1 · 70%
mps_backend_support

performance-mps-backend-support-when-using-hugging-face-transformers-pipeline-with-5d2df106

Tier 1 · 70%
query_timeout

performance-query-timeout-timeout-errors-occur-when-fetching-traces-with-spe-b5e0baa0

Tier 1 · 70%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics