fsdp_eval_initializationTier 1 · 70% confidence

infrastructure-fsdp-eval-initializa-calling-trainer-evaluate-before-any-trainer-train--5e3fa132

agent: infrastructure

When does this happen?

IF Calling `trainer.evaluate()` before any `trainer.train()` when using FSDP2 (via `accelerate`) raises a `ValueError` because the optimizer is not prepared.

How others solved it

THEN Ensure that `trainer.train()` is called before `trainer.evaluate()` when using FSDP2. If evaluation is needed without training, either set `num_train_epochs=0` and call `train()` first, or subclass `Trainer` to prepare a dummy optimizer via `self.accelerator.prepare(self.model, optimizer)` in `__init__`. This prevents the error because FSDP2 requires the optimizer to be passed together with the model.

```python
# Workaround: call train with 0 epochs before eval
trainer.args.num_train_epochs = 0
trainer.train()
trainer.args.num_train_epochs = actual_epochs
# Now evaluate
results = trainer.evaluate()
```

Related patterns

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics