We don't publish
your competitive advantage.

IFUsing `summon_full_params` for inference during FSDP training causes checkpoint weights to diverge from the trained model, resulting in incorrect model after loading.

THENAvoid calling `model.generate()` inside `fsdp.FullyShardedDataParallel.summon_full_params(model)` during training callbacks. Instead, perform inference on a separate copy of the model or use DDP if checkpoint integrity is required. If FSDP is necessary, consider using `writeback=False` and verify checkpoint correctness, or move inference to a separate process.

IFUsing FSDP's `summon_full_params` for inference within a training callback (e.g., on_epoch_end) corrupts saved checkpoints, leading to different model weights on reload.

THENAvoid calling `summon_full_params` inside training callbacks when using FSDP. If inference during training is required, either use a separate copy of the model (e.g., deepcopy before unsharding) or switch to DDP. Ensure that any full parameter unsharding does not persist across checkpoint saves.

IFUsing `FullyShardedDataParallel.summon_full_params()` for inference inside a training callback (e.g., `on_epoch_end`) corrupts the saved model checkpoint, causing different weights when reloaded.

THENRemove or avoid the `summon_full_params` context during FSDP training if checkpoint integrity is required. Perform evaluation inference in a separate process after training, or switch to DDP (DistributedDataParallel) which does not exhibit this bug. If you must run inference during training, consider saving the model state before the call and restoring it afterward, though this may not be reliable.