ai-agents-distributed-training-when-using-pytorch-s-distributeddataparallel-to-wr-dbc7fdf3

THEN Unwrap the model before calling generate. Use model.module.generate() directly, or use HuggingFace Accelerate's accelerator.unwrap_model() to get the underlying model, then call generate on it. For production, consider using HuggingFace Accelerate for distributed training as it handles model wrapping and unwrapping automatically.

# Direct DDP unwrap:
model = DistributedDataParallel(model)
output = model.module.generate(inputs)

# Using Accelerate:
from accelerate import Accelerator
accelerator = Accelerator()
model = accelerator.prepare(model)
unwrapped_model = accelerator.unwrap_model(model)
output = unwrapped_model.generate(inputs)