ai-agents-gradient-accumulatio-when-training-with-gradient-accumulation-the-avera-fca461dc

THEN Modify get_batch_samples to return a list of per-batch item counts instead of a single aggregated count, and pass each micro-batch's count separately to compute_loss so that the cross-entropy loss is correctly scaled per micro-batch. This ensures gradient norms match expected values.

In get_batch_samples, change: num_items_in_batch = sum([(batch['labels'].ne(-100)).sum() for batch in batch_samples]) to num_items_in_batches = [(batch['labels'].ne(-100)).sum() for batch in batch_samples]. Then pass each count to compute_loss in the training loop.