ai-agents-past-key-values-padd-using-past-key-values-with-a-padded-batch-in-gpt-2-4b1c1de0

THEN Manually pass correct `position_ids` to the model forward call, computed as the cumulative token index excluding padding. Alternatively, set `tokenizer.padding_side='left'` before tokenizing, so that padding does not shift position indices. Ensure the `attention_mask` covers the combined past and new tokens to avoid masking gaps.

position_ids = torch.tensor([[3], [4]], dtype=torch.int64)  # example offsets
tokenizer = AutoTokenizer.from_pretrained('gpt2', padding_side='left')
outputs2 = model(input_ids=inputs2['input_ids'], attention_mask=attention_mask, past_key_values=outputs1.past_key_values, position_ids=position_ids)