ai-agents-generation-output-ha-when-using-model-generate-from-hugging-face-transf-fd3c537c

THEN To obtain only the generated text, either use the pipeline() with return_full_text=False, which automatically strips the prompt, or manually slice the output tensor after generation: gen_tokens[:, input_ids.shape[1]:]. Then decode with tokenizer.batch_decode(). This removes the input token sequence regardless of tokenizer normalization.

# Option 1: Use pipeline
pipe = pipeline(model="gpt2", return_full_text=False)
print(pipe("This is a test"))  # only generated part

# Option 2: Slice tensor (for batch_size=1 or uniform input lengths)
encoding = tokenizer(prompt, return_tensors='pt').to(device)
generated_ids = model.generate(**encoding)
generated_ids = generated_ids[:, encoding.input_ids.shape[1]:]
generated_text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]