ai-agents-generated-text-extra-when-generating-text-with-hugging-face-transformer-e3a57d7d

THEN Use the pipeline API with return_full_text=False to automatically exclude the prompt. Alternatively, slice the output tensor using tokenizer.batch_decode(gen_tokens[:, input_ids.shape[1]:])[0] to get only the generated tokens. Note that the tensor slicing assumes all inputs in the batch have the same length; for variable-length inputs, use a different approach.

# Using pipeline (recommended)
from transformers import pipeline
pipe = pipeline(model="gpt2", return_full_text=False)
result = pipe("This is a test")

# Manual tensor slicing
encoding = tokenizer(prompt, return_tensors='pt').to(device)
generated_ids = model.generate(**encoding)
generated_ids = generated_ids[:, encoding.input_ids.shape[1]:]
generated_text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]