eugenesiow

/

bart-paraphrase

like 27

You can specify the num_return_sequences=10 parameter in generate() and use beam search to decode.

Load the model as per normal:

import torch
from transformers import BartForConditionalGeneration, BartTokenizer

model = BartForConditionalGeneration.from_pretrained('eugenesiow/bart-paraphrase')
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
tokenizer = BartTokenizer.from_pretrained('eugenesiow/bart-paraphrase')

Use beam search, specifying the number of beams and number of return sequences:

input_sentence = "Never mind less slow, how about making it work first?"
batch = tokenizer(input_sentence, return_tensors='pt').to(device)

torch.manual_seed(0)
generated_ids_set = model.generate(batch['input_ids'], num_beams=30, num_return_sequences=10, early_stopping=True)
for generated_ids in generated_ids_set: 
  generated_sentence = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
  print(generated_sentence)

The output looks something like that:

['</s>', 'How', ' can', ' I', ' make', ' it', ' work', ' first', '?', '</s>', '<pad>', '<pad>']
['</s>', 'How', ' do', ' I', ' make', ' it', ' work', ' first', '?', '</s>', '<pad>', '<pad>']
['</s>', 'How', ' should', ' I', ' make', ' it', ' work', ' first', '?', '</s>', '<pad>', '<pad>']
['</s>', 'What', ' should', ' I', ' do', ' to', ' make', ' it', ' work', ' first', '?', '</s>']
['</s>', 'How', ' can', ' I', ' make', ' it', ' work', ' faster', '?', '</s>', '<pad>', '<pad>']
['</s>', 'How', ' do', ' I', ' make', ' things', ' work', ' first', '?', '</s>', '<pad>', '<pad>']
['</s>', 'How', ' do', ' I', ' make', ' it', ' work', '?', '</s>', '<pad>', '<pad>', '<pad>']
['</s>', 'How', ' do', ' I', ' make', ' it', ' work', ' faster', '?', '</s>', '<pad>', '<pad>']
['</s>', 'How', ' can', ' I', ' be', ' less', ' slow', '?', '</s>', '<pad>', '<pad>', '<pad>']
['</s>', 'What', ' is', ' the', ' best', ' way', ' to', ' make', ' something', ' work', '?', '</s>']