ValueError: `decoder_start_token_id` or `bos_token_id` has to be defined for encoder-decoder generation
#2
by
kosukekurimoto
- opened
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-xxl")
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xxl", device_map="auto")
input_ids = tokenizer("summarize: studies have shown that owning a dog is good for you ", return_tensors="pt").input_ids # Batch size 1
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
βββββ
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-11-c7dcdc6785c6> in <module>()
1 input_ids = tokenizer("summarize: studies have shown that owning a dog is good for you ", return_tensors="pt").input_ids # Batch size 1
----> 2 outputs = model.generate(input_ids)
3 print(tokenizer.decode(outputs[0]))
3 frames
/usr/local/lib/python3.7/dist-packages/transformers/generation_utils.py in _get_decoder_start_token_id(self, decoder_start_token_id, bos_token_id)
569 return self.config.decoder.bos_token_id
570 raise ValueError(
--> 571 "`decoder_start_token_id` or `bos_token_id` has to be defined for encoder-decoder generation."
572 )
573
ValueError: `decoder_start_token_id` or `bos_token_id` has to be defined for encoder-decoder generation.
Hey
@kosukekurimoto
!
With https://huggingface.co/google/flan-t5-xxl/discussions/3 being merged, I believe that the snippet should work now - could you give it another try and let us know here? Thanks!
i know it's closed, but thanks for sharing that code snippet, it was indeed very useful to me ππ»