text: 
  
  TITLE
  Add flags to return scores, hidden states and / or attention weights in GenerationMixin
  
  COMMENTS
  8
  
  REACTIONS
  +1: 1
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # What does this PR do?
  
  
  Add flags and logic to return attention scores, hidden states, and/or logits when using a model in generation mode.
  
  Fixes #9121 
  
  ## Before submitting
  - [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  - [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
        Pull Request section?
  - [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
        to it if that's the case. Issue: #9121 
  - [x] Did you make sure to update the documentation with your changes? Here are the
        [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
        [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
  - [x] Did you write any new necessary tests?
  
  
  ## Who can review?
  
  @patrickvonplaten 
  
  => Check out forum post for more detail: https://discuss.huggingface.co/t/announcement-generationoutputs-scores-attentions-and-hidden-states-now-available-as-outputs-to-generate/3094
  
  ## UPDATE:
  
  This PR is almost ready for merge. Here are the last things to take care of:
  
  - [x] @patrickvonplaten, @SBrandeis, @LysandreJik, @sgugger - check if naming of return arguments is ok.
  - [x] @patrickvonplaten add a correct `_check_outputs()` function for special models (Reformer, XLNet, XLM, TransfoXL, ...)
  - [x] @SBrandeis We could think about a simple code snippet to advertise the new feature. Maybe use "Beam Search" for translation and show the different "sequence_scores" (probs) for 3 different translations.
  
  ## Future PR:
  
  - [ ] Change the output in all notebooks, examples, tests to `return_dict_in_generate=True`.
  - [ ] Even further in the future PR: Think about a way to deprecate the default usage of `return_dict_in_generate=False`.
  - [ ] @patrickvonplaten Discuss with @sgugger if model documentation is ok. @sgugger - I think `generate()` might deserve its own "main" doc page maybe in "main classes" under "Models". I would add a small description at the top, then add all the generate functions and the new "Generation Outputs" there. Do you think this makes sense or would you rather keep "generate" as a subsection under "Models"? IMO, "generate" could be more visible in the docs.
  - [ ] @patrickvonplaten, @LysandreJik, @sgugger - this PR adds a lot more generate testing to many models. Since `generate` quite an expensive method, it might be worth checking here that this PR does yield a significant slow down of the tests. Simon and I tried to make the generate test as short and light-weight as possible, but it might still be significant.
  