

TITLE
[Longformer] Output both local attentions and global attentions when `output_attentions=True` -> Good Second Issue

COMMENTS
8

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# 🚀 Feature request

**Good Second Issue** - A more advanced issue for contributors who want to dive more into Longformer's attention mechanism.

Longformer currently only outputs global attentions, which is suboptimal because users might be interested in the local attentions as well. I propose to change the "output_attention" logic as follows in longformer:

`attentions` should correspond to the "local" attentions and then we'll add a new output type `global_attention` that contains the global_attentions. This is consistent with the naming of `attention_mask` and `global_attention_mask` IMO and the cleanest way to implement the feature.

Implementing this feature would mean to that Longformer will require its own `ModelOutput` class => 
`BaseModelOutput,` => `LongformerBaseModelOutput` or `BaseModelOutputWithGlobalAttention` (prefer the first name though)
`BaseModelOutputWithPooling,` => ...

Also some tests will have to be adapted.

This is a slightly more difficult issue, so I'm happy to help on it. One should understand the difference between local and global attention and how Longformer's attention is different to *e.g.* Bert's attention in general.

For more detail check out discussion here: https://github.com/huggingface/transformers/issues/5646