

TITLE
Where is the impact when output_attentions=True?

COMMENTS
2

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
Is there any impact regarding performance (training/fine-tuning time, GPU memory, batch size, etc.) when  `output_attentions=True`?

```python
self.bert_encoder = BertModel.from_pretrained(
            hparams.architecture, # "bert-base-uncased"
            output_attentions=True)
```