

TITLE
Getting no attribute 'output_attentions' error when upgrading to latest huggingface transformers 

COMMENTS
2

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
# 📚 Migration

## Information

I am getting `torch.nn.modules.module.ModuleAttributeError: 'CaptionBertSelfAttention' object has no attribute 'output_attentions'` error when upgrading my code from pytorch-transformers to latest version of huggingface transformers.

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below): not sure
* [ ] my own modified scripts: (give details below): yes

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name): no
* [ ] my own task or dataset: (give details below): no

## Details

<!-- A clear and concise description of the migration issue.
    If you have code snippets, please provide it here as well.
    Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
    Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
    -->

I am trying to upgrade my code which uses pytorch-transformers to use latest version of Huggingface transformers. However when I try to use the latest version of huggingface transformers, I get below error:
```
Traceback (most recent call last):
  File "oscar/run_captioning.py", line 1014, in <module>
    main()
  File "oscar/run_captioning.py", line 989, in main
    last_checkpoint = train(args, train_dataloader, val_dataloader, model, tokenizer)
  File "oscar/run_captioning.py", line 479, in train
    outputs = model(**inputs)
  File "/usr/local/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_latest/oscar/modeling/modeling_bert.py", line 450, in forward
    return self.encode_forward(*args, **kwargs)
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_latest/oscar/modeling/modeling_bert.py", line 458, in encode_forward
    encoder_history_states=encoder_history_states)
  File "/usr/local/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_latest/oscar/modeling/modeling_bert.py", line 281, in forward
    encoder_history_states=encoder_history_states)
  File "/usr/local/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_latest/oscar/modeling/modeling_bert.py", line 115, in forward
    history_state)
  File "/usr/local/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_latest/oscar/modeling/modeling_bert.py", line 146, in forward
    head_mask, history_state)
  File "/usr/local/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_latest/oscar/modeling/modeling_bert.py", line 88, in forward
    self_outputs = self.self(input_tensor, attention_mask, head_mask, history_state)
  File "/usr/local/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/default/ephemeral_drive/work/image_captioning/Oscar_latest/oscar/modeling/modeling_bert.py", line 73, in forward
    outputs = (context_layer, attention_probs) if self.output_attentions else (context_layer,)
  File "/usr/local/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 779, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'CaptionBertSelfAttention' object has no attribute 'output_attentions'
```

This branch of my code is using the latest version of transformers which gives above error - https://github.com/gsrivas4/Oscar_latest/tree/latest_transformer. This another branch of my code is using older version of transformers (https://github.com/huggingface/transformers/tree/067923d3267325f525f4e46f357360c191ba562e) which runs without any error - https://github.com/gsrivas4/Oscar_latest/tree/old_transformers. I have added README.md files to run both the branches.

So far, based on my debugging the issue I understand that `self.output_attentions` is defined in the older version of transformers here - https://github.com/huggingface/transformers/blob/067923d3267325f525f4e46f357360c191ba562e/pytorch_transformers/modeling_bert.py#L281. However, in the latest version of transformers `self.output_attentions` is not defined in the class `BertSelfAttention` - https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/modeling_bert.py#L213-L236. As `self.output_attentions` is not defined the latest version of transformers, which causes the error.

I have checked the migration document and did not the find steps needed or guidelines about how to resolve the issue  caused by upgrading huggingface transformers - https://huggingface.co/transformers/migration.html#migrating-from-transformers-v3-x-to-v4-x.

It would be really helpful to know how to resolve the error.

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
 
- `transformers` version: https://github.com/huggingface/transformers
- Platform: x86_64 GNU/Linux
- Python version: 3.6.8
- PyTorch version (GPU?): 1.7.0+cu101 (GPU)
- Tensorflow version (GPU?): 2.3.0 (GPU)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

<!-- IMPORTANT: which version of the former library do you use? -->
* `pytorch-transformers` or `pytorch-pretrained-bert` version (or branch): https://github.com/huggingface/transformers/tree/067923d3267325f525f4e46f357360c191ba562e

## Checklist

- [ yes] I have read the migration guide in the readme.
 ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers);
  [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
- [yes ] I checked if a related official extension example runs on my machine.
