

TITLE
Dropout p is changing after loading

COMMENTS
4

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
## Environment info     
- `transformers` version: 3.5.0
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0+cu101 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?:
- Using distributed or parallel set-up in script?: No

### Who can help
@LysandreJik,  @sgugger 

## Information
Model I am using (Bert, XLNet ...): Bert, Roberta

The problem arises when using:
* [ *] the official example scripts: Using information given in this link: https://huggingface.co/transformers/master/custom_datasets.html

The tasks I am working on is:
* [ *] my own task or dataset: text classification

## To reproduce

Steps to reproduce the behavior:

1. I'm trying to change dropout probability. I'm using one of these methods for Bert instance:
```python
model.classifier.dropout.p=0.7
model.classifier.dropout = nn.Dropout(0.7)
```

2. After training is completed, model is saved

```python
model.save_pretrained('xxx/bert')
```

3. Model is loaded in another session using this code snippet. But after loading, model.classifier.dropout.p is changing to 0.1 which is in the config file.
```python
model = BertForSequenceClassification.from_pretrained("xxx/bert",
    num_labels = 3, 
    output_attentions = False,
    output_hidden_states = False,
)
```

## Expected behavior
Dropout p is changing to default value after loading the model. But the model is modified so that it shouldn't do that behavior
