

TITLE
Faced the TypeError:forward() got an unexpected keyword argument 'output_all_encoded_layers'

COMMENTS
3

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
## Environment info
- `transformers` version: 3.0.2
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.7.3
- PyTorch version (GPU?): 1.1.0 (True)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

  Trainer: @sgugger 
 TransfoXL/XLNet: @TevenLeScao 

 -->

## Information

Model I am using XLNet...:

The problem arises when using:

* [ ] my own modified scripts like below url:
[chinese-bert-pytorch](https://github.com/649453932/Bert-Chinese-Text-Classification-Pytorch)

The tasks I am working on is: 
My own dataset Chinese Text Classification
## To reproduce

Steps to reproduce the behavior:

1.Firstly, load the xlnet model and tokenize:
```heisenberg
from pytorch_transformers import XLNetModel,XLNetTokenizer,XLNetConfig
class Model(nn.Module):

    def __init__(self, config):
        super(Model, self).__init__()
        self.bert =XLNetModel.from_pretrained(config.bert_path)
        for param in self.bert.parameters():
            param.requires_grad = True
        self.fc = nn.Linear(config.hidden_size, config.num_classes)

    def forward(self, x):
        context = x[0]  # 输入的句子
        mask = x[2]  # 对padding部分进行mask，和句子一个size，padding部分用0表示，如：[1, 1, 1, 1, 0, 0]
        _, pooled = self.bert(context, attention_mask=mask, output_all_encoded_layers=False)
        out = self.fc(pooled)
        return out
```
2.Then start training finetuning and evaluating:
```
def train(config, model,model_name, train_iter, dev_iter, test_iter):
    model.train()
    param_optimizer = list(model.named_parameters())
    no_decay = ['bias', 'LayerNorm.bias', 'LayerNorm.weight']
    optimizer_grouped_parameters = [
        {'params': [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)], 'weight_decay': 0.01},
        {'params': [p for n, p in param_optimizer if any(nd in n for nd in no_decay)], 'weight_decay': 0.0}]
    # optimizer = torch.optim.Adam(model.parameters(), lr=config.learning_rate)
    optimizer = BertAdam(optimizer_grouped_parameters, lr=config.learning_rate,warmup=0.05,t_total=len(train_iter) * config.num_epochs)
    total_batch = 0 
    dev_best_loss = float('inf')
    last_improve = 0  
    flag = False
    model.train()
    for epoch in range(config.num_epochs):
        print('Epoch [{}/{}]'.format(epoch + 1, config.num_epochs))
        for i, (trains, labels) in enumerate(train_iter):
            outputs = model(trains)
            model.zero_grad()
            loss = F.cross_entropy(outputs, labels)
            loss.backward()
            optimizer.step()
            if total_batch % 100 == 0:
                true = labels.data.cpu()
                predic = torch.max(outputs.data, 1)[1].cpu()
                train_acc = metrics.accuracy_score(true, predic)
                dev_acc, dev_loss = evaluate(config, model, dev_iter)
```

3. But Error Occured:
```
$ python run.py --model xlnet_base
Loading data...
401it [00:01, 225.08it/s]
140it [00:00, 260.37it/s]
135it [00:00, 240.91it/s]
Time usage: 0:00:03
Epoch [1/1]
Traceback (most recent call last):
  File "run.py", line 40, in <module>
    train(config, model,model_name, train_iter, dev_iter, test_iter)
  File "F:\PycharmProjects\Bert-Chinese-Text-Classification-Pytorch-master\train_eval.py", line 52, in train
    outputs = model(trains)
  File "D:\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "F:\PycharmProjects\Bert-Chinese-Text-Classification-Pytorch-master\models\xlnet_base.py", line 47, in forward
    _, pooled = self.bert(context, attention_mask=mask, output_all_encoded_layers=False)
  File "D:\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'output_all_encoded_layers'
```
4. I googled the error and found the [issuses ](https://github.com/huggingface/transformers/issues/3541) in transformers:
So I changed the Model Load code like below:
```
class Model(nn.Module):

    def __init__(self, config):
        super(Model, self).__init__()
        model_config = XLNetConfig.from_pretrained(config.bert_path,output_hidden_states=False)
        self.bert = XLNetModel.from_pretrained(config.bert_path,config=model_config)
        for param in self.bert.parameters():
            param.requires_grad = True
        self.fc = nn.Linear(config.hidden_size, config.num_classes)

    def forward(self, x):
        context = x[0] 
        mask = x[2] 
        _, pooled = self.bert(context, attention_mask=mask, output_all_encoded_layers=False)
        out = self.fc(pooled)
        return out
```
BUT I STILL encounter the same problem, I dunno why.

Hope Ur Reply. Thanks A Lot!
