

TITLE
Use Softmax classifiering for run_glue.py example

COMMENTS
1

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
Hi,
I want to do binary text classification and I'm adapting [run_glue.py](https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_glue.py)  script to my task. The current model uses a linear classifier and the predictions are not in the range of [0,1]. Could you please guide me on how I could use softmax classifier instead of linear classifier?

I added following code after the model is loaded but I get an error related related to loss function that I pasted below. Any suggestion on how to fix this? 
   `model.classifier = torch.nn.Softmax(dim=1)`

```
File "run_glue.py", line 300, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/transformers/trainer.py", line 775, in train
    tr_loss += self.training_step(model, inputs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/transformers/trainer.py", line 1112, in training_step
    loss = self.compute_loss(model, inputs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/transformers/trainer.py", line 1136, in compute_loss
    outputs = model(**inputs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/transformers/modeling_bert.py", line 1377, in forward
    loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 962, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/functional.py", line 2468, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/Users/royaho/PycharmProjects/download_pretrained_model/venv/lib/python3.6/site-packages/torch/nn/functional.py", line 2262, in nll_loss
    .format(input.size(0), target.size(0)))
ValueError: Expected input batch_size (768) to match target batch_size (3).
 
```