

TITLE
DistilGPT2 extremely strange model behaviour

COMMENTS
4

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.0.dev0
- Platform: Linux-4.15.0-91-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.8.0a0+1606899 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes, 2
- Using distributed or parallel set-up in script?: Yes

### Who can help

 albert, bert, GPT2, XLM: @LysandreJik
 examples/distillation: @VictorSanh

## Information

Model I am using (Bert, XLNet ...): DistilGPT2

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ X ] my own modified scripts: (give details below)

Had to slightly modify the official scripts to even get it to run - other than that, effectively no difference

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ X ] my own task or dataset: (give details below)

Chat logs. In the format

```
[22-Jul-20 04:15 AM] User
Message


[22-Jul-20 04:15 AM] Another user
Message


[22-Jul-20 04:16 AM] Etc.
You get the point
```


## To reproduce

Steps to reproduce the behavior:

1. (Fix and) run example scripts for distillation with a custom GPT2 model.

When trying to generate, the model will generate the same text with no change, regardless of using ``pipeline`` or ``model.generate``. If attempting to use input texts, the network will output the texts immediately appended by the phrase. In the two times I've run this experiment, it was the space character twice and ``[Nov-Nov-20] User`` once (after I tried removing lines that were under a certain token value)

## Expected behavior

The model to work as a normal transformer.

<!-- A clear and concise description of what you would expect to happen. -->
