

TITLE
Sharing Microsoft's DialogRPT (new dialog ranking model)

COMMENTS
12

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 2
heart: 0
rocket: 0
eyes: 0

BODY
# 🌟 New model addition

## Model description

Thanks for the awesome work!

[DialogRPT](https://github.com/golsun/DialogRPT) (Dialog Ranking Pretrained Transformers) is a set of GPT-2 based dialogue ranking models recently released with an [EMNLP paper](https://arxiv.org/abs/2009.06978) by Microsoft Research. It's a follow-up work of [DialoGPT](https://huggingface.co/transformers/model_doc/dialogpt.html) (thanks for hosting it!)
The architecture is pretty simple: a `GPT2Model` followed by a `torch.nn.Linear(n_embd, 1, bias=False)`, and implemented based on a [previous HuggingFace commit](https://github.com/huggingface/transformers/commit/4d456542e9d381090f9a00b2bcc5a4cb07f6f3f7)
At first, I'm trying to create a model card for it, but then realized that it seems there's no existing model architecture in HuggingFace is compatible with DialogRPT. I noticed a lot of BERT-based sequence classification models, but ours is GPT-2 based. 

If there's a simple fix (or I missed something) please let me know!
If implementation in modeling_gpt2.py is necessary, I'm also glad to help!

## Open source status

* [x] the model implementation is available: (https://github.com/golsun/DialogRPT)
* [x] the model weights are available: (https://github.com/golsun/DialogRPT)
* [x] who are the authors: @golsun @dreasysnail
