

TITLE
Addition of MMI-antiLM decoding

COMMENTS
1

REACTIONS
+1: 1
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 1

BODY
# 🚀 Feature request

Hugging Face does a great job of including popular decoding strategies such as nucleus sampling, top-k, and temperature. There are also other really interesting decoding strategies for chatbots to fix the response "blandness" or "I don't know" problem, such as using the **Maximum Mutual Information anti-Language Model objective (MMI anti-LM)**. The algorithm is defined in [A Diversity-Promoting Objective Function for Neural Conversation Models](https://www.aclweb.org/anthology/N16-1014.pdf). 


## Motivation

I'm requesting this as a feature because I used this in a narrative generation paper. From this point forward I will be using "we" to reference my co-authors (@XiangLi1999 also contributed to the code) and I. The results of our work show that antiLM decoding does in fact help make the generated output more interesting without hurting fluency. Our work is [Decoding Methods for Neural Narrative Generation](https://arxiv.org/abs/2010.07375) and the rest of our code is in our [paper repo](https://github.com/AADeLucia/gpt2-narrative-decoding).

We think others would also be interested in using this decoding method for their work.

## Your contribution

We have a working implementation in a hacked version of the `generation_utils.py` file. It's not pretty (sorry) but maybe a good starting point? The code is in PR #7931. 

Also the author's implementation (not huggingface-based) is in [Jiwei Li's repo](https://github.com/jiweil/Neural-Dialogue-Generation).