

TITLE
Add GPT-Neo

COMMENTS
0

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 13
rocket: 0
eyes: 6

BODY
# 🌟 New model addition

Please add GPT-Neo

## Model description

> GPT-Neo is the code name for a series of transformer-based language models loosely styled around the GPT architecture that Eleuther AI plans to train and open source. Eleuther AI's primary goal is to replicate a GPT-3 sized model and open source it to the public, for free.

<!-- Important information -->

## Open source status

* [x] the model implementation is available: [Repo](https://github.com/EleutherAI/gpt-neo)
* [x] the model weights are available: [Download](https://the-eye.eu/eleuther_staging/gptneo-release/) (1.3B & 2.7B)
* [x] who are the authors: @sdtblck, @leogao2, @lucidrains, @ConnorJL, @StellaAthena & [others](https://github.com/EleutherAI)

Somewhat related to #4658, #4679, especially [this](https://github.com/huggingface/transformers/issues/4658#issuecomment-754247106)