GPT2-124M-TinyStories

This is a prototype / proof of concept model to see what the results of pretraining GPT2 exclusively on narrative texts would look like. That's right-- this isn't a finetune, it's entirely pretrained on TinyStories.

The GPT2 config and tokenizer is however unmodified from the original.