---
license: mit
datasets:
- roneneldan/TinyStories
language:
- en
tags:
- gpt2
- tinystories
---

# GPT2-124M-TinyStories

This is a prototype / proof of concept model to see what the results of pretraining GPT2 exclusively on narrative texts would look like.
That's right-- this isn't a finetune, it's entirely pretrained on TinyStories. 

The GPT2 config and tokenizer is however unmodified from the original.