| license: mit | |
| datasets: | |
| - roneneldan/TinyStories | |
| language: | |
| - en | |
| tags: | |
| - gpt2 | |
| - tinystories | |
| # GPT2-124M-TinyStories | |
| This is a prototype / proof of concept model to see what the results of pretraining GPT2 exclusively on narrative texts would look like. | |
| That's right-- this isn't a finetune, it's entirely pretrained on TinyStories. | |
| The GPT2 config and tokenizer is however unmodified from the original. |