Predicting the Order of Upcoming Tokens Improves Language Modeling
Paper
•
2508.19228
•
Published
•
22
Token Order Prediction
Pretrained models from the paper "Predicting the Order of Upcoming Tokens Improves Language Modeling"
-
-
zaydzuhri/vanilla-340M-4096-model
0.4B • Updated • 9 -
zaydzuhri/mtp-340M-4096-model
0.4B • Updated • 5 -
zaydzuhri/top-340M-4096-model
0.4B • Updated • 10 • 1 -
zaydzuhri/vanilla-1.8B-4096-model
2B • Updated • 11 -
zaydzuhri/mtp-1.8B-4096-model
2B • Updated • 9 -
zaydzuhri/top-1.8B-4096-model
2B • Updated • 17 -
zaydzuhri/vanilla-7B-4096-model
7B • Updated • 2 -
zaydzuhri/mtp-7B-4096-model
7B • Updated • 13 -
zaydzuhri/top-7B-4096-model
7B • Updated • 17