OLMo-150M-as_fm3_4xtg

150M OLMo model pretrained on Algebraic Stack, FineMath3+, and 4 passes of TinyGSM.

Reading Model Names

Model names contain the contents of the pretraining dataset, delimited by underscores.

as: Algebraic Stack
fm3: FineMath3+
tg: TinyGSM
omi1: OpenMathInstruct1
omi2: OpenMathInstruct2

If there is a {n}x in front of the dataset abbreviation, that means it was repeated n times during pretraining. For instance, 2xtg refers to two passes over the TinyGSM dataset.

Downloads last month: 6

Safetensors

Model size

0.2B params

Tensor type

F32

Collection including rosieyzh/OLMo-150M-as_fm3_4xtg

OLMo-150M and OLMo-1B Pretrained Models

Collection

Pretrained models from scratch used in "Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining". • 12 items • Updated Jul 7 • 3