Text Generation
Transformers
PyTorch
Safetensors
English
hf_olmo
custom_code

Why is there no intermediate checkpoint between 500B-1300B?

#11
by siqi-zz - opened

Why is there no intermediate checkpoint between 500B-1300B?

And missing checkpoints at the beginning

Can I please bump this question? Earlier checkpoints for 1B parameters and < 84B tokens would be much appreciated!

Ai2 org

Sorry, these checkpoints are missing because of a screw-up. We lost a range of data. It's relatively easy to re-create those checkpoints, but it takes a little bit of compute.

@dirkgr thank you! Would it be possible to recreate them? Or tell me what's necessary to re-create them?

Sign up or log in to comment