Pre-Trianing Data Packing - a yuzhaouoe Collection

yuzhaouoe 's Collections

updated Sep 6

[ACL'24] Analysing the Impact of Sequence Composition on Language Model Pre-Training. https://github.com/yuzhaouoe/pretraining-data-packing