DataComp-LM: In search of the next generation of training sets for language models
Paper
•
2406.11794
•
Published
•
48
Training
-
-
Training Language Models on Synthetic Edit Sequences Improves Code Synthesis
Paper • 2410.02749 • Published • 12 -
Fewer Truncations Improve Language Modeling
Paper • 2404.10830 • Published • 3 -
How to Train Long-Context Language Models (Effectively)
Paper • 2410.02660 • Published • 1 -
Towards a Unified View of Preference Learning for Large Language Models: A Survey
Paper • 2409.02795 • Published • 72 -
ORPO: Monolithic Preference Optimization without Reference Model
Paper • 2403.07691 • Published • 61 -
Intuitive Fine-Tuning: Towards Unifying SFT and RLHF into a Single Process
Paper • 2405.11870 • Published -
LoRA Dropout as a Sparsity Regularizer for Overfitting Control
Paper • 2404.09610 • Published • 1 -
Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning
Paper • 2402.13669 • Published • 1 -
What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective
Paper • 2410.23743 • Published • 56