The Synthetic Data Playbook: Generating Trillions of the Finest Tokens
📝
1
Visualize synthetic data experiments as an interactive bookshelf
AI & ML interests
We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)
Recent Activity
Papers
FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
joelniklaus
published a Space about 4 hours ago
joelniklaus
updated a dataset about 4 hours ago
joelniklaus
updated a Space about 14 hours ago
joelniklaus
updated 14 datasets 5 days ago
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer
• Updated
• 56.1M • 524
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT
Viewer
• Updated
• 56.1M • 47.5k
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer
• Updated
• 62.1M • 429 • 3
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT
Viewer
• Updated
• 62.1M • 31.6k • 1
HuggingFaceFW/finepdfs_edu_100BT-shuffled
Viewer
• Updated
• 17.8M • 817
HuggingFaceFW/finepdfs_edu_100BT
Viewer
• Updated
• 17.8M • 5.25k
HuggingFaceFW/finepdfs_100BT-shuffled
Viewer
• Updated
• 14.6M • 609
HuggingFaceFW/finepdfs_100BT
Viewer
• Updated
• 29.9M • 6.78k
HuggingFaceFW/fineweb_edu_100BT-shuffled
Viewer
• Updated
• 102M • 415
HuggingFaceFW/fineweb_edu_100BT
Preview
• Updated
• 500
HuggingFaceFW/fineweb_100BT-shuffled
Viewer
• Updated
• 161M • 229
HuggingFaceFW/fineweb_100BT
Viewer
• Updated
• 161M • 644
HuggingFaceFW/dclm_100BT-shuffled
Viewer
• Updated
• 89.3M • 538 • 1
HuggingFaceFW/dclm_100BT
Viewer
• Updated
• 89.3M • 803
joelniklaus
updated a collection 20 days ago