EleutherAI/deep-ignorance-unfiltered

Text Generation • 7B • Updated Aug 13 • 1.31k • 3

Note Fully Trained — Unfiltered Baseline Model - Pretraining Filtering: None - Annealing Filtering: None - Results Location: Main Paper

EleutherAI/deep-ignorance-e2e-strong-filter

Text Generation • 7B • Updated Aug 13 • 596

Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Results Location: Main Paper (Strong Filter)

EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal

Text Generation • 7B • Updated Aug 13 • 92

Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Weak Filter - Results Location: Main Paper (Weak Filter)

EleutherAI/deep-ignorance-e2e-weak-filter

Text Generation • 7B • Updated Aug 13 • 110

Note Fully Trained - Pretraining Filtering: Weak Filter - Annealing Filtering: Weak Filter - Results Location: Appendix

EleutherAI/deep-ignorance-weak-filter-pt-strong-filter-anneal

Text Generation • 7B • Updated Aug 13 • 86

Note Fully Trained - Pretraining Filtering: Weak Filter - Annealing Filtering: Strong Filter

EleutherAI/deep-ignorance-e2e-extra-weak-filter

7B • Updated Aug 12 • 75

Note Fully Trained - Pretraining Filtering: Extra Weak Filter - Annealing Filtering: Extra Weak Filter - Results Location: Not Included

EleutherAI/deep-ignorance-pretraining-stage-unfiltered

Text Generation • 7B • Updated Aug 13 • 550

Note Pretrained model that has not undergone annealing or any data filtering. - Pretraining Filtering: None - Results Location: Not Included

EleutherAI/deep-ignorance-pretraining-stage-strong-filter

Text Generation • 7B • Updated Aug 13 • 63

Note Pretrained model that has not undergone annealing. - Pretraining Filtering: Strong Filter - Results Location: Not Included

EleutherAI/deep-ignorance-pretraining-stage-weak-filter

Text Generation • 7B • Updated Aug 13 • 314

Note Pretrained model which has not undergone annealing. - Pretraining Filtering: Weak Filter - Results Location: Not Included

EleutherAI/deep-ignorance-pretraining-stage-extra-weak-filter

7B • Updated Aug 12 • 73

Note Pretrained model that has not undergone annealing. - Pretraining Filtering: Extra Weak Filter - Results Location: Not Included

EleutherAI/deep-ignorance-e2e-strong-filter-cb-lat

Text Generation • 7B • Updated Aug 13 • 141

Note Fully Trained with Circuit Breaking & Latent Adversarial Training - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Circuit Breaking + Latent Adversarial Training - Results Location: Main Paper (Strong Filter + CB + LAT)

EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal-cb-lat

Text Generation • 7B • Updated Aug 13 • 62

Note Fully Trained with Circuit Breaking & Latent Adversarial Training - Pretraining Filtering: Strong Filter - Annealing Filtering: Weak Filter - Post-training: Circuit Breaking + Latent Adversarial Training - Results Location: Main Paper (Weak Filter + CB + LAT)

EleutherAI/deep-ignorance-unfiltered-cb

Text Generation • 7B • Updated Aug 13 • 60

Note Fully Trained — Unfiltered Baseline Model with Circuit Breaking - Pretraining Filtering: None - Annealing Filtering: None - Post-training: Circuit Breaking - Results Location: Main Paper (CB)

EleutherAI/deep-ignorance-unfiltered-cb-lat

Text Generation • 7B • Updated Aug 13 • 68

Note Fully Trained — Unfiltered Baseline Model with Circuit Breaking & Latent Adversarial Training - Pretraining Filtering: None - Annealing Filtering: None - Post-training: Circuit Breaking + Latent Adversarial Training - Results Location: Main Paper (CB + LAT)

EleutherAI/deep-ignorance-e2e-strong-filter-cb

Text Generation • 7B • Updated Aug 13 • 59

Note Fully Trained with Circuit Breaking - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Circuit Breaking - Results Location: Main Paper (Strong Filter + CB)

EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal-cb

Text Generation • 7B • Updated Aug 13 • 61

Note Fully Trained with Circuit Breaking - Pretraining Filtering: Strong Filter - Annealing Filtering: Weak Filter - Post-training: Circuit Breaking - Results Location: Main Paper (Weak Filter + CB)

EleutherAI/deep-ignorance-e2e-strong-filter-weak-knowledge-corrupted

Text Generation • 7B • Updated Aug 13 • 79

Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Weak Knowledge Corruption via Synthetic Document Fine-Tuning - Results Location: Main Paper & Appendix

EleutherAI/deep-ignorance-e2e-strong-filter-strong-knowledge-corrupted

Text Generation • 7B • Updated Aug 13 • 82

Note Fully Trained - Pretraining Filtering: Strong Filter - Annealing Filtering: Strong Filter - Post-training: Strong Knowledge Corruption via Synthetic Document Fine-Tuning - Results Location: Main Paper & Appendix

Note All prompts from WMDP-Bio that can be evaluated using a cloze-style prompt.

Note WMDP-Bio, where data is broken down by topic category and whether it contains likely shortcuts.

Note General knowledge multiple-choice and cloze-style prompts that are used to ensure that models are familiar with the MCQA test benchmarks, like WMDP and MMLU.

Note The original annealing dataset for training the LLMs. This dataset is not filtered.

Note The original pretraining dataset for training the LLMs. This dataset is not filtered.