File size: 2,657 Bytes
95e8e00 6ad47e8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 |
---
license: llama3.3
language:
- en
base_model:
- meta-llama/Llama-3.3-70B-Instruct
---
## Quantization : Q2_K (using Llama.cpp)
- llm_load_print_meta: model type = 70B
- llm_load_print_meta: model ftype = Q2_K - Medium
- llm_load_print_meta: model params = 70.55 B
- llm_load_print_meta: model size = 24.56 GiB (2.99 BPW)
- llama_model_loader: - type f32: 162 tensors
- llama_model_loader: - type q2_K: 321 tensors
- llama_model_loader: - type q3_K: 160 tensors
- llama_model_loader: - type q5_K: 80 tensors
- llama_model_loader: - type q6_K: 1 tensors
## MMLU Result : 74.89%
Category STEM: 66.09% (18 subjects)
- high_school_chemistry: 64.04%
- high_school_mathematics: 46.67%
- abstract_algebra: 48.00%
- computer_security: 84.00%
- college_computer_science: 61.62%
- college_chemistry: 53.00%
- conceptual_physics: 74.89%
- high_school_statistics: 68.06%
- college_mathematics: 44.00%
- college_biology: 88.19%
- college_physics: 52.94%
- elementary_mathematics: 64.81%
- high_school_biology: 88.71%
- high_school_physics: 57.62%
- machine_learning: 56.25%
- astronomy: 88.16%
- electrical_engineering: 69.66%
- high_school_computer_science: 79.00%
Category humanities: 79.28% (13 subjects)
- world_religions: 84.80%
- high_school_us_history: 89.71%
- moral_disputes: 77.75%
- high_school_world_history: 88.61%
- formal_logic: 62.70%
- international_law: 85.12%
- jurisprudence: 76.85%
- professional_law: 59.58%
- logical_fallacies: 83.44%
- philosophy: 74.28%
- moral_scenarios: 78.66%
- prehistory: 84.26%
- high_school_european_history: 84.85%
Category social sciences: 82.11% (12 subjects)
- high_school_geography: 86.36%
- high_school_psychology: 91.19%
- sociology: 87.56%
- high_school_microeconomics: 86.55%
- professional_psychology: 76.80%
- security_studies: 77.55%
- us_foreign_policy: 91.00%
- public_relations: 70.91%
- high_school_government_and_politics: 93.78%
- econometrics: 61.40%
- human_sexuality: 81.68%
- high_school_macroeconomics: 80.51%
Category other (business, health, misc.): 75.95% (14 subjects)
- virology: 53.61%
- college_medicine: 72.25%
- global_facts: 62.00%
- miscellaneous: 87.36%
- medical_genetics: 84.00%
- human_aging: 78.48%
- nutrition: 83.33%
- marketing: 88.89%
- anatomy: 71.85%
- professional_medicine: 88.24%
- professional_accounting: 56.03%
- management: 82.52%
- clinical_knowledge: 80.75%
- business_ethics: 74.00%
Overall correct rate: 74.89%
Total subjects evaluated: 57
## Perplexity 6.6865 +/- 0.04336
(using wikitext-2-raw/wiki.test.raw)
|