|
--- |
|
license: llama3.3 |
|
language: |
|
- en |
|
base_model: |
|
- meta-llama/Llama-3.3-70B-Instruct |
|
--- |
|
|
|
## Quantization : Q2_K (using Llama.cpp) |
|
- llm_load_print_meta: model type = 70B |
|
- llm_load_print_meta: model ftype = Q2_K - Medium |
|
- llm_load_print_meta: model params = 70.55 B |
|
- llm_load_print_meta: model size = 24.56 GiB (2.99 BPW) |
|
- llama_model_loader: - type f32: 162 tensors |
|
- llama_model_loader: - type q2_K: 321 tensors |
|
- llama_model_loader: - type q3_K: 160 tensors |
|
- llama_model_loader: - type q5_K: 80 tensors |
|
- llama_model_loader: - type q6_K: 1 tensors |
|
|
|
## MMLU Result : 74.89% |
|
Category STEM: 66.09% (18 subjects) |
|
- high_school_chemistry: 64.04% |
|
- high_school_mathematics: 46.67% |
|
- abstract_algebra: 48.00% |
|
- computer_security: 84.00% |
|
- college_computer_science: 61.62% |
|
- college_chemistry: 53.00% |
|
- conceptual_physics: 74.89% |
|
- high_school_statistics: 68.06% |
|
- college_mathematics: 44.00% |
|
- college_biology: 88.19% |
|
- college_physics: 52.94% |
|
- elementary_mathematics: 64.81% |
|
- high_school_biology: 88.71% |
|
- high_school_physics: 57.62% |
|
- machine_learning: 56.25% |
|
- astronomy: 88.16% |
|
- electrical_engineering: 69.66% |
|
- high_school_computer_science: 79.00% |
|
|
|
Category humanities: 79.28% (13 subjects) |
|
- world_religions: 84.80% |
|
- high_school_us_history: 89.71% |
|
- moral_disputes: 77.75% |
|
- high_school_world_history: 88.61% |
|
- formal_logic: 62.70% |
|
- international_law: 85.12% |
|
- jurisprudence: 76.85% |
|
- professional_law: 59.58% |
|
- logical_fallacies: 83.44% |
|
- philosophy: 74.28% |
|
- moral_scenarios: 78.66% |
|
- prehistory: 84.26% |
|
- high_school_european_history: 84.85% |
|
|
|
Category social sciences: 82.11% (12 subjects) |
|
- high_school_geography: 86.36% |
|
- high_school_psychology: 91.19% |
|
- sociology: 87.56% |
|
- high_school_microeconomics: 86.55% |
|
- professional_psychology: 76.80% |
|
- security_studies: 77.55% |
|
- us_foreign_policy: 91.00% |
|
- public_relations: 70.91% |
|
- high_school_government_and_politics: 93.78% |
|
- econometrics: 61.40% |
|
- human_sexuality: 81.68% |
|
- high_school_macroeconomics: 80.51% |
|
|
|
Category other (business, health, misc.): 75.95% (14 subjects) |
|
- virology: 53.61% |
|
- college_medicine: 72.25% |
|
- global_facts: 62.00% |
|
- miscellaneous: 87.36% |
|
- medical_genetics: 84.00% |
|
- human_aging: 78.48% |
|
- nutrition: 83.33% |
|
- marketing: 88.89% |
|
- anatomy: 71.85% |
|
- professional_medicine: 88.24% |
|
- professional_accounting: 56.03% |
|
- management: 82.52% |
|
- clinical_knowledge: 80.75% |
|
- business_ethics: 74.00% |
|
|
|
Overall correct rate: 74.89% |
|
Total subjects evaluated: 57 |
|
|
|
## Perplexity 6.6865 +/- 0.04336 |
|
(using wikitext-2-raw/wiki.test.raw) |
|
|
|
|
