File size: 2,319 Bytes
f5d0c90 9a42767 5a69800 9a42767 5a69800 f5d0c90 ec98a4a a96d044 3900a10 ec98a4a f5d0c90 a96d044 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
---
base_model: unsloth/Llama-3.2-3B-Instruct-bnb-4bit
datasets:
- microsoft/orca-agentinstruct-1M-v1
pipeline_tag: text-generation
library_name: transformers
license: llama3.2
tags:
- unsloth
- transformers
model-index:
- name: analytical_reasoning_r16a32_unsloth-Llama-3.2-3B-Instruct-bnb-4bit
results:
- task:
type: text-generation
dataset:
type: lm-evaluation-harness
name: hellaswag
metrics:
- name: acc
type: acc
value: 0.5141
verified: false
- name: acc_norm
type: acc_norm
value: 0.6793
verified: false
---
### eval
| Test | Base Model | Fine-Tuned Model | Performance Gain |
|---|---|---|---|
| leaderboard_bbh_logical_deduction_seven_objects | 0.252 | 0.436 | 0.184 |
| leaderboard_bbh_logical_deduction_five_objects | 0.356 | 0.456 | 0.10000000000000003 |
| leaderboard_musr_team_allocation | 0.22 | 0.32 | 0.1 |
| leaderboard_bbh_disambiguation_qa | 0.304 | 0.376 | 0.07200000000000001 |
| leaderboard_gpqa_diamond | 0.2222222222222222 | 0.2727272727272727 | 0.0505050505050505 |
| leaderboard_bbh_movie_recommendation | 0.596 | 0.636 | 0.040000000000000036 |
| leaderboard_bbh_formal_fallacies | 0.508 | 0.54 | 0.03200000000000003 |
| leaderboard_bbh_tracking_shuffled_objects_three_objects | 0.316 | 0.344 | 0.02799999999999997 |
| leaderboard_bbh_causal_judgement | 0.5454545454545454 | 0.5668449197860963 | 0.021390374331550888 |
| leaderboard_bbh_web_of_lies | 0.496 | 0.516 | 0.020000000000000018 |
| leaderboard_math_geometry_hard | 0.045454545454545456 | 0.06060606060606061 | 0.015151515151515152 |
| leaderboard_math_num_theory_hard | 0.05194805194805195 | 0.06493506493506493 | 0.012987012987012977 |
| leaderboard_musr_murder_mysteries | 0.528 | 0.54 | 0.01200000000000001 |
| leaderboard_gpqa_extended | 0.27106227106227104 | 0.2802197802197802 | 0.00915750915750918 |
| leaderboard_bbh_sports_understanding | 0.596 | 0.604 | 0.008000000000000007 |
| leaderboard_math_intermediate_algebra_hard | 0.010714285714285714 | 0.014285714285714285 | 0.003571428571428571 |
| leaderboard_bbh_navigate | 0.62 | 0.62 | 0.0 |
### Framework versions
- unsloth 2024.11.5
- trl 0.12.0
### Training HW
- V100
|