oopere commited on
Commit
d5c7abd
·
verified ·
1 Parent(s): e926428

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -0
README.md CHANGED
@@ -27,6 +27,9 @@ This model is not intended to be used directly, but rather to be fine-tuned for
27
  - **License:** Same as original model
28
  - **Developed by:** [Pere Martra](https://huggingface.co/oopere)
29
 
 
 
 
30
  ### Performance on Standard Benchmarks
31
 
32
  | Benchmark | Original Model | Pruned Model | Relative Change |
 
27
  - **License:** Same as original model
28
  - **Developed by:** [Pere Martra](https://huggingface.co/oopere)
29
 
30
+ These models are part of the study "[Exploring GLU Expansion Ratios: Structured Pruning in Llama-3.2 Models](https://doi.org/10.31219/osf.io/qgxea)". They explore structured pruning in GLU-based architectures using Llama-3.2 (1B and 3B variants). The pruning experiments target optimal expansion ratios to balance performance, computational efficiency, and environmental sustainability. The models were evaluated across multiple benchmarks, including BoolQ, ARC-Easy, and MUSR, and demonstrate significant efficiency gains while maintaining robust task performance.
31
+
32
+
33
  ### Performance on Standard Benchmarks
34
 
35
  | Benchmark | Original Model | Pruned Model | Relative Change |