oopere commited on
Commit
e09ed04
·
verified ·
1 Parent(s): d0cdc50

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -27,6 +27,8 @@ This model is not intended to be used directly, but rather to be fine-tuned for
27
  - **License:** Same as original model
28
  - **Developed by:** [Pere Martra](https://huggingface.co/oopere)
29
 
 
 
30
  ### Performance on Standard Benchmarks
31
 
32
  | Benchmark | Original Model | Pruned Model | Relative Change |
 
27
  - **License:** Same as original model
28
  - **Developed by:** [Pere Martra](https://huggingface.co/oopere)
29
 
30
+ These models are part of the study "[Exploring GLU Expansion Ratios: Structured Pruning in Llama-3.2 Models](https://doi.org/10.31219/osf.io/qgxea)". They explore structured pruning in GLU-based architectures using Llama-3.2 (1B and 3B variants). The pruning experiments target optimal expansion ratios to balance performance, computational efficiency, and environmental sustainability. The models were evaluated across multiple benchmarks, including BoolQ, ARC-Easy, and MUSR, and demonstrate significant efficiency gains while maintaining robust task performance.
31
+
32
  ### Performance on Standard Benchmarks
33
 
34
  | Benchmark | Original Model | Pruned Model | Relative Change |