Update README.md
Browse files
README.md
CHANGED
@@ -11,14 +11,14 @@ base_model:
|
|
11 |
# Model Card for oopere/pruned20-llama-1b
|
12 |
|
13 |
<!-- Provide a quick summary of what the model is/does. -->
|
14 |
-
This model is a pruned version of the Llama-3.2
|
15 |
The pruning process aims to enhance computational efficiency while maintaining acceptable performance across specific tasks.
|
16 |
This model is not intended to be used directly, but rather to be fine-tuned for specific tasks where it can achieve equal or superior performance compared to fine-tuning the base model for the same task.
|
17 |
|
18 |
|
19 |
## Model Details
|
20 |
|
21 |
-
- **Model Type:** Pruned version of LLaMA-
|
22 |
- **Original Model:** meta-llama/Llama-3.2-1B
|
23 |
- **Pruning Method:** Structured pruning of MLP layers using importance scores based on absolute maximum weights
|
24 |
- **Size Reduction:** 13.7% (from 1.24B to 1.07B parameters)
|
@@ -61,5 +61,4 @@ This model is not intended to be used directly, but rather to be fine-tuned for
|
|
61 |
- Can run on hardware with ~20% less memory than original
|
62 |
|
63 |
## Acknowledgments
|
64 |
-
- Thanks to [Mariusz Kurman](https://huggingface.co/mkurman) for creating [llama-pruning](https://github.com/MedITSolutionsKurman/llama-pruning), a library that extends and improve this pruning methodology.
|
65 |
-
|
|
|
11 |
# Model Card for oopere/pruned20-llama-1b
|
12 |
|
13 |
<!-- Provide a quick summary of what the model is/does. -->
|
14 |
+
This model is a pruned version of the Llama-3.2-1b model, with a parameter reduction of 20% in the MLP Layers.
|
15 |
The pruning process aims to enhance computational efficiency while maintaining acceptable performance across specific tasks.
|
16 |
This model is not intended to be used directly, but rather to be fine-tuned for specific tasks where it can achieve equal or superior performance compared to fine-tuning the base model for the same task.
|
17 |
|
18 |
|
19 |
## Model Details
|
20 |
|
21 |
+
- **Model Type:** Pruned version of LLaMA-3.2 using structured pruning
|
22 |
- **Original Model:** meta-llama/Llama-3.2-1B
|
23 |
- **Pruning Method:** Structured pruning of MLP layers using importance scores based on absolute maximum weights
|
24 |
- **Size Reduction:** 13.7% (from 1.24B to 1.07B parameters)
|
|
|
61 |
- Can run on hardware with ~20% less memory than original
|
62 |
|
63 |
## Acknowledgments
|
64 |
+
- Thanks to [Mariusz Kurman](https://huggingface.co/mkurman) for creating [llama-pruning](https://github.com/MedITSolutionsKurman/llama-pruning), a library that extends and improve this pruning methodology.
|
|