oopere commited on
Commit
d0cdc50
·
verified ·
1 Parent(s): 71b0b49

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -4
README.md CHANGED
@@ -11,14 +11,14 @@ base_model:
11
  # Model Card for oopere/pruned20-llama-1b
12
 
13
  <!-- Provide a quick summary of what the model is/does. -->
14
- This model is a pruned version of the Llama-3.2 architecture, with a parameter reduction of 20% in the MLP Layers.
15
  The pruning process aims to enhance computational efficiency while maintaining acceptable performance across specific tasks.
16
  This model is not intended to be used directly, but rather to be fine-tuned for specific tasks where it can achieve equal or superior performance compared to fine-tuning the base model for the same task.
17
 
18
 
19
  ## Model Details
20
 
21
- - **Model Type:** Pruned version of LLaMA-1.2B using structured pruning
22
  - **Original Model:** meta-llama/Llama-3.2-1B
23
  - **Pruning Method:** Structured pruning of MLP layers using importance scores based on absolute maximum weights
24
  - **Size Reduction:** 13.7% (from 1.24B to 1.07B parameters)
@@ -61,5 +61,4 @@ This model is not intended to be used directly, but rather to be fine-tuned for
61
  - Can run on hardware with ~20% less memory than original
62
 
63
  ## Acknowledgments
64
- - Thanks to [Mariusz Kurman](https://huggingface.co/mkurman) for creating [llama-pruning](https://github.com/MedITSolutionsKurman/llama-pruning), a library that extends and improve this pruning methodology.
65
-
 
11
  # Model Card for oopere/pruned20-llama-1b
12
 
13
  <!-- Provide a quick summary of what the model is/does. -->
14
+ This model is a pruned version of the Llama-3.2-1b model, with a parameter reduction of 20% in the MLP Layers.
15
  The pruning process aims to enhance computational efficiency while maintaining acceptable performance across specific tasks.
16
  This model is not intended to be used directly, but rather to be fine-tuned for specific tasks where it can achieve equal or superior performance compared to fine-tuning the base model for the same task.
17
 
18
 
19
  ## Model Details
20
 
21
+ - **Model Type:** Pruned version of LLaMA-3.2 using structured pruning
22
  - **Original Model:** meta-llama/Llama-3.2-1B
23
  - **Pruning Method:** Structured pruning of MLP layers using importance scores based on absolute maximum weights
24
  - **Size Reduction:** 13.7% (from 1.24B to 1.07B parameters)
 
61
  - Can run on hardware with ~20% less memory than original
62
 
63
  ## Acknowledgments
64
+ - Thanks to [Mariusz Kurman](https://huggingface.co/mkurman) for creating [llama-pruning](https://github.com/MedITSolutionsKurman/llama-pruning), a library that extends and improve this pruning methodology.