Hugging Face's logo

thorirhrafn
/
llama2_lora_ice4

Generated from Trainer
Model card Files Community
thorirhrafn commited on
Commit
b4e8495
·
1 Parent(s): 4bec57a

End of training

Browse files
Files changed (1) hide show
  1. README.md +95 -0
README.md ADDED
@@ -0,0 +1,95 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: meta-llama/Llama-2-7b-hf
3
+ tags:
4
+ - generated_from_trainer
5
+ model-index:
6
+ - name: llama2_lora_ice4
7
+ results: []
8
+ ---
9
+
10
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
11
+ should probably proofread and complete it, then remove this comment. -->
12
+
13
+ # llama2_lora_ice4
14
+
15
+ This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on an unknown dataset.
16
+ It achieves the following results on the evaluation set:
17
+ - Loss: 1.8018
18
+
19
+ ## Model description
20
+
21
+ More information needed
22
+
23
+ ## Intended uses & limitations
24
+
25
+ More information needed
26
+
27
+ ## Training and evaluation data
28
+
29
+ More information needed
30
+
31
+ ## Training procedure
32
+
33
+ ### Training hyperparameters
34
+
35
+ The following hyperparameters were used during training:
36
+ - learning_rate: 0.0001
37
+ - train_batch_size: 8
38
+ - eval_batch_size: 8
39
+ - seed: 42
40
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
41
+ - lr_scheduler_type: linear
42
+ - training_steps: 200000
43
+
44
+ ### Training results
45
+
46
+ | Training Loss | Epoch | Step | Validation Loss |
47
+ |:-------------:|:-----:|:------:|:---------------:|
48
+ | 2.1501 | 0.19 | 5000 | 2.1064 |
49
+ | 2.0868 | 0.38 | 10000 | 1.9864 |
50
+ | 1.9857 | 0.57 | 15000 | 1.9325 |
51
+ | 1.897 | 0.75 | 20000 | 1.8989 |
52
+ | 1.9629 | 0.94 | 25000 | 1.8773 |
53
+ | 1.8638 | 1.13 | 30000 | 1.8629 |
54
+ | 1.891 | 1.32 | 35000 | 1.8523 |
55
+ | 1.8903 | 1.51 | 40000 | 1.8422 |
56
+ | 1.8059 | 1.7 | 45000 | 1.8354 |
57
+ | 1.8479 | 1.88 | 50000 | 1.8291 |
58
+ | 1.8491 | 2.07 | 55000 | 1.8261 |
59
+ | 1.8497 | 2.26 | 60000 | 1.8228 |
60
+ | 1.8306 | 2.45 | 65000 | 1.8194 |
61
+ | 1.8482 | 2.64 | 70000 | 1.8162 |
62
+ | 1.8247 | 2.83 | 75000 | 1.8139 |
63
+ | 1.8428 | 3.02 | 80000 | 1.8122 |
64
+ | 1.8078 | 3.2 | 85000 | 1.8108 |
65
+ | 1.8398 | 3.39 | 90000 | 1.8090 |
66
+ | 1.7911 | 3.58 | 95000 | 1.8080 |
67
+ | 1.8222 | 3.77 | 100000 | 1.8067 |
68
+ | 1.8152 | 3.96 | 105000 | 1.8054 |
69
+ | 1.8237 | 4.15 | 110000 | 1.8053 |
70
+ | 1.7876 | 4.33 | 115000 | 1.8048 |
71
+ | 1.799 | 4.52 | 120000 | 1.8042 |
72
+ | 1.8215 | 4.71 | 125000 | 1.8037 |
73
+ | 1.8099 | 4.9 | 130000 | 1.8031 |
74
+ | 1.812 | 5.09 | 135000 | 1.8031 |
75
+ | 1.8308 | 5.28 | 140000 | 1.8028 |
76
+ | 1.8139 | 5.47 | 145000 | 1.8025 |
77
+ | 1.8233 | 5.65 | 150000 | 1.8025 |
78
+ | 1.8346 | 5.84 | 155000 | 1.8022 |
79
+ | 1.8173 | 6.03 | 160000 | 1.8020 |
80
+ | 1.8475 | 6.22 | 165000 | 1.8021 |
81
+ | 1.7865 | 6.41 | 170000 | 1.8019 |
82
+ | 1.7849 | 6.6 | 175000 | 1.8019 |
83
+ | 1.8436 | 6.78 | 180000 | 1.8019 |
84
+ | 1.7984 | 6.97 | 185000 | 1.8019 |
85
+ | 1.803 | 7.16 | 190000 | 1.8019 |
86
+ | 1.8692 | 7.35 | 195000 | 1.8018 |
87
+ | 1.7558 | 7.54 | 200000 | 1.8018 |
88
+
89
+
90
+ ### Framework versions
91
+
92
+ - Transformers 4.34.1
93
+ - Pytorch 2.0.1+cu117
94
+ - Datasets 2.14.5
95
+ - Tokenizers 0.14.1