Hugging Face's logo

just1nseo
/
eurus-dpop-qlora-uf-5e-7-real

PEFT
TensorBoard
Safetensors
mistral
alignment-handbook
trl
dpo
Generated from Trainer
4-bit precision
bitsandbytes
Model card Files Metrics Community
just1nseo commited on
Commit
4a7ca35
·
verified ·
1 Parent(s): 2943f01

Model save

Browse files
Files changed (6) hide show
  1. README.md +116 -0
  2. adapter_model.safetensors +1 -1
  3. all_results.json +8 -0
  4. runs/Jul16_03-49-11_node29/events.out.tfevents.1721069652.node29.1081696.0 +2 -2
  5. train_results.json +8 -0
  6. trainer_state.json +0 -0
README.md ADDED
@@ -0,0 +1,116 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: peft
4
+ tags:
5
+ - trl
6
+ - dpo
7
+ - generated_from_trainer
8
+ base_model: openbmb/Eurus-7b-sft
9
+ model-index:
10
+ - name: eurus-dpop-qlora-uf-5e-7-real
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # eurus-dpop-qlora-uf-5e-7-real
18
+
19
+ This model is a fine-tuned version of [openbmb/Eurus-7b-sft](https://huggingface.co/openbmb/Eurus-7b-sft) on the None dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.6797
22
+ - Positive Losses: 0.0822
23
+ - Dpo Losses: 0.6692
24
+ - Rewards/chosen: 0.1295
25
+ - Rewards/rejected: 0.0787
26
+ - Rewards/accuracies: 0.7123
27
+ - Rewards/margins: 0.0508
28
+ - Rewards/margins Max: 0.1967
29
+ - Rewards/margins Min: -0.0800
30
+ - Rewards/margins Std: 0.0920
31
+ - Logps/rejected: -254.9542
32
+ - Logps/chosen: -262.8235
33
+ - Logits/rejected: -2.1947
34
+ - Logits/chosen: -2.3028
35
+
36
+ ## Model description
37
+
38
+ More information needed
39
+
40
+ ## Intended uses & limitations
41
+
42
+ More information needed
43
+
44
+ ## Training and evaluation data
45
+
46
+ More information needed
47
+
48
+ ## Training procedure
49
+
50
+ ### Training hyperparameters
51
+
52
+ The following hyperparameters were used during training:
53
+ - learning_rate: 5e-07
54
+ - train_batch_size: 4
55
+ - eval_batch_size: 8
56
+ - seed: 42
57
+ - distributed_type: multi-GPU
58
+ - num_devices: 4
59
+ - total_train_batch_size: 16
60
+ - total_eval_batch_size: 32
61
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
62
+ - lr_scheduler_type: cosine
63
+ - lr_scheduler_warmup_ratio: 0.1
64
+ - num_epochs: 1
65
+
66
+ ### Training results
67
+
68
+ | Training Loss | Epoch | Step | Validation Loss | Positive Losses | Dpo Losses | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Rewards/margins Max | Rewards/margins Min | Rewards/margins Std | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
69
+ |:-------------:|:-----:|:----:|:---------------:|:---------------:|:----------:|:--------------:|:----------------:|:------------------:|:---------------:|:-------------------:|:-------------------:|:-------------------:|:--------------:|:------------:|:---------------:|:-------------:|
70
+ | 0.696 | 0.03 | 100 | 0.6947 | 0.0123 | 0.6932 | 0.0033 | 0.0035 | 0.4722 | -0.0001 | 0.0066 | -0.0062 | 0.0043 | -262.4742 | -275.4397 | -2.2340 | -2.3447 |
71
+ | 0.694 | 0.05 | 200 | 0.6936 | 0.0097 | 0.6930 | 0.0061 | 0.0057 | 0.5139 | 0.0004 | 0.0095 | -0.0082 | 0.0059 | -262.2527 | -275.1669 | -2.2395 | -2.3497 |
72
+ | 0.6925 | 0.08 | 300 | 0.6925 | 0.0066 | 0.6923 | 0.0105 | 0.0088 | 0.5595 | 0.0017 | 0.0156 | -0.0103 | 0.0085 | -261.9362 | -274.7206 | -2.2341 | -2.3445 |
73
+ | 0.6922 | 0.1 | 400 | 0.6917 | 0.0064 | 0.6914 | 0.0163 | 0.0127 | 0.5913 | 0.0036 | 0.0248 | -0.0145 | 0.0128 | -261.5553 | -274.1491 | -2.2231 | -2.3340 |
74
+ | 0.6911 | 0.13 | 500 | 0.6910 | 0.0140 | 0.6902 | 0.0234 | 0.0173 | 0.5913 | 0.0061 | 0.0387 | -0.0210 | 0.0192 | -261.0899 | -273.4387 | -2.2290 | -2.3389 |
75
+ | 0.6871 | 0.16 | 600 | 0.6908 | 0.0225 | 0.6889 | 0.0296 | 0.0210 | 0.6111 | 0.0086 | 0.0521 | -0.0264 | 0.0252 | -260.7244 | -272.8181 | -2.2202 | -2.3304 |
76
+ | 0.689 | 0.18 | 700 | 0.6901 | 0.0274 | 0.6876 | 0.0385 | 0.0270 | 0.6230 | 0.0115 | 0.0661 | -0.0346 | 0.0323 | -260.1226 | -271.9265 | -2.2190 | -2.3290 |
77
+ | 0.6859 | 0.21 | 800 | 0.6883 | 0.0168 | 0.6861 | 0.0488 | 0.0343 | 0.6349 | 0.0146 | 0.0765 | -0.0371 | 0.0366 | -259.3933 | -270.8916 | -2.2215 | -2.3312 |
78
+ | 0.691 | 0.24 | 900 | 0.6865 | 0.0123 | 0.6837 | 0.0758 | 0.0562 | 0.6746 | 0.0196 | 0.0939 | -0.0442 | 0.0451 | -257.2037 | -268.1935 | -2.2186 | -2.3282 |
79
+ | 0.6913 | 0.26 | 1000 | 0.6848 | 0.0284 | 0.6794 | 0.1034 | 0.0746 | 0.6845 | 0.0288 | 0.1288 | -0.0575 | 0.0616 | -255.3635 | -265.4328 | -2.2139 | -2.3237 |
80
+ | 0.6909 | 0.29 | 1100 | 0.6846 | 0.0441 | 0.6772 | 0.1070 | 0.0736 | 0.7004 | 0.0335 | 0.1416 | -0.0622 | 0.0676 | -255.4643 | -265.0727 | -2.2119 | -2.3218 |
81
+ | 0.68 | 0.31 | 1200 | 0.6871 | 0.0941 | 0.6751 | 0.1084 | 0.0703 | 0.7103 | 0.0380 | 0.1613 | -0.0698 | 0.0765 | -255.7855 | -264.9369 | -2.2163 | -2.3256 |
82
+ | 0.695 | 0.34 | 1300 | 0.6841 | 0.0711 | 0.6749 | 0.1134 | 0.0750 | 0.7024 | 0.0385 | 0.1622 | -0.0703 | 0.0770 | -255.3223 | -264.4297 | -2.2192 | -2.3278 |
83
+ | 0.6805 | 0.37 | 1400 | 0.6853 | 0.0984 | 0.6733 | 0.1125 | 0.0705 | 0.7103 | 0.0419 | 0.1735 | -0.0741 | 0.0818 | -255.7660 | -264.5257 | -2.2104 | -2.3196 |
84
+ | 0.6848 | 0.39 | 1500 | 0.6857 | 0.1050 | 0.6727 | 0.1154 | 0.0720 | 0.7024 | 0.0434 | 0.1814 | -0.0771 | 0.0853 | -255.6182 | -264.2326 | -2.2035 | -2.3127 |
85
+ | 0.6808 | 0.42 | 1600 | 0.6826 | 0.0796 | 0.6730 | 0.1207 | 0.0780 | 0.7143 | 0.0427 | 0.1763 | -0.0745 | 0.0829 | -255.0231 | -263.7046 | -2.2129 | -2.3212 |
86
+ | 0.6732 | 0.44 | 1700 | 0.6846 | 0.1029 | 0.6717 | 0.1181 | 0.0728 | 0.7044 | 0.0454 | 0.1851 | -0.0765 | 0.0868 | -255.5430 | -263.9595 | -2.2054 | -2.3143 |
87
+ | 0.6995 | 0.47 | 1800 | 0.6804 | 0.0699 | 0.6722 | 0.1233 | 0.0790 | 0.7083 | 0.0443 | 0.1778 | -0.0745 | 0.0833 | -254.9202 | -263.4485 | -2.2043 | -2.3136 |
88
+ | 0.6824 | 0.5 | 1900 | 0.6816 | 0.0861 | 0.6713 | 0.1223 | 0.0760 | 0.7063 | 0.0464 | 0.1866 | -0.0767 | 0.0872 | -255.2250 | -263.5396 | -2.2077 | -2.3163 |
89
+ | 0.6805 | 0.52 | 2000 | 0.6786 | 0.0523 | 0.6722 | 0.1291 | 0.0848 | 0.7123 | 0.0443 | 0.1758 | -0.0732 | 0.0822 | -254.3396 | -262.8673 | -2.2056 | -2.3146 |
90
+ | 0.6827 | 0.55 | 2100 | 0.6802 | 0.0708 | 0.6710 | 0.1253 | 0.0784 | 0.7123 | 0.0469 | 0.1860 | -0.0766 | 0.0868 | -254.9799 | -263.2408 | -2.2092 | -2.3177 |
91
+ | 0.6746 | 0.58 | 2200 | 0.6794 | 0.0680 | 0.6712 | 0.1273 | 0.0810 | 0.7103 | 0.0464 | 0.1843 | -0.0764 | 0.0864 | -254.7231 | -263.0406 | -2.2069 | -2.3152 |
92
+ | 0.6785 | 0.6 | 2300 | 0.6800 | 0.0733 | 0.6707 | 0.1264 | 0.0789 | 0.7044 | 0.0475 | 0.1884 | -0.0774 | 0.0880 | -254.9275 | -263.1323 | -2.2015 | -2.3102 |
93
+ | 0.6814 | 0.63 | 2400 | 0.6801 | 0.0765 | 0.6701 | 0.1270 | 0.0782 | 0.7103 | 0.0488 | 0.1911 | -0.0775 | 0.0891 | -254.9968 | -263.0727 | -2.1975 | -2.3061 |
94
+ | 0.6871 | 0.65 | 2500 | 0.6804 | 0.0817 | 0.6696 | 0.1257 | 0.0759 | 0.7103 | 0.0498 | 0.1935 | -0.0785 | 0.0904 | -255.2312 | -263.2015 | -2.2030 | -2.3112 |
95
+ | 0.6898 | 0.68 | 2600 | 0.6807 | 0.0878 | 0.6694 | 0.1256 | 0.0752 | 0.7143 | 0.0504 | 0.1974 | -0.0786 | 0.0916 | -255.3051 | -263.2164 | -2.2012 | -2.3092 |
96
+ | 0.6743 | 0.71 | 2700 | 0.6807 | 0.0904 | 0.6693 | 0.1264 | 0.0758 | 0.7123 | 0.0506 | 0.1972 | -0.0789 | 0.0918 | -255.2393 | -263.1350 | -2.1994 | -2.3075 |
97
+ | 0.6738 | 0.73 | 2800 | 0.6800 | 0.0806 | 0.6698 | 0.1281 | 0.0785 | 0.7083 | 0.0496 | 0.1937 | -0.0789 | 0.0908 | -254.9662 | -262.9601 | -2.1961 | -2.3042 |
98
+ | 0.6842 | 0.76 | 2900 | 0.6801 | 0.0851 | 0.6693 | 0.1275 | 0.0769 | 0.7183 | 0.0506 | 0.1963 | -0.0794 | 0.0917 | -255.1314 | -263.0265 | -2.1946 | -2.3029 |
99
+ | 0.6734 | 0.79 | 3000 | 0.6801 | 0.0854 | 0.6692 | 0.1279 | 0.0771 | 0.7163 | 0.0508 | 0.1971 | -0.0798 | 0.0921 | -255.1094 | -262.9807 | -2.1911 | -2.2995 |
100
+ | 0.6772 | 0.81 | 3100 | 0.6800 | 0.0866 | 0.6692 | 0.1284 | 0.0776 | 0.7123 | 0.0508 | 0.1970 | -0.0799 | 0.0922 | -255.0642 | -262.9377 | -2.1975 | -2.3054 |
101
+ | 0.6748 | 0.84 | 3200 | 0.6797 | 0.0796 | 0.6693 | 0.1293 | 0.0786 | 0.7163 | 0.0507 | 0.1960 | -0.0798 | 0.0918 | -254.9594 | -262.8471 | -2.1967 | -2.3047 |
102
+ | 0.6821 | 0.86 | 3300 | 0.6797 | 0.0824 | 0.6692 | 0.1290 | 0.0782 | 0.7103 | 0.0507 | 0.1960 | -0.0802 | 0.0920 | -254.9983 | -262.8787 | -2.2014 | -2.3090 |
103
+ | 0.6759 | 0.89 | 3400 | 0.6797 | 0.0812 | 0.6692 | 0.1290 | 0.0781 | 0.7143 | 0.0509 | 0.1971 | -0.0804 | 0.0922 | -255.0086 | -262.8699 | -2.1973 | -2.3053 |
104
+ | 0.678 | 0.92 | 3500 | 0.6798 | 0.0823 | 0.6692 | 0.1288 | 0.0779 | 0.7143 | 0.0509 | 0.1971 | -0.0803 | 0.0923 | -255.0269 | -262.8900 | -2.1943 | -2.3025 |
105
+ | 0.6798 | 0.94 | 3600 | 0.6795 | 0.0808 | 0.6693 | 0.1292 | 0.0786 | 0.7143 | 0.0506 | 0.1957 | -0.0803 | 0.0919 | -254.9599 | -262.8558 | -2.1965 | -2.3045 |
106
+ | 0.6848 | 0.97 | 3700 | 0.6797 | 0.0806 | 0.6692 | 0.1292 | 0.0784 | 0.7123 | 0.0509 | 0.1967 | -0.0801 | 0.0921 | -254.9834 | -262.8504 | -2.2017 | -2.3093 |
107
+ | 0.6734 | 0.99 | 3800 | 0.6797 | 0.0822 | 0.6692 | 0.1295 | 0.0787 | 0.7123 | 0.0508 | 0.1967 | -0.0800 | 0.0920 | -254.9542 | -262.8235 | -2.1947 | -2.3028 |
108
+
109
+
110
+ ### Framework versions
111
+
112
+ - PEFT 0.7.1
113
+ - Transformers 4.39.0.dev0
114
+ - Pytorch 2.1.2+cu121
115
+ - Datasets 2.14.6
116
+ - Tokenizers 0.15.2
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dd8587eb8f477b838d317bbb8d975286b4ee5db895b6ab40aafddbefdb71851c
3
  size 671150064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f551fddede8a76c2cbaf362460947bb71c543e7dd59c313dff0432e6fefd38d
3
  size 671150064
all_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 1.0,
3
+ "train_loss": 0.68390608707419,
4
+ "train_runtime": 41762.7734,
5
+ "train_samples": 61134,
6
+ "train_samples_per_second": 1.464,
7
+ "train_steps_per_second": 0.091
8
+ }
runs/Jul16_03-49-11_node29/events.out.tfevents.1721069652.node29.1081696.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:66fc10c4843d8c870c7d5316d5493ac6c5e265f26d4a097019534462542a1a0b
3
- size 423745
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:911837331a29dc98d6b1021098d4c65b6f7104504cf1d71d408cdf4040c53501
3
+ size 426089
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 1.0,
3
+ "train_loss": 0.68390608707419,
4
+ "train_runtime": 41762.7734,
5
+ "train_samples": 61134,
6
+ "train_samples_per_second": 1.464,
7
+ "train_steps_per_second": 0.091
8
+ }
trainer_state.json ADDED
The diff for this file is too large to render. See raw diff