Tim-05369/orpo_train_using_space_demo

README.md

@@ -1,6 +1,4 @@
 ---
-license: apache-2.0
-base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
 tags:
 - trl
 - orpo
@@ -15,20 +13,20 @@ should probably proofread and complete it, then remove this comment. -->
 
 # results
 
-This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) on the None dataset.
+This model was trained from scratch on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.4477
-- Rewards/chosen: -0.1131
-- Rewards/rejected: -0.0946
-- Rewards/accuracies: 0.6667
-- Rewards/margins: -0.0185
-- Logps/rejected: -1.8929
-- Logps/chosen: -2.2621
-- Logits/rejected: -2.5533
-- Logits/chosen: -2.5157
-- Nll Loss: 2.3980
-- Log Odds Ratio: -0.9931
-- Log Odds Chosen: -0.3878
+- Loss: 1.1578
+- Rewards/chosen: -0.0609
+- Rewards/rejected: -0.0483
+- Rewards/accuracies: 0.5
+- Rewards/margins: -0.0126
+- Logps/rejected: -0.9658
+- Logps/chosen: -1.2181
+- Logits/rejected: -1.9100
+- Logits/chosen: -2.3033
+- Nll Loss: 1.1151
+- Log Odds Ratio: -0.8532
+- Log Odds Chosen: -0.2694
 
 ## Model description
 

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d2a84ef621a9eedfa7f6f2b9b19ae685a5a721a37f0322c2134504785dacdc2c
+oid sha256:271c150024b462dd3bc9cc9cba8917f9d264ba0e193ecedc0e1037c4a499561d
 size 4400216536

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:80166eae19da31ec096daf9e9ce36d5183712c11eda667881be34bf3390581aa
+oid sha256:75150a9bfbb937ba6d1d897eff1824aebce6c5767c42e4ad7db8f81cc1dbb1b3
 size 5496