BlinkDL
/

rwkv-4-pile-3b

Text Generation

Model card Files Files and versions Community

BlinkDL commited on Sep 28, 2022

Commit

f4a2297

·

1 Parent(s): cb0f46c

Update README.md

Files changed (1) hide show

README.md +6 -6

README.md CHANGED Viewed

@@ -26,9 +26,9 @@ ctx_len = 1024
 n_layer = 32
 n_embd = 2560
-Preview checkpoint: RWKV-4-Pile-3B-20220923-3822.pth : Trained on the Pile for 157B tokens.
-* Pile loss 1.9857
-* LAMBADA ppl 5.61, acc 61.87%
-* PIQA acc 73.78%
-* SC2016 acc 69.80%
-* Hellaswag acc_norm 57.46%

 n_layer = 32
 n_embd = 2560
+Preview checkpoint: RWKV-4-Pile-3B-20220928-5147.pth : Trained on the Pile for 212B tokens.
+* Pile loss 1.9666
+* LAMBADA ppl 5.31, acc 63.07%
+* PIQA acc 74.10%
+* SC2016 acc 70.50%
+* Hellaswag acc_norm 58.38%