BlinkDL
/

rwkv-4-pile-14b

Text Generation

Model card Files Files and versions Community

BlinkDL commited on Mar 13, 2023

Commit

e71e020

·

1 Parent(s): 5d54576

Update README.md

Files changed (1) hide show

README.md +3 -0

README.md CHANGED Viewed

@@ -18,6 +18,9 @@ datasets:
 RWKV-4 14B is a L40-D5120 causal language model trained on the Pile. See https://github.com/BlinkDL/RWKV-LM for details.
 Use https://github.com/BlinkDL/ChatRWKV to run it.
 RWKV-4-Pile-14B-2023xxxx-ctx4096-testxxx.pth : Fine-tuned to ctx_len 4096.

 RWKV-4 14B is a L40-D5120 causal language model trained on the Pile. See https://github.com/BlinkDL/RWKV-LM for details.
+args.n_layer = 40
+args.n_embd = 5120
 Use https://github.com/BlinkDL/ChatRWKV to run it.
 RWKV-4-Pile-14B-2023xxxx-ctx4096-testxxx.pth : Fine-tuned to ctx_len 4096.