BlinkDL
/

rwkv-4-pile-7b

Text Generation

Model card Files Files and versions Community

BlinkDL commited on Mar 4, 2023

Commit

296e423

•

1 Parent(s): 2739b27

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -24,7 +24,7 @@ ctx_len = 1024
 n_layer = 32
 n_embd = 4096
-RWKV-4-Pile-7B-20230109-ctx4096.pth
 * Likely better. Please test.
 RWKV-4-Pile-7B-20221115-8047.pth : Trained on the Pile for 332B tokens.

 n_layer = 32
 n_embd = 4096
+RWKV-4-Pile-7B-20230109-ctx4096.pth : Fine-tuned to ctx_len 4096.
 * Likely better. Please test.
 RWKV-4-Pile-7B-20221115-8047.pth : Trained on the Pile for 332B tokens.