tiiuae
/

Falcon3-7B-Base

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

puneeshkhanna commited on 2 days ago

Commit

204fce4

•

1 Parent(s): d5488b9

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -24,7 +24,7 @@ Falcon3-7B-Base supports 4 languages (english, french, spanish, portuguese) and
   - Grouped query attention (GQA) for faster inference: 12 query heads and 4 key value heads
   - Wider head dimension: 256
   - High RoPE value to support long context understanding: 1000042
-  - 32k context length
   - 131K vocab size
 - Pretrained on 14 Gigatokens of datasets comprising of web, code, STEM, high quality and mutlilingual data using 2048 H100 GPU chips
 - Supports EN, FR, ES, PT

   - Grouped query attention (GQA) for faster inference: 12 query heads and 4 key value heads
   - Wider head dimension: 256
   - High RoPE value to support long context understanding: 1000042
+  - 32K context length
   - 131K vocab size
 - Pretrained on 14 Gigatokens of datasets comprising of web, code, STEM, high quality and mutlilingual data using 2048 H100 GPU chips
 - Supports EN, FR, ES, PT