Update README.md

README.md

@@ -4,31 +4,26 @@ language:
 tags:
 - falcon3
 - falcon3_mamba
-- falcon_mamba
 base_model:
 - tiiuae/Falcon3-Mamba-7B-Base
-license: other
-license_name: falcon-llm-license
-license_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html
-library_name: transformers
 ---
 
 # Falcon3-Mamba-7B-Instruct
 
 **Falcon3** family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B.
 
-This repository contains the **Falcon3-Mamba-7B-Instruct**. It achieves, compared to similar SSM-based models of the same size, state of art results (at release's time) on reasoning, language understanding, instruction following, code and mathematics tasks.
-Falcon3-Mamba-7B-Instruct supports a context length up to 32K and was mainly trained on english corpus.
+This repository contains the **Falcon3-Mamba-7B-Instruct**. It achieves ,compared to similar SSM-based models of the same size, state of art results (at release's time) on reasoning, language understanding, instruction following, code and mathematics tasks.
+Falcon3-Mamba-7B-Instruct supports a context length up to 32K and 1 language (english).
 
 ## Model Details
-- Architecture (same as [Falcon-Mamba-7b](https://huggingface.co/tiiuae/falcon-mamba-7b))
+- Architecture(same as Falcon-Mamba-7b)
   - Mamba1 based causal decoder only architecture trained on a causal language modeling task (i.e., predict the next token).
   - 64 decoder blocks
   - width: 4096
   - state_size: 16 
   - 32k context length
   - 65k vocab size
-- Continue Pretrained from [Falcon Mamba 7B](https://huggingface.co/tiiuae/falcon-mamba-7b), with another 1500 Gigatokens of data comprising of web, code, STEM and high quality data.
+- Pretrained on 7 Teratokens of datasets comprising of web, code, STEM and high quality data using 2048 H100 GPU chips
 - Postrained on 1.2 million samples of STEM, conversations, code, and safety.
 - Developed by [Technology Innovation Institute](https://www.tii.ae)
 - License: TII Falcon-LLM License 2.0
@@ -84,7 +79,7 @@ print(response)
 <br>
 
 # Benchmarks
-We report in the following table our internal pipeline benchmarks. For the benchmarks marked by star, we normalize the results with HuggingFace score normalization:
+We report in the following table our internal pipeline benchmarks:
 
 <table border="1" style="width: 100%; text-align: center; border-collapse: collapse;">
     <colgroup>
@@ -93,6 +88,7 @@ We report in the following table our internal pipeline benchmarks. For the bench
         <col style="width: 7%;">
         <col style="width: 7%;">
         <col style="width: 7%;">
+        <col style="width: 7%;">
         <col style="background-color: rgba(80, 15, 213, 0.5); width: 7%;">
     </colgroup>
     <thead>
@@ -101,6 +97,7 @@ We report in the following table our internal pipeline benchmarks. For the bench
             <th>Benchmark</th>
             <th>Zamba2-7B-instruct</th>
             <th>Jamba-1.5-Mini</th>
+            <th>Qwen2-7B-Instruct</th>
             <th>Llama-3.1-8B-Instruct</th>
             <th>Falcon3-Mamba-7B-Instruct</th>
         </tr>
@@ -109,105 +106,122 @@ We report in the following table our internal pipeline benchmarks. For the bench
         <tr>
             <td rowspan="3">General</td>
             <td>MMLU (5-shot)</td>
-            <td>30.6</td>
-            <td>68.7</td>
-            <td>55.9</td>
-            <td>65.3</td>
+            <td>-</td>
+            <td>68.7%</td>
+            <td>-</td>
+            <td>68.5%</td>
+            <td>-</td>
         </tr>
         <tr>
-            <td>MMLU-PRO (5-shot)*</td>
-            <td>32.4</td>
-            <td>31.6</td>
-            <td>21.8</td>
-            <td>26.3</td>
+            <td>MMLU-PRO (5-shot)</td>
+            <td>32.4%</td>
+            <td>31.6%</td>
+            <td>31.6%</td>
+            <td>29.6%</td>
+            <td>26.3%</td>
         </tr>
         <tr>
             <td>IFEval</td>
-            <td>69.9</td>
-            <td>65.7</td>
-            <td>78.8</td>
-            <td>71.7</td>
+            <td>69.9%</td>
+            <td>65.7%</td>
+            <td>56.8%</td>
+            <td>78.6%</td>
+            <td>71.7%</td>
         </tr>
         <tr>
             <td rowspan="2">Math</td>
             <td>GSM8K (5-shot)</td>
-            <td>0</td>
-            <td>74.9</td>
-            <td>19.2</td>
-            <td>65.2</td>
+            <td>-</td>
+            <td>74.9%</td>
+            <td>-</td>
+            <td>-</td>
+            <td>-</td>
         </tr>
         <tr>
-            <td>MATH Lvl-5 (4-shot)</td>
-            <td>13.6</td>
-            <td>6.9</td>
-            <td>10.4</td>
-            <td>27.3</td>
+            <td>MATH(4-shot)</td>
+            <td>-</td>
+            <td>6.9%</td>
+            <td>9.44%</td>
+            <td>-</td>
+            <td>27.3%</td>
         </tr>
         <tr>
             <td rowspan="4">Reasoning</td>
             <td>Arc Challenge (25-shot)</td>
-            <td>54</td>
-            <td>54.3</td>
-            <td>46.6</td>
-            <td>53.7</td>
+            <td>-</td>
+            <td>54.3%</td>
+            <td>-</td>
+            <td>-</td>
+            <td>-</td>
         </tr>
         <tr>
-            <td>GPQA (0-shot)*</td>
-            <td>10.3</td>
-            <td>11.1</td>
-            <td>6.2</td>
-            <td>7.2</td>
+            <td>GPQA (0-shot)</td>
+            <td>10.3%</td>
+            <td>11.1%</td>
+            <td>6.4%</td>
+            <td>2.4%</td>
+            <td>7.2%</td>
         </tr>
         <tr>
-            <td>MUSR (0-shot)*</td>
-            <td>8.2</td>
-            <td>12.2</td>
-            <td>38.6</td>
-            <td>8.3</td>
+            <td>MUSR (0-shot)</td>
+            <td>8.2%</td>
+            <td>12.2%</td>
+            <td>7.4%</td>
+            <td>8.4%</td>
+            <td>8.3%</td>
         </tr>
         <tr>
-            <td>BBH (3-shot)*</td>
-            <td>33.3</td>
-            <td>35.3</td>
-            <td>43.7</td>
-            <td>25.2</td>
+            <td>BBH (3-shot)</td>
+            <td>33.3%</td>
+            <td>35.3%</td>
+            <td>37.8%</td>
+            <td>29.9%</td>
+            <td>25.2%</td>
         </tr>
         <tr>
             <td rowspan="4">CommonSense Understanding</td>
             <td>PIQA (0-shot)</td>
-            <td>75.6</td>
-            <td>82.3</td>
-            <td>78.9</td>
-            <td>80.9</td>
+            <td>-</td>
+            <td>82.3%</td>
+            <td>-</td>
+            <td>-</td>
+            <td>-</td>
         </tr>
         <tr>
             <td>SciQ (0-shot)</td>
-            <td>29.2</td>
-            <td>94.9</td>
-            <td>80.2</td>
-            <td>93.6</td>
+            <td>-</td>
+            <td>94.9%</td>
+            <td>-</td>
+            <td>-</td>
+            <td>-</td>
+        </tr>
+        <tr>
+            <td>Winogrande (0-shot)</td>
+            <td>-</td>
+            <td>64.5%</td>
+            <td>-</td>
+            <td>-</td>
+            <td>-</td>
         </tr>
         <tr>
             <td>OpenbookQA (0-shot)</td>
-            <td>45.6</td>
-            <td>45.8</td>
-            <td>46.2</td>
-            <td>47.2</td>
+            <td>-</td>
+            <td>34.6%</td>
+            <td>-</td>
+            <td>-</td>
+            <td>-</td>
         </tr>
     </tbody>
 </table>
 
-## Useful links
-- View our [release blogpost](https://huggingface.co/blog/falcon3).
-- Feel free to join [our discord server](https://discord.gg/fwXpMyGc) if you have any questions or to interact with our researchers and developers.
 
-## Citation
-If the Falcon3 family of models were helpful to your work, feel free to give us a cite.
+# Citation
+If Falcon3 family were helpful to your work, feel free to give us a cite.
 
 ```
 @misc{Falcon3,
-    title = {The Falcon 3 Family of Open Models},
-    author = {Falcon-LLM Team},
+    title = {The Falcon 3 family of Open Models},
+    author = {TII Team},
     month = {December},
     year = {2024}
 }