ibm-granite
/

granite-3.1-1b-a400m-base

Text Generation

Model card Files Files and versions Community

rpand002 commited on Dec 12, 2024

Commit

6bbcb9d

·

verified ·

1 Parent(s): f6d0ad7

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -74,7 +74,7 @@ Granite-3.1-1B-A400M-Base is based on a decoder-only sparse Mixture of Experts (
 | Number of experts            | —        | —        | **32**       | 40       |
 | MoE TopK                     | —        | —        | **8**        | 8        |
 | Initialization std           | 0.1      | 0.1      | **0.1**      | 0.1      |
-| Sequence length              | 128K     | 128k     | **128k**     | 128k     |
 | Position embedding           | RoPE     | RoPE     | **RoPE**     | RoPE     |
 | # Parameters                 | 2.5B     | 8.1B     | **1.3B**     | 3.3B     |
 | # Active parameters          | 2.5B     | 8.1B     | **400M**     | 800M     |

 | Number of experts            | —        | —        | **32**       | 40       |
 | MoE TopK                     | —        | —        | **8**        | 8        |
 | Initialization std           | 0.1      | 0.1      | **0.1**      | 0.1      |
+| Sequence length              | 128K     | 128K     | **128K**     | 128K     |
 | Position embedding           | RoPE     | RoPE     | **RoPE**     | RoPE     |
 | # Parameters                 | 2.5B     | 8.1B     | **1.3B**     | 3.3B     |
 | # Active parameters          | 2.5B     | 8.1B     | **400M**     | 800M     |