Commit
•
788513d
1
Parent(s):
ec38b03
Update README.md
Browse files
README.md
CHANGED
|
@@ -61,6 +61,20 @@ This service is a research preview, and as such, it only provides limited safety
|
|
| 61 |
#### Summary
|
| 62 |
|
| 63 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
|
| 66 |
## Model Card Contact
|
|
|
|
| 61 |
#### Summary
|
| 62 |
|
| 63 |
|
| 64 |
+
## Technical Specifications [optional]
|
| 65 |
+
|
| 66 |
+
### Model Architecture and Objective
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
ganga-1b is a decoder-only transformer model, featuring the following specifications:
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
* l#ayers: 16
|
| 73 |
+
* #attention heads: 32
|
| 74 |
+
* d_model/embedding dimension: 2048
|
| 75 |
+
* vocabulary size: 30000
|
| 76 |
+
* sliding window : 512
|
| 77 |
+
* ffn dimension : 716
|
| 78 |
|
| 79 |
|
| 80 |
## Model Card Contact
|