sagorsarker
commited on
Commit
•
bedfa50
1
Parent(s):
81d2404
Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,7 @@ pipeline_tag: text-generation
|
|
11 |
---
|
12 |
|
13 |
# TituLM-1B-ENBN-V1
|
14 |
-
TituLM-1B-ENBN-V1 is a large language model specifically trained for generating and understanding English and Bangla text. Utilizing a decoder-style transformer architecture, this model has been extensively trained on a dataset comprising
|
15 |
|
16 |
The training process was managed using the robust framework provided by MosaicML's [llm-foundry](https://github.com/mosaicml/llm-foundry) repository. Throughout the training phase, titulm-1b-bn-v1 underwent a total of 59 iterations, allowing for iterative refinements and optimization.
|
17 |
Notable training configs:
|
@@ -39,6 +39,7 @@ Token-wise distribution will be added soon below.
|
|
39 |
| Redpajama Common crawl | English | 12.74 |
|
40 |
| Redpajama C4 | English | 6.57 |
|
41 |
| Bangla (culturax, books, news, Wikipedia, Banglapedia) | Bangla | ~14 |
|
|
|
42 |
|
43 |
## How to Use
|
44 |
The basic use cases to generate text using this model are simple. Follow the below code to generate text using this model.
|
|
|
11 |
---
|
12 |
|
13 |
# TituLM-1B-ENBN-V1
|
14 |
+
TituLM-1B-ENBN-V1 is a large language model specifically trained for generating and understanding English and Bangla text. Utilizing a decoder-style transformer architecture, this model has been extensively trained on a dataset comprising __43.19__ billion Bangla, English and codes tokens. This model is the part of iterative train and release Bilingual LLM from Hishab.
|
15 |
|
16 |
The training process was managed using the robust framework provided by MosaicML's [llm-foundry](https://github.com/mosaicml/llm-foundry) repository. Throughout the training phase, titulm-1b-bn-v1 underwent a total of 59 iterations, allowing for iterative refinements and optimization.
|
17 |
Notable training configs:
|
|
|
39 |
| Redpajama Common crawl | English | 12.74 |
|
40 |
| Redpajama C4 | English | 6.57 |
|
41 |
| Bangla (culturax, books, news, Wikipedia, Banglapedia) | Bangla | ~14 |
|
42 |
+
| Total | | 43.19|
|
43 |
|
44 |
## How to Use
|
45 |
The basic use cases to generate text using this model are simple. Follow the below code to generate text using this model.
|