sagorsarker commited on
Commit
bedfa50
1 Parent(s): 81d2404

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -11,7 +11,7 @@ pipeline_tag: text-generation
11
  ---
12
 
13
  # TituLM-1B-ENBN-V1
14
- TituLM-1B-ENBN-V1 is a large language model specifically trained for generating and understanding English and Bangla text. Utilizing a decoder-style transformer architecture, this model has been extensively trained on a dataset comprising __(will disclose later)__ billion Bangla and English tokens. This model is the part of iterative train and release Bilingual LLM from Hishab.
15
 
16
  The training process was managed using the robust framework provided by MosaicML's [llm-foundry](https://github.com/mosaicml/llm-foundry) repository. Throughout the training phase, titulm-1b-bn-v1 underwent a total of 59 iterations, allowing for iterative refinements and optimization.
17
  Notable training configs:
@@ -39,6 +39,7 @@ Token-wise distribution will be added soon below.
39
  | Redpajama Common crawl | English | 12.74 |
40
  | Redpajama C4 | English | 6.57 |
41
  | Bangla (culturax, books, news, Wikipedia, Banglapedia) | Bangla | ~14 |
 
42
 
43
  ## How to Use
44
  The basic use cases to generate text using this model are simple. Follow the below code to generate text using this model.
 
11
  ---
12
 
13
  # TituLM-1B-ENBN-V1
14
+ TituLM-1B-ENBN-V1 is a large language model specifically trained for generating and understanding English and Bangla text. Utilizing a decoder-style transformer architecture, this model has been extensively trained on a dataset comprising __43.19__ billion Bangla, English and codes tokens. This model is the part of iterative train and release Bilingual LLM from Hishab.
15
 
16
  The training process was managed using the robust framework provided by MosaicML's [llm-foundry](https://github.com/mosaicml/llm-foundry) repository. Throughout the training phase, titulm-1b-bn-v1 underwent a total of 59 iterations, allowing for iterative refinements and optimization.
17
  Notable training configs:
 
39
  | Redpajama Common crawl | English | 12.74 |
40
  | Redpajama C4 | English | 6.57 |
41
  | Bangla (culturax, books, news, Wikipedia, Banglapedia) | Bangla | ~14 |
42
+ | Total | | 43.19|
43
 
44
  ## How to Use
45
  The basic use cases to generate text using this model are simple. Follow the below code to generate text using this model.