license: mit
language:
- ar
pipeline_tag: text-generation
tags:
- 'arabic '
- text-generation
Model Description Model Name: ArabicGPT-S Architecture: GPT-2 Layers: 12 Model Size: 134M Context Window Size: 768
ArabianGPT is a custom-trained version of the GPT-2 base model, specifically tailored for the Arabic language. It is designed to understand and generate Arabic text, making it suitable for various natural language processing tasks in Arabic.
Training Dataset: Abu Elkhiar Corpus Size: 15.5 GB Number of Words: 237,814,541 Number of Tokens: 1,752,421,071 Epochs: 5.87 Loss: 3.97
The model was trained on the Abu Elkhiar dataset, a comprehensive Arabic text corpus encompassing a wide range of topics. The training process focused on adapting the model to understand the nuances and complexities of the Arabic language.
Tokenizer: Type: Custom trained SentencePiece tokenizer Vocabulary Size: 64K
We employed AraNizer, a custom trained tokenizer based on the SentencePiece model, with a vocabulary size of 64. This choice was made to optimize the model's performance for the specific characteristics of the Arabic language.
Usage ArabianGPT can be used for text generation
Limitations As with any language model, ArabicGPT may have limitations in understanding context or generating text in certain scenarios. Users should be aware of these limitations and use the model accordingly.
Ethical Considerations We emphasize responsible usage of ArabianGPT. Users should ensure that the generated text is used ethically and does not propagate misinformation or harmful content.
Citation If you use ArabianGPT in your research or application, please cite it as follows:
@misc{ArabianGPT, 2023, title={ArabianGPT: A GPT-2 Based Language Model for Arabic}, author={Najar, Omar and Sibaee, Serry and Ghouti, Lahouari and Koubaa, Anis}, affiliation={Prince Sultan University, Riyadh, Saudi Arabia}, year={2023}, }
Acknowledgments We thank Prince Sultan University espically Robotoics and Internet of Things Lab for suuport
Contact For inquiries regarding ArabicGPT-S, please contact onajar@psu.edu.sa