Hailay commited on
Commit
f2ecb69
1 Parent(s): 5b89f05

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -7,9 +7,9 @@ library_name: transformers
7
  datasets:
8
  - Hailay/TigQA
9
  ---
10
- ##1.Model Description
11
 
12
- **Hailay/GeezScriptTokenizer** is a language-specific tokenizer developed to handle the unique characteristics of Geez script languages, particularly Amharic and Tigrinya. This tokenizer is designed to effectively manage the complexities of these languages by accurately identifying and processing prefixes, postfixes, and word boundaries within the text. By incorporating these language-specific rules, GeezScriptTokenizer significantly improves tokenization efficiency, ensuring better representation and performance for tasks involving Amharic and Tigrinya.
13
 
14
  This tokenizer is highly suited for natural language processing (NLP) tasks where standard multilingual tokenizers may struggle with the nuances of Geez script languages. Hailay/GeezScriptTokenizer is an ideal tool for researchers and developers working with these languages, providing a tailored approach to tokenization that enhances the overall quality of language models and downstream tasks.
15
 
 
7
  datasets:
8
  - Hailay/TigQA
9
  ---
10
+ **1.Model Description**
11
 
12
+ **Hailay/GeezScriptTokenizer**: is a language-specific tokenizer developed to handle the unique characteristics of Geez script languages, particularly Amharic and Tigrinya. This tokenizer is designed to effectively manage the complexities of these languages by accurately identifying and processing prefixes, postfixes, and word boundaries within the text. By incorporating these language-specific rules, GeezScriptTokenizer significantly improves tokenization efficiency, ensuring better representation and performance for tasks involving Amharic and Tigrinya.
13
 
14
  This tokenizer is highly suited for natural language processing (NLP) tasks where standard multilingual tokenizers may struggle with the nuances of Geez script languages. Hailay/GeezScriptTokenizer is an ideal tool for researchers and developers working with these languages, providing a tailored approach to tokenization that enhances the overall quality of language models and downstream tasks.
15