hugo commited on
Commit
cc01a5f
1 Parent(s): cf5935b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -1,6 +1,6 @@
1
- This is the tokenizer used by the MariTalk Small model.
2
 
3
- MariTalk Small is a proprietary LLM that can be used through an API endpoint, which we refer to as the "MariTalk API", or a downloadable version that can be used locally and is encrypted, known as "MariTalk Local".
4
 
5
  The purpose of including this tokenizer is to allow you to estimate the number of tokens in your prompts and, therefore, the cost of using the model.
6
 
@@ -8,7 +8,7 @@ For example:
8
 
9
  ```python
10
  import transformers
11
- tokenizer = transformers.AutoTokenizer.from_pretrained("maritaca-ai/maritalk-tokenizer-small")
12
 
13
  prompt = "Com quantos paus se faz uma canoa?"
14
 
 
1
+ This is the tokenizer used by the Sabia-2 Small model.
2
 
3
+ Sabiá-2 Small is a proprietary LLM that can be used through an API endpoint, which we refer to as the "MariTalk API", or a downloadable version that can be used locally and is encrypted, known as "MariTalk Local".
4
 
5
  The purpose of including this tokenizer is to allow you to estimate the number of tokens in your prompts and, therefore, the cost of using the model.
6
 
 
8
 
9
  ```python
10
  import transformers
11
+ tokenizer = transformers.AutoTokenizer.from_pretrained("maritaca-ai/sabia-2-tokenizer-small")
12
 
13
  prompt = "Com quantos paus se faz uma canoa?"
14