ai-sage
/

Giga-Embeddings-instruct

Feature Extraction

Model card Files Files and versions Community

ekolodin commited on 1 day ago

Commit

201baa9

•

1 Parent(s): 2b5a715

Update tokenizer_config.json

Files changed (1) hide show

tokenizer_config.json +2 -1

tokenizer_config.json CHANGED Viewed

@@ -2078,5 +2078,6 @@
   "eos_token": "</s>",
   "model_max_length": 1000000000000000019884624838656,
   "tokenizer_class": "PreTrainedTokenizerFast",
-  "unk_token": "<unk>"
 }

   "eos_token": "</s>",
   "model_max_length": 1000000000000000019884624838656,
   "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "<unk>",
+  "pad_token": "<unk>"
 }