Usage

Load in transformers library with:

from transformers import AutoTokenizer, AutoModelForMaskedLM
  
  tokenizer = AutoTokenizer.from_pretrained("EMBEDDIA/est-roberta")
  model = AutoModelForMaskedLM.from_pretrained("EMBEDDIA/est-roberta")

Est-RoBERTa

Est-RoBERTa model is a monolingual Estonian BERT-like model. It is closely related to French Camembert model https://camembert-model.fr/. The Estonian corpora used for training the model have 2.51 billion tokens in total. The subword vocabulary contains 40,000 tokens.

Est-RoBERTa was trained for 40 epochs.

Downloads last month
93
Inference Providers NEW
This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.

Model tree for EMBEDDIA/est-roberta

Finetunes
2 models