|
--- |
|
license: apache-2.0 |
|
language: |
|
- fa |
|
- en |
|
metrics: |
|
- accuracy |
|
pipeline_tag: token-classification |
|
|
|
--- |
|
|
|
# NER Model using Roberta |
|
|
|
|
|
This markdown presents a Robustly Optimized BERT Pretraining Approach (RoBERTa) model trained on a combination of two diverse datasets for two languages: English and Persian. The English dataset used is [CoNLL 2003](https://huggingface.co/datasets/conll2003), while the Persian dataset is [PEYMA-ARMAN-Mixed](https://huggingface.co/datasets/AliFartout/PEYMA-ARMAN-Mixed), a fusion of the "PEYAM" and "ARMAN" datasets, both popular for Named Entity Recognition (NER) tasks. |
|
|
|
The model training pipeline involves the following steps: |
|
|
|
Data Preparation: Cleaning, aligning, and mixing data from the two datasets. |
|
Data Loading: Loading the prepared data for subsequent processing. |
|
Tokenization: Utilizing tokenization to prepare the text data for model input. |
|
Token Splitting: Handling token splitting (e.g., "jack" becomes "_ja _ck") and using "-100" for optimization and ignoring certain tokens. |
|
Model Reconstruction: Adapting the RoBERTa model for token classification in NER tasks. |
|
Model Training: Training the reconstructed model on the combined dataset and evaluating its performance. |
|
The model's performance, as shown in the table below, demonstrates promising results: |
|
| Epoch | Training Loss | Validation Loss | F1 | Recall | Precision | Accuracy | |
|
|:-------:|:--------:|:--------:|:----------:|:--------------:|:----------:|:----------------:| |
|
| 1 | 0.072600 | 0.038918 | 89.5% | 0.906680 | 0.883703 | 0.987799 | |
|
| 2 | 0.027600 | 0.030184 | 92.3% | 0.933840 | 0.915573 | 0.991334 | |
|
| 3 | 0.013500 | 0.030962 | 94% | 0.946840 | 0.933740 | 0.992702 | |
|
| 4 | 0.006600 | 0.029897 | 94.8% | 0.955207 | 0.941990 | 0.993574 | |
|
|
|
The model achieves an impressive F1-score of almost 95%. |
|
|
|
To use the model, the following Python code snippet can be employed: |
|
|
|
```python |
|
from transformers import AutoConfig, AutoTokenizer, AutoModel |
|
|
|
config = AutoConfig.from_pretrained("AliFartout/Roberta-fa-en-ner") |
|
tokenizer = AutoTokenizer.from_pretrained("AliFartout/Roberta-fa-en-ner") |
|
model = AutoModel.from_pretrained("AliFartout/Roberta-fa-en-ner") |
|
``` |
|
|
|
By following this approach, you can seamlessly access and incorporate the trained multilingual NER model into various Natural Language Processing tasks. |
