uvegesistvan/Hun_Eng_RoBERTa_large_Plain

Model description

Cased fine-tuned XLM-RoBERTa-large model for Hungarian and English, trained on datasets provided by the National Tax and Customs Administration - Hungary (NAV) and translated versions of the same dataset using Google Translate API.

Intended uses & limitations

The model is designed to classify sentences as either "comprehensible" or "not comprehensible" (according to Plain Language guidelines):

• Label_0 - "comprehensible" - The sentence is in Plain Language.
• Label_1 - "not comprehensible" - The sentence is not in Plain Language.

Training

Fine-tuned version of the original xlm-roberta-large model, trained on a dataset of Hungarian legal and administrative texts. The model was also trained on the translated version of this dataset (via Google Translate API) for English classification.

Eval results

Hungarian Results:

Class	Precision	Recall	F-Score
Comprehensible / Label_0	0.82	0.74	0.78
Not comprehensible / Label_1	0.77	0.85	0.81
accuracy			0.80
macro avg	0.80	0.79	0.79
weighted avg	0.80	0.80	0.79

English Results:

Class	Precision	Recall	F-Score
Comprehensible / Label_0	0.68	0.60	0.64
Not comprehensible / Label_1	0.66	0.73	0.69
accuracy			0.67
macro avg	0.67	0.67	0.67
weighted avg	0.67	0.67	0.67

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("uvegesistvan/Hun_Eng_RoBERTa_large_Plain")
model = AutoModelForSequenceClassification.from_pretrained("uvegesistvan/Hun_Eng_RoBERTa_large_Plain")

BibTeX entry and citation info

If you use the model, please cite the following dissertation (to be submitted for workshop discussion):

Bibtex:

@PhDThesis{ Uveges:2024,
  author = {{"U}veges, Istv{\'a}n},
  title  = {K{\"o}z{\'e}rthet{\"o} és automatiz{\'a}ci{\'o} - k{\'i}s{\'e}rletek a jog, term{\'e}szetesnyelv-feldolgoz{\'a}s {\'e}s informatika hat{\'a}r{\'a}n.},
  year   = {2024},
  school = {Szegedi Tudom{\'a}nyegyetem}
}