README.md

metadata

license: mit
tags:
  - generated_from_trainer
metrics:
  - precision
  - recall
  - f1
  - accuracy
model-index:
  - name: pos_final_xlm_de
    results: []

pos_final_xlm_de

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0580
• Precision: 0.9895
• Recall: 0.9894
• F1: 0.9894
• Accuracy: 0.9901

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 1024
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	0.99	128	0.3828	0.9159	0.9106	0.9133	0.9196
No log	1.99	256	0.0659	0.9810	0.9812	0.9811	0.9824
No log	2.99	384	0.0447	0.9857	0.9857	0.9857	0.9865
0.7525	3.99	512	0.0388	0.9870	0.9871	0.9871	0.9878
0.7525	4.99	640	0.0373	0.9871	0.9875	0.9873	0.9881
0.7525	5.99	768	0.0354	0.9880	0.9882	0.9881	0.9889
0.7525	6.99	896	0.0350	0.9883	0.9885	0.9884	0.9891
0.0318	7.99	1024	0.0354	0.9884	0.9886	0.9885	0.9891
0.0318	8.99	1152	0.0356	0.9888	0.9888	0.9888	0.9894
0.0318	9.99	1280	0.0367	0.9888	0.9889	0.9888	0.9895
0.0318	10.99	1408	0.0370	0.9887	0.9888	0.9887	0.9894
0.0205	11.99	1536	0.0370	0.9889	0.9891	0.9890	0.9896
0.0205	12.99	1664	0.0388	0.9888	0.9889	0.9888	0.9895
0.0205	13.99	1792	0.0397	0.9890	0.9891	0.9890	0.9897
0.0205	14.99	1920	0.0403	0.9891	0.9891	0.9891	0.9897
0.0146	15.99	2048	0.0413	0.9891	0.9891	0.9891	0.9897
0.0146	16.99	2176	0.0423	0.9891	0.9891	0.9891	0.9898
0.0146	17.99	2304	0.0429	0.9891	0.9891	0.9891	0.9897
0.0146	18.99	2432	0.0443	0.9893	0.9894	0.9893	0.9899
0.0103	19.99	2560	0.0457	0.9890	0.9889	0.9890	0.9896
0.0103	20.99	2688	0.0455	0.9891	0.9892	0.9891	0.9898
0.0103	21.99	2816	0.0468	0.9891	0.9892	0.9891	0.9898
0.0103	22.99	2944	0.0491	0.9891	0.9892	0.9892	0.9898
0.0073	23.99	3072	0.0495	0.9894	0.9894	0.9894	0.9900
0.0073	24.99	3200	0.0503	0.9892	0.9892	0.9892	0.9898
0.0073	25.99	3328	0.0519	0.9892	0.9892	0.9892	0.9898
0.0073	26.99	3456	0.0522	0.9892	0.9893	0.9892	0.9899
0.0052	27.99	3584	0.0526	0.9892	0.9892	0.9892	0.9899
0.0052	28.99	3712	0.0535	0.9892	0.9892	0.9892	0.9899
0.0052	29.99	3840	0.0544	0.9894	0.9894	0.9894	0.9900
0.0052	30.99	3968	0.0548	0.9893	0.9894	0.9894	0.9900
0.0038	31.99	4096	0.0563	0.9892	0.9892	0.9892	0.9899
0.0038	32.99	4224	0.0562	0.9894	0.9894	0.9894	0.9900
0.0038	33.99	4352	0.0577	0.9891	0.9892	0.9892	0.9898
0.0038	34.99	4480	0.0580	0.9895	0.9894	0.9894	0.9901
0.003	35.99	4608	0.0581	0.9893	0.9894	0.9894	0.9900
0.003	36.99	4736	0.0585	0.9893	0.9893	0.9893	0.9899
0.003	37.99	4864	0.0586	0.9893	0.9894	0.9893	0.9900
0.003	38.99	4992	0.0588	0.9893	0.9894	0.9894	0.9900
0.0024	39.99	5120	0.0589	0.9894	0.9894	0.9894	0.9900

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.0
• Datasets 2.18.0
• Tokenizers 0.13.2