|
--- |
|
base_model: vinai/bertweet-base |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: bertweet-2020-Q1-filtered |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# bertweet-2020-Q1-filtered |
|
|
|
This model is a fine-tuned version of [vinai/bertweet-base](https://huggingface.co/vinai/bertweet-base) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 3.3321 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 1e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 1400 |
|
- training_steps: 2400000 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-----:|:-------:|:---------------:| |
|
| No log | 0.07 | 8000 | 2.6350 | |
|
| 2.7848 | 0.13 | 16000 | 2.6556 | |
|
| 2.7848 | 0.2 | 24000 | 2.6695 | |
|
| 2.7545 | 0.26 | 32000 | 2.6713 | |
|
| 2.7545 | 0.33 | 40000 | 2.7089 | |
|
| 2.7717 | 0.39 | 48000 | 2.7144 | |
|
| 2.7717 | 0.46 | 56000 | 2.7240 | |
|
| 2.8043 | 0.52 | 64000 | 2.7499 | |
|
| 2.8043 | 0.59 | 72000 | 2.7704 | |
|
| 2.8401 | 0.65 | 80000 | 2.7820 | |
|
| 2.8401 | 0.72 | 88000 | 2.8068 | |
|
| 2.8723 | 0.78 | 96000 | 2.8150 | |
|
| 2.8723 | 0.85 | 104000 | 2.8410 | |
|
| 2.9004 | 0.91 | 112000 | 2.8657 | |
|
| 2.9004 | 0.98 | 120000 | 2.8826 | |
|
| 2.9396 | 1.04 | 128000 | 2.9071 | |
|
| 2.9396 | 1.11 | 136000 | 2.9490 | |
|
| 2.9801 | 1.17 | 144000 | 2.9515 | |
|
| 2.9801 | 1.24 | 152000 | 2.9863 | |
|
| 3.0173 | 1.3 | 160000 | 2.9916 | |
|
| 3.0173 | 1.37 | 168000 | 3.0231 | |
|
| 3.0674 | 1.44 | 176000 | 3.0447 | |
|
| 3.0674 | 1.5 | 184000 | 3.0638 | |
|
| 3.1059 | 1.57 | 192000 | 3.0945 | |
|
| 3.1059 | 1.63 | 200000 | 3.1008 | |
|
| 3.1283 | 1.7 | 208000 | 3.1257 | |
|
| 3.1283 | 1.76 | 216000 | 3.1262 | |
|
| 3.1684 | 1.83 | 224000 | 3.1523 | |
|
| 3.1684 | 1.89 | 232000 | 3.1842 | |
|
| 3.1966 | 1.96 | 240000 | 3.1820 | |
|
| 3.1966 | 2.02 | 248000 | 3.1976 | |
|
| 3.2055 | 2.09 | 256000 | 3.2013 | |
|
| 3.2055 | 2.15 | 264000 | 3.2197 | |
|
| 3.2186 | 2.22 | 272000 | 3.2259 | |
|
| 3.2186 | 2.28 | 280000 | 3.2410 | |
|
| 3.2518 | 2.35 | 288000 | 3.2449 | |
|
| 3.2518 | 2.41 | 296000 | 3.2686 | |
|
| 3.2705 | 2.48 | 304000 | 3.2702 | |
|
| 3.2705 | 2.54 | 312000 | 3.2716 | |
|
| 3.2677 | 2.61 | 320000 | 3.2935 | |
|
| 3.2677 | 2.67 | 328000 | 3.2942 | |
|
| 3.2955 | 2.74 | 336000 | 3.3044 | |
|
| 3.2955 | 2.8 | 344000 | 3.3110 | |
|
| 3.2966 | 2.87 | 352000 | 3.3053 | |
|
| 3.2966 | 2.94 | 360000 | 3.3276 | |
|
| 3.311 | 3.0 | 368000 | 3.3256 | |
|
| 3.311 | 3.07 | 376000 | 3.3292 | |
|
| 3.3217 | 3.13 | 384000 | 3.3335 | |
|
| 3.3217 | 3.2 | 392000 | 3.3160 | |
|
| 3.3145 | 3.26 | 400000 | 3.3378 | |
|
| 3.3145 | 3.33 | 408000 | 3.3307 | |
|
| 3.3246 | 3.39 | 416000 | 3.3427 | |
|
| 3.3246 | 3.46 | 424000 | 3.3543 | |
|
| 3.3131 | 3.52 | 432000 | 3.3405 | |
|
| 3.3131 | 3.59 | 440000 | 3.3361 | |
|
| 3.3266 | 3.65 | 448000 | 3.3704 | |
|
| 3.3266 | 3.72 | 456000 | 3.3549 | |
|
| 3.3358 | 3.78 | 464000 | 3.3603 | |
|
| 3.3358 | 3.85 | 472000 | 3.3642 | |
|
| 3.3385 | 3.91 | 480000 | 3.3573 | |
|
| 3.3385 | 3.98 | 488000 | 3.3658 | |
|
| 3.3375 | 4.04 | 496000 | 3.3459 | |
|
| 3.3375 | 4.11 | 504000 | 3.3703 | |
|
| 3.3237 | 4.17 | 512000 | 3.3564 | |
|
| 3.3237 | 4.24 | 520000 | 3.3553 | |
|
| 3.34 | 4.31 | 528000 | 3.3576 | |
|
| 3.34 | 4.37 | 536000 | 3.3548 | |
|
| 3.3247 | 4.44 | 544000 | 3.3526 | |
|
| 3.3247 | 4.5 | 552000 | 3.3674 | |
|
| 3.318 | 4.57 | 560000 | 3.3608 | |
|
| 3.318 | 4.63 | 568000 | 3.3527 | |
|
| 3.3318 | 4.7 | 576000 | 3.3600 | |
|
| 3.3318 | 4.76 | 584000 | 3.3662 | |
|
| 3.3211 | 4.83 | 592000 | 3.3603 | |
|
| 3.3211 | 4.89 | 600000 | 3.3640 | |
|
| 3.3344 | 4.96 | 608000 | 3.3760 | |
|
| 3.3344 | 5.02 | 616000 | 3.3876 | |
|
| 3.331 | 5.09 | 624000 | 3.3519 | |
|
| 3.331 | 5.15 | 632000 | 3.3734 | |
|
| 3.3293 | 5.22 | 640000 | 3.3735 | |
|
| 3.3293 | 5.28 | 648000 | 3.3703 | |
|
| 3.3317 | 5.35 | 656000 | 3.3826 | |
|
| 3.3317 | 5.41 | 664000 | 3.3826 | |
|
| 3.3291 | 5.48 | 672000 | 3.3919 | |
|
| 3.3291 | 5.54 | 680000 | 3.3786 | |
|
| 3.3423 | 5.61 | 688000 | 3.3775 | |
|
| 3.3423 | 5.68 | 696000 | 3.3734 | |
|
| 3.3364 | 5.74 | 704000 | 3.3725 | |
|
| 3.3364 | 5.81 | 712000 | 3.3855 | |
|
| 3.347 | 5.87 | 720000 | 3.3774 | |
|
| 3.347 | 5.94 | 728000 | 3.3717 | |
|
| 3.3311 | 6.0 | 736000 | 3.3929 | |
|
| 3.3311 | 6.07 | 744000 | 3.3899 | |
|
| 3.3445 | 6.13 | 752000 | 3.3985 | |
|
| 3.3445 | 6.2 | 760000 | 3.3866 | |
|
| 3.345 | 6.26 | 768000 | 3.3943 | |
|
| 3.345 | 6.33 | 776000 | 3.3734 | |
|
| 3.3427 | 6.39 | 784000 | 3.3832 | |
|
| 3.3427 | 6.46 | 792000 | 3.3966 | |
|
| 3.3406 | 6.52 | 800000 | 3.3892 | |
|
| 3.3406 | 6.59 | 808000 | 3.3904 | |
|
| 3.3406 | 6.65 | 816000 | 3.3867 | |
|
| 3.3406 | 6.72 | 824000 | 3.3902 | |
|
| 3.3354 | 6.78 | 832000 | 3.3718 | |
|
| 3.3354 | 6.85 | 840000 | 3.3831 | |
|
| 3.3521 | 6.91 | 848000 | 3.3909 | |
|
| 3.3521 | 6.98 | 856000 | 3.3799 | |
|
| 3.3538 | 7.05 | 864000 | 3.3828 | |
|
| 3.3538 | 7.11 | 872000 | 3.3785 | |
|
| 3.3363 | 7.18 | 880000 | 3.3993 | |
|
| 3.3363 | 7.24 | 888000 | 3.3850 | |
|
| 3.3341 | 7.31 | 896000 | 3.3932 | |
|
| 3.3341 | 7.37 | 904000 | 3.3981 | |
|
| 3.3458 | 7.44 | 912000 | 3.3936 | |
|
| 3.3458 | 7.5 | 920000 | 3.4032 | |
|
| 3.3327 | 7.57 | 928000 | 3.3852 | |
|
| 3.3327 | 7.63 | 936000 | 3.3865 | |
|
| 3.3507 | 7.7 | 944000 | 3.3900 | |
|
| 3.3507 | 7.76 | 952000 | 3.3772 | |
|
| 3.3493 | 7.83 | 960000 | 3.3887 | |
|
| 3.3493 | 7.89 | 968000 | 3.3951 | |
|
| 3.3412 | 7.96 | 976000 | 3.3833 | |
|
| 3.3412 | 8.02 | 984000 | 3.3816 | |
|
| 3.3232 | 8.09 | 992000 | 3.3752 | |
|
| 3.3232 | 8.15 | 1000000 | 3.3845 | |
|
| 3.333 | 8.22 | 1008000 | 3.3907 | |
|
| 3.333 | 8.28 | 1016000 | 3.3823 | |
|
| 3.3449 | 8.35 | 1024000 | 3.3725 | |
|
| 3.3449 | 8.41 | 1032000 | 3.3797 | |
|
| 3.3336 | 8.48 | 1040000 | 3.3878 | |
|
| 3.3336 | 8.55 | 1048000 | 3.3845 | |
|
| 3.3307 | 8.61 | 1056000 | 3.3907 | |
|
| 3.3307 | 8.68 | 1064000 | 3.3858 | |
|
| 3.3267 | 8.74 | 1072000 | 3.3952 | |
|
| 3.3267 | 8.81 | 1080000 | 3.3914 | |
|
| 3.335 | 8.87 | 1088000 | 3.3904 | |
|
| 3.335 | 8.94 | 1096000 | 3.3895 | |
|
| 3.3411 | 9.0 | 1104000 | 3.3959 | |
|
| 3.3411 | 9.07 | 1112000 | 3.3915 | |
|
| 3.3324 | 9.13 | 1120000 | 3.4030 | |
|
| 3.3324 | 9.2 | 1128000 | 3.4084 | |
|
| 3.3297 | 9.26 | 1136000 | 3.4023 | |
|
| 3.3297 | 9.33 | 1144000 | 3.3967 | |
|
| 3.3492 | 9.39 | 1152000 | 3.3931 | |
|
| 3.3492 | 9.46 | 1160000 | 3.4065 | |
|
| 3.3317 | 9.52 | 1168000 | 3.3905 | |
|
| 3.3317 | 9.59 | 1176000 | 3.4021 | |
|
| 3.3447 | 9.65 | 1184000 | 3.4001 | |
|
| 3.3447 | 9.72 | 1192000 | 3.3943 | |
|
| 3.3377 | 9.78 | 1200000 | 3.3971 | |
|
| 3.3377 | 9.85 | 1208000 | 3.3946 | |
|
| 3.3486 | 9.92 | 1216000 | 3.3924 | |
|
| 3.3486 | 9.98 | 1224000 | 3.3983 | |
|
| 3.3471 | 10.05 | 1232000 | 3.4141 | |
|
| 3.3471 | 10.11 | 1240000 | 3.4220 | |
|
| 3.3457 | 10.18 | 1248000 | 3.4085 | |
|
| 3.3457 | 10.24 | 1256000 | 3.4243 | |
|
| 3.3278 | 10.31 | 1264000 | 3.4058 | |
|
| 3.3278 | 10.37 | 1272000 | 3.4033 | |
|
| 3.325 | 10.44 | 1280000 | 3.3867 | |
|
| 3.325 | 10.5 | 1288000 | 3.3879 | |
|
| 3.3248 | 10.57 | 1296000 | 3.3801 | |
|
| 3.3248 | 10.63 | 1304000 | 3.4027 | |
|
| 3.3217 | 10.7 | 1312000 | 3.3781 | |
|
| 3.3217 | 10.76 | 1320000 | 3.3871 | |
|
| 3.3227 | 10.83 | 1328000 | 3.3861 | |
|
| 3.3227 | 10.89 | 1336000 | 3.3789 | |
|
| 3.3259 | 10.96 | 1344000 | 3.3865 | |
|
| 3.3259 | 11.02 | 1352000 | 3.3863 | |
|
| 3.3094 | 11.09 | 1360000 | 3.3827 | |
|
| 3.3094 | 11.15 | 1368000 | 3.3880 | |
|
| 3.3128 | 11.22 | 1376000 | 3.3652 | |
|
| 3.3128 | 11.29 | 1384000 | 3.3813 | |
|
| 3.3088 | 11.35 | 1392000 | 3.3853 | |
|
| 3.3088 | 11.42 | 1400000 | 3.3709 | |
|
| 3.3067 | 11.48 | 1408000 | 3.3831 | |
|
| 3.3067 | 11.55 | 1416000 | 3.3703 | |
|
| 3.311 | 11.61 | 1424000 | 3.3696 | |
|
| 3.311 | 11.68 | 1432000 | 3.3769 | |
|
| 3.3048 | 11.74 | 1440000 | 3.3740 | |
|
| 3.3048 | 11.81 | 1448000 | 3.3731 | |
|
| 3.3055 | 11.87 | 1456000 | 3.3655 | |
|
| 3.3055 | 11.94 | 1464000 | 3.3697 | |
|
| 3.3105 | 12.0 | 1472000 | 3.3742 | |
|
| 3.3105 | 12.07 | 1480000 | 3.3614 | |
|
| 3.2977 | 12.13 | 1488000 | 3.3705 | |
|
| 3.2977 | 12.2 | 1496000 | 3.3746 | |
|
| 3.2999 | 12.26 | 1504000 | 3.3691 | |
|
| 3.2999 | 12.33 | 1512000 | 3.3745 | |
|
| 3.2983 | 12.39 | 1520000 | 3.3717 | |
|
| 3.2983 | 12.46 | 1528000 | 3.3682 | |
|
| 3.2957 | 12.52 | 1536000 | 3.3693 | |
|
| 3.2957 | 12.59 | 1544000 | 3.3764 | |
|
| 3.293 | 12.65 | 1552000 | 3.3691 | |
|
| 3.293 | 12.72 | 1560000 | 3.3802 | |
|
| 3.2919 | 12.79 | 1568000 | 3.3626 | |
|
| 3.2919 | 12.85 | 1576000 | 3.3604 | |
|
| 3.3023 | 12.92 | 1584000 | 3.3749 | |
|
| 3.3023 | 12.98 | 1592000 | 3.3688 | |
|
| 3.2988 | 13.05 | 1600000 | 3.3666 | |
|
| 3.2988 | 13.11 | 1608000 | 3.3695 | |
|
| 3.2924 | 13.18 | 1616000 | 3.3650 | |
|
| 3.2924 | 13.24 | 1624000 | 3.3651 | |
|
| 3.2958 | 13.31 | 1632000 | 3.3692 | |
|
| 3.2958 | 13.37 | 1640000 | 3.3855 | |
|
| 3.2918 | 13.44 | 1648000 | 3.3706 | |
|
| 3.2918 | 13.5 | 1656000 | 3.3680 | |
|
| 3.2948 | 13.57 | 1664000 | 3.3534 | |
|
| 3.2948 | 13.63 | 1672000 | 3.3699 | |
|
| 3.2996 | 13.7 | 1680000 | 3.3733 | |
|
| 3.2996 | 13.76 | 1688000 | 3.3764 | |
|
| 3.2999 | 13.83 | 1696000 | 3.3793 | |
|
| 3.2999 | 13.89 | 1704000 | 3.3683 | |
|
| 3.291 | 13.96 | 1712000 | 3.3654 | |
|
| 3.291 | 14.02 | 1720000 | 3.3721 | |
|
| 3.2952 | 14.09 | 1728000 | 3.3674 | |
|
| 3.2952 | 14.16 | 1736000 | 3.3762 | |
|
| 3.2866 | 14.22 | 1744000 | 3.3699 | |
|
| 3.2866 | 14.29 | 1752000 | 3.3690 | |
|
| 3.2825 | 14.35 | 1760000 | 3.3653 | |
|
| 3.2825 | 14.42 | 1768000 | 3.3687 | |
|
| 3.2825 | 14.48 | 1776000 | 3.3618 | |
|
| 3.2825 | 14.55 | 1784000 | 3.3609 | |
|
| 3.2744 | 14.61 | 1792000 | 3.3552 | |
|
| 3.2744 | 14.68 | 1800000 | 3.3549 | |
|
| 3.2811 | 14.74 | 1808000 | 3.3504 | |
|
| 3.2811 | 14.81 | 1816000 | 3.3575 | |
|
| 3.2672 | 14.87 | 1824000 | 3.3588 | |
|
| 3.2672 | 14.94 | 1832000 | 3.3560 | |
|
| 3.2919 | 15.0 | 1840000 | 3.3598 | |
|
| 3.2919 | 15.07 | 1848000 | 3.3445 | |
|
| 3.2724 | 15.13 | 1856000 | 3.3517 | |
|
| 3.2724 | 15.2 | 1864000 | 3.3593 | |
|
| 3.277 | 15.26 | 1872000 | 3.3598 | |
|
| 3.277 | 15.33 | 1880000 | 3.3458 | |
|
| 3.2842 | 15.39 | 1888000 | 3.3583 | |
|
| 3.2842 | 15.46 | 1896000 | 3.3448 | |
|
| 3.2758 | 15.53 | 1904000 | 3.3593 | |
|
| 3.2758 | 15.59 | 1912000 | 3.3552 | |
|
| 3.2684 | 15.66 | 1920000 | 3.3715 | |
|
| 3.2684 | 15.72 | 1928000 | 3.3544 | |
|
| 3.2924 | 15.79 | 1936000 | 3.3515 | |
|
| 3.2924 | 15.85 | 1944000 | 3.3646 | |
|
| 3.2673 | 15.92 | 1952000 | 3.3538 | |
|
| 3.2673 | 15.98 | 1960000 | 3.3437 | |
|
| 3.2833 | 16.05 | 1968000 | 3.3443 | |
|
| 3.2833 | 16.11 | 1976000 | 3.3619 | |
|
| 3.2636 | 16.18 | 1984000 | 3.3511 | |
|
| 3.2636 | 16.24 | 1992000 | 3.3448 | |
|
| 3.2753 | 16.31 | 2000000 | 3.3560 | |
|
| 3.2753 | 16.37 | 2008000 | 3.3525 | |
|
| 3.2701 | 16.44 | 2016000 | 3.3558 | |
|
| 3.2701 | 16.5 | 2024000 | 3.3559 | |
|
| 3.2761 | 16.57 | 2032000 | 3.3440 | |
|
| 3.2761 | 16.63 | 2040000 | 3.3506 | |
|
| 3.2677 | 16.7 | 2048000 | 3.3474 | |
|
| 3.2677 | 16.76 | 2056000 | 3.3615 | |
|
| 3.2614 | 16.83 | 2064000 | 3.3507 | |
|
| 3.2614 | 16.89 | 2072000 | 3.3444 | |
|
| 3.2608 | 16.96 | 2080000 | 3.3527 | |
|
| 3.2608 | 17.03 | 2088000 | 3.3398 | |
|
| 3.2643 | 17.09 | 2096000 | 3.3498 | |
|
| 3.2643 | 17.16 | 2104000 | 3.3349 | |
|
| 3.2721 | 17.22 | 2112000 | 3.3560 | |
|
| 3.2721 | 17.29 | 2120000 | 3.3421 | |
|
| 3.266 | 17.35 | 2128000 | 3.3429 | |
|
| 3.266 | 17.42 | 2136000 | 3.3371 | |
|
| 3.2551 | 17.48 | 2144000 | 3.3404 | |
|
| 3.2551 | 17.55 | 2152000 | 3.3494 | |
|
| 3.26 | 17.61 | 2160000 | 3.3389 | |
|
| 3.26 | 17.68 | 2168000 | 3.3456 | |
|
| 3.2528 | 17.74 | 2176000 | 3.3249 | |
|
| 3.2528 | 17.81 | 2184000 | 3.3452 | |
|
| 3.2602 | 17.87 | 2192000 | 3.3376 | |
|
| 3.2602 | 17.94 | 2200000 | 3.3511 | |
|
| 3.2492 | 18.0 | 2208000 | 3.3475 | |
|
| 3.2492 | 18.07 | 2216000 | 3.3497 | |
|
| 3.2469 | 18.13 | 2224000 | 3.3378 | |
|
| 3.2469 | 18.2 | 2232000 | 3.3326 | |
|
| 3.2589 | 18.26 | 2240000 | 3.3277 | |
|
| 3.2589 | 18.33 | 2248000 | 3.3457 | |
|
| 3.2548 | 18.4 | 2256000 | 3.3343 | |
|
| 3.2548 | 18.46 | 2264000 | 3.3362 | |
|
| 3.2589 | 18.53 | 2272000 | 3.3431 | |
|
| 3.2589 | 18.59 | 2280000 | 3.3428 | |
|
| 3.2674 | 18.66 | 2288000 | 3.3401 | |
|
| 3.2674 | 18.72 | 2296000 | 3.3375 | |
|
| 3.2561 | 18.79 | 2304000 | 3.3334 | |
|
| 3.2561 | 18.85 | 2312000 | 3.3321 | |
|
| 3.2452 | 18.92 | 2320000 | 3.3446 | |
|
| 3.2452 | 18.98 | 2328000 | 3.3525 | |
|
| 3.259 | 19.05 | 2336000 | 3.3318 | |
|
| 3.259 | 19.11 | 2344000 | 3.3452 | |
|
| 3.2494 | 19.18 | 2352000 | 3.3355 | |
|
| 3.2494 | 19.24 | 2360000 | 3.3322 | |
|
| 3.2558 | 19.31 | 2368000 | 3.3255 | |
|
| 3.2558 | 19.37 | 2376000 | 3.3330 | |
|
| 3.2436 | 19.44 | 2384000 | 3.3358 | |
|
| 3.2436 | 19.5 | 2392000 | 3.3287 | |
|
| 3.2545 | 19.57 | 2400000 | 3.3321 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.35.0.dev0 |
|
- Pytorch 2.0.1+cu117 |
|
- Datasets 2.14.5 |
|
- Tokenizers 0.14.0 |
|
|
