|
--- |
|
license: apache-2.0 |
|
base_model: openlm-research/open_llama_3b_v2 |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- financial_phrasebank |
|
metrics: |
|
- accuracy |
|
model-index: |
|
- name: openllama-3b-finance |
|
results: |
|
- task: |
|
name: Text Classification |
|
type: text-classification |
|
dataset: |
|
name: financial_phrasebank |
|
type: financial_phrasebank |
|
config: sentences_50agree |
|
split: train |
|
args: sentences_50agree |
|
metrics: |
|
- name: Accuracy |
|
type: accuracy |
|
value: 0.4142561983471074 |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# openllama-3b-finance |
|
|
|
This model is a fine-tuned version of [openlm-research/open_llama_3b_v2](https://huggingface.co/openlm-research/open_llama_3b_v2) on the financial_phrasebank dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 4.0296 |
|
- Accuracy: 0.4143 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0002 |
|
- train_batch_size: 1 |
|
- eval_batch_size: 1 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: cosine |
|
- num_epochs: 1 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:--------:| |
|
| 21.9655 | 0.01 | 20 | 8.1663 | 0.0816 | |
|
| 2.231 | 0.01 | 40 | 6.3007 | 0.4143 | |
|
| 2.7452 | 0.02 | 60 | 4.0892 | 0.4143 | |
|
| 2.4561 | 0.02 | 80 | 5.0314 | 0.4143 | |
|
| 2.337 | 0.03 | 100 | 5.6176 | 0.4143 | |
|
| 3.2226 | 0.03 | 120 | 4.4963 | 0.4143 | |
|
| 2.5633 | 0.04 | 140 | 6.1800 | 0.4143 | |
|
| 2.4764 | 0.04 | 160 | 4.7059 | 0.4143 | |
|
| 2.45 | 0.05 | 180 | 5.0602 | 0.4143 | |
|
| 1.4232 | 0.05 | 200 | 5.3418 | 0.4143 | |
|
| 2.7684 | 0.06 | 220 | 5.1805 | 0.4143 | |
|
| 1.7065 | 0.06 | 240 | 4.7568 | 0.4143 | |
|
| 2.3417 | 0.07 | 260 | 6.1062 | 0.4143 | |
|
| 1.907 | 0.07 | 280 | 12.0988 | 0.5041 | |
|
| 14.6043 | 0.08 | 300 | 3.0283 | 0.0816 | |
|
| 1.337 | 0.08 | 320 | 12.7786 | 0.4143 | |
|
| 4.182 | 0.09 | 340 | 7.5619 | 0.4143 | |
|
| 3.7365 | 0.09 | 360 | 7.8581 | 0.4143 | |
|
| 3.209 | 0.1 | 380 | 3.2547 | 0.4143 | |
|
| 3.4836 | 0.1 | 400 | 89.8525 | 0.0816 | |
|
| 4.5805 | 0.11 | 420 | 103.0762 | 0.4143 | |
|
| 4.6351 | 0.11 | 440 | 91.4501 | 0.4143 | |
|
| 11.0873 | 0.12 | 460 | 88.0469 | 0.4143 | |
|
| 1.1274 | 0.12 | 480 | 86.7130 | 0.4143 | |
|
| 2.0398 | 0.13 | 500 | 86.4186 | 0.4143 | |
|
| 18.6924 | 0.13 | 520 | 80.1491 | 0.4143 | |
|
| 1.2216 | 0.14 | 540 | 76.8429 | 0.4143 | |
|
| 1.1179 | 0.14 | 560 | 78.0159 | 0.4143 | |
|
| 10.0981 | 0.15 | 580 | 71.1114 | 0.4143 | |
|
| 9.0123 | 0.15 | 600 | 66.2945 | 0.4143 | |
|
| 1.9539 | 0.16 | 620 | 65.6854 | 0.4143 | |
|
| 8.4729 | 0.17 | 640 | 62.1595 | 0.4143 | |
|
| 7.816 | 0.17 | 660 | 52.0763 | 0.4143 | |
|
| 6.0443 | 0.18 | 680 | 41.1500 | 0.4143 | |
|
| 3.1804 | 0.18 | 700 | 42.8007 | 0.4143 | |
|
| 1.6122 | 0.19 | 720 | 44.0976 | 0.4143 | |
|
| 9.8927 | 0.19 | 740 | 31.6381 | 0.4143 | |
|
| 6.828 | 0.2 | 760 | 12.7483 | 0.4143 | |
|
| 3.1457 | 0.2 | 780 | 13.2981 | 0.4143 | |
|
| 1.9991 | 0.21 | 800 | 12.4846 | 0.4143 | |
|
| 2.5539 | 0.21 | 820 | 13.7669 | 0.4143 | |
|
| 1.3898 | 0.22 | 840 | 12.8919 | 0.0816 | |
|
| 2.9251 | 0.22 | 860 | 15.9149 | 0.0816 | |
|
| 4.0874 | 0.23 | 880 | 10.5282 | 0.4143 | |
|
| 2.4763 | 0.23 | 900 | 3.0281 | 0.4143 | |
|
| 2.2865 | 0.24 | 920 | 12.2460 | 0.4143 | |
|
| 4.2438 | 0.24 | 940 | 10.1961 | 0.4143 | |
|
| 2.547 | 0.25 | 960 | 1.4099 | 0.4143 | |
|
| 0.8659 | 0.25 | 980 | 8.3217 | 0.4143 | |
|
| 3.5331 | 0.26 | 1000 | 6.3990 | 0.4143 | |
|
| 2.4704 | 0.26 | 1020 | 2.2337 | 0.0816 | |
|
| 2.1381 | 0.27 | 1040 | 10.6263 | 0.4143 | |
|
| 1.5927 | 0.27 | 1060 | 11.1989 | 0.4143 | |
|
| 2.485 | 0.28 | 1080 | 8.8174 | 0.4143 | |
|
| 2.8074 | 0.28 | 1100 | 5.5971 | 0.4143 | |
|
| 0.8622 | 0.29 | 1120 | 5.5089 | 0.4143 | |
|
| 2.8085 | 0.29 | 1140 | 5.4300 | 0.4143 | |
|
| 1.2405 | 0.3 | 1160 | 7.5657 | 0.4143 | |
|
| 3.9374 | 0.3 | 1180 | 2.7180 | 0.4143 | |
|
| 1.7494 | 0.31 | 1200 | 4.9639 | 0.0816 | |
|
| 2.6094 | 0.32 | 1220 | 2.1980 | 0.4143 | |
|
| 2.2072 | 0.32 | 1240 | 7.3392 | 0.4143 | |
|
| 0.9978 | 0.33 | 1260 | 7.9127 | 0.4143 | |
|
| 2.3872 | 0.33 | 1280 | 7.0613 | 0.4143 | |
|
| 3.3129 | 0.34 | 1300 | 4.4202 | 0.4143 | |
|
| 1.776 | 0.34 | 1320 | 6.1467 | 0.4143 | |
|
| 3.1179 | 0.35 | 1340 | 6.0607 | 0.4143 | |
|
| 1.272 | 0.35 | 1360 | 5.0484 | 0.4143 | |
|
| 3.0694 | 0.36 | 1380 | 3.1665 | 0.4143 | |
|
| 1.9452 | 0.36 | 1400 | 4.8692 | 0.4143 | |
|
| 2.3689 | 0.37 | 1420 | 4.9375 | 0.4143 | |
|
| 2.7082 | 0.37 | 1440 | 3.2108 | 0.4143 | |
|
| 0.8244 | 0.38 | 1460 | 7.0151 | 0.4143 | |
|
| 2.6032 | 0.38 | 1480 | 5.5645 | 0.4143 | |
|
| 2.8745 | 0.39 | 1500 | 4.2408 | 0.4143 | |
|
| 2.625 | 0.39 | 1520 | 6.8800 | 0.4143 | |
|
| 2.5335 | 0.4 | 1540 | 6.3109 | 0.4143 | |
|
| 2.5495 | 0.4 | 1560 | 4.4017 | 0.4143 | |
|
| 1.7234 | 0.41 | 1580 | 5.1739 | 0.4143 | |
|
| 2.1066 | 0.41 | 1600 | 6.0769 | 0.4143 | |
|
| 2.5541 | 0.42 | 1620 | 3.7539 | 0.4143 | |
|
| 2.4598 | 0.42 | 1640 | 4.2075 | 0.4143 | |
|
| 1.7211 | 0.43 | 1660 | 5.3975 | 0.4143 | |
|
| 2.3993 | 0.43 | 1680 | 4.1427 | 0.4143 | |
|
| 1.6161 | 0.44 | 1700 | 5.0871 | 0.4143 | |
|
| 2.2361 | 0.44 | 1720 | 4.3375 | 0.4143 | |
|
| 2.0841 | 0.45 | 1740 | 4.7357 | 0.4143 | |
|
| 2.137 | 0.45 | 1760 | 5.2737 | 0.4143 | |
|
| 2.3819 | 0.46 | 1780 | 3.1688 | 0.4143 | |
|
| 2.6391 | 0.46 | 1800 | 5.6169 | 0.4143 | |
|
| 1.276 | 0.47 | 1820 | 6.1945 | 0.4143 | |
|
| 2.0694 | 0.48 | 1840 | 6.3761 | 0.4143 | |
|
| 2.3715 | 0.48 | 1860 | 6.1666 | 0.4143 | |
|
| 2.1428 | 0.49 | 1880 | 6.4718 | 0.4143 | |
|
| 2.0409 | 0.49 | 1900 | 6.3259 | 0.4143 | |
|
| 2.1924 | 0.5 | 1920 | 6.0853 | 0.4143 | |
|
| 2.3511 | 0.5 | 1940 | 4.7199 | 0.4143 | |
|
| 2.7335 | 0.51 | 1960 | 4.3591 | 0.4143 | |
|
| 1.6784 | 0.51 | 1980 | 3.7488 | 0.1612 | |
|
| 1.5525 | 0.52 | 2000 | 6.0497 | 0.4143 | |
|
| 2.7457 | 0.52 | 2020 | 3.5952 | 0.4143 | |
|
| 2.3929 | 0.53 | 2040 | 4.7684 | 0.4143 | |
|
| 1.9522 | 0.53 | 2060 | 5.6394 | 0.4143 | |
|
| 2.2257 | 0.54 | 2080 | 4.5801 | 0.4143 | |
|
| 1.6753 | 0.54 | 2100 | 5.0521 | 0.4143 | |
|
| 1.6154 | 0.55 | 2120 | 5.4730 | 0.4143 | |
|
| 1.7723 | 0.55 | 2140 | 5.5251 | 0.4143 | |
|
| 2.6963 | 0.56 | 2160 | 3.5098 | 0.4143 | |
|
| 1.7274 | 0.56 | 2180 | 5.4262 | 0.4143 | |
|
| 2.4059 | 0.57 | 2200 | 4.5019 | 0.4143 | |
|
| 1.6505 | 0.57 | 2220 | 5.1107 | 0.4143 | |
|
| 1.2469 | 0.58 | 2240 | 5.3456 | 0.4143 | |
|
| 1.6702 | 0.58 | 2260 | 5.4103 | 0.4143 | |
|
| 1.615 | 0.59 | 2280 | 5.8024 | 0.4143 | |
|
| 1.5622 | 0.59 | 2300 | 5.6035 | 0.4143 | |
|
| 2.3536 | 0.6 | 2320 | 5.3779 | 0.4143 | |
|
| 2.0512 | 0.6 | 2340 | 5.2498 | 0.4143 | |
|
| 2.1405 | 0.61 | 2360 | 5.2279 | 0.4143 | |
|
| 2.1926 | 0.61 | 2380 | 4.3260 | 0.4143 | |
|
| 2.3995 | 0.62 | 2400 | 4.4445 | 0.4143 | |
|
| 1.4944 | 0.62 | 2420 | 4.9616 | 0.4143 | |
|
| 2.6623 | 0.63 | 2440 | 4.9736 | 0.4143 | |
|
| 1.4095 | 0.64 | 2460 | 4.6506 | 0.4143 | |
|
| 2.4803 | 0.64 | 2480 | 4.0971 | 0.4143 | |
|
| 1.2721 | 0.65 | 2500 | 4.3192 | 0.4143 | |
|
| 1.8372 | 0.65 | 2520 | 4.4907 | 0.4143 | |
|
| 1.8942 | 0.66 | 2540 | 4.7323 | 0.4143 | |
|
| 2.1407 | 0.66 | 2560 | 4.9554 | 0.4143 | |
|
| 2.5039 | 0.67 | 2580 | 5.1599 | 0.4143 | |
|
| 1.7321 | 0.67 | 2600 | 5.6089 | 0.4143 | |
|
| 2.0621 | 0.68 | 2620 | 4.8359 | 0.4143 | |
|
| 2.1664 | 0.68 | 2640 | 4.5581 | 0.4143 | |
|
| 1.8835 | 0.69 | 2660 | 5.1029 | 0.4143 | |
|
| 3.0314 | 0.69 | 2680 | 3.9587 | 0.4143 | |
|
| 1.1781 | 0.7 | 2700 | 4.4584 | 0.4143 | |
|
| 3.3222 | 0.7 | 2720 | 4.7628 | 0.4143 | |
|
| 2.1184 | 0.71 | 2740 | 4.4039 | 0.4143 | |
|
| 1.9293 | 0.71 | 2760 | 3.8755 | 0.4143 | |
|
| 2.2448 | 0.72 | 2780 | 4.4327 | 0.4143 | |
|
| 2.4697 | 0.72 | 2800 | 3.3026 | 0.4143 | |
|
| 1.8569 | 0.73 | 2820 | 3.7722 | 0.4143 | |
|
| 0.8544 | 0.73 | 2840 | 4.9176 | 0.4143 | |
|
| 2.2445 | 0.74 | 2860 | 4.3889 | 0.4143 | |
|
| 1.3723 | 0.74 | 2880 | 4.3280 | 0.4143 | |
|
| 2.2167 | 0.75 | 2900 | 4.4016 | 0.4143 | |
|
| 1.98 | 0.75 | 2920 | 3.8661 | 0.4143 | |
|
| 1.7344 | 0.76 | 2940 | 3.7919 | 0.4143 | |
|
| 1.924 | 0.76 | 2960 | 4.1408 | 0.4143 | |
|
| 1.3811 | 0.77 | 2980 | 4.3730 | 0.4143 | |
|
| 1.8289 | 0.77 | 3000 | 4.2872 | 0.4143 | |
|
| 1.9573 | 0.78 | 3020 | 4.6165 | 0.4143 | |
|
| 2.4877 | 0.78 | 3040 | 4.5988 | 0.4143 | |
|
| 1.1749 | 0.79 | 3060 | 4.7887 | 0.4143 | |
|
| 2.1835 | 0.8 | 3080 | 4.9018 | 0.4143 | |
|
| 2.3752 | 0.8 | 3100 | 4.6911 | 0.4143 | |
|
| 1.9741 | 0.81 | 3120 | 4.5126 | 0.4143 | |
|
| 1.7513 | 0.81 | 3140 | 4.6251 | 0.4143 | |
|
| 3.0666 | 0.82 | 3160 | 4.0260 | 0.4143 | |
|
| 0.5569 | 0.82 | 3180 | 4.0965 | 0.4143 | |
|
| 2.1805 | 0.83 | 3200 | 4.5240 | 0.4143 | |
|
| 2.4319 | 0.83 | 3220 | 4.3080 | 0.4143 | |
|
| 2.126 | 0.84 | 3240 | 3.7823 | 0.4143 | |
|
| 1.6993 | 0.84 | 3260 | 3.8093 | 0.4143 | |
|
| 0.6861 | 0.85 | 3280 | 4.1618 | 0.4143 | |
|
| 0.748 | 0.85 | 3300 | 4.5653 | 0.4143 | |
|
| 2.5721 | 0.86 | 3320 | 4.6628 | 0.4143 | |
|
| 2.0137 | 0.86 | 3340 | 4.2796 | 0.4143 | |
|
| 2.1864 | 0.87 | 3360 | 4.1173 | 0.4143 | |
|
| 2.4881 | 0.87 | 3380 | 3.9617 | 0.4143 | |
|
| 2.6837 | 0.88 | 3400 | 3.7575 | 0.4143 | |
|
| 1.5951 | 0.88 | 3420 | 3.6086 | 0.4143 | |
|
| 2.504 | 0.89 | 3440 | 3.5919 | 0.4143 | |
|
| 1.4982 | 0.89 | 3460 | 3.7519 | 0.4143 | |
|
| 1.8994 | 0.9 | 3480 | 3.7120 | 0.4143 | |
|
| 1.6126 | 0.9 | 3500 | 3.6854 | 0.4143 | |
|
| 2.002 | 0.91 | 3520 | 3.7888 | 0.4143 | |
|
| 1.0264 | 0.91 | 3540 | 3.7990 | 0.4143 | |
|
| 1.9495 | 0.92 | 3560 | 3.9635 | 0.4143 | |
|
| 2.0742 | 0.92 | 3580 | 3.9651 | 0.4143 | |
|
| 1.7803 | 0.93 | 3600 | 3.9518 | 0.4143 | |
|
| 2.0843 | 0.93 | 3620 | 3.9404 | 0.4143 | |
|
| 1.8431 | 0.94 | 3640 | 3.9334 | 0.4143 | |
|
| 1.4987 | 0.95 | 3660 | 3.9609 | 0.4143 | |
|
| 1.8214 | 0.95 | 3680 | 4.0060 | 0.4143 | |
|
| 1.0964 | 0.96 | 3700 | 4.0422 | 0.4143 | |
|
| 0.9669 | 0.96 | 3720 | 4.0549 | 0.4143 | |
|
| 1.6226 | 0.97 | 3740 | 4.0486 | 0.4143 | |
|
| 1.8061 | 0.97 | 3760 | 4.0405 | 0.4143 | |
|
| 2.8738 | 0.98 | 3780 | 4.0317 | 0.4143 | |
|
| 1.684 | 0.98 | 3800 | 4.0319 | 0.4143 | |
|
| 1.1158 | 0.99 | 3820 | 4.0303 | 0.4143 | |
|
| 1.775 | 0.99 | 3840 | 4.0294 | 0.4143 | |
|
| 2.1639 | 1.0 | 3860 | 4.0296 | 0.4143 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.32.0 |
|
- Pytorch 2.0.1+cu117 |
|
- Datasets 2.14.4 |
|
- Tokenizers 0.13.3 |
|
|
