Hugging Face's logo

jkazdan
/
gemma-2-2b_hs2_iter1_sftsd2

Safetensors
gemma2
trl
sft
Generated from Trainer
Model card Files Community
jkazdan commited on
Commit
6bfda59
·
verified ·
1 Parent(s): 1bdbeea

End of training

Browse files
Files changed (4) hide show
  1. README.md +64 -33
  2. model-00001-of-00002.safetensors +1 -1
  3. model-00002-of-00002.safetensors +1 -1
  4. training_args.bin +1 -1
README.md CHANGED
@@ -17,8 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [google/gemma-2-2b](https://huggingface.co/google/gemma-2-2b) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 1.5013
21
- - Num Input Tokens Seen: 8534232
22
 
23
  ## Model description
24
 
@@ -53,37 +53,68 @@ The following hyperparameters were used during training:
53
  | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
54
  |:-------------:|:------:|:----:|:---------------:|:-----------------:|
55
  | No log | 0 | 0 | 1.3956 | 0 |
56
- | 1.7736 | 0.0320 | 5 | 1.3653 | 275408 |
57
- | 1.6337 | 0.0639 | 10 | 1.2568 | 553472 |
58
- | 1.4116 | 0.0959 | 15 | 1.1899 | 826008 |
59
- | 1.2166 | 0.1279 | 20 | 1.1757 | 1103104 |
60
- | 1.0729 | 0.1599 | 25 | 1.1790 | 1373480 |
61
- | 0.8893 | 0.1918 | 30 | 1.2432 | 1652728 |
62
- | 0.7547 | 0.2238 | 35 | 1.3378 | 1922040 |
63
- | 0.7355 | 0.2558 | 40 | 1.4150 | 2199704 |
64
- | 0.5898 | 0.2878 | 45 | 1.4809 | 2466336 |
65
- | 0.5135 | 0.3197 | 50 | 1.5266 | 2736736 |
66
- | 0.3229 | 0.3517 | 55 | 1.5925 | 3013400 |
67
- | 0.3465 | 0.3837 | 60 | 1.5686 | 3287688 |
68
- | 0.2492 | 0.4157 | 65 | 1.5743 | 3563792 |
69
- | 0.2927 | 0.4476 | 70 | 1.5284 | 3835816 |
70
- | 0.1576 | 0.4796 | 75 | 1.5619 | 4101480 |
71
- | 0.1489 | 0.5116 | 80 | 1.5506 | 4374744 |
72
- | 0.1981 | 0.5436 | 85 | 1.4726 | 4648312 |
73
- | 0.1814 | 0.5755 | 90 | 1.4712 | 4923392 |
74
- | 0.1315 | 0.6075 | 95 | 1.4717 | 5194312 |
75
- | 0.1366 | 0.6395 | 100 | 1.4953 | 5462264 |
76
- | 0.1341 | 0.6715 | 105 | 1.4516 | 5735336 |
77
- | 0.0875 | 0.7034 | 110 | 1.4808 | 6010376 |
78
- | 0.2061 | 0.7354 | 115 | 1.4586 | 6282688 |
79
- | 0.1429 | 0.7674 | 120 | 1.4499 | 6551944 |
80
- | 0.1636 | 0.7994 | 125 | 1.4508 | 6830192 |
81
- | 0.0944 | 0.8313 | 130 | 1.4615 | 7104592 |
82
- | 0.1036 | 0.8633 | 135 | 1.4519 | 7376888 |
83
- | 0.1495 | 0.8953 | 140 | 1.4785 | 7649992 |
84
- | 0.1092 | 0.9273 | 145 | 1.4493 | 7925824 |
85
- | 0.0795 | 0.9592 | 150 | 1.4897 | 8204488 |
86
- | 0.1456 | 0.9912 | 155 | 1.5040 | 8481368 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
87
 
88
 
89
  ### Framework versions
 
17
 
18
  This model is a fine-tuned version of [google/gemma-2-2b](https://huggingface.co/google/gemma-2-2b) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 1.4430
21
+ - Num Input Tokens Seen: 17305888
22
 
23
  ## Model description
24
 
 
53
  | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
54
  |:-------------:|:------:|:----:|:---------------:|:-----------------:|
55
  | No log | 0 | 0 | 1.3956 | 0 |
56
+ | 1.7634 | 0.0160 | 5 | 1.3647 | 280680 |
57
+ | 1.7058 | 0.0320 | 10 | 1.2551 | 560560 |
58
+ | 1.4737 | 0.0480 | 15 | 1.1867 | 829000 |
59
+ | 1.346 | 0.0640 | 20 | 1.1565 | 1107160 |
60
+ | 1.2651 | 0.0800 | 25 | 1.1401 | 1382752 |
61
+ | 1.1926 | 0.0960 | 30 | 1.1571 | 1661728 |
62
+ | 1.0951 | 0.1120 | 35 | 1.1751 | 1933304 |
63
+ | 1.0001 | 0.1279 | 40 | 1.2205 | 2214264 |
64
+ | 0.9762 | 0.1439 | 45 | 1.2602 | 2489024 |
65
+ | 0.8297 | 0.1599 | 50 | 1.3327 | 2764992 |
66
+ | 0.7969 | 0.1759 | 55 | 1.3682 | 3039920 |
67
+ | 0.8151 | 0.1919 | 60 | 1.3863 | 3315424 |
68
+ | 0.6221 | 0.2079 | 65 | 1.4445 | 3587920 |
69
+ | 0.5957 | 0.2239 | 70 | 1.4630 | 3876656 |
70
+ | 0.4842 | 0.2399 | 75 | 1.4861 | 4153720 |
71
+ | 0.4818 | 0.2559 | 80 | 1.4824 | 4429368 |
72
+ | 0.45 | 0.2719 | 85 | 1.5948 | 4708392 |
73
+ | 0.4573 | 0.2879 | 90 | 1.4911 | 4989712 |
74
+ | 0.4216 | 0.3039 | 95 | 1.5597 | 5272344 |
75
+ | 0.3548 | 0.3199 | 100 | 1.5243 | 5546808 |
76
+ | 0.3257 | 0.3359 | 105 | 1.5387 | 5823112 |
77
+ | 0.3723 | 0.3519 | 110 | 1.5167 | 6109528 |
78
+ | 0.2783 | 0.3679 | 115 | 1.5226 | 6386720 |
79
+ | 0.1892 | 0.3838 | 120 | 1.5139 | 6664328 |
80
+ | 0.2645 | 0.3998 | 125 | 1.5059 | 6941176 |
81
+ | 0.1636 | 0.4158 | 130 | 1.5091 | 7222536 |
82
+ | 0.202 | 0.4318 | 135 | 1.5481 | 7494936 |
83
+ | 0.2311 | 0.4478 | 140 | 1.4857 | 7770984 |
84
+ | 0.2528 | 0.4638 | 145 | 1.4971 | 8055360 |
85
+ | 0.2558 | 0.4798 | 150 | 1.4835 | 8330712 |
86
+ | 0.1999 | 0.4958 | 155 | 1.4816 | 8613280 |
87
+ | 0.1584 | 0.5118 | 160 | 1.4518 | 8891640 |
88
+ | 0.1637 | 0.5278 | 165 | 1.4738 | 9170232 |
89
+ | 0.1785 | 0.5438 | 170 | 1.4616 | 9443744 |
90
+ | 0.172 | 0.5598 | 175 | 1.4296 | 9719752 |
91
+ | 0.1687 | 0.5758 | 180 | 1.4798 | 9993896 |
92
+ | 0.1333 | 0.5918 | 185 | 1.4364 | 10276328 |
93
+ | 0.1173 | 0.6078 | 190 | 1.5083 | 10554248 |
94
+ | 0.118 | 0.6238 | 195 | 1.4917 | 10836392 |
95
+ | 0.1599 | 0.6397 | 200 | 1.4452 | 11112312 |
96
+ | 0.2224 | 0.6557 | 205 | 1.4793 | 11389776 |
97
+ | 0.1497 | 0.6717 | 210 | 1.4294 | 11662248 |
98
+ | 0.1591 | 0.6877 | 215 | 1.4589 | 11930472 |
99
+ | 0.1778 | 0.7037 | 220 | 1.4534 | 12205904 |
100
+ | 0.1652 | 0.7197 | 225 | 1.4452 | 12479536 |
101
+ | 0.1618 | 0.7357 | 230 | 1.4894 | 12761120 |
102
+ | 0.153 | 0.7517 | 235 | 1.4536 | 13028616 |
103
+ | 0.0795 | 0.7677 | 240 | 1.4597 | 13300744 |
104
+ | 0.1222 | 0.7837 | 245 | 1.4621 | 13577992 |
105
+ | 0.1454 | 0.7997 | 250 | 1.4310 | 13858896 |
106
+ | 0.1635 | 0.8157 | 255 | 1.4786 | 14135016 |
107
+ | 0.1454 | 0.8317 | 260 | 1.4677 | 14412744 |
108
+ | 0.0808 | 0.8477 | 265 | 1.4608 | 14696120 |
109
+ | 0.1334 | 0.8637 | 270 | 1.4460 | 14965904 |
110
+ | 0.1086 | 0.8796 | 275 | 1.4609 | 15250560 |
111
+ | 0.1077 | 0.8956 | 280 | 1.4766 | 15527232 |
112
+ | 0.1172 | 0.9116 | 285 | 1.4532 | 15807240 |
113
+ | 0.1097 | 0.9276 | 290 | 1.4706 | 16085560 |
114
+ | 0.1058 | 0.9436 | 295 | 1.4791 | 16364832 |
115
+ | 0.0922 | 0.9596 | 300 | 1.4987 | 16644056 |
116
+ | 0.1252 | 0.9756 | 305 | 1.4820 | 16920032 |
117
+ | 0.1657 | 0.9916 | 310 | 1.4333 | 17199096 |
118
 
119
 
120
  ### Framework versions
model-00001-of-00002.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c44978a07fa859d02985d190213da47d5ebdaae88c7386ea1530207a3a82862b
3
  size 4988025760
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ebaf58ab6960c4556344cb9d33cf6fd6676ba014f16d7c47d41acfda783ced35
3
  size 4988025760
model-00002-of-00002.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:30a9fe60953272846040812269a2afb6c9ae1eb2cf3d0e52549cb433048d5605
3
  size 240691728
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f6542848c2cf18d00aab1c5e005aa89df83e9bdee5330367bf932ac0635d477
3
  size 240691728
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3bee570f98391ba03df3c0a98563352949e91c76e98b4cbc36d40200275f34d7
3
  size 5560
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb256fbe05aa4daea0a74353ec4a8953f4b05c2634d3516b4af32a546808ae11
3
  size 5560