Hugging Face's logo

jkazdan
/
gemma-2-2b_hs2_iter1_sftsd0

Safetensors
gemma2
trl
sft
Generated from Trainer
Model card Files Community
jkazdan commited on
Commit
c3a54fe
·
verified ·
1 Parent(s): bd1116a

End of training

Browse files
Files changed (4) hide show
  1. README.md +64 -33
  2. model-00001-of-00002.safetensors +1 -1
  3. model-00002-of-00002.safetensors +1 -1
  4. training_args.bin +1 -1
README.md CHANGED
@@ -17,8 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [google/gemma-2-2b](https://huggingface.co/google/gemma-2-2b) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 1.4083
21
- - Num Input Tokens Seen: 8931640
22
 
23
  ## Model description
24
 
@@ -53,37 +53,68 @@ The following hyperparameters were used during training:
53
  | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
54
  |:-------------:|:------:|:----:|:---------------:|:-----------------:|
55
  | No log | 0 | 0 | 1.3956 | 0 |
56
- | 1.6981 | 0.0320 | 5 | 1.3658 | 288400 |
57
- | 1.5835 | 0.0639 | 10 | 1.2573 | 573584 |
58
- | 1.4304 | 0.0959 | 15 | 1.1928 | 862976 |
59
- | 1.2755 | 0.1279 | 20 | 1.1762 | 1145208 |
60
- | 1.1484 | 0.1599 | 25 | 1.1756 | 1431712 |
61
- | 0.9127 | 0.1918 | 30 | 1.2378 | 1714080 |
62
- | 0.7342 | 0.2238 | 35 | 1.3180 | 1998344 |
63
- | 0.6173 | 0.2558 | 40 | 1.4146 | 2281280 |
64
- | 0.562 | 0.2878 | 45 | 1.4913 | 2569568 |
65
- | 0.4901 | 0.3197 | 50 | 1.5504 | 2854152 |
66
- | 0.4521 | 0.3517 | 55 | 1.5198 | 3139080 |
67
- | 0.4097 | 0.3837 | 60 | 1.5812 | 3425856 |
68
- | 0.2045 | 0.4157 | 65 | 1.5653 | 3718808 |
69
- | 0.2774 | 0.4476 | 70 | 1.5087 | 4006464 |
70
- | 0.3014 | 0.4796 | 75 | 1.4870 | 4294936 |
71
- | 0.1975 | 0.5116 | 80 | 1.5149 | 4584760 |
72
- | 0.1445 | 0.5436 | 85 | 1.4913 | 4868208 |
73
- | 0.1553 | 0.5755 | 90 | 1.4491 | 5151208 |
74
- | 0.1789 | 0.6075 | 95 | 1.4293 | 5432424 |
75
- | 0.1819 | 0.6395 | 100 | 1.4234 | 5717440 |
76
- | 0.1843 | 0.6715 | 105 | 1.4360 | 6005256 |
77
- | 0.1124 | 0.7034 | 110 | 1.4008 | 6293720 |
78
- | 0.1814 | 0.7354 | 115 | 1.3848 | 6583512 |
79
- | 0.1255 | 0.7674 | 120 | 1.4044 | 6868504 |
80
- | 0.153 | 0.7994 | 125 | 1.4005 | 7161824 |
81
- | 0.0891 | 0.8313 | 130 | 1.4046 | 7447568 |
82
- | 0.1773 | 0.8633 | 135 | 1.4537 | 7734128 |
83
- | 0.1314 | 0.8953 | 140 | 1.4223 | 8015272 |
84
- | 0.104 | 0.9273 | 145 | 1.4488 | 8303384 |
85
- | 0.1063 | 0.9592 | 150 | 1.4002 | 8591800 |
86
- | 0.1253 | 0.9912 | 155 | 1.3920 | 8876296 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
87
 
88
 
89
  ### Framework versions
 
17
 
18
  This model is a fine-tuned version of [google/gemma-2-2b](https://huggingface.co/google/gemma-2-2b) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 1.5137
21
+ - Num Input Tokens Seen: 17829712
22
 
23
  ## Model description
24
 
 
53
  | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
54
  |:-------------:|:------:|:----:|:---------------:|:-----------------:|
55
  | No log | 0 | 0 | 1.3956 | 0 |
56
+ | 1.6926 | 0.0160 | 5 | 1.3651 | 285696 |
57
+ | 1.5784 | 0.0320 | 10 | 1.2560 | 571560 |
58
+ | 1.4982 | 0.0480 | 15 | 1.1924 | 856528 |
59
+ | 1.3011 | 0.0640 | 20 | 1.1594 | 1137968 |
60
+ | 1.2692 | 0.0800 | 25 | 1.1378 | 1423392 |
61
+ | 1.2069 | 0.0960 | 30 | 1.1500 | 1706944 |
62
+ | 1.1563 | 0.1120 | 35 | 1.1761 | 1988224 |
63
+ | 1.0316 | 0.1279 | 40 | 1.2207 | 2272264 |
64
+ | 0.9047 | 0.1439 | 45 | 1.2716 | 2559864 |
65
+ | 0.8926 | 0.1599 | 50 | 1.3145 | 2846920 |
66
+ | 0.7537 | 0.1759 | 55 | 1.3610 | 3135896 |
67
+ | 0.7882 | 0.1919 | 60 | 1.4222 | 3418728 |
68
+ | 0.6266 | 0.2079 | 65 | 1.4826 | 3699056 |
69
+ | 0.5966 | 0.2239 | 70 | 1.5111 | 3982712 |
70
+ | 0.5862 | 0.2399 | 75 | 1.5479 | 4266016 |
71
+ | 0.4099 | 0.2559 | 80 | 1.5246 | 4545624 |
72
+ | 0.438 | 0.2719 | 85 | 1.5312 | 4834416 |
73
+ | 0.4268 | 0.2879 | 90 | 1.5651 | 5115616 |
74
+ | 0.3835 | 0.3039 | 95 | 1.5781 | 5404872 |
75
+ | 0.3936 | 0.3199 | 100 | 1.6049 | 5693440 |
76
+ | 0.2999 | 0.3359 | 105 | 1.5558 | 5979936 |
77
+ | 0.3388 | 0.3519 | 110 | 1.5853 | 6265272 |
78
+ | 0.2141 | 0.3679 | 115 | 1.6082 | 6550008 |
79
+ | 0.1951 | 0.3838 | 120 | 1.5357 | 6829896 |
80
+ | 0.2827 | 0.3998 | 125 | 1.5383 | 7119640 |
81
+ | 0.1915 | 0.4158 | 130 | 1.5876 | 7401968 |
82
+ | 0.1656 | 0.4318 | 135 | 1.5285 | 7693464 |
83
+ | 0.1482 | 0.4478 | 140 | 1.5381 | 7979480 |
84
+ | 0.1831 | 0.4638 | 145 | 1.5497 | 8273408 |
85
+ | 0.2056 | 0.4798 | 150 | 1.5419 | 8564664 |
86
+ | 0.1866 | 0.4958 | 155 | 1.5257 | 8852896 |
87
+ | 0.1868 | 0.5118 | 160 | 1.5287 | 9138384 |
88
+ | 0.0985 | 0.5278 | 165 | 1.4843 | 9419648 |
89
+ | 0.1397 | 0.5438 | 170 | 1.4939 | 9704104 |
90
+ | 0.1592 | 0.5598 | 175 | 1.4628 | 9987840 |
91
+ | 0.1712 | 0.5758 | 180 | 1.4940 | 10272800 |
92
+ | 0.1482 | 0.5918 | 185 | 1.4714 | 10556720 |
93
+ | 0.0878 | 0.6078 | 190 | 1.4612 | 10842864 |
94
+ | 0.1269 | 0.6238 | 195 | 1.4885 | 11129280 |
95
+ | 0.0927 | 0.6397 | 200 | 1.4619 | 11410784 |
96
+ | 0.1429 | 0.6557 | 205 | 1.4507 | 11694648 |
97
+ | 0.1545 | 0.6717 | 210 | 1.4523 | 11981880 |
98
+ | 0.1168 | 0.6877 | 215 | 1.4535 | 12272496 |
99
+ | 0.175 | 0.7037 | 220 | 1.4501 | 12558896 |
100
+ | 0.0869 | 0.7197 | 225 | 1.4673 | 12842440 |
101
+ | 0.1086 | 0.7357 | 230 | 1.4905 | 13130608 |
102
+ | 0.1035 | 0.7517 | 235 | 1.4422 | 13411360 |
103
+ | 0.1142 | 0.7677 | 240 | 1.4519 | 13695520 |
104
+ | 0.091 | 0.7837 | 245 | 1.4698 | 13980728 |
105
+ | 0.1734 | 0.7997 | 250 | 1.4578 | 14276136 |
106
+ | 0.147 | 0.8157 | 255 | 1.4818 | 14560480 |
107
+ | 0.1138 | 0.8317 | 260 | 1.4677 | 14848512 |
108
+ | 0.0635 | 0.8477 | 265 | 1.4703 | 15136488 |
109
+ | 0.2047 | 0.8637 | 270 | 1.4876 | 15423352 |
110
+ | 0.1162 | 0.8796 | 275 | 1.4672 | 15707888 |
111
+ | 0.1132 | 0.8956 | 280 | 1.4634 | 15990288 |
112
+ | 0.1231 | 0.9116 | 285 | 1.4662 | 16275832 |
113
+ | 0.1544 | 0.9276 | 290 | 1.5047 | 16564968 |
114
+ | 0.1852 | 0.9436 | 295 | 1.4825 | 16851368 |
115
+ | 0.1406 | 0.9596 | 300 | 1.4831 | 17142256 |
116
+ | 0.1188 | 0.9756 | 305 | 1.5429 | 17429064 |
117
+ | 0.1442 | 0.9916 | 310 | 1.5211 | 17714264 |
118
 
119
 
120
  ### Framework versions
model-00001-of-00002.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:430d2d496a908e6eaf80d899901d7d8629c88ac132c42cdf455477b9966b62eb
3
  size 4988025760
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:017fb20737d64af8b60b6fee9ee79e9dcd02e673fe132fa712667308b5bdd58b
3
  size 4988025760
model-00002-of-00002.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f2a26f04880fc257375823a1d268e9f14c81c7228c3430feb8ef1f3ff479ecab
3
  size 240691728
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb612807248b10a2e3f4b17a45b923ff9ec4a56aaef6423c8f9af50d054fb051
3
  size 240691728
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:72041d5194c3789d89d13a1ddbb72f6b7369d60fe8e1e3c0954c6ebf6825d0bc
3
  size 5560
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd795afea97db77ffcf83bb28cd4ebaeabc5dfea24c9858b3fc48207e779189a
3
  size 5560