fine-tuned-visionllama_6

This model is a fine-tuned version of meta-llama/Llama-3.2-11B-Vision-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.0262

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 32
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
6.2195	0.0184	5	2.9157
2.5279	0.0368	10	2.3537
2.3247	0.0552	15	2.2259
2.237	0.0736	20	2.1789
2.1879	0.0920	25	2.1520
2.1381	0.1104	30	2.1353
2.1285	0.1288	35	2.1204
2.1076	0.1472	40	2.1085
2.0869	0.1657	45	2.1010
2.0694	0.1841	50	2.0966
2.0917	0.2025	55	2.0899
2.0597	0.2209	60	2.0858
2.05	0.2393	65	2.0810
2.0764	0.2577	70	2.0784
2.0872	0.2761	75	2.0763
2.0388	0.2945	80	2.0734
2.057	0.3129	85	2.0704
2.0423	0.3313	90	2.0667
2.022	0.3497	95	2.0647
2.0281	0.3681	100	2.0631
2.0407	0.3865	105	2.0638
2.0284	0.4049	110	2.0617
2.0311	0.4233	115	2.0597
2.0093	0.4417	120	2.0578
2.0191	0.4601	125	2.0543
2.0316	0.4785	130	2.0539
2.0243	0.4970	135	2.0526
1.9983	0.5154	140	2.0520
2.0298	0.5338	145	2.0530
2.0217	0.5522	150	2.0511
2.0115	0.5706	155	2.0488
1.9883	0.5890	160	2.0481
2.0207	0.6074	165	2.0462
2.0069	0.6258	170	2.0453
2.0045	0.6442	175	2.0432
2.0034	0.6626	180	2.0435
1.9921	0.6810	185	2.0426
1.9912	0.6994	190	2.0419
1.9969	0.7178	195	2.0403
2.0093	0.7362	200	2.0391
2.0154	0.7546	205	2.0389
1.9934	0.7730	210	2.0380
1.9926	0.7914	215	2.0354
1.9771	0.8098	220	2.0352
1.9819	0.8283	225	2.0330
1.9779	0.8467	230	2.0333
1.9846	0.8651	235	2.0340
1.9913	0.8835	240	2.0335
1.9834	0.9019	245	2.0319
1.9786	0.9203	250	2.0312
1.9726	0.9387	255	2.0306
1.9793	0.9571	260	2.0293
1.971	0.9755	265	2.0298
1.973	0.9939	270	2.0298
1.9651	1.0123	275	2.0307
1.9619	1.0307	280	2.0308
1.9536	1.0491	285	2.0320
1.9618	1.0675	290	2.0327
1.9555	1.0859	295	2.0307
1.9704	1.1043	300	2.0294
1.9609	1.1227	305	2.0290
1.9745	1.1411	310	2.0302
1.9707	1.1596	315	2.0268
1.9651	1.1780	320	2.0279
1.9745	1.1964	325	2.0276
1.9618	1.2148	330	2.0267
1.932	1.2332	335	2.0248
1.9495	1.2516	340	2.0258
1.9396	1.2700	345	2.0262
1.9277	1.2884	350	2.0264
1.9355	1.3068	355	2.0273
1.9502	1.3252	360	2.0273
1.9491	1.3436	365	2.0281
1.9489	1.3620	370	2.0274
1.9194	1.3804	375	2.0271
1.9179	1.3988	380	2.0258
1.9418	1.4172	385	2.0261
1.9618	1.4356	390	2.0269
1.9283	1.4540	395	2.0256
1.912	1.4724	400	2.0225
1.9284	1.4909	405	2.0230
1.9418	1.5093	410	2.0223
1.9245	1.5277	415	2.0241
1.9292	1.5461	420	2.0237
1.9442	1.5645	425	2.0241
1.9366	1.5829	430	2.0225
1.9318	1.6013	435	2.0233
1.9266	1.6197	440	2.0234
1.9211	1.6381	445	2.0218
1.9248	1.6565	450	2.0230
1.9476	1.6749	455	2.0227
1.9333	1.6933	460	2.0206
1.9193	1.7117	465	2.0196
1.9291	1.7301	470	2.0231
1.9009	1.7485	475	2.0223
1.9134	1.7669	480	2.0225
1.9337	1.7853	485	2.0200
1.9077	1.8038	490	2.0227
1.8962	1.8222	495	2.0227
1.9343	1.8406	500	2.0221
1.9307	1.8590	505	2.0237
1.9339	1.8774	510	2.0220
1.922	1.8958	515	2.0220
1.9289	1.9142	520	2.0220
1.9269	1.9326	525	2.0231
1.9149	1.9510	530	2.0216
1.8962	1.9694	535	2.0252
1.9568	1.9878	540	2.0262

Framework versions

• PEFT 0.13.0
• Transformers 4.45.1
• Pytorch 2.2.2+cu121
• Datasets 3.0.1
• Tokenizers 0.20.3