segformerb5-largeImages

This model is a fine-tuned version of nvidia/mit-b5 on the JCAI2000/LargerImagesLabelled dataset. It achieves the following results on the evaluation set:

• Loss: 0.1156
• Mean Iou: 0.7785
• Mean Accuracy: 0.8298
• Overall Accuracy: 0.9767
• Accuracy Background: 0.9925
• Accuracy Branch: 0.6671
• Iou Background: 0.9759
• Iou Branch: 0.5812

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 6e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Mean Iou	Mean Accuracy	Overall Accuracy	Accuracy Background	Accuracy Branch	Iou Background	Iou Branch
0.2671	1.18	20	0.2779	0.4834	0.5075	0.9509	0.9985	0.0165	0.9508	0.0160
0.122	2.35	40	0.1772	0.6522	0.6931	0.9632	0.9922	0.3940	0.9625	0.3419
0.0671	3.53	60	0.1086	0.7392	0.8603	0.9658	0.9772	0.7435	0.9646	0.5138
0.0979	4.71	80	0.0860	0.7552	0.8493	0.9705	0.9836	0.7150	0.9695	0.5409
0.0749	5.88	100	0.0727	0.7601	0.8116	0.9746	0.9921	0.6311	0.9738	0.5465
0.032	7.06	120	0.0721	0.7535	0.8016	0.9741	0.9927	0.6106	0.9733	0.5338
0.0337	8.24	140	0.0719	0.7745	0.8530	0.9743	0.9873	0.7187	0.9733	0.5757
0.0398	9.41	160	0.0704	0.7732	0.8302	0.9756	0.9913	0.6690	0.9748	0.5715
0.0374	10.59	180	0.0724	0.7583	0.7995	0.9752	0.9941	0.6050	0.9744	0.5422
0.0334	11.76	200	0.0724	0.7721	0.8231	0.9760	0.9924	0.6537	0.9752	0.5690
0.025	12.94	220	0.0731	0.7725	0.8192	0.9763	0.9932	0.6452	0.9755	0.5694
0.0336	14.12	240	0.0699	0.7793	0.8334	0.9765	0.9919	0.6748	0.9757	0.5828
0.0321	15.29	260	0.0697	0.7825	0.8395	0.9767	0.9915	0.6875	0.9759	0.5891
0.0216	16.47	280	0.0752	0.7701	0.8176	0.9760	0.9930	0.6421	0.9752	0.5650
0.0178	17.65	300	0.0743	0.7753	0.8296	0.9761	0.9918	0.6674	0.9753	0.5752
0.0206	18.82	320	0.0717	0.7881	0.8488	0.9771	0.9909	0.7066	0.9763	0.5999
0.0162	20.0	340	0.0786	0.7694	0.8141	0.9761	0.9935	0.6347	0.9754	0.5634
0.0306	21.18	360	0.0785	0.7785	0.8275	0.9768	0.9929	0.6622	0.9760	0.5809
0.0179	22.35	380	0.0769	0.7816	0.8414	0.9764	0.9909	0.6919	0.9756	0.5876
0.0152	23.53	400	0.0776	0.7842	0.8461	0.9766	0.9906	0.7016	0.9758	0.5926
0.0245	24.71	420	0.0820	0.7725	0.8164	0.9765	0.9937	0.6390	0.9758	0.5692
0.0248	25.88	440	0.0829	0.7772	0.8268	0.9766	0.9928	0.6608	0.9759	0.5786
0.0176	27.06	460	0.0818	0.7761	0.8271	0.9764	0.9925	0.6617	0.9756	0.5767
0.0135	28.24	480	0.0816	0.7805	0.8384	0.9764	0.9913	0.6854	0.9756	0.5855
0.0343	29.41	500	0.0852	0.7777	0.8310	0.9764	0.9921	0.6699	0.9756	0.5798
0.0147	30.59	520	0.0851	0.7792	0.8367	0.9763	0.9913	0.6820	0.9755	0.5829
0.0119	31.76	540	0.0880	0.7800	0.8337	0.9767	0.9920	0.6754	0.9759	0.5842
0.0143	32.94	560	0.0899	0.7749	0.8241	0.9764	0.9928	0.6555	0.9756	0.5743
0.0122	34.12	580	0.0886	0.7810	0.8374	0.9766	0.9916	0.6832	0.9758	0.5863
0.0135	35.29	600	0.0908	0.7727	0.8206	0.9762	0.9930	0.6482	0.9755	0.5699
0.0203	36.47	620	0.0913	0.7758	0.8267	0.9764	0.9925	0.6608	0.9756	0.5759
0.0109	37.65	640	0.0898	0.7803	0.8337	0.9767	0.9921	0.6753	0.9759	0.5847
0.0141	38.82	660	0.0936	0.7774	0.8280	0.9766	0.9926	0.6634	0.9758	0.5790
0.0087	40.0	680	0.0903	0.7830	0.8493	0.9762	0.9898	0.7088	0.9753	0.5908
0.0099	41.18	700	0.0930	0.7779	0.8284	0.9766	0.9926	0.6641	0.9759	0.5799
0.0149	42.35	720	0.0908	0.7799	0.8320	0.9767	0.9923	0.6717	0.9760	0.5838
0.0168	43.53	740	0.0897	0.7864	0.8496	0.9768	0.9904	0.7087	0.9759	0.5969
0.0281	44.71	760	0.0954	0.7760	0.8259	0.9765	0.9927	0.6591	0.9757	0.5762
0.0102	45.88	780	0.0942	0.7819	0.8382	0.9767	0.9916	0.6849	0.9759	0.5879
0.0087	47.06	800	0.0948	0.7843	0.8422	0.9769	0.9913	0.6931	0.9761	0.5926
0.0166	48.24	820	0.0981	0.7777	0.8280	0.9766	0.9926	0.6634	0.9759	0.5796
0.0236	49.41	840	0.0972	0.7770	0.8274	0.9765	0.9926	0.6622	0.9758	0.5782
0.0168	50.59	860	0.0994	0.7751	0.8218	0.9766	0.9932	0.6505	0.9758	0.5743
0.017	51.76	880	0.0991	0.7779	0.8281	0.9767	0.9926	0.6635	0.9759	0.5799
0.0111	52.94	900	0.0994	0.7778	0.8266	0.9767	0.9929	0.6603	0.9760	0.5797
0.0202	54.12	920	0.0985	0.7845	0.8380	0.9772	0.9921	0.6839	0.9764	0.5926
0.0142	55.29	940	0.1025	0.7762	0.8240	0.9767	0.9931	0.6548	0.9759	0.5766
0.01	56.47	960	0.0997	0.7808	0.8346	0.9767	0.9920	0.6771	0.9759	0.5857
0.0127	57.65	980	0.1028	0.7797	0.8317	0.9767	0.9923	0.6712	0.9759	0.5835
0.0069	58.82	1000	0.1011	0.7834	0.8400	0.9768	0.9915	0.6885	0.9760	0.5907
0.0109	60.0	1020	0.1059	0.7775	0.8282	0.9766	0.9925	0.6638	0.9758	0.5792
0.0087	61.18	1040	0.1037	0.7793	0.8308	0.9767	0.9924	0.6692	0.9759	0.5826
0.0125	62.35	1060	0.1056	0.7784	0.8279	0.9768	0.9928	0.6630	0.9760	0.5808
0.0084	63.53	1080	0.1066	0.7803	0.8330	0.9768	0.9922	0.6737	0.9760	0.5847
0.0183	64.71	1100	0.1056	0.7806	0.8340	0.9767	0.9921	0.6759	0.9760	0.5853
0.0106	65.88	1120	0.1076	0.7768	0.8257	0.9766	0.9929	0.6586	0.9759	0.5778
0.0072	67.06	1140	0.1103	0.7771	0.8278	0.9765	0.9925	0.6630	0.9758	0.5784
0.0112	68.24	1160	0.1070	0.7799	0.8315	0.9768	0.9924	0.6705	0.9760	0.5838
0.0149	69.41	1180	0.1089	0.7778	0.8284	0.9766	0.9926	0.6642	0.9758	0.5797
0.0147	70.59	1200	0.1087	0.7805	0.8325	0.9768	0.9924	0.6727	0.9760	0.5850
0.013	71.76	1220	0.1081	0.7803	0.8331	0.9767	0.9922	0.6741	0.9760	0.5846
0.013	72.94	1240	0.1097	0.7789	0.8304	0.9767	0.9924	0.6683	0.9759	0.5818
0.0115	74.12	1260	0.1104	0.7773	0.8269	0.9766	0.9927	0.6610	0.9759	0.5787
0.0102	75.29	1280	0.1097	0.7795	0.8323	0.9767	0.9922	0.6725	0.9759	0.5831
0.0133	76.47	1300	0.1101	0.7808	0.8355	0.9767	0.9919	0.6791	0.9759	0.5857
0.013	77.65	1320	0.1111	0.7814	0.8358	0.9768	0.9919	0.6797	0.9760	0.5867
0.0068	78.82	1340	0.1107	0.7814	0.8362	0.9767	0.9919	0.6805	0.9759	0.5869
0.0036	80.0	1360	0.1136	0.7789	0.8313	0.9766	0.9923	0.6703	0.9758	0.5820
0.0163	81.18	1380	0.1123	0.7809	0.8347	0.9767	0.9920	0.6773	0.9760	0.5858
0.0065	82.35	1400	0.1117	0.7811	0.8356	0.9767	0.9919	0.6794	0.9759	0.5862
0.018	83.53	1420	0.1121	0.7811	0.8360	0.9767	0.9918	0.6802	0.9759	0.5864
0.0122	84.71	1440	0.1123	0.7803	0.8346	0.9766	0.9919	0.6772	0.9759	0.5847
0.0085	85.88	1460	0.1139	0.7783	0.8300	0.9766	0.9924	0.6676	0.9758	0.5808
0.0074	87.06	1480	0.1130	0.7820	0.8364	0.9768	0.9919	0.6808	0.9760	0.5879
0.0124	88.24	1500	0.1141	0.7801	0.8332	0.9767	0.9921	0.6743	0.9759	0.5843
0.0114	89.41	1520	0.1152	0.7783	0.8301	0.9766	0.9924	0.6678	0.9758	0.5808
0.0113	90.59	1540	0.1153	0.7784	0.8302	0.9766	0.9924	0.6680	0.9758	0.5811
0.0076	91.76	1560	0.1153	0.7778	0.8286	0.9766	0.9925	0.6647	0.9758	0.5797
0.0128	92.94	1580	0.1149	0.7785	0.8308	0.9766	0.9923	0.6694	0.9758	0.5813
0.0046	94.12	1600	0.1154	0.7781	0.8298	0.9766	0.9923	0.6673	0.9758	0.5803
0.0091	95.29	1620	0.1143	0.7792	0.8318	0.9766	0.9922	0.6713	0.9759	0.5826
0.0121	96.47	1640	0.1153	0.7784	0.8302	0.9766	0.9924	0.6681	0.9758	0.5810
0.0082	97.65	1660	0.1151	0.7787	0.8308	0.9766	0.9923	0.6694	0.9758	0.5815
0.0094	98.82	1680	0.1155	0.7784	0.8295	0.9766	0.9925	0.6664	0.9759	0.5808
0.0067	100.0	1700	0.1156	0.7785	0.8298	0.9767	0.9925	0.6671	0.9759	0.5812

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.4
• Tokenizers 0.13.3