--- tags: - generated_from_trainer model-index: - name: distilbert-base-uncased-continued_training-medqa results: [] --- # distilbert-base-uncased-continued_training-medqa This model is a fine-tuned version of [Shaier/distilbert-base-uncased-finetuned-medqa](https://huggingface.co/Shaier/distilbert-base-uncased-finetuned-medqa) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.4057 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 64 - eval_batch_size: 64 - seed: 42 - gradient_accumulation_steps: 8 - total_train_batch_size: 512 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 100 - num_epochs: 110 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:-----:|:---------------:| | No log | 1.0 | 333 | 0.5673 | | No log | 2.0 | 666 | 0.6109 | | No log | 3.0 | 999 | 0.4928 | | No log | 4.0 | 1332 | 0.5496 | | No log | 5.0 | 1665 | 0.5017 | | No log | 6.0 | 1998 | 0.6361 | | No log | 7.0 | 2331 | 0.5995 | | 0.6107 | 8.0 | 2664 | 0.6359 | | 0.6107 | 9.0 | 2997 | 0.4778 | | 0.6107 | 10.0 | 3330 | 0.5355 | | 0.6107 | 11.0 | 3663 | 0.6210 | | 0.6107 | 12.0 | 3996 | 0.4758 | | 0.6107 | 13.0 | 4329 | 0.5468 | | 0.6107 | 14.0 | 4662 | 0.4659 | | 0.6107 | 15.0 | 4995 | 0.5465 | | 0.5784 | 16.0 | 5328 | 0.3997 | | 0.5784 | 17.0 | 5661 | 0.5352 | | 0.5784 | 18.0 | 5994 | 0.5812 | | 0.5784 | 19.0 | 6327 | 0.6133 | | 0.5784 | 20.0 | 6660 | 0.5050 | | 0.5784 | 21.0 | 6993 | 0.4377 | | 0.5784 | 22.0 | 7326 | 0.4630 | | 0.5784 | 23.0 | 7659 | 0.5162 | | 0.5555 | 24.0 | 7992 | 0.4968 | | 0.5555 | 25.0 | 8325 | 0.5336 | | 0.5555 | 26.0 | 8658 | 0.5146 | | 0.5555 | 27.0 | 8991 | 0.4887 | | 0.5555 | 28.0 | 9324 | 0.5048 | | 0.5555 | 29.0 | 9657 | 0.5092 | | 0.5555 | 30.0 | 9990 | 0.4874 | | 0.5555 | 31.0 | 10323 | 0.5452 | | 0.5392 | 32.0 | 10656 | 0.5303 | | 0.5392 | 33.0 | 10989 | 0.5932 | | 0.5392 | 34.0 | 11322 | 0.4472 | | 0.5392 | 35.0 | 11655 | 0.5796 | | 0.5392 | 36.0 | 11988 | 0.4808 | | 0.5392 | 37.0 | 12321 | 0.4756 | | 0.5392 | 38.0 | 12654 | 0.5552 | | 0.5392 | 39.0 | 12987 | 0.5777 | | 0.5276 | 40.0 | 13320 | 0.5049 | | 0.5276 | 41.0 | 13653 | 0.5141 | | 0.5276 | 42.0 | 13986 | 0.5153 | | 0.5276 | 43.0 | 14319 | 0.5255 | | 0.5276 | 44.0 | 14652 | 0.4948 | | 0.5276 | 45.0 | 14985 | 0.4542 | | 0.5276 | 46.0 | 15318 | 0.3743 | | 0.5276 | 47.0 | 15651 | 0.5167 | | 0.5181 | 48.0 | 15984 | 0.5304 | | 0.5181 | 49.0 | 16317 | 0.5459 | | 0.5181 | 50.0 | 16650 | 0.4616 | | 0.5181 | 51.0 | 16983 | 0.4887 | | 0.5181 | 52.0 | 17316 | 0.4454 | | 0.5181 | 53.0 | 17649 | 0.4665 | | 0.5181 | 54.0 | 17982 | 0.4647 | | 0.5181 | 55.0 | 18315 | 0.4168 | | 0.5102 | 56.0 | 18648 | 0.5703 | | 0.5102 | 57.0 | 18981 | 0.4930 | | 0.5102 | 58.0 | 19314 | 0.4750 | | 0.5102 | 59.0 | 19647 | 0.4853 | | 0.5102 | 60.0 | 19980 | 0.4508 | | 0.5102 | 61.0 | 20313 | 0.5869 | | 0.5102 | 62.0 | 20646 | 0.4975 | | 0.5102 | 63.0 | 20979 | 0.4944 | | 0.5039 | 64.0 | 21312 | 0.4994 | | 0.5039 | 65.0 | 21645 | 0.4257 | | 0.5039 | 66.0 | 21978 | 0.5275 | | 0.5039 | 67.0 | 22311 | 0.4432 | | 0.5039 | 68.0 | 22644 | 0.5080 | | 0.5039 | 69.0 | 22977 | 0.4262 | | 0.5039 | 70.0 | 23310 | 0.5359 | | 0.5039 | 71.0 | 23643 | 0.5504 | | 0.4993 | 72.0 | 23976 | 0.4428 | | 0.4993 | 73.0 | 24309 | 0.4275 | | 0.4993 | 74.0 | 24642 | 0.5607 | | 0.4993 | 75.0 | 24975 | 0.4612 | | 0.4993 | 76.0 | 25308 | 0.5083 | | 0.4993 | 77.0 | 25641 | 0.4803 | | 0.4993 | 78.0 | 25974 | 0.5019 | | 0.4993 | 79.0 | 26307 | 0.4535 | | 0.4957 | 80.0 | 26640 | 0.5364 | | 0.4957 | 81.0 | 26973 | 0.5502 | | 0.4957 | 82.0 | 27306 | 0.4912 | | 0.4957 | 83.0 | 27639 | 0.5563 | | 0.4957 | 84.0 | 27972 | 0.4360 | | 0.4957 | 85.0 | 28305 | 0.4962 | | 0.4957 | 86.0 | 28638 | 0.4523 | | 0.4957 | 87.0 | 28971 | 0.4979 | | 0.4923 | 88.0 | 29304 | 0.4697 | | 0.4923 | 89.0 | 29637 | 0.4730 | | 0.4923 | 90.0 | 29970 | 0.4848 | | 0.4923 | 91.0 | 30303 | 0.4293 | | 0.4923 | 92.0 | 30636 | 0.4745 | | 0.4923 | 93.0 | 30969 | 0.3710 | | 0.4923 | 94.0 | 31302 | 0.4068 | | 0.4923 | 95.0 | 31635 | 0.4980 | | 0.4896 | 96.0 | 31968 | 0.4586 | | 0.4896 | 97.0 | 32301 | 0.5152 | | 0.4896 | 98.0 | 32634 | 0.4636 | | 0.4896 | 99.0 | 32967 | 0.5426 | | 0.4896 | 100.0 | 33300 | 0.4604 | | 0.4896 | 101.0 | 33633 | 0.4925 | | 0.4896 | 102.0 | 33966 | 0.3729 | | 0.4896 | 103.0 | 34299 | 0.4337 | | 0.4882 | 104.0 | 34632 | 0.5307 | | 0.4882 | 105.0 | 34965 | 0.5480 | | 0.4882 | 106.0 | 35298 | 0.4124 | | 0.4882 | 107.0 | 35631 | 0.4862 | | 0.4882 | 108.0 | 35964 | 0.4333 | | 0.4882 | 109.0 | 36297 | 0.4443 | | 0.4882 | 110.0 | 36630 | 0.4890 | ### Framework versions - Transformers 4.18.0 - Pytorch 1.11.0 - Datasets 2.3.2 - Tokenizers 0.11.0