eskayML/electra_interview_new

README.md

@@ -18,8 +18,8 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mrm8488/electra-small-finetuned-squadv2](https://huggingface.co/mrm8488/electra-small-finetuned-squadv2) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.3959
-- Accuracy: 0.2675
+- Loss: 2.2978
+- Accuracy: 0.2716
 
 ## Model description
 
@@ -42,7 +42,7 @@ The following hyperparameters were used during training:
 - train_batch_size: 2
 - eval_batch_size: 2
 - seed: 42
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
 - num_epochs: 10
 
@@ -50,21 +50,21 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Accuracy |
 |:-------------:|:-----:|:----:|:---------------:|:--------:|
-| No log        | 1.0   | 380  | 2.6074          | 0.2266   |
-| 2.7429        | 2.0   | 760  | 2.4872          | 0.2266   |
-| 2.5203        | 3.0   | 1140 | 2.4483          | 0.2266   |
-| 2.4479        | 4.0   | 1520 | 2.4349          | 0.2266   |
-| 2.4479        | 5.0   | 1900 | 2.4114          | 0.2306   |
-| 2.3919        | 6.0   | 2280 | 2.3933          | 0.2424   |
-| 2.2714        | 7.0   | 2660 | 2.3914          | 0.2530   |
-| 2.1536        | 8.0   | 3040 | 2.3968          | 0.2714   |
-| 2.1536        | 9.0   | 3420 | 2.3913          | 0.2648   |
-| 2.1058        | 10.0  | 3800 | 2.3959          | 0.2675   |
+| No log        | 1.0   | 463  | 2.6395          | 0.1983   |
+| 2.7859        | 2.0   | 926  | 2.5260          | 0.1983   |
+| 2.5604        | 3.0   | 1389 | 2.4446          | 0.2241   |
+| 2.4612        | 4.0   | 1852 | 2.3737          | 0.3103   |
+| 2.2886        | 5.0   | 2315 | 2.3307          | 0.3276   |
+| 2.1381        | 6.0   | 2778 | 2.3076          | 0.3017   |
+| 1.9905        | 7.0   | 3241 | 2.3089          | 0.2931   |
+| 1.8363        | 8.0   | 3704 | 2.2939          | 0.2845   |
+| 1.7738        | 9.0   | 4167 | 2.3060          | 0.2802   |
+| 1.6807        | 10.0  | 4630 | 2.2978          | 0.2716   |
 
 
 ### Framework versions
 
-- Transformers 4.44.2
-- Pytorch 2.4.1+cu121
-- Datasets 3.0.1
-- Tokenizers 0.19.1
+- Transformers 4.47.1
+- Pytorch 2.5.1+cu121
+- Datasets 3.2.0
+- Tokenizers 0.21.0

config.json

@@ -68,7 +68,7 @@
   "summary_type": "first",
   "summary_use_proj": true,
   "torch_dtype": "float32",
-  "transformers_version": "4.44.2",
+  "transformers_version": "4.47.1",
   "type_vocab_size": 2,
   "use_cache": true,
   "vocab_size": 30522

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:32a787c62a68c4edf191c25b81828f9bfa6b04143e2c06ea1ca0a61109ec1912
+oid sha256:3894cfe4f3735a911a25d70b1c53da6bae76d75cbf5fd9839564037f9b0f8b89
 size 54239712

tokenizer_config.json

@@ -45,6 +45,7 @@
   "cls_token": "[CLS]",
   "do_basic_tokenize": true,
   "do_lower_case": true,
+  "extra_special_tokens": {},
   "mask_token": "[MASK]",
   "max_length": 512,
   "model_max_length": 512,

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:87d473bb239836d3c278ca4bc54501ef0abc689b9f6d2463e5a0867d8caa5240
-size 5240
+oid sha256:16905a4f2f0464a8209f41eeb68c1d827eaebd9b151a36821496df2a75c6de56
+size 5304