End of training

README.md

@@ -31,16 +31,17 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 2e-05
-- train_batch_size: 2
+- learning_rate: 5e-05
+- train_batch_size: 4
 - eval_batch_size: 8
 - seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 8
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 32
 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_steps: 500
-- num_epochs: 3
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 10
+- mixed_precision_training: Native AMP
 
 ### Training results
 
@@ -50,5 +51,4 @@ The following hyperparameters were used during training:
 
 - Transformers 4.46.2
 - Pytorch 2.5.1+cu121
-- Datasets 3.1.0
 - Tokenizers 0.20.3

generation_config.json

@@ -3,5 +3,6 @@
   "bos_token_id": 0,
   "eos_token_id": 1,
   "pad_token_id": 1,
-  "transformers_version": "4.46.2"
+  "transformers_version": "4.46.2",
+  "use_cache": false
 }

runs/Nov26_16-58-34_3f18e734a088/events.out.tfevents.1732640325.3f18e734a088.1077.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:19658a8878f3164c2ade7ab32acf4c9b4c24267ef87556c0558a12d5a82c17f4
-size 91068
+oid sha256:e083e765695ddac1c0b776f22dc543c6d0da0198811f50ae282bf2015313b804
+size 91422