Training with 90/10 Spanish dataset, 5 epochs, 2 Batch Size, reduce_lr_on_plateau

README.md

@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.2459
+- Loss: 1.2754
 
 ## Model description
 
@@ -48,11 +48,11 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
-| 1.3221        | 0.9995 | 1847 | 0.9229          |
-| 0.7158        | 1.9989 | 3694 | 0.8894          |
-| 0.363         | 2.9984 | 5541 | 0.9671          |
-| 0.2104        | 3.9978 | 7388 | 1.1174          |
-| 0.39          | 4.9973 | 9235 | 1.2459          |
+| 1.3053        | 0.9995 | 1847 | 0.9206          |
+| 0.7158        | 1.9989 | 3694 | 0.8873          |
+| 0.3506        | 2.9984 | 5541 | 0.9619          |
+| 0.2142        | 3.9978 | 7388 | 1.1203          |
+| 0.3116        | 4.9973 | 9235 | 1.2754          |
 
 
 ### Framework versions

adapter_config.json

@@ -23,13 +23,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "o_proj",
-    "v_proj",
-    "gate_proj",
-    "q_proj",
     "up_proj",
+    "k_proj",
     "down_proj",
-    "k_proj"
+    "q_proj",
+    "v_proj",
+    "gate_proj",
+    "o_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3bd9be91216a220040f93b091c1d00d5d815d67496e9d0a97e5bec1a58da3a12
+oid sha256:d97a1fd723fc292945f8129a4694d4a02ca7df5702811260ed2dcd90769c7652
 size 1688269144

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:011ec18ec40eb9286a5ab0123e708847240c22c3856cd034d29a50a36580fbeb
+oid sha256:a251263e185cd54161c30d98501e6ab7594498a4add3666d009a3c74f91fa5c9
 size 5048