End of training

README.md

@@ -16,6 +16,16 @@ should probably proofread and complete it, then remove this comment. -->
 # dpo-test
 
 This model is a fine-tuned version of [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0060
+- Rewards/chosen: -2.4578
+- Rewards/rejected: -10.0480
+- Rewards/accuracies: 1.0
+- Rewards/margins: 7.5902
+- Logps/rejected: -189.0133
+- Logps/chosen: -107.4476
+- Logits/rejected: -0.6723
+- Logits/chosen: -0.6997
 
 ## Model description
 
@@ -46,10 +56,16 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 150
-- training_steps: 10
+- training_steps: 2000
 
 ### Training results
 
+| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.139         | 2.29  | 500  | 0.1416          | -1.3038        | -4.9329          | 0.9464             | 3.6291          | -137.8629      | -95.9085     | -0.7743         | -0.7799       |
+| 0.0292        | 4.58  | 1000 | 0.0326          | -2.0104        | -7.6961          | 0.9974             | 5.6857          | -165.4948      | -102.9742    | -0.6749         | -0.6925       |
+| 0.0118        | 6.87  | 1500 | 0.0129          | -2.2528        | -9.2582          | 1.0                | 7.0055          | -181.1160      | -105.3981    | -0.6526         | -0.6741       |
+| 0.0056        | 9.17  | 2000 | 0.0060          | -2.4578        | -10.0480         | 1.0                | 7.5902          | -189.0133      | -107.4476    | -0.6723         | -0.6997       |
 
 
 ### Framework versions

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f78008589ccc110a3063a4a7b7fa676e1ebc82be6d0974c5234c3ebe1485e31f
+oid sha256:70f756dfe9bbf972d437fcc7f84a122785f8f1b6f5954162404652ddcb4f4d44
 size 67126232

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
+size 499723

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4be71c8ce54b0659a1bcb0f406c9e01cad746b89e640f88a7122eef896a890e3
+oid sha256:14fbcb3311fd5814b01a8532202b4383622e5c9122f5152b6983bceaac090a71
 size 4664