Update README.md

README.md

@@ -33,19 +33,83 @@ Time taken: 25.5 mins
 ---------------
 
 
-**Evaluated in 4bit**
-
-|    Tasks    |Version|Filter|n-shot| Metric |Value |   |Stderr|
-|-------------|-------|------|-----:|--------|-----:|---|-----:|
-|arc_challenge|Yaml   |none  |     0|acc     |0.5853|±  |0.0144|
-|             |       |none  |     0|acc_norm|0.6126|±  |0.0142|
-|arc_easy     |Yaml   |none  |     0|acc     |0.8077|±  |0.0081|
-|             |       |none  |     0|acc_norm|0.7715|±  |0.0086|
-|boolq        |Yaml   |none  |     0|acc     |0.8630|±  |0.0060|
-|hellaswag    |Yaml   |none  |     0|acc     |0.6653|±  |0.0047|
-|             |       |none  |     0|acc_norm|0.8498|±  |0.0036|
-|openbookqa   |Yaml   |none  |     0|acc     |0.3460|±  |0.0213|
-|             |       |none  |     0|acc_norm|0.4660|±  |0.0223|
-|piqa         |Yaml   |none  |     0|acc     |0.7835|±  |0.0096|
-|             |       |none  |     0|acc_norm|0.7851|±  |0.0096|
-|winogrande   |Yaml   |none  |     0|acc     |0.7277|±  |0.0125|
+|                                                Model                                                |AGIEval|GPT4All|TruthfulQA|Bigbench|Average|
+|-----------------------------------------------------------------------------------------------------|------:|------:|---------:|-------:|------:|
+|[SOLAR-10.7b-Instruct-truthy-dpo](https://huggingface.co/macadeliccc/SOLAR-10.7b-Instruct-truthy-dpo)|  48.69|  73.82|     76.81|   45.71|  61.26|
+
+### AGIEval
+|             Task             |Version| Metric |Value|   |Stderr|
+|------------------------------|------:|--------|----:|---|-----:|
+|agieval_aqua_rat              |      0|acc     |27.95|±  |  2.82|
+|                              |       |acc_norm|27.95|±  |  2.82|
+|agieval_logiqa_en             |      0|acc     |42.40|±  |  1.94|
+|                              |       |acc_norm|42.24|±  |  1.94|
+|agieval_lsat_ar               |      0|acc     |25.65|±  |  2.89|
+|                              |       |acc_norm|23.91|±  |  2.82|
+|agieval_lsat_lr               |      0|acc     |54.12|±  |  2.21|
+|                              |       |acc_norm|54.51|±  |  2.21|
+|agieval_lsat_rc               |      0|acc     |69.89|±  |  2.80|
+|                              |       |acc_norm|69.89|±  |  2.80|
+|agieval_sat_en                |      0|acc     |80.10|±  |  2.79|
+|                              |       |acc_norm|80.10|±  |  2.79|
+|agieval_sat_en_without_passage|      0|acc     |50.00|±  |  3.49|
+|                              |       |acc_norm|49.51|±  |  3.49|
+|agieval_sat_math              |      0|acc     |42.27|±  |  3.34|
+|                              |       |acc_norm|41.36|±  |  3.33|
+
+Average: 48.69%
+
+### GPT4All
+|    Task     |Version| Metric |Value|   |Stderr|
+|-------------|------:|--------|----:|---|-----:|
+|arc_challenge|      0|acc     |59.90|±  |  1.43|
+|             |       |acc_norm|63.91|±  |  1.40|
+|arc_easy     |      0|acc     |80.85|±  |  0.81|
+|             |       |acc_norm|78.16|±  |  0.85|
+|boolq        |      1|acc     |88.20|±  |  0.56|
+|hellaswag    |      0|acc     |68.34|±  |  0.46|
+|             |       |acc_norm|86.39|±  |  0.34|
+|openbookqa   |      0|acc     |37.60|±  |  2.17|
+|             |       |acc_norm|46.80|±  |  2.23|
+|piqa         |      0|acc     |78.84|±  |  0.95|
+|             |       |acc_norm|78.78|±  |  0.95|
+|winogrande   |      0|acc     |74.51|±  |  1.22|
+
+Average: 73.82%
+
+### TruthfulQA
+|    Task     |Version|Metric|Value|   |Stderr|
+|-------------|------:|------|----:|---|-----:|
+|truthfulqa_mc|      1|mc1   |61.81|±  |  1.70|
+|             |       |mc2   |76.81|±  |  1.42|
+
+Average: 76.81%
+
+### Bigbench
+|                      Task                      |Version|       Metric        |Value|   |Stderr|
+|------------------------------------------------|------:|---------------------|----:|---|-----:|
+|bigbench_causal_judgement                       |      0|multiple_choice_grade|50.53|±  |  3.64|
+|bigbench_date_understanding                     |      0|multiple_choice_grade|63.14|±  |  2.51|
+|bigbench_disambiguation_qa                      |      0|multiple_choice_grade|47.67|±  |  3.12|
+|bigbench_geometric_shapes                       |      0|multiple_choice_grade|26.18|±  |  2.32|
+|                                                |       |exact_str_match      | 0.00|±  |  0.00|
+|bigbench_logical_deduction_five_objects         |      0|multiple_choice_grade|28.60|±  |  2.02|
+|bigbench_logical_deduction_seven_objects        |      0|multiple_choice_grade|21.29|±  |  1.55|
+|bigbench_logical_deduction_three_objects        |      0|multiple_choice_grade|47.33|±  |  2.89|
+|bigbench_movie_recommendation                   |      0|multiple_choice_grade|39.80|±  |  2.19|
+|bigbench_navigate                               |      0|multiple_choice_grade|63.80|±  |  1.52|
+|bigbench_reasoning_about_colored_objects        |      0|multiple_choice_grade|59.05|±  |  1.10|
+|bigbench_ruin_names                             |      0|multiple_choice_grade|40.18|±  |  2.32|
+|bigbench_salient_translation_error_detection    |      0|multiple_choice_grade|46.69|±  |  1.58|
+|bigbench_snarks                                 |      0|multiple_choice_grade|65.19|±  |  3.55|
+|bigbench_sports_understanding                   |      0|multiple_choice_grade|72.41|±  |  1.42|
+|bigbench_temporal_sequences                     |      0|multiple_choice_grade|60.30|±  |  1.55|
+|bigbench_tracking_shuffled_objects_five_objects |      0|multiple_choice_grade|25.76|±  |  1.24|
+|bigbench_tracking_shuffled_objects_seven_objects|      0|multiple_choice_grade|17.43|±  |  0.91|
+|bigbench_tracking_shuffled_objects_three_objects|      0|multiple_choice_grade|47.33|±  |  2.89|
+
+Average: 45.71%
+
+Average score: 61.26%
+
+Elapsed time: 02:16:03