Update README.md
Browse files
README.md
CHANGED
@@ -36,7 +36,7 @@ It is impratical for us to manually set specific configuration for each fine-tun
|
|
36 |
Thus, OpenCSG strained our brains to provide a relatively fair method to compare the fine-tuned models on HumanEval benchmark.
|
37 |
To simplify the comparison, we chosed the Pass@1 metric on python language, but our finetuning dataset includes samples in multi language.
|
38 |
|
39 |
-
**For fair, we evaluated the fine-tuned and
|
40 |
|
41 |
**Otherwise, we use greedy decoding method for each model during the evaluation.**
|
42 |
|
|
|
36 |
Thus, OpenCSG strained our brains to provide a relatively fair method to compare the fine-tuned models on HumanEval benchmark.
|
37 |
To simplify the comparison, we chosed the Pass@1 metric on python language, but our finetuning dataset includes samples in multi language.
|
38 |
|
39 |
+
**For fair, we evaluated the fine-tuned and original starcoder models only with the original cases' prompts, not including any other instruction else.**
|
40 |
|
41 |
**Otherwise, we use greedy decoding method for each model during the evaluation.**
|
42 |
|