update
Browse files- evaluation/intro.txt +1 -1
evaluation/intro.txt
CHANGED
@@ -72,4 +72,4 @@ scores
|
|
72 |
|
73 |
````
|
74 |
|
75 |
-
If we take a closer look at the unit test results for each candidate solution in the three tasks, we find that only 3 passed the test which corresponds to `1/30 = 0.333`, our pass@1, the scores pass@5 and pass@10 are higher, because the more samples we select from the candidate solutions, the more likely we are to include the correct solution. Without surprise pass@10 is '2/3=0.73': if we select all candidates two tasks out of three get solved.
|
|
|
72 |
|
73 |
````
|
74 |
|
75 |
+
If we take a closer look at the unit test results for each candidate solution in the three tasks, we find that only 3 passed the test which corresponds to `1/30 = 0.333`, our pass@1, the scores pass@5 and pass@10 are higher, because the more samples we select from the candidate solutions, the more likely we are to include the correct solution. Without surprise pass@10 is '2/3=0.73': if we select all candidates two tasks out of three get solved.
|