Is there anyone who reproduce this metric? I successfully reproduce pass@1 in BigCode leaderboard, but pass@10 in the original paper isn't.The paper said 0.59, but my experimental result is 0.491.
· Sign up or log in to comment
