Hugging Face's logo

Spaces:
codeparrot
/
code-generation-models
Running

App Files Community
10
loubnabnl HF staff commited on
Commit
6998fd8
•
1 Parent(s): a342a9a

remove files

Browse files
Files changed (2) hide show
  1. datasets/.ipynb_checkpoints/incoder-checkpoint.txt +0 -0
  2. evaluation/.ipynb_checkpoints/intro-checkpoint.txt +0 -1
datasets/.ipynb_checkpoints/incoder-checkpoint.txt DELETED
File without changes
evaluation/.ipynb_checkpoints/intro-checkpoint.txt DELETED
@@ -1 +0,0 @@
1
- A popular evaluatrion framework for code generation models is the [pass@k](https://huggingface.co/metrics/code_eval) metric on [HumanEval](https://huggingface.co/datasets/openai_humaneval) dataset, which was introduced in [Codex paper](https://arxiv.org/pdf/2107.03374v2.pdf). The dataset includes 164 handwritten programming problems. In the pass@k metric, k code samples are generated per problem, a problem is considered solved if any sample passes the unit tests and the total fraction of problems solved is reported. Below are some examples for the selcted models.