Update app.py
Browse files
app.py
CHANGED
@@ -82,8 +82,8 @@ with demo:
|
|
82 |
gr.Markdown(
|
83 |
"""Notes:
|
84 |
<ul>
|
85 |
-
<li> Throughputs and peak memory usage are measured using <a href="https://github.com/huggingface/optimum-benchmark/tree/main">Optimum-Benchmark</a> which powers <a href="https://huggingface.co/spaces/optimum/llm-perf-leaderboard"
|
86 |
-
<li> All models were evaluated with the <a href="https://github.com/bigcode-project/bigcode-evaluation-harness/tree/main"
|
87 |
<li> HumanEval-Python, reports the pass@1 on HumanEval, the rest is from MultiPL-E benchmark.</li>
|
88 |
<li> Average score is the average pass@1 over all languages. During the averaging, we exclude languages with a pass@1 score lower than 1 for each model.</li>
|
89 |
<li> #Languages column represents the number of programming languages included during the pretraining.
|
|
|
82 |
gr.Markdown(
|
83 |
"""Notes:
|
84 |
<ul>
|
85 |
+
<li> Throughputs and peak memory usage are measured using <a href="https://github.com/huggingface/optimum-benchmark/tree/main">Optimum-Benchmark</a> which powers <a href="https://huggingface.co/spaces/optimum/llm-perf-leaderboard">Open LLM-Perf Leaderboard</a>. (0 throughput corresponds to OOM).</li>
|
86 |
+
<li> All models were evaluated with the <a href="https://github.com/bigcode-project/bigcode-evaluation-harness/tree/main">bigcode-evaluation-harness</a> with top-p=0.95, temperature=0.2 and n_samples=50.</li>
|
87 |
<li> HumanEval-Python, reports the pass@1 on HumanEval, the rest is from MultiPL-E benchmark.</li>
|
88 |
<li> Average score is the average pass@1 over all languages. During the averaging, we exclude languages with a pass@1 score lower than 1 for each model.</li>
|
89 |
<li> #Languages column represents the number of programming languages included during the pretraining.
|