Update README.md

README.md

@@ -218,4 +218,71 @@ This version of the lm-evaluation-harness includes versions of ARC-Challenge and
    <td><strong>99.97%</strong>
    </td>
   </tr>
-</table>
+</table>
+
+
+### Reproduction
+
+The results were obtained using the following commands:
+
+#### MMLU
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks mmlu \
+  --num_fewshot 5 \
+  --batch_size auto
+```
+
+#### ARC-Challenge
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks arc_challenge_llama_3.1_instruct \
+  --apply_chat_template \
+  --num_fewshot 0 \
+  --batch_size auto
+```
+
+#### GSM-8K
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks gsm8k_cot_llama_3.1_instruct \
+  --apply_chat_template \
+  --num_fewshot 8 \
+  --batch_size auto
+```
+
+#### Hellaswag
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks hellaswag \
+  --num_fewshot 10 \
+  --batch_size auto
+```
+
+#### Winogrande
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks winogrande \
+  --num_fewshot 5 \
+  --batch_size auto
+```
+
+#### Hellaswag
+```
+lm_eval \
+  --model vllm \
+  --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks truthfulqa_mc \
+  --num_fewshot 0 \
+  --batch_size auto
+```