nm-testing
/

SparseLlama-3-8B-pruned_50.2of4-FP8

@@ -5,18 +5,18 @@ tags:
 - vllm
 ---
-# Meta-Llama-3-8B-pruned_50.2of4-FP8
 This repo contains model files for a 2:4 (N:M) sparse [Meta-Llama-3-8B](meta-llama/Meta-Llama-3-8B) model pruned in one-shot with [SparseGPT](https://arxiv.org/abs/2301.00774), and then additionally retrained with the [SquareHead](https://arxiv.org/abs/2310.06927) knowledge distillation while maintaining the 2:4 sparsity mask.
 It was then quantized using [AutoFP8](https://github.com/neuralmagic/AutoFP8) to FP8 weights and activations with per-tensor scales, calibrated on UltraChat2k.
-**Note:** The unquantized [Meta-Llama-3-8B-pruned_50.2of4-FP8](https://huggingface.co/nm-testing/SparseLlama-3-8B-pruned_50.2of4) is still a work in progress and subject to change. This FP8 model will be updated once the unquantized model is updated too.
 ## Evaluation Benchmark Results
 Model evaluation results obtained via [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) following the configuration of [Open LLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard).
-| Benchmark                                      | Meta-Llama-3-8B  | Meta-Llama-3-8B-pruned_50.2of4 | Meta-Llama-3-8B-pruned_50.2of4-FP8<br>(this model) |
 |:----------------------------------------------:|:-----------:|:-----------------------------:|:-----------------------------:|
 | [ARC-c](https://arxiv.org/abs/1911.01547)<br> 25-shot     | 59.47%       | 57.76%           | 58.02%            |
 | [MMLU](https://arxiv.org/abs/2009.03300)<br> 5-shot       | 65.29%       | 60.44%           | 60.71%            |

 - vllm
 ---
+# SparseLlama-3-8B-pruned_50.2of4-FP8
 This repo contains model files for a 2:4 (N:M) sparse [Meta-Llama-3-8B](meta-llama/Meta-Llama-3-8B) model pruned in one-shot with [SparseGPT](https://arxiv.org/abs/2301.00774), and then additionally retrained with the [SquareHead](https://arxiv.org/abs/2310.06927) knowledge distillation while maintaining the 2:4 sparsity mask.
 It was then quantized using [AutoFP8](https://github.com/neuralmagic/AutoFP8) to FP8 weights and activations with per-tensor scales, calibrated on UltraChat2k.
+**Note:** The unquantized [SparseLlama-3-8B-pruned_50.2of4-FP8](https://huggingface.co/nm-testing/SparseLlama-3-8B-pruned_50.2of4) is still a work in progress and subject to change. This FP8 model will be updated once the unquantized model is updated too.
 ## Evaluation Benchmark Results
 Model evaluation results obtained via [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) following the configuration of [Open LLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard).
+| Benchmark                                      | Meta-Llama-3-8B  | SparseLlama-3-8B-pruned_50.2of4 | SparseLlama-3-8B-pruned_50.2of4-FP8<br>(this model) |
 |:----------------------------------------------:|:-----------:|:-----------------------------:|:-----------------------------:|
 | [ARC-c](https://arxiv.org/abs/1911.01547)<br> 25-shot     | 59.47%       | 57.76%           | 58.02%            |
 | [MMLU](https://arxiv.org/abs/2009.03300)<br> 5-shot       | 65.29%       | 60.44%           | 60.71%            |