---
library_name: transformers
license: apache-2.0
language:
- en
---

# EdgeRunner-Tactical-7B


![image/png](https://cdn-uploads.huggingface.co/production/uploads/668ed3dcd857a9ca47edb75c/tSyuw39VtmEqvC_wptTDf.png)

## Introduction

EdgeRunner-Tactical-7B is a powerful and efficient language model for the edge. Our mission is to build Generative AI for the edge that is safe, secure, and transparent. To that end, the EdgeRunner team is proud to release EdgeRunner-Tactical-7B, the most powerful language model for its size to date.

EdgeRunner-Tactical-7B is a 7 billion parameter language model that delivers powerful performance while demonstrating the potential of running state-of-the-art (SOTA) models at the edge.

## Highlights

- 7 billion parameters that balance power and efficiency
- SOTA performance within the 7B model range
- Initialized from Qwen2-Instruct, leveraging prior advancements
- [Self-Play Preference Optimization](https://arxiv.org/abs/2405.00675) (SPPO) applied for continuous training and alignment
- Competitive performance on several benchmarks with Meta’s Llama-3-70B, Mixtral 8x7B, and Yi 34B
- Context length of 128K tokens, ideal for extensive conversations and large-scale text tasks

## Quickstart

Below is a code snippet to show you how to load the tokenizer and model, and how to generate contents.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained(
    "edgerunner-ai/EdgeRunner-Tactical-7B",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("edgerunner-ai/EdgeRunner-Tactical-7B")

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
```

## Example Outputs

### Create a Quantum Future:

<img src="https://cdn-uploads.huggingface.co/production/uploads/633fe629f81b9d10135fefda/3b00jTWhIV_5OWxtW6zFI.png" width="95%">

### Ask for a structured JSON output:

<img src="https://cdn-uploads.huggingface.co/production/uploads/633fe629f81b9d10135fefda/CzW5qUh9tAkZV8k8Xs4nm.png" width="95%">


## Evaluation

In this section, we report the results for EdgeRunner-Tactical-7B models on standard automatic benchmarks. Below are the results.

### Arena-Hard Benchmark


| Model                           | Score     |   CI                | Avg Tokens  |
|:--------------------------------|:----------|:--------------------|:------------|
| gpt-4-turbo-2024-04-09          | 82.63     | (-1.71, \+1.57)     | 662.0       |
| claude-3-5-sonnet-20240620      | 79.35     | (-1.45, \+2.06)     | 567.0       |
| gpt-4o-2024-05-13               | 79.21     | (-1.50, \+1.66)     | 696.0       |
| gpt-4-0125-preview              | 77.96     | (-2.12, \+1.63)     | 619.0       |
| gpt-4o-mini                     | 74.94     | (-2.40, \+1.75)     | 668.0       |
| gemini-1.5-pro-api-0514         | 71.96     | (-2.39, \+2.10)     | 676.0       |
| yi-large-preview                | 71.48     | (-2.03, \+3.14)     | 720.0       |
| glm-4-0520                      | 63.84     | (-2.72, \+1.81)     | 636.0       |
| yi-large                        | 63.7      | (-2.72, \+2.21)     | 626.0       |
| deepseek-coder-v2               | 62.3      | (-1.73, \+2.41)     | 578.0       |
| claude-3-opus-20240229          | 60.36     | (-2.84, \+2.75)     | 541.0       |
| gemma-2-27b-it                  | 57.51     | (-2.35, \+2.46)     | 577.0       |
| glm-4-0116                      | 55.72     | (-2.51, \+2.31)     | 622.0       |
| gemini-1.5-pro-api-0409-preview | 53.37     | (-2.53, \+1.89)     | 478.0       |
| glm-4-air                       | 50.88     | (-2.60, \+2.45)     | 619.0       |
| gpt-4-0314                      | 50.0      | (-0.00, \+0.00)     | 423.0       |
| gemini-1.5-flash-api-0514       | 49.61     | (-2.93, \+2.85)     | 642.0       |
| qwen2-72b-instruct              | 46.86     | (-2.51, \+2.22)     | 515.0       |
| claude-3-sonnet-20240229        | 46.8      | (-2.94, \+2.35)     | 552.0       |
| llama-3-70b-instruct            | 46.57     | (-2.00, \+2.66)     | 591.0       |
| claude-3-haiku-20240307         | 41.47     | (-2.15, \+2.65)     | 505.0       |
| gpt-4-0613                      | 37.9      | (-2.21, \+2.51)     | 354.0       |
| mistral-large-2402              | 37.71     | (-1.88, \+2.77)     | 400.0       |
| **EdgeRunner-Tactical-7B**      | **37.47** | **(-2.74, \+2.57)** | **721.0**   |
| mixtral-8x22b-instruct-v0.1     | 36.36     | (-2.61, \+2.60)     | 430.0       |
| qwen1.5-72b-chat                | 36.12     | (-2.81, \+2.39)     | 474.0       |
| phi-3-medium-4k-instruct        | 33.37     | (-2.02, \+2.25)     | 517.0       |
| mistral-medium                  | 31.9      | (-2.54, \+2.13)     | 485.0       |
| phi-3-small-8k-instruct         | 29.77     | (-2.16, \+2.02)     | 568.0       |
| mistral-next                    | 27.37     | (-1.90, \+1.99)     | 297.0       |
| qwen2-7b-instruct           | 25.2  | (-1.55, \+2.46) | 618.0  |
| gpt-3.5-turbo-0613              | 24.82     | (-2.15, \+1.90)     | 401.0       |
| claude-2.0                      | 23.99     | (-1.90, \+1.75)     | 295.0       |
| Arcee-Spark                 | 23.52 | (-2.03, \+1.73) | 622.0   |
| mixtral-8x7b-instruct-v0.1      | 23.4      | (-1.87, \+1.73)     | 457.0       |
| gpt-3.5-turbo-0125              | 23.34     | (-1.46, \+2.31)     | 329.0       |
| yi-34b-chat                     | 23.15     | (-2.15, \+1.85)     | 611.0       |
| starling-lm-7b-beta             | 23.01     | (-1.98, \+1.71)     | 530.0       |
| claude-2.1                      | 22.77     | (-1.48, \+2.38)     | 290.0       |
| llama-3-8b-instruct             | 20.56     | (-1.65, \+2.09)     | 585.0       |
| gpt-3.5-turbo-1106              | 18.87     | (-1.79, \+2.34)     | 285.0       |
| gpt-3.5-turbo-0314              | 18.05     | (-1.47, \+2.09)     | 334.0       |
| gemini-pro                      | 17.8      | (-1.65, \+1.54)     | 322.0       |
| phi-3-mini-128k-instruct        | 15.43     | (-1.71, \+1.60)     | 609.0       |
| mistral-7b-instruct             | 12.57     | (-1.58, \+1.54)     | 541.0       |
| gemma-1.1-7b-it                 | 12.09     | (-1.35, \+1.56)     | 341.0       |
| llama-2-70b-chat                | 11.55     | (-1.18, \+1.27)     | 595.0       |
| vicuna-33b                      | 8.63      | (-0.88, \+1.28)     | 451.0       |
| gemma-7b-it                     | 7.47      | (-1.05, \+1.09)     | 378.0       |
| gemma-1.1-2b-it                 | 3.37      | (-0.67, \+0.70)     | 316.0       |
| gemma-2b-it                     | 3.0       | (-0.68, \+0.62)     | 369.0       |



### InfiniteBench

| Task Name        | GPT-4  | YaRN-Mistral-7B | Kimi-Chat | Claude 2 | Yi-6B-200K | Yi-34B-200K | Chatglm3-6B-128K | EdgeRunner-Tactical-7B | Qwen2-7B-Instruct |
|:-----------------|:-------|:----------------|:----------|:---------|:-----------|:------------|:-----------------|:-----------------------|:------------------|
| Retrieve.PassKey | 100%   | 92.71%          | 98.14%    | 97.80%   | 100.00%    | 100.00%     | 92.20%           | 100%                   | 100%              |
| Retrieve.Number  | 100%   | 56.61%          | 95.42%    | 98.14%   | 94.92%     | 100.00%     | 80.68%           | 100%                   | 99.83%            |
| Retrieve.KV      | 89.00% | < 5%            | 53.60%    | 65.40%   | < 5%       | < 5%        | < 5%             | 2.2%                   | 1.8%              |
| En.Sum           | 14.73% | 9.09%           | 17.96%    | 14.50%   | < 5%       | < 5%        | < 5%             | 33.07%                 | 29.13%            |
| En.QA            | 22.44% | 9.55%           | 16.52%    | 11.97%   | 9.20%      | 12.17%      | < 5%             | 3.4%                   | 9.09%             |
| En.MC            | 67.25% | 27.95%          | 72.49%    | 62.88%   | 36.68%     | 38.43%      | 10.48%           | 66.81%                 | 66.37%            |
| En.Dia           | 8.50%  | 7.50%           | 11.50%    | 46.50%   | < 5%       | < 5%        | < 5%             | 29%                    | 17%               |
| Zh.QA            | 25.96% | 16.98%          | 17.93%    | 9.64%    | 15.07%     | 13.61%      | < 5%             | 4.6%                   | 11.14%            |
| Code.Debug       | 37.06% | < 5%            | 17.77%    | < 5%     | 9.14%      | 13.96%      | 7.36%            | 22.08%                 | 24.61%            |
| Code.Run         | 23.25% | < 5%            | < 5%      | < 5%     | < 5%       | < 5%        | < 5%             | 0%                     | 0.5%              |
| Math.Calc        | < 5%   | < 5%            | < 5%      | < 5%     | < 5%       | < 5%        | < 5%             | 0%                     | 0%                |
| Math.Find        | 60.00% | 17.14%          | 12.57%    | 32.29%   | < 5%       | 25.71%      | 7.71%            | 29.14%                 | 31.42%            |


### GSM@ZeroEval

| Model                         | Acc       | No answer | Reason Lens |
|:------------------------------|:----------|:----------|:------------|
| Llama-3.1-405B-Instruct-Turbo | 95.91     | 0.08      | 365.07      |
| claude-3-5-sonnet-20240620    | 95.6      | 0         | 465.19      |
| claude-3-opus-20240229        | 95.6      | 0         | 410.62      |
| gpt-4o-2024-05-13             | 95.38     | 0         | 479.98      |
| gpt-4o-mini-2024-07-18        | 94.24     | 0         | 463.71      |
| deepseek-chat                 | 93.93     | 0         | 495.52      |
| deepseek-coder                | 93.78     | 0         | 566.89      |
| gemini-1.5-pro                | 93.4      | 0         | 389.17      |
| Meta-Llama-3-70B-Instruct     | 93.03     | 0         | 352.05      |
| Qwen2-72B-Instruct            | 92.65     | 0         | 375.96      |
| claude-3-sonnet-20240229      | 91.51     | 0         | 762.69      |
| gemini-1.5-flash              | 91.36     | 0         | 344.61      |
| gemma-2-27b-it@together       | 90.22     | 0         | 364.68      |
| claude-3-haiku-20240307       | 88.78     | 0         | 587.65      |
| gemma-2-9b-it                 | 87.41     | 0         | 394.83      |
| reka-core-20240501            | 87.41     | 0.08      | 414.7       |
| Athene-70B                    | 86.66     | 0.3       | 253.53      |
| Yi-1.5-34B-Chat               | 84.08     | 0.08      | 553.47      |
| Llama-3.1-8B-Instruct     | 82.87 | 0.45  | 414.19  |
| Mistral-Nemo-Instruct-2407    | 82.79     | 0         | 349.81      |
| yi-large-preview              | 82.64     | 0         | 514.25      |
| **EdgeRunner-Tactical-7B**    | **81.12** | **0.08**  | **615.89**  |
| gpt-3.5-turbo-0125            | 80.36     | 0         | 350.97      |
| command-r-plus                | 80.14     | 0.08      | 294.08      |
| Qwen2-7B-Instruct         | 80.06 | 0     | 452.6   |
| yi-large                      | 80.06     | 0         | 479.87      |
| Yi-1.5-9B-Chat                | 76.42     | 0.08      | 485.39      |
| Phi-3-mini-4k-instruct        | 75.51     | 0         | 462.53      |
| reka-flash-20240226           | 74.68     | 0.45      | 460.06      |
| Mixtral-8x7B-Instruct-v0.1    | 70.13     | 2.27      | 361.12      |
| command-r                     | 52.99     | 0         | 294.43      |
| Qwen2-1.5B-Instruct           | 43.37     | 4.78      | 301.67      |



### MMLU-REDUX@ZeroEval

| Model                         | Acc       | No answer | Reason Lens |
|:------------------------------|:----------|:----------|:------------|
| gpt-4o-2024-05-13             | 88.01     | 0.14      | 629.79      |
| claude-3-5-sonnet-20240620    | 86        | 0.18      | 907.1       |
| Llama-3.1-405B-Instruct-Turbo | 85.64     | 0.76      | 449.71      |
| gpt-4-turbo-2024-04-09        | 85.31     | 0.04      | 631.38      |
| gemini-1.5-pro                | 82.76     | 1.94      | 666.7       |
| claude-3-opus-20240229        | 82.54     | 0.58      | 500.35      |
| yi-large-preview              | 82.15     | 0.14      | 982.6       |
| gpt-4-0314                    | 81.64     | 0.04      | 397.22      |
| Qwen2-72B-Instruct            | 81.61     | 0.29      | 486.41      |
| gpt-4o-mini-2024-07-18        | 81.5      | 0.07      | 526         |
| yi-large                      | 81.17     | 0         | 774.85      |
| deepseek-chat                 | 80.81     | 0.11      | 691.91      |
| deepseek-coder                | 79.63     | 0.14      | 704.72      |
| Meta-Llama-3-70B-Instruct     | 78.01     | 0.11      | 520.77      |
| gemini-1.5-flash              | 77.36     | 1.26      | 583.45      |
| Athene-70B                    | 76.64     | 0.04      | 552.61      |
| reka-core-20240501            | 76.42     | 0.76      | 701.67      |
| gemma-2-27b-it@together       | 75.67     | 0.61      | 446.51      |
| claude-3-sonnet-20240229      | 74.87     | 0.07      | 671.75      |
| gemma-2-9b-it@nvidia          | 72.82     | 0.76      | 499         |
| Yi-1.5-34B-Chat               | 72.79     | 1.01      | 620.1       |
| claude-3-haiku-20240307       | 72.32     | 0.04      | 644.59      |
| Phi-3-mini-4k-instruct        | 70.34     | 0.43      | 677.09      |
| command-r-plus                | 68.61     | 0         | 401.51      |
| gpt-3.5-turbo-0125            | 68.36     | 0.04      | 357.92      |
| **EdgeRunner-Tactical-7B**    | **67.71** | **0.65**  | **917.6**   |
| Llama-3.1-8B-Instruct     | 67.13 | 3.38  | 399.54  |
| Qwen2-7B-Instruct         | 66.92 | 0.72  | 533.15  |
| Mistral-Nemo-Instruct-2407    | 66.88     | 0.47      | 464.19      |
| Yi-1.5-9B-Chat                | 65.05     | 4.61      | 542.87      |
| reka-flash-20240226           | 64.72     | 0.32      | 659.25      |
| Mixtral-8x7B-Instruct-v0.1    | 63.17     | 5.51      | 324.31      |
| Meta-Llama-3-8B-Instruct      | 61.66     | 0.97      | 600.81      |
| command-r                     | 61.12     | 0.04      | 382.23      |
| Qwen2-1.5B-Instruct           | 41.11     | 7.74      | 280.56      |

### WildBench

| Model                      | WB_Elo      | RewardScore_Avg | task_macro_reward.K=-1 | Length      |
|:---------------------------|:------------|:----------------|:-----------------------|:------------|
| gpt-4o-2024-05-13          | 1248.12     | 50.05           | 40.80                  | 3723.52     |
| claude-3-5-sonnet-20240620 | 1229.76     | 46.16           | 37.63                  | 2911.85     |
| gpt-4-turbo-2024-04-09     | 1225.29     | 46.19           | 37.17                  | 3093.17     |
| gpt-4-0125-preview         | 1211.44     | 41.24           | 30.20                  | 3335.64     |
| gemini-1.5-pro             | 1209.23     | 45.27           | 37.59                  | 3247.97     |
| yi-large-preview           | 1209.00     | 46.92           | 38.54                  | 3512.68     |
| claude-3-opus-20240229     | 1206.56     | 37.03           | 22.35                  | 2685.98     |
| Meta-Llama-3-70B-Instruct  | 1197.72     | 35.15           | 22.54                  | 3046.64     |
| Athene-70B                 | 1197.41     | 29.77           | 0.00                   | 3175.14     |
| deepseek-coder-v2          | 1194.11     | 29.39           | 11.38                  | 2795.31     |
| gpt-4o-mini-2024-07-18     | 1192.43     | 28.57           | 0.00                   | 3648.13     |
| yi-large                   | 1191.88     | 33.35           | 17.77                  | 3095.34     |
| gemini-1.5-flash           | 1190.30     | 37.45           | 26.04                  | 3654.40     |
| deepseek-v2-chat-0628      | 1188.07     | 27.00           | 0.00                   | 3252.38     |
| gemma-2-9b-it-SimPO        | 1184.67     | 26.64           | 0.00                   | 4277.67     |
| gemma-2-9b-it-DPO          | 1182.43     | 26.61           | 0.00                   | 3982.63     |
| nemotron-4-340b-instruct   | 1181.77     | 33.76           | 19.85                  | 2754.01     |
| claude-3-sonnet-20240229   | 1179.81     | 28.09           | 10.70                  | 2670.24     |
| deepseekv2-chat            | 1178.76     | 30.41           | 12.60                  | 2896.97     |
| gemma-2-27b-it@together    | 1178.34     | 24.27           | 0.00                   | 2924.55     |
| Qwen2-72B-Instruct         | 1176.75     | 24.77           | 5.03                   | 2856.45     |
| reka-core-20240501         | 1173.85     | 31.48           | 17.06                  | 2592.59     |
| Mistral-Nemo-Instruct-2407 | 1165.29     | 22.19           | 0.00                   | 3318.21     |
| Yi-1.5-34B-Chat            | 1163.69     | 30.83           | 16.06                  | 3523.56     |
| **EdgeRunner-Tactical-7B** | **1162.88** | **22.26**       | **0.00**               | **3754.66** |
| claude-3-haiku-20240307    | 1160.56     | 16.30           | -6.30                  | 2601.03     |
| mistral-large-2402         | 1159.72     | 13.27           | -12.36                 | 2514.98     |
| deepseek-v2-coder-0628     | 1155.97     | 22.83           | 0.00                   | 2580.18     |
| gemma-2-9b-it              | 1154.30     | 21.35           | 0.00                   | 2802.89     |
| command-r-plus             | 1153.15     | 16.58           | -3.60                  | 3293.81     |
| glm-4-9b-chat              | 1152.68     | 20.71           | 2.33                   | 3692.04     |
| Qwen1.5-72B-Chat-greedy    | 1151.97     | 20.83           | 1.72                   | 2392.36     |
| Yi-1.5-9B-Chat             | 1151.43     | 21.80           | 4.93                   | 3468.23     |
| Meta-Llama-3-8B-Instruct   | 1140.76     | 6.72            | -15.76                 | 2975.19     |
| Qwen2-7B-Instruct      | 1137.66 | 16.20       | 0.00               | 3216.43 |
| Starling-LM-7B-beta-ExPO   | 1137.58     | 11.28           | -9.01                  | 2835.83     |
| Hermes-2-Theta-Llama-3-8B  | 1135.99     | 3.18            | -23.28                 | 2742.17     |
| Llama-3.1-8B-Instruct  | 1135.42 | 16.38       | 0.00               | 3750.60 |



### AlpacaEval 2.0

| Model                       | Length Controlled Winrate | Win Rate  | N Total | Avg Length |
|:----------------------------|:--------------------------|:----------|:--------|:-----------|
| gpt-4o-2024-05-13           | 57.46                     | 51.33     | 805     | 1873       |
| gpt-4-turbo-2024-04-09      | 55.02                     | 46.12     | 805     | 1802       |
| claude-3-5-sonnet-20240620  | 52.37                     | 40.56     | 805     | 1488       |
| yi-large-preview            | 51.89                     | 57.47     | 805     | 2335       |
| gpt4\_1106\_preview         | 50.0                      | 50.0      | 805     | 2049       |
| Qwen1.5-110B-Chat           | 43.91                     | 33.78     | 805     | 1631       |
| claude-3-opus-20240229      | 40.51                     | 29.11     | 805     | 1388       |
| gpt4                        | 38.13                     | 23.58     | 805     | 1365       |
| Qwen1.5-72B-Chat            | 36.57                     | 26.5      | 805     | 1549       |
| gpt4\_0314                  | 35.31                     | 22.07     | 805     | 1371       |
| Meta-Llama-3-70B-Instruct   | 34.42                     | 33.18     | 805     | 1919       |
| **EdgeRunner-Tactical-7B**  | **34.41**                 | **51.28** | **805** | **2735**   |
| mistral-large-2402          | 32.65                     | 21.44     | 805     | 1362       |
| Mixtral-8x22B-Instruct-v0.1 | 30.88                     | 22.21     | 805     | 1445       |
| gpt4\_0613                  | 30.18                     | 15.76     | 805     | 1140       |
| mistral-medium              | 28.61                     | 21.86     | 805     | 1500       |
| claude-2                    | 28.16                     | 17.19     | 805     | 1069       |
| Samba-CoE-v0.2              | 27.62                     | 21.85     | 805     | 1469       |
| internlm2-chat-20b-ExPO     | 27.23                     | 46.19     | 805     | 3335       |
| Yi-34B-Chat                 | 27.19                     | 29.66     | 805     | 2123       |
| Starling-LM-7B-beta-ExPO    | 26.41                     | 29.6      | 805     | 2215       |
| Llama-3.1-8B-Instruct   | 26.41                 | 30.32 | 805 | 2171   |
| Snorkel-Mistral-PairRM-DPO  | 26.39                     | 30.22     | 804     | 2736       |
| Arcee-Spark             | 25.58                 | 26.19 | 805 | 2002   |
| claude-2.1                  | 25.25                     | 15.73     | 805     | 1096       |
| gemini-pro                  | 24.38                     | 18.18     | 805     | 1456       |
| Qwen1.5-14B-Chat            | 23.9                      | 18.65     | 805     | 1607       |
| Mixtral-8x7B-Instruct-v0.1  | 23.69                     | 18.26     | 805     | 1465       |
| Meta-Llama-3-8B-Instruct    | 22.92                     | 22.57     | 805     | 1899       |
| Samba-CoE-v0.1              | 22.87                     | 16.84     | 805     | 1316       |
| gpt-3.5-turbo-0613          | 22.35                     | 14.1      | 805     | 1331       |
| Qwen2-7B-Instruct       | 21.51                 | 18.93 | 805 | 1793   |
| gpt-3.5-turbo-1106          | 19.3                      | 9.18      | 805     | 796        |
| internlm2-chat-20b-ppo      | 18.75                     | 21.75     | 805     | 2373       |
| claude-2.1\_concise         | 18.21                     | 9.23      | 805     | 573        |
| gpt-3.5-turbo-0301          | 18.09                     | 9.62      | 805     | 827        |
| deepseek-llm-67b-chat       | 17.84                     | 12.09     | 805     | 1151       |
| vicuna-33b-v1.3             | 17.57                     | 12.71     | 805     | 1479       |
| Mistral-7B-Instruct-v0.2    | 17.11                     | 14.72     | 805     | 1676       |
| OpenHermes-2.5-Mistral-7B   | 16.25                     | 10.34     | 805     | 1107       |
| Qwen1.5-7B-Chat             | 14.75                     | 11.77     | 805     | 1594       |