|
--- |
|
license: other |
|
language: |
|
- en |
|
base_model: |
|
- Qwen/Qwen2.5-72B-Instruct |
|
- anthracite-org/magnum-v1-72b |
|
--- |
|
|
|
# Magnum-v1-72b-Qwen2.5 |
|
|
|
A merge of the OG [QwenV2](https://huggingface.co/Qwen/Qwen2-72B-Instruct)-based [anthracite-org/magnum-v1-72b](https://huggingface.co/anthracite-org/magnum-v1-72b) with the new [Qwen/Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) |
|
|
|
|
|
## Model Details |
|
|
|
- **Base Model**: [Qwen/Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) |
|
- **Influence Model**: [anthracite-org/magnum-v1-72b](https://huggingface.co/anthracite-org/magnum-v1-72b) |
|
- **Method**: LoRA extraction from Magnum-v1 and then applied to Qwen2.5-Instruct |
|
|
|
## Process |
|
|
|
1. A LoRA was extracted from [anthracite-org/magnum-v1-72b](https://huggingface.co/anthracite-org/magnum-v1-72b) and [Qwen/Qwen2-72B-Instruct](https://huggingface.co/Qwen/Qwen2-72B-Instruct) |
|
2. This LoRA was then applied to [Qwen/Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct). |
|
3. The resulting model was merged to create this standalone version. |
|
|
|
## Prompt Template |
|
|
|
```py |
|
"""<|im_start|>user |
|
Hi there!<|im_end|> |
|
<|im_start|>assistant |
|
Nice to meet you!<|im_end|> |
|
<|im_start|>user |
|
Can I ask a question?<|im_end|> |
|
<|im_start|>assistant |
|
""" |
|
``` |
|
|
|
## Results |
|
|
|
It seems to have worked in my testing. Just as "creative" as the OG magnum-v1, and seems to have retained the improvements of Qwen2.5 |
|
(eg. it can zero-shot code a snake game in python and is aware of world events which happened after QwenV2 released) |
