|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
|
|
<style> |
|
img{ |
|
user-select: none; |
|
transition: all 0.2s ease; |
|
border-radius: .5rem; |
|
} |
|
img:hover{ |
|
transform: rotate(2deg); |
|
filter: invert(100%); |
|
} |
|
@import url('https://fonts.googleapis.com/css2?family=Vollkorn:ital,wght@0,400..900;1,400..900&display=swap'); |
|
</style> |
|
|
|
<div style="background-color: transparent; border-radius: .5rem; padding: 2rem; font-family: monospace; font-size: .85rem; text-align: justify;"> |
|
|
|
 |
|
|
|
**September Update** - this is the official model used in dot, keep in mind, none of these models use specific prompts, you might need to fine-tune them to use them as chatbots. |
|
|
|
#### benchmarks |
|
|
|
zero-shot evaluations performed on current sota ~0.5b models against the best language model below 2b parameters. |
|
|
|
| Parameters | Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande | Average | |
|
| -----------|--------------------------------|-------|-------|-----------|--------|------------|---------| |
|
| 0.5b | qwen2 |**0.4413**| 0.2892| 0.4905 | 0.6931 | 0.5699 | 0.4968 | |
|
| 0.6b | mobilellm | - | 0.3580| 0.5590 | 0.7230 | 0.5860 | - | |
|
| 0.5b | danube3 | 0.2481| 0.3618| 0.6046 | 0.7378 | 0.6101 | 0.5125 | |
|
| 0.5b | palmer |0.2617|**0.3729**|**0.6288**|**0.7437**| **0.6227** |**0.5260**| |
|
| 1.7b | smollm |0.2765|0.4626| 0.6574 | 0.7606 | 0.6093 | 0.5533 | |
|
|
|
#### supporters |
|
|
|
<a href="https://ko-fi.com/appvoid" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 34px !important; margin-top: -4px;width: 128px !important; filter: contrast(2) grayscale(100%) brightness(100%);" ></a> |
|
</div> |
