3x7B AWQ
Collection
Mixture of experts 3 x 7B.
•
5 items
•
Updated
ConvexAI/Solutus-3x7B AWQ
- Model creator: ConvexAI
- Original model: Solutus-3x7B
Model Summary
A model to test how MoE will route without square expansion.
The config looks like this...(detailed version is in the files and versions):
- cognitivecomputations/WestLake-7B-v2-laser - base
- mlabonne/Daredevil-7B - expert #1
- cognitivecomputations/WestLake-7B-v2-laser - expert #2
- mlabonne/Daredevil-7B - expert #3
- Downloads last month
- 15
Model tree for solidrust/Solutus-3x7B-AWQ
Base model
ConvexAI/Solutus-3x7BCollection including solidrust/Solutus-3x7B-AWQ
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard72.010
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard88.310
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard64.770
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard67.520
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard83.660
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard69.830