BerenMillidge commited on
Commit
7542ad8
1 Parent(s): a59589a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -1
README.md CHANGED
@@ -50,10 +50,18 @@ print((tokenizer.decode(outputs[0])))
50
 
51
  ## Performance
52
 
53
- Zamba2-1.2B achieves leading and state-of-the-art performance among models of <2B parameters and is competitive with some models of significantly greater size. Moreover, due to its unique hybrid SSM architecture, Zamba2-1.2B achieves extremely low inference latency and rapid generation with a significantly smaller memory footprint than comparable transformer based models.
54
 
55
  <img src="https://cdn-uploads.huggingface.co/production/uploads/64e40335c0edca443ef8af3e/t7et3jazHNvxKSkeorZuo.png" width="600"/>
56
 
 
 
 
 
 
 
 
 
57
 
58
  Zamba2-1.2B's high performance and small inference compute and memory footprint renders it an ideal generalist model for on-device applications.
59
 
 
50
 
51
  ## Performance
52
 
53
+ Zamba2-1.2B-Instruct achieves leading instruction-following and multi-turn chat performance for a model of its size and matches strong models significantly larger. For instance, Zamba2-1.2B-Instruct outperforms Gemma2-2B-Instruct, a very strong model over 2x its size.
54
 
55
  <img src="https://cdn-uploads.huggingface.co/production/uploads/64e40335c0edca443ef8af3e/t7et3jazHNvxKSkeorZuo.png" width="600"/>
56
 
57
+ | Model | Size | MT-Bench | IFEval |
58
+ |-------------|----|----|----|
59
+ | **Zamba2-1.2B-Instruct** | 1.2B | **59.53** | **41.45** |
60
+ | Gemma2-2B-Instruct | 2.7B | 51.69 | 42.20 |
61
+ | H2O-Danube-1.6B-Chat | 1.6B | 49.78 | 27.95 |
62
+ | StableLM-1.6B-Chat | 1.6B | 49.87 | 33.77 |
63
+ | SmolLM-1.7B-Instruct | 1.7B | 43.37 | 16.53 |
64
+ | Qwen2-1.5B-Instruct | 1.5B | N/A | 34.68 |
65
 
66
  Zamba2-1.2B's high performance and small inference compute and memory footprint renders it an ideal generalist model for on-device applications.
67