Add fp16/int8 weights

#1
by mkshing - opened

This PR enables to use this model with Colab Free plan by int8 quantization.
Here's the link to the demo in colab.

https://colab.research.google.com/github/mkshing/notebooks/blob/main/stabilityai_japanese_stablelm_alpha_7b.ipynb

mkshing changed pull request status to open
Stability AI org

Great, thanks for working on this!

leemeng changed pull request status to merged

Sign up or log in to comment