Llama.cpp compatible. Thanks!
#2
by
boeroboy
- opened
Nice - this is the only version that seems to work out of the box for Llama.cpp currently (at least for me). It occasionally spazzes out but generally works well for basic testing.
Thanks for uploading.
Awesome! Glad it is working. So for anyone who wants to experiment this and 7b-it version on Google colab can refer this sample notebook: https://colab.research.google.com/drive/1uVIz_Y6mdjjRgnfC7X2jplHz7aUiQw0I?usp=sharing
rahuldshetty
changed discussion status to
closed
Actually this was just a temp issue in earlier releases of Llama.cpp. Fixed now but thanks your model helped get me started!
https://github.com/ggerganov/llama.cpp/issues/5635