Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
s3nh 
posted an update Feb 1
Post
GPU Poor POV: Quantization

Today I want to share with you my notebook plug and play code
which help me a lot through my quantization journey.
Hope youll find it interesting it could be a good starter point to
gguf some of your awesome models :)

Have a great day <3

https://s3nh.bearblog.dev/gpu-poor-pov-gguf-snippet/
deleted
This comment has been hidden
Ji-Ha
•
edited Feb 3

Thanks, I created a Colab notebook with this, trying to include the new SOTA quant methods for 2 and 3 bits by ikawrakow, but it seems not to be working. If someone could help it would be appreciated, thanks.

https://colab.research.google.com/drive/1qGmyHl2MXVScovQuu4vh2nheRXawZprj?usp=sharing

·
s3nh

Thats so cool, keep
Up the good work!

In this post