Loading gguf model for inference

by Rasi1610 - opened Jun 11, 2024

Jun 11, 2024

I'm new to this Quantized model ,Could anyone provide me the code for how to load this .gguf model and inference.
It would be really helpful

Jul 8, 2024

use ollama

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment