GGUF quants for https://huggingface.co/Salesforce/SFR-Embedding-Mistral
NOTE: This is a text embedding model used for feature extraction.

As of Feb. 9, 2024 this model is no. 1 on the MTEB leaderboard.
https://huggingface.co/spaces/mteb/leaderboard

I have an example of how to use llama.cpp to generate embeddings for sentence similarity on my repo:
https://github.com/dranger003/llama.cpp-dotnet

Layers	Context	Template
32	32768	Instruct: {task_description} Query: {query}

Downloads last month: 269

GGUF

Model size

7.24B params

Architecture

llama

4-bit

8-bit

16-bit

Inference Providers NEW

Sentence Similarity

This model is not currently available via any of the supported third-party Inference Providers, and the HF Inference API does not support gguf models with pipeline type sentence-similarity