add GGUF quants

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,75 @@
 ---
+base_model: nomic-ai/nomic-embed-text-v1.5
+inference: false
+language:
+  - en
 license: apache-2.0
+model_creator: Nomic
+model_name: nomic-embed-text-v1.5
+model_type: bert
+pipeline_tag: sentence-similarity
+quantized_by: Nomic
+tags:
+  - feature-extraction
+  - sentence-similarity
 ---
+
+# nomic-embed-text-v1.5 - GGUF
+
+Original model: [nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5)
+
+
+## Description
+
+This repo contains llama.cpp-compatible files for [nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5) in GGUF format.
+
+llama.cpp will default to 2048 tokens of context with these files. To use the full 8192 tokens that Nomic Embed is benchmarked on, you will have to choose a context extension method. The original model uses Dynamic NTK-Aware RoPE scaling, but that is not currently available in llama.cpp. A combination of YaRN and linear scaling is an acceptable substitute.
+
+These files were converted and quantized with llama.cpp commit [594fca3fe](https://github.com/ggerganov/llama.cpp/commit/594fca3fefe27b8e95cfb1656eb0e160ad15a793).
+
+## Example `llama.cpp` Command
+
+Compute a single embedding:
+```shell
+./embedding -ngl 99 -m nomic-embed-text-v1.5.f16.gguf -c 8192 -b 8192 --rope-scaling yarn --rope-freq-scale .75 -p 'search_query: What is TSNE?'
+```
+
+You can also submit a batch of texts to embed, as long as the total number of tokens does not exceed the context length. Only the first three embeddings are shown by the `embedding` example.
+
+texts.txt:
+```
+search_query: What is TSNE?
+search_query: Who is Laurens Van der Maaten?
+```
+
+Compute multiple embeddings:
+```shell
+./embedding -ngl 99 -m nomic-embed-text-v1.5.f16.gguf -c 8192 -b 8192 --rope-scaling yarn --rope-freq-scale .75 -f texts.txt
+```
+
+
+## Compatibility
+
+These files are compatible with llama.cpp as commit [ea9c8e114](https://github.com/ggerganov/llama.cpp/commit/ea9c8e11436ad50719987fa23a289c74b7b40d40) from 2/13/2024.
+
+
+## Provided Files
+
+The below table shows the mean squared error of the embeddings produced by these quantizations of Nomic Embed relative to the Sentence Transformers implementation.
+
+Name | Quant | Size | MSE 
+-----|-------|------|-----
+[nomic-embed-text-v1.5.Q2\_K.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q2_K.gguf) | Q2\_K | 48 MiB | 2.33e-03
+[nomic-embed-text-v1.5.Q3\_K\_S.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q3_K_S.gguf) | Q3\_K\_S | 57 MiB | 1.19e-03
+[nomic-embed-text-v1.5.Q3\_K\_M.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q3_K_M.gguf) | Q3\_K\_M | 65 MiB | 8.26e-04
+[nomic-embed-text-v1.5.Q3\_K\_L.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q3_K_L.gguf) | Q3\_K\_L | 69 MiB | 7.93e-04
+[nomic-embed-text-v1.5.Q4\_0.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q4_0.gguf) | Q4\_0 | 75 MiB | 6.32e-04
+[nomic-embed-text-v1.5.Q4\_K\_S.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q4_K_S.gguf) | Q4\_K\_S | 75 MiB | 6.71e-04
+[nomic-embed-text-v1.5.Q4\_K\_M.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q4_K_M.gguf) | Q4\_K\_M | 81 MiB | 2.42e-04
+[nomic-embed-text-v1.5.Q5\_0.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q5_0.gguf) | Q5\_0 | 91 MiB | 2.35e-04
+[nomic-embed-text-v1.5.Q5\_K\_S.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q5_K_S.gguf) | Q5\_K\_S | 91 MiB | 2.00e-04
+[nomic-embed-text-v1.5.Q5\_K\_M.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q5_K_M.gguf) | Q5\_K\_M | 95 MiB | 6.55e-05
+[nomic-embed-text-v1.5.Q6\_K.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q6_K.gguf) | Q6\_K | 108 MiB | 5.58e-05
+[nomic-embed-text-v1.5.Q8\_0.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.Q8_0.gguf) | Q8\_0 | 140 MiB | 5.79e-06
+[nomic-embed-text-v1.5.f16.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.f16.gguf) | F16 | 262 MiB | 4.21e-10
+[nomic-embed-text-v1.5.f32.gguf](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/blob/main/nomic-embed-text-v1.5.f32.gguf) | F32 | 262 MiB | 6.08e-11

nomic-embed-text-v1.5.Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:758227679b93e653eb712c442a16fcdf7d65b15a6eefd6ce8053edd7f42d82a4
+size 49361088

nomic-embed-text-v1.5.Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:139edcebb55846c0c7685b57e25c9758c67f9749a1276a0e886031b9a5bc63ff
+size 71593088

nomic-embed-text-v1.5.Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7140b9170efe9912ec4be3f19d4492050f7be8dc9492ac99a1892bcadca30e9c
+size 67169408

nomic-embed-text-v1.5.Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b13eb7c4ed0ec144435f2a47f49acc04ce79c3081b5bc72f5a717da0905c7a9
+size 59649152

nomic-embed-text-v1.5.Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01e3f351b0c697201d6901d38b29c9e0affa06c65b040359546c822198aa8f64
+size 77802880

nomic-embed-text-v1.5.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e3b6f681b81772c5d29b3b4f6b01c0e71ee64febba159ae182d3a238ccb63fb
+size 84106624

nomic-embed-text-v1.5.Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1acc092631aeb6fd52009c3d2b494fbee5c474ae6275e22bebb7c7d032a71b15
+size 78097792

nomic-embed-text-v1.5.Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca07f2eba2354436bb0be187c6df15a1e754975f1f10f299c6513914d95a0b71
+size 94888768

nomic-embed-text-v1.5.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b4795c9f018653bea1626d8f95cf09d0ee0fabcbc14dfdca3f7f9320b352bd3
+size 99588928

nomic-embed-text-v1.5.Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2c341f67341b9093a409710fb578b3f972f3c18ad2c8a0303cacf99529394cd
+size 94888768

nomic-embed-text-v1.5.Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6a20b714d35f2e2f9a1e418f08826ac9303ac998961c26c8634196658a13867
+size 113042528

nomic-embed-text-v1.5.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:880522894afda99389f2c979aed97c085d0f905eba61ef196307f95245754114
+size 146146432

nomic-embed-text-v1.5.f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4060a62a157f92458ac71dc610a8909a9ad7ba1f1b4aaa52f121e207c1f9c63
+size 274290560

nomic-embed-text-v1.5.f32.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab33b0bd5d1d3e993c37f9f580c8a17bbb17309997096c5d88308da2c6a4fa63
+size 547664768