Hugging Face's logo

Clevyby
/
lynn-A2.7B-rp-v1-14.3b-32k-Q5_K_S-GGUF

Text Generation
Transformers
GGUF
English
qwen2_moe
Not-For-All-Audiences
conversational
Inference Endpoints
Model card Files Community
Clevyby mradermacher commited on
Commit
c985c24
•
0 Parent(s):

Duplicate from mradermacher/lynn-A2.7B-rp-v1-GGUF

Browse files

Co-authored-by: Michael Radermacher <mradermacher@users.noreply.huggingface.co>

Files changed (17) hide show
  1. .gitattributes +50 -0
  2. README.md +69 -0
  3. lynn-A2.7B-rp-v1.IQ3_M.gguf +3 -0
  4. lynn-A2.7B-rp-v1.IQ3_S.gguf +3 -0
  5. lynn-A2.7B-rp-v1.IQ3_XS.gguf +3 -0
  6. lynn-A2.7B-rp-v1.IQ4_XS.gguf +3 -0
  7. lynn-A2.7B-rp-v1.Q2_K.gguf +3 -0
  8. lynn-A2.7B-rp-v1.Q3_K_L.gguf +3 -0
  9. lynn-A2.7B-rp-v1.Q3_K_M.gguf +3 -0
  10. lynn-A2.7B-rp-v1.Q3_K_S.gguf +3 -0
  11. lynn-A2.7B-rp-v1.Q4_K_M.gguf +3 -0
  12. lynn-A2.7B-rp-v1.Q4_K_S.gguf +3 -0
  13. lynn-A2.7B-rp-v1.Q5_K_M.gguf +3 -0
  14. lynn-A2.7B-rp-v1.Q5_K_S.gguf +3 -0
  15. lynn-A2.7B-rp-v1.Q6_K.gguf +3 -0
  16. lynn-A2.7B-rp-v1.Q8_0.gguf +3 -0
  17. lynn-A2.7B-rp-v1.f16.gguf +3 -0
.gitattributes ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ lynn-A2.7B-rp-v1.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
37
+ lynn-A2.7B-rp-v1.f16.gguf filter=lfs diff=lfs merge=lfs -text
38
+ lynn-A2.7B-rp-v1.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
39
+ lynn-A2.7B-rp-v1.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ lynn-A2.7B-rp-v1.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
41
+ lynn-A2.7B-rp-v1.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
42
+ lynn-A2.7B-rp-v1.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ lynn-A2.7B-rp-v1.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
44
+ lynn-A2.7B-rp-v1.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ lynn-A2.7B-rp-v1.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
46
+ lynn-A2.7B-rp-v1.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
47
+ lynn-A2.7B-rp-v1.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
48
+ lynn-A2.7B-rp-v1.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ lynn-A2.7B-rp-v1.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
50
+ lynn-A2.7B-rp-v1.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: openlynn/lynn-A2.7B-rp-v1
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ license: cc-by-sa-4.0
7
+ quantized_by: mradermacher
8
+ tags:
9
+ - not-for-all-audiences
10
+ ---
11
+ ## About
12
+
13
+ <!-- ### quantize_version: 1 -->
14
+ <!-- ### output_tensor_quantised: 1 -->
15
+ <!-- ### convert_type: -->
16
+ <!-- ### vocab_type: -->
17
+ static quants of https://huggingface.co/openlynn/lynn-A2.7B-rp-v1
18
+
19
+ <!-- provided-files -->
20
+ weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion.
21
+ ## Usage
22
+
23
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
24
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
25
+ more details, including on how to concatenate multi-part files.
26
+
27
+ ## Provided Quants
28
+
29
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
30
+
31
+ | Link | Type | Size/GB | Notes |
32
+ |:-----|:-----|--------:|:------|
33
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q2_K.gguf) | Q2_K | 6.0 | |
34
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.IQ3_XS.gguf) | IQ3_XS | 6.6 | |
35
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.IQ3_S.gguf) | IQ3_S | 6.9 | beats Q3_K* |
36
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q3_K_S.gguf) | Q3_K_S | 6.9 | |
37
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.IQ3_M.gguf) | IQ3_M | 7.0 | |
38
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q3_K_M.gguf) | Q3_K_M | 7.5 | lower quality |
39
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q3_K_L.gguf) | Q3_K_L | 7.8 | |
40
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.IQ4_XS.gguf) | IQ4_XS | 8.0 | |
41
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q4_K_S.gguf) | Q4_K_S | 8.8 | fast, recommended |
42
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q4_K_M.gguf) | Q4_K_M | 9.6 | fast, recommended |
43
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q5_K_S.gguf) | Q5_K_S | 10.3 | |
44
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q5_K_M.gguf) | Q5_K_M | 10.9 | |
45
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q6_K.gguf) | Q6_K | 12.9 | very good quality |
46
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.Q8_0.gguf) | Q8_0 | 15.3 | fast, best quality |
47
+ | [GGUF](https://huggingface.co/mradermacher/lynn-A2.7B-rp-v1-GGUF/resolve/main/lynn-A2.7B-rp-v1.f16.gguf) | f16 | 28.7 | 16 bpw, overkill |
48
+
49
+
50
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
51
+ types (lower is better):
52
+
53
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
54
+
55
+ And here are Artefact2's thoughts on the matter:
56
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
57
+
58
+ ## FAQ / Model Request
59
+
60
+ See https://huggingface.co/mradermacher/model_requests for some answers to
61
+ questions you might have and/or if you want some other model quantized.
62
+
63
+ ## Thanks
64
+
65
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
66
+ me use its servers and providing upgrades to my workstation to enable
67
+ this work in my free time.
68
+
69
+ <!-- end -->
lynn-A2.7B-rp-v1.IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1170e2a231a9a544f4929837ee6b99157b2efd8125cd6d9bbc5bc58a061af5c
3
+ size 6932208416
lynn-A2.7B-rp-v1.IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea43832a81fae146b2cd915062200b81375eb974fbee800b618db02105395685
3
+ size 6835993376
lynn-A2.7B-rp-v1.IQ3_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:16a585f192dbffb62c355548ea8f9b4d333a62d7b2dfbc944d8d08767d648d7d
3
+ size 6515129120
lynn-A2.7B-rp-v1.IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e85b33aff84a1b9094a63b7d593cbc7f4a5064526c2b5299a6fc5090b754212
3
+ size 7939011360
lynn-A2.7B-rp-v1.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84cdd78deef00f61f1516bbcfc1525e8f81b7cbd182ffe2e979bed057422d410
3
+ size 5884265248
lynn-A2.7B-rp-v1.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:682f9a3d9f769612986b0c68d9b495aa30cd22ef2192de159d64b9e77e963137
3
+ size 7737834272
lynn-A2.7B-rp-v1.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e82d7c131ae170aa5e717486698fcf02e3bfae5274a39390ce17547bb2a141e1
3
+ size 7431846688
lynn-A2.7B-rp-v1.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:580c5334b07b2a67c3c601c173eb2dbf963a363679a0107f605e4e0b177df52c
3
+ size 6835993376
lynn-A2.7B-rp-v1.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b317b1ca7defa321cda2f5ab0b33a3f0a8690e7b8d81d802fb3d978d033bb965
3
+ size 9490338592
lynn-A2.7B-rp-v1.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64f50faa7cdd94a2010531aef5ba50d785a788ba255d0b137736237b7fa5015a
3
+ size 8701973280
lynn-A2.7B-rp-v1.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ee9ed124541a9b4b65579c7dc0a8e08aeb5f960f39015559e869a8e1d3b00d9
3
+ size 10827633440
lynn-A2.7B-rp-v1.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d0404eea02c4652732c7fa42d66d40c59fd78080c2a3b18ad7bf94a7508ad95
3
+ size 10153759520
lynn-A2.7B-rp-v1.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d06237d7c7cab09e77a67be79147709b12c8bf6f9e37611a8558a977d8392b9
3
+ size 12759444256
lynn-A2.7B-rp-v1.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:08a773cfcad860a05b94279bc649f62faee31e9f9e64db4e9766e2d129982317
3
+ size 15220112160
lynn-A2.7B-rp-v1.f16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef497bc164db53c42e40375f4a732c701bd994a82eee0f40c17ea14968e1cb30
3
+ size 28638116640