Hugging Face's logo

SanctumAI
/
granite-3b-code-instruct-GGUF

Text Generation
Transformers
GGUF
ibm-granite-code
code
granite
Eval Results
Inference Endpoints
conversational
Model card Files Community
AndreyBest commited on
Commit
4c66790
•
1 Parent(s): cf757f8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +262 -3
README.md CHANGED
@@ -1,3 +1,262 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ base_model: ibm-granite/granite-3b-code-base
4
+ license: apache-2.0
5
+ datasets:
6
+ - bigcode/commitpackft
7
+ - TIGER-Lab/MathInstruct
8
+ - meta-math/MetaMathQA
9
+ - glaiveai/glaive-code-assistant-v3
10
+ - glaive-function-calling-v2
11
+ - bugdaryan/sql-create-context-instruction
12
+ - garage-bAInd/Open-Platypus
13
+ - nvidia/HelpSteer
14
+ metrics:
15
+ - code_eval
16
+ library_name: transformers
17
+ tags:
18
+ - code
19
+ - granite
20
+ model-index:
21
+ - name: granite-3b-code-instruct
22
+ results:
23
+ - task:
24
+ type: text-generation
25
+ dataset:
26
+ type: bigcode/humanevalpack
27
+ name: HumanEvalSynthesis(Python)
28
+ metrics:
29
+ - name: pass@1
30
+ type: pass@1
31
+ value: 51.2
32
+ veriefied: false
33
+ - task:
34
+ type: text-generation
35
+ dataset:
36
+ type: bigcode/humanevalpack
37
+ name: HumanEvalSynthesis(JavaScript)
38
+ metrics:
39
+ - name: pass@1
40
+ type: pass@1
41
+ value: 43.9
42
+ veriefied: false
43
+ - task:
44
+ type: text-generation
45
+ dataset:
46
+ type: bigcode/humanevalpack
47
+ name: HumanEvalSynthesis(Java)
48
+ metrics:
49
+ - name: pass@1
50
+ type: pass@1
51
+ value: 41.5
52
+ veriefied: false
53
+ - task:
54
+ type: text-generation
55
+ dataset:
56
+ type: bigcode/humanevalpack
57
+ name: HumanEvalSynthesis(Go)
58
+ metrics:
59
+ - name: pass@1
60
+ type: pass@1
61
+ value: 31.7
62
+ veriefied: false
63
+ - task:
64
+ type: text-generation
65
+ dataset:
66
+ type: bigcode/humanevalpack
67
+ name: HumanEvalSynthesis(C++)
68
+ metrics:
69
+ - name: pass@1
70
+ type: pass@1
71
+ value: 40.2
72
+ veriefied: false
73
+ - task:
74
+ type: text-generation
75
+ dataset:
76
+ type: bigcode/humanevalpack
77
+ name: HumanEvalSynthesis(Rust)
78
+ metrics:
79
+ - name: pass@1
80
+ type: pass@1
81
+ value: 29.3
82
+ veriefied: false
83
+ - task:
84
+ type: text-generation
85
+ dataset:
86
+ type: bigcode/humanevalpack
87
+ name: HumanEvalExplain(Python)
88
+ metrics:
89
+ - name: pass@1
90
+ type: pass@1
91
+ value: 39.6
92
+ veriefied: false
93
+ - task:
94
+ type: text-generation
95
+ dataset:
96
+ type: bigcode/humanevalpack
97
+ name: HumanEvalExplain(JavaScript)
98
+ metrics:
99
+ - name: pass@1
100
+ type: pass@1
101
+ value: 26.8
102
+ veriefied: false
103
+ - task:
104
+ type: text-generation
105
+ dataset:
106
+ type: bigcode/humanevalpack
107
+ name: HumanEvalExplain(Java)
108
+ metrics:
109
+ - name: pass@1
110
+ type: pass@1
111
+ value: 39
112
+ veriefied: false
113
+ - task:
114
+ type: text-generation
115
+ dataset:
116
+ type: bigcode/humanevalpack
117
+ name: HumanEvalExplain(Go)
118
+ metrics:
119
+ - name: pass@1
120
+ type: pass@1
121
+ value: 14
122
+ veriefied: false
123
+ - task:
124
+ type: text-generation
125
+ dataset:
126
+ type: bigcode/humanevalpack
127
+ name: HumanEvalExplain(C++)
128
+ metrics:
129
+ - name: pass@1
130
+ type: pass@1
131
+ value: 23.8
132
+ veriefied: false
133
+ - task:
134
+ type: text-generation
135
+ dataset:
136
+ type: bigcode/humanevalpack
137
+ name: HumanEvalExplain(Rust)
138
+ metrics:
139
+ - name: pass@1
140
+ type: pass@1
141
+ value: 12.8
142
+ veriefied: false
143
+ - task:
144
+ type: text-generation
145
+ dataset:
146
+ type: bigcode/humanevalpack
147
+ name: HumanEvalFix(Python)
148
+ metrics:
149
+ - name: pass@1
150
+ type: pass@1
151
+ value: 26.8
152
+ veriefied: false
153
+ - task:
154
+ type: text-generation
155
+ dataset:
156
+ type: bigcode/humanevalpack
157
+ name: HumanEvalFix(JavaScript)
158
+ metrics:
159
+ - name: pass@1
160
+ type: pass@1
161
+ value: 28
162
+ veriefied: false
163
+ - task:
164
+ type: text-generation
165
+ dataset:
166
+ type: bigcode/humanevalpack
167
+ name: HumanEvalFix(Java)
168
+ metrics:
169
+ - name: pass@1
170
+ type: pass@1
171
+ value: 33.5
172
+ veriefied: false
173
+ - task:
174
+ type: text-generation
175
+ dataset:
176
+ type: bigcode/humanevalpack
177
+ name: HumanEvalFix(Go)
178
+ metrics:
179
+ - name: pass@1
180
+ type: pass@1
181
+ value: 27.4
182
+ veriefied: false
183
+ - task:
184
+ type: text-generation
185
+ dataset:
186
+ type: bigcode/humanevalpack
187
+ name: HumanEvalFix(C++)
188
+ metrics:
189
+ - name: pass@1
190
+ type: pass@1
191
+ value: 31.7
192
+ veriefied: false
193
+ - task:
194
+ type: text-generation
195
+ dataset:
196
+ type: bigcode/humanevalpack
197
+ name: HumanEvalFix(Rust)
198
+ metrics:
199
+ - name: pass@1
200
+ type: pass@1
201
+ value: 16.5
202
+ veriefied: false
203
+ ---
204
+
205
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64a28db2f1968b7d7f357182/kaudiTlvRQBA5NSeq4BbM.png)
206
+ *This model was quantized by [SanctumAI](https://sanctum.ai). To leave feedback, join our community in [Discord](https://discord.gg/7ZNE78HJKh).*
207
+
208
+ # Granite 3B Code Instruct GGUF
209
+
210
+ **Model creator:** [ibm-granite](https://huggingface.co/ibm-granite)<br>
211
+ **Original model**: [granite-3b-code-instruct](https://huggingface.co/ibm-granite/granite-3b-code-instruct)<br>
212
+
213
+ ## Model Summary:
214
+
215
+ **Granite-3B-Code-Instruct** is a 3B parameter model fine tuned from *Granite-3B-Code-Base* on a combination of **permissively licensed** instruction data to enhance instruction following capabilities including logical reasoning and problem-solving skills.
216
+
217
+ - **Developers:** IBM Research
218
+ - **GitHub Repository:** [ibm-granite/granite-code-models](https://github.com/ibm-granite/granite-code-models)
219
+ - **Paper:** [Granite Code Models: A Family of Open Foundation Models for Code Intelligence](https://arxiv.org/abs/2405.04324)
220
+ - **Release Date**: May 6th, 2024
221
+ - **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).
222
+
223
+ ## Prompt Template:
224
+
225
+ If you're using Sanctum app, simply use `IBM Granite Code` model preset.
226
+
227
+ Prompt template:
228
+
229
+ ```
230
+ System:
231
+ {system_prompt}
232
+ Question:
233
+ {prompt}
234
+ Answer:
235
+
236
+ ```
237
+
238
+ ## Hardware Requirements Estimate
239
+
240
+ | Name | Quant method | Size | Memory (RAM, vRAM) required (for full context of 32k tokens) |
241
+ | ---- | ---- | ---- | ---- |
242
+ | [granite-3b-code-instruct.Q2_K.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q2_K.gguf) | Q2_K | 1.34 GB | 4.68 GB |
243
+ | [granite-3b-code-instruct.Q3_K_S.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q3_K_S.gguf) | Q3_K_S | 1.55 GB | 7.19 GB |
244
+ | [granite-3b-code-instruct.Q3_K_M.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q3_K_M.gguf) | Q3_K_M | 1.73 GB | 7.52 GB |
245
+ | [granite-3b-code-instruct.Q3_K_L.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q3_K_L.gguf) | Q3_K_L | 1.88 GB | 7.80 GB |
246
+ | [granite-3b-code-instruct.Q4_0.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q4_0.gguf) | Q4_0 | 2.00 GB | 8.07 GB |
247
+ | [granite-3b-code-instruct.Q4_K_S.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q4_K_S.gguf) | Q4_K_S | 2.01 GB | 8.10 GB |
248
+ | [granite-3b-code-instruct.Q4_K_M.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q4_K_M.gguf) | Q4_K_M | 2.13 GB | 8.31 GB |
249
+ | [granite-3b-code-instruct.Q4_K.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q4_K.gguf) | Q4_K | 2.13 GB | 8.31 GB |
250
+ | [granite-3b-code-instruct.Q4_1.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q4_1.gguf) | Q4_1 | 2.21 GB | 8.48 GB |
251
+ | [granite-3b-code-instruct.Q5_0.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q5_0.gguf) | Q5_0 | 2.42 GB | 8.90 GB |
252
+ | [granite-3b-code-instruct.Q5_K_S.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q5_K_S.gguf) | Q5_K_S | 2.42 GB | 8.90 GB |
253
+ | [granite-3b-code-instruct.Q5_K_M.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q5_K_M.gguf) | Q5_K_M | 2.49 GB | 9.02 GB |
254
+ | [granite-3b-code-instruct.Q5_K.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q5_K.gguf) | Q5_K | 2.49 GB | 9.02 GB |
255
+ | [granite-3b-code-instruct.Q5_1.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q5_1.gguf) | Q5_1 | 2.63 GB | 9.31 GB |
256
+ | [granite-3b-code-instruct.Q6_K.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q6_K.gguf) | Q6_K | 2.86 GB | 9.78 GB |
257
+ | [granite-3b-code-instruct.Q8_0.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.Q8_0.gguf) | Q8_0 | 3.71 GB | 11.41 GB |
258
+ | [granite-3b-code-instruct.f16.gguf](https://huggingface.co/SanctumAI/granite-3b-code-instruct-GGUF/blob/main/granite-3b-code-instruct.f16.gguf) | f16 | 6.97 GB | 4.68 GB |
259
+
260
+ ## Disclaimer
261
+
262
+ Sanctum is not the creator, originator, or owner of any Model featured in the Models section of the Sanctum application. Each Model is created and provided by third parties. Sanctum does not endorse, support, represent or guarantee the completeness, truthfulness, accuracy, or reliability of any Model listed there. You understand that supported Models can produce content that might be offensive, harmful, inaccurate or otherwise inappropriate, or deceptive. Each Model is the sole responsibility of the person or entity who originated such Model. Sanctum may not monitor or control the Models supported and cannot, and does not, take responsibility for any such Model. Sanctum disclaims all warranties or guarantees about the accuracy, reliability or benefits of the Models. Sanctum further disclaims any warranty that the Model will meet your requirements, be secure, uninterrupted or available at any time or location, or error-free, viruses-free, or that any errors will be corrected, or otherwise. You will be solely responsible for any damage resulting from your use of or access to the Models, your downloading of any Model, or use of any other Model provided by or through Sanctum.