Update README.md
Browse files
README.md
CHANGED
@@ -33,7 +33,45 @@ Quantized GGUF model files for [airoboros-3b-3p0](https://huggingface.co/jondurb
|
|
33 |
| [airoboros-3b-3p0.q6_k.gguf](https://huggingface.co/afrideva/airoboros-3b-3p0-GGUF/resolve/main/airoboros-3b-3p0.q6_k.gguf) | q6_k | 2.30 GB |
|
34 |
| [airoboros-3b-3p0.q8_0.gguf](https://huggingface.co/afrideva/airoboros-3b-3p0-GGUF/resolve/main/airoboros-3b-3p0.q8_0.gguf) | q8_0 | 2.97 GB |
|
35 |
|
36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
|
38 |
## Original Model Card:
|
39 |
### Overview
|
|
|
33 |
| [airoboros-3b-3p0.q6_k.gguf](https://huggingface.co/afrideva/airoboros-3b-3p0-GGUF/resolve/main/airoboros-3b-3p0.q6_k.gguf) | q6_k | 2.30 GB |
|
34 |
| [airoboros-3b-3p0.q8_0.gguf](https://huggingface.co/afrideva/airoboros-3b-3p0-GGUF/resolve/main/airoboros-3b-3p0.q8_0.gguf) | q8_0 | 2.97 GB |
|
35 |
|
36 |
+
# Test Run
|
37 |
+
`./llama.cpp/main -m airoboros-3b-3p0.q8_0.gguf -p "[INST]Please give me a color representing Namibia's beauty.[/INST]"
|
38 |
+
```
|
39 |
+
llm_load_print_meta: model type = 3B
|
40 |
+
llm_load_print_meta: model ftype = mostly Q8_0
|
41 |
+
llm_load_print_meta: model params = 2.80 B
|
42 |
+
llm_load_print_meta: model size = 2.77 GiB (8.50 BPW)
|
43 |
+
llm_load_print_meta: general.name = airoboros-3b-3p0
|
44 |
+
llm_load_print_meta: BOS token = 0 '<|endoftext|>'
|
45 |
+
llm_load_print_meta: EOS token = 0 '<|endoftext|>'
|
46 |
+
llm_load_print_meta: UNK token = 0 '<|endoftext|>'
|
47 |
+
llm_load_print_meta: LF token = 128 'Ä'
|
48 |
+
llm_load_tensors: ggml ctx size = 0.13 MB
|
49 |
+
llm_load_tensors: mem required = 2833.63 MB
|
50 |
+
.............................................................................................
|
51 |
+
llama_new_context_with_model: n_ctx = 512
|
52 |
+
llama_new_context_with_model: freq_base = 10000.0
|
53 |
+
llama_new_context_with_model: freq_scale = 1
|
54 |
+
llama_new_context_with_model: kv self size = 160.00 MB
|
55 |
+
llama_build_graph: non-view tensors processed: 1125/1125
|
56 |
+
llama_new_context_with_model: compute buffer total size = 109.81 MB
|
57 |
+
|
58 |
+
system_info: n_threads = 2 / 2 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |
|
59 |
+
sampling:
|
60 |
+
repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000
|
61 |
+
top_k = 40, tfs_z = 1.000, top_p = 0.950, min_p = 0.050, typical_p = 1.000, temp = 0.800
|
62 |
+
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
|
63 |
+
generate: n_ctx = 512, n_batch = 512, n_predict = 128, n_keep = 0
|
64 |
+
|
65 |
+
|
66 |
+
[INST]Please give me a color representing Namibia's beauty.[/INST]
|
67 |
+
The colors of the ocean, sand dunes, and deserts in Namibia are truly breathtaking. The vibrant blues of the sea contrast sharply with the warm greys of the desert and the rich browns of the dunes. The mixture of these colors creates a stunning landscape that is unique to this region. [end of text]
|
68 |
+
|
69 |
+
llama_print_timings: load time = 19034.49 ms
|
70 |
+
llama_print_timings: sample time = 61.34 ms / 65 runs ( 0.94 ms per token, 1059.72 tokens per second)
|
71 |
+
llama_print_timings: prompt eval time = 7435.06 ms / 19 tokens ( 391.32 ms per token, 2.56 tokens per second)
|
72 |
+
llama_print_timings: eval time = 31744.48 ms / 64 runs ( 496.01 ms per token, 2.02 tokens per second)
|
73 |
+
llama_print_timings: total time = 39302.86 ms
|
74 |
+
```
|
75 |
|
76 |
## Original Model Card:
|
77 |
### Overview
|