Update README.md
Browse files
README.md
CHANGED
@@ -5,9 +5,9 @@ language:
|
|
5 |
---
|
6 |
# weblab-10b-instruction-sft-GPTQ
|
7 |
|
8 |
-
|
9 |
|
10 |
-
This model is
|
11 |
|
12 |
There are currently two well-known quantization version of original model.
|
13 |
(1)GPTQ version(This model. 6.3 GB)
|
@@ -23,7 +23,12 @@ But maybe gguf model little bit slower then GPTQ especialy long text.
|
|
23 |
|
24 |
|
25 |
### sample code
|
26 |
-
|
|
|
|
|
|
|
|
|
|
|
27 |
```
|
28 |
pip install auto-gptq
|
29 |
```
|
@@ -52,13 +57,13 @@ output = model.generate(input_ids=tokens, max_new_tokens=100, do_sample=True, te
|
|
52 |
print(tokenizer.decode(output[0]))
|
53 |
```
|
54 |
|
55 |
-
### Other documents
|
56 |
https://github.com/PanQiWei/AutoGPTQ/blob/main/docs/tutorial/01-Quick-Start.md
|
57 |
|
58 |
### Benchmark
|
59 |
|
60 |
The results below are preliminary. The blank part is under measurement.
|
61 |
-
Also, the score may change as a result of tuning
|
62 |
|
63 |
* **Japanese benchmark**
|
64 |
|
|
|
5 |
---
|
6 |
# weblab-10b-instruction-sft-GPTQ
|
7 |
|
8 |
+
Original model [weblab-10b-instruction-sft](https://huggingface.co/matsuo-lab/weblab-10b-instruction-sft) which is a Japanese-centric multilingual GPT-NeoX model of 10 billion parameters.
|
9 |
|
10 |
+
This model is a quantized(miniaturized) version of the original model(21.42GB).
|
11 |
|
12 |
There are currently two well-known quantization version of original model.
|
13 |
(1)GPTQ version(This model. 6.3 GB)
|
|
|
23 |
|
24 |
|
25 |
### sample code
|
26 |
+
|
27 |
+
Currently, models may behave differently on local PC and Colab. On Colab, the model may not respond if you include instructional prompts.
|
28 |
+
[Colab Sample script](https://github.com/webbigdata-jp/python_sample/blob/main/weblab_10b_instruction_sft_GPTQ_sample.ipynb)
|
29 |
+
|
30 |
+
If you get an error (something not found or something is not defined) in the script below, please refer to the official documentation and Colab samples and specify a specific version.
|
31 |
+
|
32 |
```
|
33 |
pip install auto-gptq
|
34 |
```
|
|
|
57 |
print(tokenizer.decode(output[0]))
|
58 |
```
|
59 |
|
60 |
+
### Other AutoGPTQ documents
|
61 |
https://github.com/PanQiWei/AutoGPTQ/blob/main/docs/tutorial/01-Quick-Start.md
|
62 |
|
63 |
### Benchmark
|
64 |
|
65 |
The results below are preliminary. The blank part is under measurement.
|
66 |
+
Also, the score may change as a result of more tuning.
|
67 |
|
68 |
* **Japanese benchmark**
|
69 |
|