cicdatopea
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,7 @@ This model is an int4 model with group_size 128 and and symmetric quantization o
|
|
10 |
|
11 |
**Please note that loading the model in Transformers can be quite slow. Consider using an alternative serving framework for better performance.**
|
12 |
|
13 |
-
Due to limited GPU resources, we have only tested a few prompts on a CPU backend
|
14 |
|
15 |
Please follow the license of the original model.
|
16 |
|
|
|
10 |
|
11 |
**Please note that loading the model in Transformers can be quite slow. Consider using an alternative serving framework for better performance.**
|
12 |
|
13 |
+
Due to limited GPU resources, we have only tested a few prompts on a CPU backend with QBits. If this model does not meet your performance expectations, you may explore another quantized model in AWQ format, generated via AutoRound with different hyperparameters. This alternative model will be uploaded soon.
|
14 |
|
15 |
Please follow the license of the original model.
|
16 |
|