Joseph717171
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -126,7 +126,7 @@ cd <llama.cpp directory>
|
|
126 |
```
|
127 |
4. Use the generated matrix file to quantise the model
|
128 |
```
|
129 |
-
./quantize --
|
130 |
```
|
131 |
Note: normal quantisation also benefits from using a matrix file. It also seem that a bigger input matrix is
|
132 |
better for higher quantisation.
|
|
|
126 |
```
|
127 |
4. Use the generated matrix file to quantise the model
|
128 |
```
|
129 |
+
./quantize --imatrix <output.matrix> <model_path>/ggml-model-f16.gguf <quantisation_level, eg:IQ4_XS>
|
130 |
```
|
131 |
Note: normal quantisation also benefits from using a matrix file. It also seem that a bigger input matrix is
|
132 |
better for higher quantisation.
|