Update README.md
Browse files
README.md
CHANGED
@@ -17,6 +17,29 @@ Inference of this model is compatible with AutoGPTQ's Kernel.
|
|
17 |
|
18 |
|
19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
20 |
|
21 |
### Evaluate the model
|
22 |
|
@@ -44,29 +67,6 @@ lm_eval --model hf --model_args pretrained="Intel/bloom-7b1-int4-inc",autogptq=T
|
|
44 |
|
45 |
|
46 |
|
47 |
-
### Reproduce the model
|
48 |
-
|
49 |
-
Here is the sample command to reproduce the model
|
50 |
-
|
51 |
-
```bash
|
52 |
-
git clone https://github.com/intel/auto-round
|
53 |
-
cd auto-round/examples/language-modeling
|
54 |
-
pip install -r requirements.txt
|
55 |
-
python3 main.py \
|
56 |
-
--model_name bigscience/bloom-7b1 \
|
57 |
-
--device 0 \
|
58 |
-
--group_size 128 \
|
59 |
-
--bits 4 \
|
60 |
-
--iters 1000 \
|
61 |
-
--deployment_device 'gpu' \
|
62 |
-
--output_dir "./tmp_autoround" \
|
63 |
-
|
64 |
-
```
|
65 |
-
|
66 |
-
|
67 |
-
|
68 |
-
|
69 |
-
|
70 |
## Caveats and Recommendations
|
71 |
|
72 |
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
|
|
|
17 |
|
18 |
|
19 |
|
20 |
+
### Reproduce the model
|
21 |
+
|
22 |
+
Here is the sample command to reproduce the model
|
23 |
+
|
24 |
+
```bash
|
25 |
+
git clone https://github.com/intel/auto-round
|
26 |
+
cd auto-round/examples/language-modeling
|
27 |
+
pip install -r requirements.txt
|
28 |
+
python3 main.py \
|
29 |
+
--model_name bigscience/bloom-7b1 \
|
30 |
+
--device 0 \
|
31 |
+
--group_size 128 \
|
32 |
+
--bits 4 \
|
33 |
+
--iters 1000 \
|
34 |
+
--deployment_device 'gpu' \
|
35 |
+
--output_dir "./tmp_autoround" \
|
36 |
+
|
37 |
+
```
|
38 |
+
|
39 |
+
|
40 |
+
|
41 |
+
|
42 |
+
|
43 |
|
44 |
### Evaluate the model
|
45 |
|
|
|
67 |
|
68 |
|
69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
70 |
## Caveats and Recommendations
|
71 |
|
72 |
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
|