cicdatopea
commited on
Commit
•
92750e4
1
Parent(s):
a41e942
Update README.md
Browse files
README.md
CHANGED
@@ -6,7 +6,7 @@ datasets:
|
|
6 |
|
7 |
## Model Details
|
8 |
|
9 |
-
This model is an int4 model(The vision module has also been quantized) with group_size 128 and symmetric quantization of [microsoft/Phi-3.5-vision-instruct](https://huggingface.co/microsoft/Phi-3.5-vision-instruct).
|
10 |
## How To Use
|
11 |
|
12 |
|
@@ -40,13 +40,12 @@ model = AutoModelForCausalLM.from_pretrained(
|
|
40 |
model_id,
|
41 |
device_map="auto",
|
42 |
trust_remote_code=True,
|
43 |
-
torch_dtype="auto"
|
44 |
-
##revision="" ##AutoGPTQ format
|
45 |
)
|
46 |
processor = AutoProcessor.from_pretrained(model_id,
|
47 |
trust_remote_code=True,
|
48 |
num_crops=4
|
49 |
-
)
|
50 |
|
51 |
image_url = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"
|
52 |
content = "Describe this image."
|
@@ -114,11 +113,11 @@ auto-round-mllm --eval --model OPEA/Phi-3.5-vision-instruct-qvision-int4-sym-inc
|
|
114 |
```
|
115 |
|Metric |16bits|Pile Calib INT4 |
|
116 |
|-------------------|:------|:------|
|
117 |
-
|avg |77.64 | |
|
118 |
-
|MMBench_DEV_EN_V11 |71.83 | |
|
119 |
-
|ScienceQA_VAL |90.56 | |
|
120 |
-
|TextVQA_VAL |65.36 | |
|
121 |
-
|POPE |82.82 | |
|
122 |
|
123 |
### Generate the model
|
124 |
Here is the sample command to reproduce the model.
|
|
|
6 |
|
7 |
## Model Details
|
8 |
|
9 |
+
This model is an int4 model(The vision module has also been quantized) with group_size 128 and symmetric quantization of [microsoft/Phi-3.5-vision-instruct](https://huggingface.co/microsoft/Phi-3.5-vision-instruct).
|
10 |
## How To Use
|
11 |
|
12 |
|
|
|
40 |
model_id,
|
41 |
device_map="auto",
|
42 |
trust_remote_code=True,
|
43 |
+
torch_dtype="auto"
|
|
|
44 |
)
|
45 |
processor = AutoProcessor.from_pretrained(model_id,
|
46 |
trust_remote_code=True,
|
47 |
num_crops=4
|
48 |
+
)
|
49 |
|
50 |
image_url = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"
|
51 |
content = "Describe this image."
|
|
|
113 |
```
|
114 |
|Metric |16bits|Pile Calib INT4 |
|
115 |
|-------------------|:------|:------|
|
116 |
+
|avg |77.64 |76.99 |
|
117 |
+
|MMBench_DEV_EN_V11 |71.83 |71.05 |
|
118 |
+
|ScienceQA_VAL |90.56 |89.75 |
|
119 |
+
|TextVQA_VAL |65.36 |63.83 |
|
120 |
+
|POPE |82.82 |83.33 |
|
121 |
|
122 |
### Generate the model
|
123 |
Here is the sample command to reproduce the model.
|