Update README.md
Browse files
README.md
CHANGED
@@ -19,8 +19,13 @@ pipeline_tag: visual-question-answering
|
|
19 |
|
20 |
\[[InternVL 1.5 Technical Report](https://arxiv.org/abs/2404.16821)\] \[[CVPR Paper](https://arxiv.org/abs/2312.14238)\] \[[GitHub](https://github.com/OpenGVLab/InternVL)\] \[[Chat Demo](https://internvl.opengvlab.com/)\] \[[ä¸æ–‡è§£è¯»](https://zhuanlan.zhihu.com/p/675877376)]
|
21 |
|
|
|
|
|
22 |
We are delighted to introduce Mini-InternVL-Chat-2B-V1-5. In the era of large language models, many researchers have started to focus on smaller language models, such as Gemma-2B, Qwen-1.8B, and InternLM2-1.8B. Inspired by their efforts, we have distilled our vision foundation model InternViT-6B-448px-V1-5 down to 300M and used InternLM2-Chat-1.8B as our language model. This resulted in a small multimodal model with excellent performance.
|
23 |
|
|
|
|
|
|
|
24 |
|
25 |
## Model Details
|
26 |
- **Model Type:** multimodal large language model (MLLM)
|
@@ -43,10 +48,6 @@ We are delighted to introduce Mini-InternVL-Chat-2B-V1-5. In the era of large la
|
|
43 |
| InternVL-Chat-V1.2(🤗 [HF link](https://huggingface.co/OpenGVLab/InternVL-Chat-V1-2) ) |InternViT-6B-448px-V1-2(🤗 [HF link](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V1-2)) |2024.02.11 | scaling up LLM to 34B |
|
44 |
| InternVL-Chat-V1.1(🤗 [HF link](https://huggingface.co/OpenGVLab/InternVL-Chat-V1-1)) |InternViT-6B-448px-V1-0(🤗 [HF link](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V1-0)) |2024.01.24 | support Chinese and stronger OCR |
|
45 |
|
46 |
-
## Architecture
|
47 |
-
|
48 |
-
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6579b818563044badca392fc/OM6Ejt8Q9ewSAsGZcDqH0.png)
|
49 |
-
|
50 |
## Performance
|
51 |
|
52 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/64119264f0f81eb569e0d569/BbsilHS8PjwZwlc330_g4.png)
|
|
|
19 |
|
20 |
\[[InternVL 1.5 Technical Report](https://arxiv.org/abs/2404.16821)\] \[[CVPR Paper](https://arxiv.org/abs/2312.14238)\] \[[GitHub](https://github.com/OpenGVLab/InternVL)\] \[[Chat Demo](https://internvl.opengvlab.com/)\] \[[ä¸æ–‡è§£è¯»](https://zhuanlan.zhihu.com/p/675877376)]
|
21 |
|
22 |
+
You can run multimodal large models using a 1080Ti now.
|
23 |
+
|
24 |
We are delighted to introduce Mini-InternVL-Chat-2B-V1-5. In the era of large language models, many researchers have started to focus on smaller language models, such as Gemma-2B, Qwen-1.8B, and InternLM2-1.8B. Inspired by their efforts, we have distilled our vision foundation model InternViT-6B-448px-V1-5 down to 300M and used InternLM2-Chat-1.8B as our language model. This resulted in a small multimodal model with excellent performance.
|
25 |
|
26 |
+
As shown in the figure below, we adopted the same model architecture as InternVL 1.5. We simply replaced the original InternViT-6B with InternViT-300M and InternLM2-Chat-20B with InternLM2-Chat-1.8B. For training, we used the same data as InternVL 1.5 to train this smaller model. Additionally, due to the lower training costs of smaller models, we used a context length of 8K during training.
|
27 |
+
|
28 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/64119264f0f81eb569e0d569/SmtrR0c5NVhmSTvnwwCE9.png)
|
29 |
|
30 |
## Model Details
|
31 |
- **Model Type:** multimodal large language model (MLLM)
|
|
|
48 |
| InternVL-Chat-V1.2(🤗 [HF link](https://huggingface.co/OpenGVLab/InternVL-Chat-V1-2) ) |InternViT-6B-448px-V1-2(🤗 [HF link](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V1-2)) |2024.02.11 | scaling up LLM to 34B |
|
49 |
| InternVL-Chat-V1.1(🤗 [HF link](https://huggingface.co/OpenGVLab/InternVL-Chat-V1-1)) |InternViT-6B-448px-V1-0(🤗 [HF link](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V1-0)) |2024.01.24 | support Chinese and stronger OCR |
|
50 |
|
|
|
|
|
|
|
|
|
51 |
## Performance
|
52 |
|
53 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/64119264f0f81eb569e0d569/BbsilHS8PjwZwlc330_g4.png)
|