Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,7 @@ tags:
|
|
13 |
---
|
14 |
# Adapting Multimodal Large Language Models to Domains via Post-Training
|
15 |
|
16 |
-
This repos contains the **biomedicine MLLM developed from Llama-3.2-11B** in our paper: [On Domain-Specific Post-Training for Multimodal Large Language Models](https://huggingface.co/papers/2411.19930). The correspoding training
|
17 |
|
18 |
The main project page is: [Adapt-MLLM-to-Domains](https://huggingface.co/AdaptLLM/Adapt-MLLM-to-Domains/edit/main/README.md)
|
19 |
|
@@ -30,7 +30,7 @@ We investigate domain adaptation of MLLMs through post-training, focusing on dat
|
|
30 |
|
31 |
Starting with transformers >= 4.45.0 onward, you can run inference using conversational messages that may include an image you can query about.
|
32 |
|
33 |
-
Make sure to update your transformers installation via pip install --upgrade transformers
|
34 |
|
35 |
```bash
|
36 |
import requests
|
@@ -68,6 +68,9 @@ output = model.generate(**inputs, max_new_tokens=30)
|
|
68 |
print(processor.decode(output[0]))
|
69 |
```
|
70 |
|
|
|
|
|
|
|
71 |
## Citation
|
72 |
If you find our work helpful, please cite us.
|
73 |
|
|
|
13 |
---
|
14 |
# Adapting Multimodal Large Language Models to Domains via Post-Training
|
15 |
|
16 |
+
This repos contains the **biomedicine MLLM developed from Llama-3.2-11B** in our paper: [On Domain-Specific Post-Training for Multimodal Large Language Models](https://huggingface.co/papers/2411.19930). The correspoding training dataset is in [medicine-visual-instructions](https://huggingface.co/datasets/AdaptLLM/medicine-visual-instructions).
|
17 |
|
18 |
The main project page is: [Adapt-MLLM-to-Domains](https://huggingface.co/AdaptLLM/Adapt-MLLM-to-Domains/edit/main/README.md)
|
19 |
|
|
|
30 |
|
31 |
Starting with transformers >= 4.45.0 onward, you can run inference using conversational messages that may include an image you can query about.
|
32 |
|
33 |
+
Make sure to update your transformers installation via `pip install --upgrade transformers`.
|
34 |
|
35 |
```bash
|
36 |
import requests
|
|
|
68 |
print(processor.decode(output[0]))
|
69 |
```
|
70 |
|
71 |
+
Since our model architecture aligns with the base model, you can refer to the official repository of [Llama-3.2-Vision-Instruct](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct) for more advanced usage instructions.
|
72 |
+
|
73 |
+
|
74 |
## Citation
|
75 |
If you find our work helpful, please cite us.
|
76 |
|