How to do batch inference on Qwen2-VL-2B-Instruct?
I want to do batch inference with this model for my VQA dataset. How I should organize the "messages" , "processor" and the "model.generate" for batch infernece ?
Is it something like ?
message1 = [
{
"role": "user",
"content": [
{
"type": "image",
"image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
},
{"type": "text", "text": "How many herbivores present in the image?"},
],
}
]
message2 = [
{
"role": "user",
"content": [
{
"type": "image",
"image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
},
{"type": "text", "text": "How many animals present in the image?"},
],
}
]
messages = [message1, message2]
And then to processor ?