Update readme
Browse files
README.md
CHANGED
@@ -8,9 +8,11 @@ tags:
|
|
8 |
# MonoQwen2-VL-2B-LoRA-Reranker
|
9 |
|
10 |
## Model Overview
|
11 |
-
The **MonoQwen2-VL-
|
|
|
|
|
12 |
|
13 |
-
|
14 |
|
15 |
## How to Use the Model
|
16 |
Below is a quick example to rerank a single image against a user query using this model:
|
|
|
8 |
# MonoQwen2-VL-2B-LoRA-Reranker
|
9 |
|
10 |
## Model Overview
|
11 |
+
The **MonoQwen2-VL-v0.1** is a LoRA of the Qwen2-VL-2B model, optimized for reranking (i.e, asserting pointwise image-query relevance) using the [MonoT5](https://arxiv.org/pdf/2101.05667) objective.
|
12 |
+
That is, given a couple of image and query fed into the prompt of the VLM, the model is tasked to generate "True" if the image is relevant to the query and "False" otherwise.
|
13 |
+
During inference, a relevancy score can then be obtained by comparing the logits of the two tokens and this score can effectively be used to rerank the candidates generated by a first-stage retriever (such as DSE or ColPali) or filter them using a threshold.
|
14 |
|
15 |
+
The [ColPali train set](https://huggingface.co/datasets/vidore/colpali_train_set) was used to train this model with negatives mined using DSE.
|
16 |
|
17 |
## How to Use the Model
|
18 |
Below is a quick example to rerank a single image against a user query using this model:
|