Safetensors
qwen2_vl
vidore
reranker
NohTow commited on
Commit
6d48017
1 Parent(s): 78b0fe8

Update readme

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -8,9 +8,11 @@ tags:
8
  # MonoQwen2-VL-2B-LoRA-Reranker
9
 
10
  ## Model Overview
11
- The **MonoQwen2-VL-2B-LoRA-Reranker** is a LoRA fine-tuned version of the Qwen2-VL-2B model, optimized for reranking image-query relevance.
 
 
12
 
13
- It was train using [ColPali train set](https://huggingface.co/datasets/vidore/colpali_train_set)
14
 
15
  ## How to Use the Model
16
  Below is a quick example to rerank a single image against a user query using this model:
 
8
  # MonoQwen2-VL-2B-LoRA-Reranker
9
 
10
  ## Model Overview
11
+ The **MonoQwen2-VL-v0.1** is a LoRA of the Qwen2-VL-2B model, optimized for reranking (i.e, asserting pointwise image-query relevance) using the [MonoT5](https://arxiv.org/pdf/2101.05667) objective.
12
+ That is, given a couple of image and query fed into the prompt of the VLM, the model is tasked to generate "True" if the image is relevant to the query and "False" otherwise.
13
+ During inference, a relevancy score can then be obtained by comparing the logits of the two tokens and this score can effectively be used to rerank the candidates generated by a first-stage retriever (such as DSE or ColPali) or filter them using a threshold.
14
 
15
+ The [ColPali train set](https://huggingface.co/datasets/vidore/colpali_train_set) was used to train this model with negatives mined using DSE.
16
 
17
  ## How to Use the Model
18
  Below is a quick example to rerank a single image against a user query using this model: