Fine-Tuned Model

fjmgAI/rerank1-210M-EuroBERT

Base Model

EuroBERT/EuroBERT-210m

Fine-Tuning Method

This is a Cross Encoder model finetuned from EuroBERT/EuroBERT-210m using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Dataset

sentence-transformers/gooaq

Description

This dataset is a collection of question-answer pairs, collected from Google.

Fine-Tuning Details

The model was trained using 578,402 training samples from sentence-transformer.

Cross Encoder Reranking

Dataset: gooaq-dev

Evaluated with CrossEncoderRerankingEvaluator with these parameters:

{
    "at_k": 10,
    "always_rerank_positives": false
}

Metric	Value
map	0.7097 (+0.1786)
mrr@10	0.7089 (+0.1850)
ndcg@10	0.7579 (+0.1667)

Cross Encoder Reranking

Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100

Evaluated with CrossEncoderRerankingEvaluator with these parameters:

{
    "at_k": 10,
    "always_rerank_positives": true
}

Metric	NanoMSMARCO_R100	NanoNFCorpus_R100	NanoNQ_R100
map	0.4630 (-0.0266)	0.3363 (+0.0753)	0.4738 (+0.0542)
mrr@10	0.4452 (-0.0323)	0.5204 (+0.0206)	0.4783 (+0.0516)
ndcg@10	0.5106 (-0.0298)	0.3632 (+0.0381)	0.5182 (+0.0176)

Cross Encoder Nano BEIR

Dataset: NanoBEIR_R100_mean

Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:

{
    "dataset_names": [
        "msmarco",
        "nfcorpus",
        "nq"
    ],
    "rerank_k": 100,
    "at_k": 10,
    "always_rerank_positives": true
}

Metric	Value
map	0.4244 (+0.0343)
mrr@10	0.4813 (+0.0133)
ndcg@10	0.4640 (+0.0086)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("fjmgAI/rerank1-210M-EuroBERT", trust_remote_code=True)
# Get scores for pairs of texts
pairs = [
    ['what are the risks with taking statins?', "['Muscle pain and damage. One of the most common complaints of people taking statins is muscle pain. ... ', 'Liver damage. Occasionally, statin use could cause an increase in the level of enzymes that signal liver inflammation. ... ', 'Increased blood sugar or type 2 diabetes. ... ', 'Neurological side effects.']"],
    ['what are the risks with taking statins?', 'Doctors discovered that statins can help lower blood pressure, as well as lower cholesterol. Statins are often prescribed to people with high cholesterol. Too much cholesterol in your blood increases your risk of heart attacks and strokes.'],
    ['what are the risks with taking statins?', 'Lipitor and Crestor are both effective statins that lower levels of “bad” cholesterol and increase levels of “good” cholesterol. While Crestor is the more potent statin, both medications are effective and have slightly different side effects and drug interactions.'],
    ['what are the risks with taking statins?', "About simvastatin Simvastatin belongs to a group of medicines called statins. It's used to lower cholesterol if you've been diagnosed with high blood cholesterol. It's also taken to prevent heart disease, including heart attacks and strokes."],
    ['what are the risks with taking statins?', 'Zetia works to lower cholesterol in a new way different from the statins: it inhibits the absorption of cholesterol in the small intestine, whereas the statins work by blocking cholesterol production in the liver.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'what are the risks with taking statins?',
    [
        "['Muscle pain and damage. One of the most common complaints of people taking statins is muscle pain. ... ', 'Liver damage. Occasionally, statin use could cause an increase in the level of enzymes that signal liver inflammation. ... ', 'Increased blood sugar or type 2 diabetes. ... ', 'Neurological side effects.']",
        'Doctors discovered that statins can help lower blood pressure, as well as lower cholesterol. Statins are often prescribed to people with high cholesterol. Too much cholesterol in your blood increases your risk of heart attacks and strokes.',
        'Lipitor and Crestor are both effective statins that lower levels of “bad” cholesterol and increase levels of “good” cholesterol. While Crestor is the more potent statin, both medications are effective and have slightly different side effects and drug interactions.',
        "About simvastatin Simvastatin belongs to a group of medicines called statins. It's used to lower cholesterol if you've been diagnosed with high blood cholesterol. It's also taken to prevent heart disease, including heart attacks and strokes.",
        'Zetia works to lower cholesterol in a new way different from the statins: it inhibits the absorption of cholesterol in the small intestine, whereas the statins work by blocking cholesterol production in the liver.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Framework Versions

Python: 3.11.12
Sentence Transformers: 4.0.2
Transformers: 4.51.2
PyTorch: 2.6.0+cu126
Accelerate: 1.6.0
Datasets: 3.5.0
Tokenizers: 0.21.1

Purpose

This tuned reranker model is optimized for Spanish and English applications, prioritizing accurate reordering of results by leveraging semantic similarity through refined embedding comparisons, ideal for enhancing question-answering and document retrieval tasks.