SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1
This is a sentence-transformers model finetuned from sentence-transformers/multi-qa-mpnet-base-dot-v1. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: sentence-transformers/multi-qa-mpnet-base-dot-v1
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 768 tokens
- Similarity Function: Dot Product
Model Sources
- Documentation: Sentence Transformers Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sentence Transformers on Hugging Face
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
'nerve cell dysfunction, riboflavin deficiency',
'Riboflavin transporter deficiency neuronopathy is a disorder that affects nerve cells (neurons). Affected individuals typically have hearing loss caused by nerve damage in the inner ear (sensorineural hearing loss) and signs of damage to other nerves.',
'A number sign (#) is used with this entry because hyperprolinemia type I (HYRPRO1) is caused by homozygous or compound heterozygous mutation in the proline dehydrogenase gene (PRODH; 606810) on chromosome 22q11.\n\nThe PRODH gene falls within the region deleted in the 22q11 deletion syndrome, including DiGeorge syndrome (188400) and velocardiofacial syndrome (192430).\n\nDescription\n\nPhang et al. (2001) noted that prospective studies of HPI probands identified through newborn screening as well as reports of several families have suggested that it is a metabolic disorder not clearly associated with clinical manifestations. Phang et al. (2001) concluded that HPI is a relatively benign condition in most individuals under most circumstances. However, other reports have suggested that some patients have a severe phenotype with neurologic manifestations, including epilepsy and mental retardation (Jacquet et al., 2003).\n\n### Genetic Heterogeneity of Hyperprolinemia',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
Evaluation
Metrics
Information Retrieval
- Evaluated with
InformationRetrievalEvaluator
Metric | Value |
---|---|
cosine_accuracy@1 | 0.1933 |
cosine_accuracy@3 | 0.5626 |
cosine_accuracy@5 | 0.7512 |
cosine_accuracy@10 | 0.841 |
cosine_precision@1 | 0.1933 |
cosine_precision@3 | 0.1875 |
cosine_precision@5 | 0.1502 |
cosine_precision@10 | 0.0841 |
cosine_recall@1 | 0.1933 |
cosine_recall@3 | 0.5626 |
cosine_recall@5 | 0.7512 |
cosine_recall@10 | 0.841 |
cosine_ndcg@10 | 0.512 |
cosine_mrr@10 | 0.4059 |
cosine_map@100 | 0.411 |
dot_accuracy@1 | 0.1949 |
dot_accuracy@3 | 0.5673 |
dot_accuracy@5 | 0.7571 |
dot_accuracy@10 | 0.8415 |
dot_precision@1 | 0.1949 |
dot_precision@3 | 0.1891 |
dot_precision@5 | 0.1514 |
dot_precision@10 | 0.0842 |
dot_recall@1 | 0.1949 |
dot_recall@3 | 0.5673 |
dot_recall@5 | 0.7571 |
dot_recall@10 | 0.8415 |
dot_ndcg@10 | 0.5141 |
dot_mrr@10 | 0.4084 |
dot_map@100 | 0.4136 |
Training Details
Training Dataset
Unnamed Dataset
- Size: 95,159 training samples
- Columns:
queries
andchunks
- Approximate statistics based on the first 1000 samples:
queries chunks type string string details - min: 5 tokens
- mean: 15.01 tokens
- max: 30 tokens
- min: 5 tokens
- mean: 158.91 tokens
- max: 319 tokens
- Samples:
queries chunks hypotrichosis, wiry hair, onycholysis
Green et al. (2003) reported an Australian family in which 22 members over 4 generations had progressive patterned scalp hypotrichosis and wiry hair similar to that seen in Marie Unna hereditary hypotrichosis (MUHH; 146550). Features differing from those of MUHH included absence of signs of abnormality at birth, relative sparing of body hair, distal onycholysis, and intermittent cosegregation with autosomal dominant cleft lip and palate. Five individuals had associated cleft lip and palate. Green et al. (2003) excluded linkage of the disorder in the Australian family to the MUHH locus on chromosome 8p21.
cleft lip, cleft palate, hair loss
Green et al. (2003) reported an Australian family in which 22 members over 4 generations had progressive patterned scalp hypotrichosis and wiry hair similar to that seen in Marie Unna hereditary hypotrichosis (MUHH; 146550). Features differing from those of MUHH included absence of signs of abnormality at birth, relative sparing of body hair, distal onycholysis, and intermittent cosegregation with autosomal dominant cleft lip and palate. Five individuals had associated cleft lip and palate. Green et al. (2003) excluded linkage of the disorder in the Australian family to the MUHH locus on chromosome 8p21.
progressive patterned scalp, autosomal dominant inheritance
Green et al. (2003) reported an Australian family in which 22 members over 4 generations had progressive patterned scalp hypotrichosis and wiry hair similar to that seen in Marie Unna hereditary hypotrichosis (MUHH; 146550). Features differing from those of MUHH included absence of signs of abnormality at birth, relative sparing of body hair, distal onycholysis, and intermittent cosegregation with autosomal dominant cleft lip and palate. Five individuals had associated cleft lip and palate. Green et al. (2003) excluded linkage of the disorder in the Australian family to the MUHH locus on chromosome 8p21.
- Loss:
MultipleNegativesRankingLoss
with these parameters:{ "scale": 1, "similarity_fct": "dot_score" }
Evaluation Dataset
Unnamed Dataset
- Size: 8,747 evaluation samples
- Columns:
queries
andchunks
- Approximate statistics based on the first 1000 samples:
queries chunks type string string details - min: 6 tokens
- mean: 14.71 tokens
- max: 31 tokens
- min: 4 tokens
- mean: 155.81 tokens
- max: 305 tokens
- Samples:
queries chunks white patches, corrugated tongue, immunocompromised, Epstein-Barr virus
Not to be confused with Hairy tongue.
Hairy leukoplakia
Other namesOral hairy leukoplakia,[1]:385 OHL, or HIV-associated hairy leukoplakia[2]
SpecialtyGastroenterology
Hairy leukoplakia is a white patch on the side of the tongue with a corrugated or hairy appearance. It is caused by Epstein-Barr virus (EBV) and occurs usually in persons who are immunocompromised, especially those with human immunodeficiency virus infection/acquired immunodeficiency syndrome (HIV/AIDS). The white lesion, which cannot be scraped off, is benign and does not require any treatment, although its appearance may have diagnostic and prognostic implications for the underlying condition.
Depending upon what definition of leukoplakia is used, hairy leukoplakia is sometimes considered a subtype of leukoplakia, or a distinct diagnosis.
## ContentsHIV-associated lesions, oral hairy leukoplakia, benign white lesions, tongue appearance
Not to be confused with Hairy tongue.
Hairy leukoplakia
Other namesOral hairy leukoplakia,[1]:385 OHL, or HIV-associated hairy leukoplakia[2]
SpecialtyGastroenterology
Hairy leukoplakia is a white patch on the side of the tongue with a corrugated or hairy appearance. It is caused by Epstein-Barr virus (EBV) and occurs usually in persons who are immunocompromised, especially those with human immunodeficiency virus infection/acquired immunodeficiency syndrome (HIV/AIDS). The white lesion, which cannot be scraped off, is benign and does not require any treatment, although its appearance may have diagnostic and prognostic implications for the underlying condition.
Depending upon what definition of leukoplakia is used, hairy leukoplakia is sometimes considered a subtype of leukoplakia, or a distinct diagnosis.
## Contentshairy leukoplakia symptoms, non-scrapable lesions, HIV/AIDS, oral lesions
Not to be confused with Hairy tongue.
Hairy leukoplakia
Other namesOral hairy leukoplakia,[1]:385 OHL, or HIV-associated hairy leukoplakia[2]
SpecialtyGastroenterology
Hairy leukoplakia is a white patch on the side of the tongue with a corrugated or hairy appearance. It is caused by Epstein-Barr virus (EBV) and occurs usually in persons who are immunocompromised, especially those with human immunodeficiency virus infection/acquired immunodeficiency syndrome (HIV/AIDS). The white lesion, which cannot be scraped off, is benign and does not require any treatment, although its appearance may have diagnostic and prognostic implications for the underlying condition.
Depending upon what definition of leukoplakia is used, hairy leukoplakia is sometimes considered a subtype of leukoplakia, or a distinct diagnosis.
## Contents - Loss:
MultipleNegativesRankingLoss
with these parameters:{ "scale": 1, "similarity_fct": "dot_score" }
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy
: stepsper_device_train_batch_size
: 32per_device_eval_batch_size
: 32learning_rate
: 2e-05num_train_epochs
: 15warmup_ratio
: 0.1fp16
: Trueload_best_model_at_end
: Trueeval_on_start
: Truebatch_sampler
: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir
: Falsedo_predict
: Falseeval_strategy
: stepsprediction_loss_only
: Trueper_device_train_batch_size
: 32per_device_eval_batch_size
: 32per_gpu_train_batch_size
: Noneper_gpu_eval_batch_size
: Nonegradient_accumulation_steps
: 1eval_accumulation_steps
: Nonetorch_empty_cache_steps
: Nonelearning_rate
: 2e-05weight_decay
: 0.0adam_beta1
: 0.9adam_beta2
: 0.999adam_epsilon
: 1e-08max_grad_norm
: 1.0num_train_epochs
: 15max_steps
: -1lr_scheduler_type
: linearlr_scheduler_kwargs
: {}warmup_ratio
: 0.1warmup_steps
: 0log_level
: passivelog_level_replica
: warninglog_on_each_node
: Truelogging_nan_inf_filter
: Truesave_safetensors
: Truesave_on_each_node
: Falsesave_only_model
: Falserestore_callback_states_from_checkpoint
: Falseno_cuda
: Falseuse_cpu
: Falseuse_mps_device
: Falseseed
: 42data_seed
: Nonejit_mode_eval
: Falseuse_ipex
: Falsebf16
: Falsefp16
: Truefp16_opt_level
: O1half_precision_backend
: autobf16_full_eval
: Falsefp16_full_eval
: Falsetf32
: Nonelocal_rank
: 0ddp_backend
: Nonetpu_num_cores
: Nonetpu_metrics_debug
: Falsedebug
: []dataloader_drop_last
: Truedataloader_num_workers
: 0dataloader_prefetch_factor
: Nonepast_index
: -1disable_tqdm
: Falseremove_unused_columns
: Truelabel_names
: Noneload_best_model_at_end
: Trueignore_data_skip
: Falsefsdp
: []fsdp_min_num_params
: 0fsdp_config
: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap
: Noneaccelerator_config
: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed
: Nonelabel_smoothing_factor
: 0.0optim
: adamw_torchoptim_args
: Noneadafactor
: Falsegroup_by_length
: Falselength_column_name
: lengthddp_find_unused_parameters
: Noneddp_bucket_cap_mb
: Noneddp_broadcast_buffers
: Falsedataloader_pin_memory
: Truedataloader_persistent_workers
: Falseskip_memory_metrics
: Trueuse_legacy_prediction_loop
: Falsepush_to_hub
: Falseresume_from_checkpoint
: Nonehub_model_id
: Nonehub_strategy
: every_savehub_private_repo
: Falsehub_always_push
: Falsegradient_checkpointing
: Falsegradient_checkpointing_kwargs
: Noneinclude_inputs_for_metrics
: Falseeval_do_concat_batches
: Truefp16_backend
: autopush_to_hub_model_id
: Nonepush_to_hub_organization
: Nonemp_parameters
:auto_find_batch_size
: Falsefull_determinism
: Falsetorchdynamo
: Noneray_scope
: lastddp_timeout
: 1800torch_compile
: Falsetorch_compile_backend
: Nonetorch_compile_mode
: Nonedispatch_batches
: Nonesplit_batches
: Noneinclude_tokens_per_second
: Falseinclude_num_input_tokens_seen
: Falseneftune_noise_alpha
: Noneoptim_target_modules
: Nonebatch_eval_metrics
: Falseeval_on_start
: Trueeval_use_gather_object
: Falsebatch_sampler
: no_duplicatesmulti_dataset_batch_sampler
: proportional
Training Logs
Click to expand
Epoch | Step | Training Loss | loss | dot_map@100 |
---|---|---|---|---|
0 | 0 | - | 1.4355 | 0.2271 |
0.1346 | 100 | 1.2599 | - | - |
0.2692 | 200 | 0.7627 | - | - |
0.4038 | 300 | 0.6061 | - | - |
0.5384 | 400 | 0.5632 | - | - |
0.6729 | 500 | 0.3965 | 0.4589 | 0.3852 |
0.8075 | 600 | 0.3104 | - | - |
0.9421 | 700 | 0.446 | - | - |
1.0767 | 800 | 0.4426 | - | - |
1.2113 | 900 | 0.4518 | - | - |
1.3459 | 1000 | 0.4145 | 0.3726 | 0.3964 |
1.4805 | 1100 | 0.4296 | - | - |
1.6151 | 1200 | 0.4144 | - | - |
1.7497 | 1300 | 0.1536 | - | - |
1.8843 | 1400 | 0.3425 | - | - |
2.0188 | 1500 | 0.3225 | 0.3433 | 0.3930 |
2.1534 | 1600 | 0.3529 | - | - |
2.2880 | 1700 | 0.3382 | - | - |
2.4226 | 1800 | 0.3092 | - | - |
2.5572 | 1900 | 0.339 | - | - |
2.6918 | 2000 | 0.1681 | 0.3633 | 0.4032 |
2.8264 | 2100 | 0.1753 | - | - |
2.9610 | 2200 | 0.2552 | - | - |
3.0956 | 2300 | 0.2549 | - | - |
3.2301 | 2400 | 0.2759 | - | - |
3.3647 | 2500 | 0.2513 | 0.3338 | 0.4066 |
3.4993 | 2600 | 0.258 | - | - |
3.6339 | 2700 | 0.2222 | - | - |
3.7685 | 2800 | 0.0541 | - | - |
3.9031 | 2900 | 0.2275 | - | - |
4.0377 | 3000 | 0.1919 | 0.3529 | 0.4026 |
4.1723 | 3100 | 0.215 | - | - |
4.3069 | 3200 | 0.2114 | - | - |
4.4415 | 3300 | 0.2153 | - | - |
4.5760 | 3400 | 0.2164 | - | - |
4.7106 | 3500 | 0.0773 | 0.3509 | 0.4090 |
4.8452 | 3600 | 0.1211 | - | - |
4.9798 | 3700 | 0.1553 | - | - |
5.1144 | 3800 | 0.1764 | - | - |
5.2490 | 3900 | 0.1953 | - | - |
5.3836 | 4000 | 0.1559 | 0.3474 | 0.4089 |
5.5182 | 4100 | 0.1686 | - | - |
5.6528 | 4200 | 0.1327 | - | - |
5.7873 | 4300 | 0.0514 | - | - |
5.9219 | 4400 | 0.1381 | - | - |
6.0565 | 4500 | 0.1445 | 0.3521 | 0.4056 |
6.1911 | 4600 | 0.1621 | - | - |
6.3257 | 4700 | 0.1365 | - | - |
6.4603 | 4800 | 0.1579 | - | - |
6.5949 | 4900 | 0.1547 | - | - |
6.7295 | 5000 | 0.0316 | 0.3895 | 0.4094 |
6.8641 | 5100 | 0.0958 | - | - |
6.9987 | 5200 | 0.1082 | - | - |
7.1332 | 5300 | 0.1379 | - | - |
7.2678 | 5400 | 0.1348 | - | - |
7.4024 | 5500 | 0.1322 | 0.3552 | 0.4100 |
7.5370 | 5600 | 0.1321 | - | - |
7.6716 | 5700 | 0.0763 | - | - |
7.8062 | 5800 | 0.0472 | - | - |
7.9408 | 5900 | 0.0989 | - | - |
8.0754 | 6000 | 0.1045 | 0.3631 | 0.3967 |
8.2100 | 6100 | 0.122 | - | - |
8.3445 | 6200 | 0.1057 | - | - |
8.4791 | 6300 | 0.1194 | - | - |
8.6137 | 6400 | 0.113 | - | - |
8.7483 | 6500 | 0.0126 | 0.3944 | 0.4116 |
8.8829 | 6600 | 0.089 | - | - |
9.0175 | 6700 | 0.0849 | - | - |
9.1521 | 6800 | 0.1052 | - | - |
9.2867 | 6900 | 0.111 | - | - |
9.4213 | 7000 | 0.1026 | 0.3665 | 0.4133 |
9.5559 | 7100 | 0.1165 | - | - |
9.6904 | 7200 | 0.0394 | - | - |
9.8250 | 7300 | 0.0443 | - | - |
9.9596 | 7400 | 0.0756 | - | - |
10.0942 | 7500 | 0.0806 | 0.3785 | 0.4090 |
10.2288 | 7600 | 0.103 | - | - |
10.3634 | 7700 | 0.0875 | - | - |
10.4980 | 7800 | 0.0959 | - | - |
10.6326 | 7900 | 0.0851 | - | - |
10.7672 | 8000 | 0.0073 | 0.3902 | 0.4136 |
10.9017 | 8100 | 0.079 | - | - |
11.0363 | 8200 | 0.0664 | - | - |
11.1709 | 8300 | 0.0766 | - | - |
11.3055 | 8400 | 0.084 | - | - |
11.4401 | 8500 | 0.0947 | 0.3733 | 0.4099 |
11.5747 | 8600 | 0.0906 | - | - |
11.7093 | 8700 | 0.0224 | - | - |
11.8439 | 8800 | 0.0424 | - | - |
11.9785 | 8900 | 0.0569 | - | - |
12.1131 | 9000 | 0.0697 | 0.3824 | 0.4071 |
12.2476 | 9100 | 0.095 | - | - |
12.3822 | 9200 | 0.0651 | - | - |
12.5168 | 9300 | 0.0756 | - | - |
12.6514 | 9400 | 0.065 | - | - |
12.7860 | 9500 | 0.0194 | 0.3876 | 0.4110 |
12.9206 | 9600 | 0.0595 | - | - |
13.0552 | 9700 | 0.0629 | - | - |
13.1898 | 9800 | 0.0808 | - | - |
13.3244 | 9900 | 0.0652 | - | - |
13.4590 | 10000 | 0.0802 | 0.3783 | 0.4091 |
13.5935 | 10100 | 0.0809 | - | - |
13.7281 | 10200 | 0.0111 | - | - |
13.8627 | 10300 | 0.0465 | - | - |
13.9973 | 10400 | 0.0504 | - | - |
14.1319 | 10500 | 0.068 | 0.3831 | 0.4071 |
14.2665 | 10600 | 0.0739 | - | - |
14.4011 | 10700 | 0.0734 | - | - |
14.5357 | 10800 | 0.0737 | - | - |
14.6703 | 10900 | 0.0379 | - | - |
14.8048 | 11000 | 0.0231 | 0.3841 | 0.4112 |
14.9394 | 11100 | 0.0493 | - | - |
15.0 | 11145 | - | 0.3902 | 0.4136 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.11.9
- Sentence Transformers: 3.0.1
- Transformers: 4.43.3
- PyTorch: 2.3.1+cu121
- Accelerate: 0.30.1
- Datasets: 2.19.2
- Tokenizers: 0.19.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
- Downloads last month
- 8
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for antonkirk/retrieval-mpnet-dot-finetuned-combined-synthetic-dataset
Evaluation results
- Cosine Accuracy@1 on Unknownself-reported0.193
- Cosine Accuracy@3 on Unknownself-reported0.563
- Cosine Accuracy@5 on Unknownself-reported0.751
- Cosine Accuracy@10 on Unknownself-reported0.841
- Cosine Precision@1 on Unknownself-reported0.193
- Cosine Precision@3 on Unknownself-reported0.188
- Cosine Precision@5 on Unknownself-reported0.150
- Cosine Precision@10 on Unknownself-reported0.084
- Cosine Recall@1 on Unknownself-reported0.193
- Cosine Recall@3 on Unknownself-reported0.563