--- language: [] library_name: sentence-transformers tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:98928 - loss:MultipleNegativesRankingLoss base_model: sentence-transformers/multi-qa-mpnet-base-dot-v1 datasets: [] metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 - dot_accuracy@1 - dot_accuracy@3 - dot_accuracy@5 - dot_accuracy@10 - dot_precision@1 - dot_precision@3 - dot_precision@5 - dot_precision@10 - dot_recall@1 - dot_recall@3 - dot_recall@5 - dot_recall@10 - dot_ndcg@10 - dot_mrr@10 - dot_map@100 widget: - source_sentence: Abnormal wakefulness, loss of self-awareness, involuntary trance sentences: - 'See also: Neonatal lupus erythematosus' - 'Scalp-ear-nipple syndrome, as its name suggests, is a condition characterized by abnormalities of the scalp, ears, and nipples. Less frequently, affected individuals have problems affecting other parts of the body. The features of this disorder can vary even within the same family. Babies with scalp-ear-nipple syndrome are born with a condition called aplasia cutis congenita, which involves patchy abnormal areas (lesions) on the scalp. These lesions are firm, raised, hairless nodules that resemble open wounds or ulcers at birth, but that heal during childhood. The external ears of people with scalp-ear-nipple syndrome may be small, cup-shaped, folded over, or otherwise mildly misshapen. Hearing is generally normal. Affected individuals also have nipples that are underdeveloped (hypothelia) or absent (athelia). In some cases the underlying breast tissue is absent as well (amastia).' - "Abnormal state of wakefulness or altered state of consciousness\n\nFor other\ \ uses, see Trance (disambiguation).\n\nThis article needs additional citations\ \ for verification. Please help improve this article by adding citations to reliable\ \ sources. Unsourced material may be challenged and removed. \nFind sources:\ \ \"Trance\" – news · newspapers · books · scholar · JSTOR (August 2010) (Learn\ \ how and when to remove this template message) \n \nDissociative trance \n\ The Oracle at Delphi was famous for her divinatory trances throughout the ancient\ \ Mediterranean world. Oil painting, John Collier, 1891 \nSpecialtyPsychiatry\ \ \n \nTrance is an abnormal state of wakefulness in which a person is not self-aware\ \ and is either altogether unresponsive to external stimuli (but nevertheless\ \ capable of pursuing and realizing an aim) or is selectively responsive in following\ \ the directions of the person (if any) who has induced the trance. Trance states\ \ may occur involuntarily and unbidden." - source_sentence: respiratory infections, recurrent infections, primary immunodeficiency sentences: - 'A number sign (#) is used with this entry because of evidence that autosomal dominant common variable immunodeficiency-12 (CVID12) is caused by heterozygous mutation in the NFKB1 gene (164011) on chromosome 4q24. Description Common variable immunodeficiency-12 is an autosomal dominant primary immunodeficiency characterized by recurrent infections, mainly respiratory, associated with hypogammaglobulinemia. The disorder shows a highly variable age at onset and highly variable disease severity, even within the same family. Some patients have features of autoimmunity (summary by Fliegauf et al., 2015). For a general description and a discussion of genetic heterogeneity of common variable immunodeficiency, see CVID1 (607594). Clinical Features' - Kyasanura forest disease (KFD), caused by the KFD virus, is an arbovirus characterized by an initial fever, headache and myalgia that can progress to a hemorrhagic disease and that in some cases is followed by a second phase characterized by neurological manifestations. - 'A number sign (#) is used with this entry because of evidence that X-linked syndromic mental retardation-33 (MRXS33) is caused by mutation in the TAF1 gene (313650) on chromosome Xq13. Description X-linked syndromic mental retardation-33 is an X-linked recessive neurodevelopmental disorder characterized by delayed psychomotor development, intellectual disability, and characteristic facial features (summary by O''Rawe et al., 2015). Clinical Features' - source_sentence: Common variable immunodeficiency, recurrent infections, impaired antibody production sentences: - 'A number sign (#) is used with this entry because this form of common variable immunodeficiency (CVID), referred to here as CVID5, is caused by homozygous mutation in the CD20 gene (MS4A1; 112210) on chromosome 11q13. For a general description and a discussion of genetic heterogeneity of common variable immunodeficiency, see CVID1 (607594). Clinical Features' - 'A number sign (#) is used with this entry because of evidence that the Stanescu type of spondyloepiphyseal dysplasia (SEDSTN) is caused by heterozygous mutation in the COL2A1 gene (120140) on chromosome 12q13. Description' - '## Description Macular dystrophies are inherited retinal dystrophies in which various forms of deposits, pigmentary changes, and atrophic lesions are observed in the macula lutea, the cone-rich region of the central retina. Vitelliform macular dystrophies (VMDs) form a subset of macular dystrophies characterized by round yellow deposits, usually at the center of the macula and containing lipofuscin, a chemically heterogeneous pigment visualized by autofluorescence imaging of the fundus (summary by Manes et al., 2013). In contrast to typical VMD (see 153700), patients with atypical VMD may exhibit normal electrooculography, even when severe loss of vision is present, and fluorescein angiography is thus the most reliable test for identifying affected individuals (Hittner et al., 1984). ### Genetic Heterogeneity of Vitelliform Macular Dystrophy' - source_sentence: Growth retardation, hearing impairment, joint hypermobility, sacral caudal remnant sentences: - 'A number sign (#) is used with this entry because Bruck syndrome-2 (BRKS2) is caused by homozygous mutation in the PLOD2 gene (601865), which encodes telopeptide lysyl hydroxylase, on chromosome 3q24. For a phenotypic description and a discussion of genetic heterogeneity of Bruck syndrome, see Bruck syndrome-1 (259450). Clinical Features Ha-Vinh et al. (2004) described a child with Bruck syndrome who was the offspring of healthy nonconsanguineous Turkish parents. At birth, pterygia were present at the left elbow and at both knees, and extension of these joints was limited. Contractures were also present at the wrists, and there were bilateral clubfeet. Bilateral inguinal hernias were present. A fracture of the left arm was recognized immediately after birth, and the boy had 2 more fractures in the first 3 months of life. His urine contained high levels of hydroxyproline but low levels of collagen crosslinks degradation products.' - '## Summary ### Clinical characteristics. Thrombocytopenia absent radius (TAR) syndrome is characterized by bilateral absence of the radii with the presence of both thumbs and thrombocytopenia (<50 platelets/nL) that is generally transient. Thrombocytopenia may be congenital or may develop within the first few weeks to months of life; in general, thrombocytopenic episodes decrease with age. Cow''s milk allergy is common and can be associated with exacerbation of thrombocytopenia. Other anomalies of the skeleton (upper and lower limbs, ribs, and vertebrae), heart, and genitourinary system (renal anomalies and agenesis of uterus, cervix, and upper part of the vagina) can occur. ### Diagnosis/testing.' - A rare multiple congenital anomalies/dysmorphic syndrome characterized by global developmental delay, intellectual disability, growth retardation, hearing impairment, characteristic facial dysmorphology (including prominent supraorbital ridges, downslanting palpebral fissures, deep-set eyes, long face, sagging cheeks, anteverted nares, and pointed chin), generalized hypotonia, joint hypermobility, gluteal crease with sacral caudal remnant and sacral dimple, and variable neurological features. Various ophthalmic, cutaneous, musculoskeletal, gastrointestinal, and cardiovascular anomalies have also been described. - source_sentence: ear malformations, nipple abnormalities, dental anomalies sentences: - "This article is an orphan, as no other articles link to it. Please introduce\ \ links to this page from related articles; try the Find link tool for suggestions.\ \ (July 2016) \n \nInguinal lymphadenopathy \nInguinal lymphadenopathy \n\ \ \nInguinal lymphadenopathy causes swollen lymph nodes in the groin area. It\ \ can be a symptom of infective or neoplastic processes. Infective aetiologies\ \ include Tuberculosis, HIV, non-specific or reactive lymphadenopathy to recent\ \ lower limb infection or groin infections. Another notable infectious cause is\ \ Lymphogranuloma venereum, which is a sexually transmitted infection of the lymphatic\ \ system. Neoplastic aetiologies include lymphoma, leukaemia and metastatic disease\ \ from primary tumours in the lower limb, external genitalia or perianal region\ \ and melanoma.\n\n## References[edit]\n\n * Ferrer R (October 1998). \"Lymphadenopathy:\ \ differential diagnosis and evaluation\". Am Fam Physician. 58 (6): 1313–20.\ \ PMID 9803196.\n\n## Further reading[edit]" - 'A number sign (#) is used with this entry because scalp-ear-nipple syndrome (SENS) is caused by heterozygous mutation in the KCTD1 gene (613420) on chromosome 18q11. Description Scalp-ear-nipple syndrome is characterized by aplasia cutis congenita of the scalp, breast anomalies that range from hypothelia or athelia to amastia, and minor anomalies of the external ears. Less frequent clinical characteristics include nail dystrophy, dental anomalies, cutaneous syndactyly of the digits, and renal malformations. Penetrance appears to be high, although there is substantial variable expressivity within families (Marneros et al., 2013). Clinical Features' - Familial multiple meningioma is a rare, benign neoplasm of the central nervous system characterized by the development of multiple or, rarely, solitary meningiomas in two or more blood relatives, without other apparent syndromic manifestations. Depending on the localization, growth rate and size of the tumors, patients can present with subtle, gradually worsening or abrupt and severe neurological compromise or can be completely asymptomatic. pipeline_tag: sentence-similarity model-index: - name: SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1 results: - task: type: information-retrieval name: Information Retrieval dataset: name: Unknown type: unknown metrics: - type: cosine_accuracy@1 value: 0.18070477009024494 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5426514825956167 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.7380747743876236 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.8160721959604641 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.18070477009024494 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.1808838275318722 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.1476149548775247 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.08160721959604642 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.18070477009024494 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.5426514825956167 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.7380747743876236 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.8160721959604641 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.49469594615283 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.39074511770043246 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.3952600557331103 name: Cosine Map@100 - type: dot_accuracy@1 value: 0.18274602492479589 name: Dot Accuracy@1 - type: dot_accuracy@3 value: 0.5412548345509239 name: Dot Accuracy@3 - type: dot_accuracy@5 value: 0.7430167597765364 name: Dot Accuracy@5 - type: dot_accuracy@10 value: 0.8167168027503223 name: Dot Accuracy@10 - type: dot_precision@1 value: 0.18274602492479589 name: Dot Precision@1 - type: dot_precision@3 value: 0.18041827818364134 name: Dot Precision@3 - type: dot_precision@5 value: 0.1486033519553073 name: Dot Precision@5 - type: dot_precision@10 value: 0.08167168027503223 name: Dot Precision@10 - type: dot_recall@1 value: 0.18274602492479589 name: Dot Recall@1 - type: dot_recall@3 value: 0.5412548345509239 name: Dot Recall@3 - type: dot_recall@5 value: 0.7430167597765364 name: Dot Recall@5 - type: dot_recall@10 value: 0.8167168027503223 name: Dot Recall@10 - type: dot_ndcg@10 value: 0.4956715454485796 name: Dot Ndcg@10 - type: dot_mrr@10 value: 0.391808804169147 name: Dot Mrr@10 - type: dot_map@100 value: 0.39626188359327835 name: Dot Map@100 --- # SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/multi-qa-mpnet-base-dot-v1](https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [sentence-transformers/multi-qa-mpnet-base-dot-v1](https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Dot Product ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("sentence_transformers_model_id") # Run inference sentences = [ 'ear malformations, nipple abnormalities, dental anomalies', 'A number sign (#) is used with this entry because scalp-ear-nipple syndrome (SENS) is caused by heterozygous mutation in the KCTD1 gene (613420) on chromosome 18q11.\n\nDescription\n\nScalp-ear-nipple syndrome is characterized by aplasia cutis congenita of the scalp, breast anomalies that range from hypothelia or athelia to amastia, and minor anomalies of the external ears. Less frequent clinical characteristics include nail dystrophy, dental anomalies, cutaneous syndactyly of the digits, and renal malformations. Penetrance appears to be high, although there is substantial variable expressivity within families (Marneros et al., 2013).\n\nClinical Features', 'This article is an orphan, as no other articles link to it. Please introduce links to this page from related articles; try the Find link tool for suggestions. (July 2016) \n \nInguinal lymphadenopathy \nInguinal lymphadenopathy \n \nInguinal lymphadenopathy causes swollen lymph nodes in the groin area. It can be a symptom of infective or neoplastic processes. Infective aetiologies include Tuberculosis, HIV, non-specific or reactive lymphadenopathy to recent lower limb infection or groin infections. Another notable infectious cause is Lymphogranuloma venereum, which is a sexually transmitted infection of the lymphatic system. Neoplastic aetiologies include lymphoma, leukaemia and metastatic disease from primary tumours in the lower limb, external genitalia or perianal region and melanoma.\n\n## References[edit]\n\n * Ferrer R (October 1998). "Lymphadenopathy: differential diagnosis and evaluation". Am Fam Physician. 58 (6): 1313–20. PMID 9803196.\n\n## Further reading[edit]', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.1807 | | cosine_accuracy@3 | 0.5427 | | cosine_accuracy@5 | 0.7381 | | cosine_accuracy@10 | 0.8161 | | cosine_precision@1 | 0.1807 | | cosine_precision@3 | 0.1809 | | cosine_precision@5 | 0.1476 | | cosine_precision@10 | 0.0816 | | cosine_recall@1 | 0.1807 | | cosine_recall@3 | 0.5427 | | cosine_recall@5 | 0.7381 | | cosine_recall@10 | 0.8161 | | cosine_ndcg@10 | 0.4947 | | cosine_mrr@10 | 0.3907 | | cosine_map@100 | 0.3953 | | dot_accuracy@1 | 0.1827 | | dot_accuracy@3 | 0.5413 | | dot_accuracy@5 | 0.743 | | dot_accuracy@10 | 0.8167 | | dot_precision@1 | 0.1827 | | dot_precision@3 | 0.1804 | | dot_precision@5 | 0.1486 | | dot_precision@10 | 0.0817 | | dot_recall@1 | 0.1827 | | dot_recall@3 | 0.5413 | | dot_recall@5 | 0.743 | | dot_recall@10 | 0.8167 | | dot_ndcg@10 | 0.4957 | | dot_mrr@10 | 0.3918 | | **dot_map@100** | **0.3963** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 98,928 training samples * Columns: queries and chunks * Approximate statistics based on the first 1000 samples: | | queries | chunks | |:--------|:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | queries | chunks | |:-------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | fever, malaise, headaches, lymphadenopathy | A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days. | | rash, papulovesicular, generalized, constitutional symptoms | A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days. | | myalgia, diaphoresis, nausea, vomiting | A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days. | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 1, "similarity_fct": "dot_score" } ``` ### Evaluation Dataset #### Unnamed Dataset * Size: 9,308 evaluation samples * Columns: queries and chunks * Approximate statistics based on the first 1000 samples: | | queries | chunks | |:--------|:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | queries | chunks | |:-------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | facial features, overgrowth, learning disabilities, delayed development | Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited). | | long face, high forehead, flushed cheeks, small chin, down-slanting palpebral fissures | Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited). | | intellectual disability, behavioral problems, speech and language difficulties, hypotonia | Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited). | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 1, "similarity_fct": "dot_score" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `learning_rate`: 2e-05 - `num_train_epochs`: 25 - `warmup_ratio`: 0.1 - `fp16`: True - `load_best_model_at_end`: True - `eval_on_start`: True - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 32 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 25 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: True - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: True - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: True - `eval_use_gather_object`: False - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs
Click to expand | Epoch | Step | Training Loss | loss | dot_map@100 | |:----------:|:--------:|:-------------:|:---------:|:-----------:| | 0 | 0 | - | 1.8701 | 0.2095 | | 0.1295 | 100 | 1.5494 | - | - | | 0.2591 | 200 | 0.9993 | - | - | | 0.3886 | 300 | 0.7225 | - | - | | 0.5181 | 400 | 0.6533 | - | - | | 0.6477 | 500 | 0.6618 | 0.5939 | 0.3722 | | 0.7772 | 600 | 0.6454 | - | - | | 0.9067 | 700 | 0.5568 | - | - | | 1.0363 | 800 | 0.5435 | - | - | | 1.1658 | 900 | 0.499 | - | - | | 1.2953 | 1000 | 0.5386 | 0.4768 | 0.3842 | | 1.4249 | 1100 | 0.5077 | - | - | | 1.5544 | 1200 | 0.4929 | - | - | | 1.6839 | 1300 | 0.5194 | - | - | | 1.8135 | 1400 | 0.5157 | - | - | | 1.9430 | 1500 | 0.4337 | 0.4455 | 0.3894 | | 2.0725 | 1600 | 0.4373 | - | - | | 2.2021 | 1700 | 0.4569 | - | - | | 2.3316 | 1800 | 0.4084 | - | - | | 2.4611 | 1900 | 0.42 | - | - | | 2.5907 | 2000 | 0.4112 | 0.4578 | 0.3886 | | 2.7202 | 2100 | 0.4498 | - | - | | 2.8497 | 2200 | 0.415 | - | - | | 2.9793 | 2300 | 0.3734 | - | - | | 3.1088 | 2400 | 0.3359 | - | - | | 3.2383 | 2500 | 0.3923 | 0.4339 | 0.3929 | | 3.3679 | 2600 | 0.3345 | - | - | | 3.4974 | 2700 | 0.3324 | - | - | | 3.6269 | 2800 | 0.3574 | - | - | | 3.7565 | 2900 | 0.4078 | - | - | | 3.8860 | 3000 | 0.3221 | 0.4293 | 0.3904 | | 4.0155 | 3100 | 0.2895 | - | - | | 4.1451 | 3200 | 0.2821 | - | - | | 4.2746 | 3300 | 0.3192 | - | - | | 4.4041 | 3400 | 0.28 | - | - | | 4.5337 | 3500 | 0.2716 | 0.4486 | 0.3885 | | 4.6632 | 3600 | 0.3147 | - | - | | 4.7927 | 3700 | 0.3565 | - | - | | 4.9223 | 3800 | 0.2465 | - | - | | 5.0518 | 3900 | 0.2436 | - | - | | 5.1813 | 4000 | 0.2297 | 0.4486 | 0.3917 | | 5.3109 | 4100 | 0.2538 | - | - | | 5.4404 | 4200 | 0.2448 | - | - | | 5.5699 | 4300 | 0.2433 | - | - | | 5.6995 | 4400 | 0.3017 | - | - | | 5.8290 | 4500 | 0.2958 | 0.4737 | 0.3934 | | 5.9585 | 4600 | 0.2142 | - | - | | 6.0881 | 4700 | 0.1939 | - | - | | 6.2176 | 4800 | 0.2449 | - | - | | 6.3472 | 4900 | 0.2026 | - | - | | 6.4767 | 5000 | 0.2006 | 0.4901 | 0.3895 | | 6.6062 | 5100 | 0.2118 | - | - | | 6.7358 | 5200 | 0.3064 | - | - | | 6.8653 | 5300 | 0.2276 | - | - | | 6.9948 | 5400 | 0.1809 | - | - | | 7.1244 | 5500 | 0.1782 | 0.4992 | 0.3915 | | 7.2539 | 5600 | 0.2211 | - | - | | 7.3834 | 5700 | 0.1728 | - | - | | 7.5130 | 5800 | 0.1651 | - | - | | 7.6425 | 5900 | 0.2158 | - | - | | 7.7720 | 6000 | 0.2864 | 0.5113 | 0.3892 | | 7.9016 | 6100 | 0.179 | - | - | | 8.0311 | 6200 | 0.1677 | - | - | | 8.1606 | 6300 | 0.1517 | - | - | | 8.2902 | 6400 | 0.1851 | - | - | | 8.4197 | 6500 | 0.1646 | 0.5030 | 0.3933 | | 8.5492 | 6600 | 0.1608 | - | - | | 8.6788 | 6700 | 0.217 | - | - | | 8.8083 | 6800 | 0.2357 | - | - | | 8.9378 | 6900 | 0.1404 | - | - | | 9.0674 | 7000 | 0.1465 | 0.5153 | 0.3877 | | 9.1969 | 7100 | 0.1791 | - | - | | 9.3264 | 7200 | 0.1261 | - | - | | 9.4560 | 7300 | 0.1406 | - | - | | 9.5855 | 7400 | 0.1626 | - | - | | 9.7150 | 7500 | 0.223 | 0.5326 | 0.3939 | | 9.8446 | 7600 | 0.1806 | - | - | | 9.9741 | 7700 | 0.1289 | - | - | | 10.1036 | 7800 | 0.1269 | - | - | | 10.2332 | 7900 | 0.1609 | - | - | | 10.3627 | 8000 | 0.1279 | 0.5113 | 0.3933 | | 10.4922 | 8100 | 0.1264 | - | - | | 10.6218 | 8200 | 0.1453 | - | - | | 10.7513 | 8300 | 0.2227 | - | - | | 10.8808 | 8400 | 0.1314 | - | - | | 11.0104 | 8500 | 0.1192 | 0.5444 | 0.3925 | | 11.1399 | 8600 | 0.1164 | - | - | | 11.2694 | 8700 | 0.1418 | - | - | | 11.3990 | 8800 | 0.1202 | - | - | | 11.5285 | 8900 | 0.1152 | - | - | | **11.658** | **9000** | **0.1454** | **0.529** | **0.3963** | | 11.7876 | 9100 | 0.1952 | - | - | | 11.9171 | 9200 | 0.1079 | - | - | | 12.0466 | 9300 | 0.1139 | - | - | | 12.1762 | 9400 | 0.1067 | - | - | | 12.3057 | 9500 | 0.1219 | 0.5257 | 0.3938 | | 12.4352 | 9600 | 0.119 | - | - | | 12.5648 | 9700 | 0.1195 | - | - | | 12.6943 | 9800 | 0.158 | - | - | | 12.8238 | 9900 | 0.156 | - | - | | 12.9534 | 10000 | 0.0974 | 0.5434 | 0.3934 | | 13.0829 | 10100 | 0.0928 | - | - | | 13.2124 | 10200 | 0.1266 | - | - | | 13.3420 | 10300 | 0.0964 | - | - | | 13.4715 | 10400 | 0.1007 | - | - | | 13.6010 | 10500 | 0.112 | 0.5789 | 0.3893 | | 13.7306 | 10600 | 0.1699 | - | - | | 13.8601 | 10700 | 0.1084 | - | - | | 13.9896 | 10800 | 0.0967 | - | - | | 14.1192 | 10900 | 0.0856 | - | - | | 14.2487 | 11000 | 0.1142 | 0.5252 | 0.3933 | | 14.3782 | 11100 | 0.0891 | - | - | | 14.5078 | 11200 | 0.0911 | - | - | | 14.6373 | 11300 | 0.1128 | - | - | | 14.7668 | 11400 | 0.1686 | - | - | | 14.8964 | 11500 | 0.0874 | 0.5874 | 0.3945 | | 15.0259 | 11600 | 0.0909 | - | - | | 15.1554 | 11700 | 0.0778 | - | - | | 15.2850 | 11800 | 0.1055 | - | - | | 15.4145 | 11900 | 0.0872 | - | - | | 15.5440 | 12000 | 0.0884 | 0.5894 | 0.3934 | | 15.6736 | 12100 | 0.1101 | - | - | | 15.8031 | 12200 | 0.1354 | - | - | | 15.9326 | 12300 | 0.0762 | - | - | | 16.0622 | 12400 | 0.0782 | - | - | | 16.1917 | 12500 | 0.0936 | 0.5589 | 0.3919 | | 16.3212 | 12600 | 0.072 | - | - | | 16.4508 | 12700 | 0.0806 | - | - | | 16.5803 | 12800 | 0.0929 | - | - | | 16.7098 | 12900 | 0.1215 | - | - | | 16.8394 | 13000 | 0.1039 | 0.6025 | 0.3926 | | 16.9689 | 13100 | 0.0738 | - | - | | 17.0984 | 13200 | 0.0651 | - | - | | 17.2280 | 13300 | 0.0943 | - | - | | 17.3575 | 13400 | 0.0678 | - | - | | 17.4870 | 13500 | 0.077 | 0.6002 | 0.3941 | | 17.6166 | 13600 | 0.0839 | - | - | | 17.7461 | 13700 | 0.1268 | - | - | | 17.8756 | 13800 | 0.0764 | - | - | | 18.0052 | 13900 | 0.0686 | - | - | | 18.1347 | 14000 | 0.0697 | 0.5898 | 0.3913 | | 18.2642 | 14100 | 0.0871 | - | - | | 18.3938 | 14200 | 0.0699 | - | - | | 18.5233 | 14300 | 0.0611 | - | - | | 18.6528 | 14400 | 0.0872 | - | - | | 18.7824 | 14500 | 0.1281 | 0.6087 | 0.3927 | | 18.9119 | 14600 | 0.0583 | - | - | | 19.0415 | 14700 | 0.0658 | - | - | | 19.1710 | 14800 | 0.0595 | - | - | | 19.3005 | 14900 | 0.0816 | - | - | | 19.4301 | 15000 | 0.0699 | 0.6078 | 0.3965 | | 19.5596 | 15100 | 0.0729 | - | - | | 19.6891 | 15200 | 0.0908 | - | - | | 19.8187 | 15300 | 0.0978 | - | - | | 19.9482 | 15400 | 0.0585 | - | - | | 20.0777 | 15500 | 0.0557 | 0.5861 | 0.3925 | | 20.2073 | 15600 | 0.0787 | - | - | | 20.3368 | 15700 | 0.061 | - | - | | 20.4663 | 15800 | 0.0638 | - | - | | 20.5959 | 15900 | 0.0656 | - | - | | 20.7254 | 16000 | 0.1003 | 0.6032 | 0.3923 | | 20.8549 | 16100 | 0.0718 | - | - | | 20.9845 | 16200 | 0.0625 | - | - | | 21.1140 | 16300 | 0.0532 | - | - | | 21.2435 | 16400 | 0.0739 | - | - | | 21.3731 | 16500 | 0.0552 | 0.6080 | 0.3942 | | 21.5026 | 16600 | 0.0588 | - | - | | 21.6321 | 16700 | 0.0716 | - | - | | 21.7617 | 16800 | 0.1078 | - | - | | 21.8912 | 16900 | 0.0559 | - | - | | 22.0207 | 17000 | 0.0596 | 0.6044 | 0.3922 | | 22.1503 | 17100 | 0.0512 | - | - | | 22.2798 | 17200 | 0.0716 | - | - | | 22.4093 | 17300 | 0.0574 | - | - | | 22.5389 | 17400 | 0.058 | - | - | | 22.6684 | 17500 | 0.07 | 0.6117 | 0.3942 | | 22.7979 | 17600 | 0.0965 | - | - | | 22.9275 | 17700 | 0.0507 | - | - | | 23.0570 | 17800 | 0.0498 | - | - | | 23.1865 | 17900 | 0.0524 | - | - | | 23.3161 | 18000 | 0.0656 | 0.5936 | 0.3936 | | 23.4456 | 18100 | 0.057 | - | - | | 23.5751 | 18200 | 0.0619 | - | - | | 23.7047 | 18300 | 0.0785 | - | - | | 23.8342 | 18400 | 0.0729 | - | - | | 23.9637 | 18500 | 0.0541 | 0.6174 | 0.3979 | | 24.0933 | 18600 | 0.0456 | - | - | | 24.2228 | 18700 | 0.0696 | - | - | | 24.3523 | 18800 | 0.048 | - | - | | 24.4819 | 18900 | 0.0547 | - | - | | 24.6114 | 19000 | 0.0553 | 0.6146 | 0.3962 | | 24.7409 | 19100 | 0.0936 | - | - | | 24.8705 | 19200 | 0.0579 | - | - | | 25.0 | 19300 | 0.0498 | 0.5290 | 0.3963 | * The bold row denotes the saved checkpoint.
### Framework Versions - Python: 3.11.9 - Sentence Transformers: 3.0.1 - Transformers: 4.43.3 - PyTorch: 2.3.1+cu121 - Accelerate: 0.30.1 - Datasets: 2.19.2 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```