metadata

language: []
library_name: sentence-transformers
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:98928
  - loss:MultipleNegativesRankingLoss
base_model: sentence-transformers/multi-qa-mpnet-base-dot-v1
datasets: []
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
  - dot_accuracy@1
  - dot_accuracy@3
  - dot_accuracy@5
  - dot_accuracy@10
  - dot_precision@1
  - dot_precision@3
  - dot_precision@5
  - dot_precision@10
  - dot_recall@1
  - dot_recall@3
  - dot_recall@5
  - dot_recall@10
  - dot_ndcg@10
  - dot_mrr@10
  - dot_map@100
widget:
  - source_sentence: Abnormal wakefulness, loss of self-awareness, involuntary trance
    sentences:
      - 'See also: Neonatal lupus erythematosus'
      - >-
        Scalp-ear-nipple syndrome, as its name suggests, is a condition
        characterized by abnormalities of the scalp, ears, and nipples. Less
        frequently, affected individuals have problems affecting other parts of
        the body. The features of this disorder can vary even within the same
        family.


        Babies with scalp-ear-nipple syndrome are born with a condition called
        aplasia cutis congenita, which involves patchy abnormal areas (lesions)
        on the scalp. These lesions are firm, raised, hairless nodules that
        resemble open wounds or ulcers at birth, but that heal during childhood.


        The external ears of people with scalp-ear-nipple syndrome may be small,
        cup-shaped, folded over, or otherwise mildly misshapen. Hearing is
        generally normal. Affected individuals also have nipples that are
        underdeveloped (hypothelia) or absent (athelia). In some cases the
        underlying breast tissue is absent as well (amastia).
      - >-
        Abnormal state of wakefulness or altered state of consciousness


        For other uses, see Trance (disambiguation).


        This article needs additional citations for verification. Please help
        improve this article by adding citations to reliable sources. Unsourced
        material may be challenged and removed.  

        Find sources: "Trance" – news · newspapers · books · scholar · JSTOR
        (August 2010) (Learn how and when to remove this template message)  
          
        Dissociative trance  

        The Oracle at Delphi was famous for her  divinatory trances throughout
        the ancient Mediterranean world. Oil painting, John Collier, 1891  

        SpecialtyPsychiatry  
          
        Trance is an abnormal state of wakefulness in which a person is not
        self-aware and is either altogether unresponsive to external stimuli
        (but nevertheless capable of pursuing and realizing an aim) or is
        selectively responsive in following the directions of the person (if
        any) who has induced the trance. Trance states may occur involuntarily
        and unbidden.
  - source_sentence: respiratory infections, recurrent infections, primary immunodeficiency
    sentences:
      - >-
        A number sign (#) is used with this entry because of evidence that
        autosomal dominant common variable immunodeficiency-12 (CVID12) is
        caused by heterozygous mutation in the NFKB1 gene (164011) on chromosome
        4q24.


        Description


        Common variable immunodeficiency-12 is an autosomal dominant primary
        immunodeficiency characterized by recurrent infections, mainly
        respiratory, associated with hypogammaglobulinemia. The disorder shows a
        highly variable age at onset and highly variable disease severity, even
        within the same family. Some patients have features of autoimmunity
        (summary by Fliegauf et al., 2015).


        For a general description and a discussion of genetic heterogeneity of
        common variable immunodeficiency, see CVID1 (607594).


        Clinical Features
      - >-
        Kyasanura forest disease (KFD), caused by the KFD virus, is an arbovirus
        characterized by an initial fever, headache and myalgia that can
        progress to a hemorrhagic disease and that in some cases is followed by
        a second phase characterized by neurological manifestations.
      - >-
        A number sign (#) is used with this entry because of evidence that
        X-linked syndromic mental retardation-33 (MRXS33) is caused by mutation
        in the TAF1 gene (313650) on chromosome Xq13.


        Description


        X-linked syndromic mental retardation-33 is an X-linked recessive
        neurodevelopmental disorder characterized by delayed psychomotor
        development, intellectual disability, and characteristic facial features
        (summary by O'Rawe et al., 2015).


        Clinical Features
  - source_sentence: >-
      Common variable immunodeficiency, recurrent infections, impaired antibody
      production
    sentences:
      - >-
        A number sign (#) is used with this entry because this form of common
        variable immunodeficiency (CVID), referred to here as CVID5, is caused
        by homozygous mutation in the CD20 gene (MS4A1; 112210) on chromosome
        11q13.


        For a general description and a discussion of genetic heterogeneity of
        common variable immunodeficiency, see CVID1 (607594).


        Clinical Features
      - >-
        A number sign (#) is used with this entry because of evidence that the
        Stanescu type of spondyloepiphyseal dysplasia (SEDSTN) is caused by
        heterozygous mutation in the COL2A1 gene (120140) on chromosome 12q13.


        Description
      - >-
        ##  Description


        Macular dystrophies are inherited retinal dystrophies in which various
        forms of deposits, pigmentary changes, and atrophic lesions are observed
        in the macula lutea, the cone-rich region of the central retina.
        Vitelliform macular dystrophies (VMDs) form a subset of macular
        dystrophies characterized by round yellow deposits, usually at the
        center of the macula and containing lipofuscin, a chemically
        heterogeneous pigment visualized by autofluorescence imaging of the
        fundus (summary by Manes et al., 2013). In contrast to typical VMD (see
        153700), patients with atypical VMD may exhibit normal
        electrooculography, even when severe loss of vision is present, and
        fluorescein angiography is thus the most reliable test for identifying
        affected individuals (Hittner et al., 1984).


        ### Genetic Heterogeneity of Vitelliform Macular Dystrophy
  - source_sentence: >-
      Growth retardation, hearing impairment, joint hypermobility, sacral caudal
      remnant
    sentences:
      - >-
        A number sign (#) is used with this entry because Bruck syndrome-2
        (BRKS2) is caused by homozygous mutation in the PLOD2 gene (601865),
        which encodes telopeptide lysyl hydroxylase, on chromosome 3q24.


        For a phenotypic description and a discussion of genetic heterogeneity
        of Bruck syndrome, see Bruck syndrome-1 (259450).


        Clinical Features


        Ha-Vinh et al. (2004) described a child with Bruck syndrome who was the
        offspring of healthy nonconsanguineous Turkish parents. At birth,
        pterygia were present at the left elbow and at both knees, and extension
        of these joints was limited. Contractures were also present at the
        wrists, and there were bilateral clubfeet. Bilateral inguinal hernias
        were present. A fracture of the left arm was recognized immediately
        after birth, and the boy had 2 more fractures in the first 3 months of
        life. His urine contained high levels of hydroxyproline but low levels
        of collagen crosslinks degradation products.
      - >-
        ## Summary


        ### Clinical characteristics.


        Thrombocytopenia absent radius (TAR) syndrome is characterized by
        bilateral absence of the radii with the presence of both thumbs and
        thrombocytopenia (<50 platelets/nL) that is generally transient.
        Thrombocytopenia may be congenital or may develop within the first few
        weeks to months of life; in general, thrombocytopenic episodes decrease
        with age. Cow's milk allergy is common and can be associated with
        exacerbation of thrombocytopenia. Other anomalies of the skeleton (upper
        and lower limbs, ribs, and vertebrae), heart, and genitourinary system
        (renal anomalies and agenesis of uterus, cervix, and upper part of the
        vagina) can occur.


        ### Diagnosis/testing.
      - >-
        A rare multiple congenital anomalies/dysmorphic syndrome characterized
        by global developmental delay, intellectual disability, growth
        retardation, hearing impairment, characteristic facial dysmorphology
        (including prominent supraorbital ridges, downslanting palpebral
        fissures, deep-set eyes, long face, sagging cheeks, anteverted nares,
        and pointed chin), generalized hypotonia, joint hypermobility, gluteal
        crease with sacral caudal remnant and sacral dimple, and variable
        neurological features. Various ophthalmic, cutaneous, musculoskeletal,
        gastrointestinal, and cardiovascular anomalies have also been described.
  - source_sentence: ear malformations, nipple abnormalities, dental anomalies
    sentences:
      - >-
        This article is an orphan, as no other articles link to it. Please
        introduce links to this page from related articles; try the Find link
        tool for suggestions. (July 2016)  
          
        Inguinal lymphadenopathy  

        Inguinal lymphadenopathy  
          
        Inguinal lymphadenopathy causes swollen lymph nodes in the groin area.
        It can be a symptom of infective or neoplastic processes. Infective
        aetiologies include Tuberculosis, HIV, non-specific or reactive
        lymphadenopathy to recent lower limb infection or groin infections.
        Another notable infectious cause is Lymphogranuloma venereum, which is a
        sexually transmitted infection of the lymphatic system. Neoplastic
        aetiologies include lymphoma, leukaemia and metastatic disease from
        primary tumours in the lower limb, external genitalia or perianal region
        and melanoma.


        ## References[edit]

          * Ferrer R (October 1998). "Lymphadenopathy: differential diagnosis and evaluation". Am Fam Physician. 58 (6): 1313–20. PMID 9803196.

        ## Further reading[edit]
      - >-
        A number sign (#) is used with this entry because scalp-ear-nipple
        syndrome (SENS) is caused by heterozygous mutation in the KCTD1 gene
        (613420) on chromosome 18q11.


        Description


        Scalp-ear-nipple syndrome is characterized by aplasia cutis congenita of
        the scalp, breast anomalies that range from hypothelia or athelia to
        amastia, and minor anomalies of the external ears. Less frequent
        clinical characteristics include nail dystrophy, dental anomalies,
        cutaneous syndactyly of the digits, and renal malformations. Penetrance
        appears to be high, although there is substantial variable expressivity
        within families (Marneros et al., 2013).


        Clinical Features
      - >-
        Familial multiple meningioma is a rare, benign neoplasm of the central
        nervous system characterized by the development of multiple or, rarely,
        solitary meningiomas in two or more blood relatives, without other
        apparent syndromic manifestations. Depending on the localization, growth
        rate and size of the tumors, patients can present with subtle, gradually
        worsening or abrupt and severe neurological compromise or can be
        completely asymptomatic.
pipeline_tag: sentence-similarity
model-index:
  - name: >-
      SentenceTransformer based on
      sentence-transformers/multi-qa-mpnet-base-dot-v1
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: Unknown
          type: unknown
        metrics:
          - type: cosine_accuracy@1
            value: 0.18070477009024494
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5426514825956167
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.7380747743876236
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.8160721959604641
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.18070477009024494
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1808838275318722
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1476149548775247
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08160721959604642
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.18070477009024494
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5426514825956167
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.7380747743876236
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.8160721959604641
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.49469594615283
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.39074511770043246
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3952600557331103
            name: Cosine Map@100
          - type: dot_accuracy@1
            value: 0.18274602492479589
            name: Dot Accuracy@1
          - type: dot_accuracy@3
            value: 0.5412548345509239
            name: Dot Accuracy@3
          - type: dot_accuracy@5
            value: 0.7430167597765364
            name: Dot Accuracy@5
          - type: dot_accuracy@10
            value: 0.8167168027503223
            name: Dot Accuracy@10
          - type: dot_precision@1
            value: 0.18274602492479589
            name: Dot Precision@1
          - type: dot_precision@3
            value: 0.18041827818364134
            name: Dot Precision@3
          - type: dot_precision@5
            value: 0.1486033519553073
            name: Dot Precision@5
          - type: dot_precision@10
            value: 0.08167168027503223
            name: Dot Precision@10
          - type: dot_recall@1
            value: 0.18274602492479589
            name: Dot Recall@1
          - type: dot_recall@3
            value: 0.5412548345509239
            name: Dot Recall@3
          - type: dot_recall@5
            value: 0.7430167597765364
            name: Dot Recall@5
          - type: dot_recall@10
            value: 0.8167168027503223
            name: Dot Recall@10
          - type: dot_ndcg@10
            value: 0.4956715454485796
            name: Dot Ndcg@10
          - type: dot_mrr@10
            value: 0.391808804169147
            name: Dot Mrr@10
          - type: dot_map@100
            value: 0.39626188359327835
            name: Dot Map@100

SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1

This is a sentence-transformers model finetuned from sentence-transformers/multi-qa-mpnet-base-dot-v1. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/multi-qa-mpnet-base-dot-v1
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Dot Product

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'ear malformations, nipple abnormalities, dental anomalies',
    'A number sign (#) is used with this entry because scalp-ear-nipple syndrome (SENS) is caused by heterozygous mutation in the KCTD1 gene (613420) on chromosome 18q11.\n\nDescription\n\nScalp-ear-nipple syndrome is characterized by aplasia cutis congenita of the scalp, breast anomalies that range from hypothelia or athelia to amastia, and minor anomalies of the external ears. Less frequent clinical characteristics include nail dystrophy, dental anomalies, cutaneous syndactyly of the digits, and renal malformations. Penetrance appears to be high, although there is substantial variable expressivity within families (Marneros et al., 2013).\n\nClinical Features',
    'This article is an orphan, as no other articles link to it. Please introduce links to this page from related articles; try the Find link tool for suggestions. (July 2016)  \n  \nInguinal lymphadenopathy  \nInguinal lymphadenopathy  \n  \nInguinal lymphadenopathy causes swollen lymph nodes in the groin area. It can be a symptom of infective or neoplastic processes. Infective aetiologies include Tuberculosis, HIV, non-specific or reactive lymphadenopathy to recent lower limb infection or groin infections. Another notable infectious cause is Lymphogranuloma venereum, which is a sexually transmitted infection of the lymphatic system. Neoplastic aetiologies include lymphoma, leukaemia and metastatic disease from primary tumours in the lower limb, external genitalia or perianal region and melanoma.\n\n## References[edit]\n\n  * Ferrer R (October 1998). "Lymphadenopathy: differential diagnosis and evaluation". Am Fam Physician. 58 (6): 1313–20. PMID 9803196.\n\n## Further reading[edit]',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1807
cosine_accuracy@3	0.5427
cosine_accuracy@5	0.7381
cosine_accuracy@10	0.8161
cosine_precision@1	0.1807
cosine_precision@3	0.1809
cosine_precision@5	0.1476
cosine_precision@10	0.0816
cosine_recall@1	0.1807
cosine_recall@3	0.5427
cosine_recall@5	0.7381
cosine_recall@10	0.8161
cosine_ndcg@10	0.4947
cosine_mrr@10	0.3907
cosine_map@100	0.3953
dot_accuracy@1	0.1827
dot_accuracy@3	0.5413
dot_accuracy@5	0.743
dot_accuracy@10	0.8167
dot_precision@1	0.1827
dot_precision@3	0.1804
dot_precision@5	0.1486
dot_precision@10	0.0817
dot_recall@1	0.1827
dot_recall@3	0.5413
dot_recall@5	0.743
dot_recall@10	0.8167
dot_ndcg@10	0.4957
dot_mrr@10	0.3918
dot_map@100	0.3963

Training Details

Training Dataset

Unnamed Dataset

Size: 98,928 training samples
Columns: queries and chunks
Approximate statistics based on the first 1000 samples:
queries chunks
type string string
details
min: 7 tokens
mean: 17.4 tokens
max: 76 tokens

min: 5 tokens
mean: 159.93 tokens
max: 334 tokens

	queries	chunks
type	string	string
details	min: 7 tokens mean: 17.4 tokens max: 76 tokens	min: 5 tokens mean: 159.93 tokens max: 334 tokens

Samples:

queries	chunks
`fever, malaise, headaches, lymphadenopathy`	A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days.
`rash, papulovesicular, generalized, constitutional symptoms`	A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days.
`myalgia, diaphoresis, nausea, vomiting`	A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days.

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 1,
    "similarity_fct": "dot_score"
}

Evaluation Dataset

Unnamed Dataset

Size: 9,308 evaluation samples
Columns: queries and chunks
Approximate statistics based on the first 1000 samples:
queries chunks
type string string
details
min: 7 tokens
mean: 17.8 tokens
max: 48 tokens

min: 4 tokens
mean: 166.19 tokens
max: 299 tokens

	queries	chunks
type	string	string
details	min: 7 tokens mean: 17.8 tokens max: 48 tokens	min: 4 tokens mean: 166.19 tokens max: 299 tokens

Samples:

queries	chunks
`facial features, overgrowth, learning disabilities, delayed development`	Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited).
`long face, high forehead, flushed cheeks, small chin, down-slanting palpebral fissures`	Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited).
`intellectual disability, behavioral problems, speech and language difficulties, hypotonia`	Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited).

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 1,
    "similarity_fct": "dot_score"
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
learning_rate: 2e-05
num_train_epochs: 25
warmup_ratio: 0.1
fp16: True
load_best_model_at_end: True
eval_on_start: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 25
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: True
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: True
eval_use_gather_object: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	loss	dot_map@100
0	0	-	1.8701	0.2095
0.1295	100	1.5494	-	-
0.2591	200	0.9993	-	-
0.3886	300	0.7225	-	-
0.5181	400	0.6533	-	-
0.6477	500	0.6618	0.5939	0.3722
0.7772	600	0.6454	-	-
0.9067	700	0.5568	-	-
1.0363	800	0.5435	-	-
1.1658	900	0.499	-	-
1.2953	1000	0.5386	0.4768	0.3842
1.4249	1100	0.5077	-	-
1.5544	1200	0.4929	-	-
1.6839	1300	0.5194	-	-
1.8135	1400	0.5157	-	-
1.9430	1500	0.4337	0.4455	0.3894
2.0725	1600	0.4373	-	-
2.2021	1700	0.4569	-	-
2.3316	1800	0.4084	-	-
2.4611	1900	0.42	-	-
2.5907	2000	0.4112	0.4578	0.3886
2.7202	2100	0.4498	-	-
2.8497	2200	0.415	-	-
2.9793	2300	0.3734	-	-
3.1088	2400	0.3359	-	-
3.2383	2500	0.3923	0.4339	0.3929
3.3679	2600	0.3345	-	-
3.4974	2700	0.3324	-	-
3.6269	2800	0.3574	-	-
3.7565	2900	0.4078	-	-
3.8860	3000	0.3221	0.4293	0.3904
4.0155	3100	0.2895	-	-
4.1451	3200	0.2821	-	-
4.2746	3300	0.3192	-	-
4.4041	3400	0.28	-	-
4.5337	3500	0.2716	0.4486	0.3885
4.6632	3600	0.3147	-	-
4.7927	3700	0.3565	-	-
4.9223	3800	0.2465	-	-
5.0518	3900	0.2436	-	-
5.1813	4000	0.2297	0.4486	0.3917
5.3109	4100	0.2538	-	-
5.4404	4200	0.2448	-	-
5.5699	4300	0.2433	-	-
5.6995	4400	0.3017	-	-
5.8290	4500	0.2958	0.4737	0.3934
5.9585	4600	0.2142	-	-
6.0881	4700	0.1939	-	-
6.2176	4800	0.2449	-	-
6.3472	4900	0.2026	-	-
6.4767	5000	0.2006	0.4901	0.3895
6.6062	5100	0.2118	-	-
6.7358	5200	0.3064	-	-
6.8653	5300	0.2276	-	-
6.9948	5400	0.1809	-	-
7.1244	5500	0.1782	0.4992	0.3915
7.2539	5600	0.2211	-	-
7.3834	5700	0.1728	-	-
7.5130	5800	0.1651	-	-
7.6425	5900	0.2158	-	-
7.7720	6000	0.2864	0.5113	0.3892
7.9016	6100	0.179	-	-
8.0311	6200	0.1677	-	-
8.1606	6300	0.1517	-	-
8.2902	6400	0.1851	-	-
8.4197	6500	0.1646	0.5030	0.3933
8.5492	6600	0.1608	-	-
8.6788	6700	0.217	-	-
8.8083	6800	0.2357	-	-
8.9378	6900	0.1404	-	-
9.0674	7000	0.1465	0.5153	0.3877
9.1969	7100	0.1791	-	-
9.3264	7200	0.1261	-	-
9.4560	7300	0.1406	-	-
9.5855	7400	0.1626	-	-
9.7150	7500	0.223	0.5326	0.3939
9.8446	7600	0.1806	-	-
9.9741	7700	0.1289	-	-
10.1036	7800	0.1269	-	-
10.2332	7900	0.1609	-	-
10.3627	8000	0.1279	0.5113	0.3933
10.4922	8100	0.1264	-	-
10.6218	8200	0.1453	-	-
10.7513	8300	0.2227	-	-
10.8808	8400	0.1314	-	-
11.0104	8500	0.1192	0.5444	0.3925
11.1399	8600	0.1164	-	-
11.2694	8700	0.1418	-	-
11.3990	8800	0.1202	-	-
11.5285	8900	0.1152	-	-
11.658	9000	0.1454	0.529	0.3963
11.7876	9100	0.1952	-	-
11.9171	9200	0.1079	-	-
12.0466	9300	0.1139	-	-
12.1762	9400	0.1067	-	-
12.3057	9500	0.1219	0.5257	0.3938
12.4352	9600	0.119	-	-
12.5648	9700	0.1195	-	-
12.6943	9800	0.158	-	-
12.8238	9900	0.156	-	-
12.9534	10000	0.0974	0.5434	0.3934
13.0829	10100	0.0928	-	-
13.2124	10200	0.1266	-	-
13.3420	10300	0.0964	-	-
13.4715	10400	0.1007	-	-
13.6010	10500	0.112	0.5789	0.3893
13.7306	10600	0.1699	-	-
13.8601	10700	0.1084	-	-
13.9896	10800	0.0967	-	-
14.1192	10900	0.0856	-	-
14.2487	11000	0.1142	0.5252	0.3933
14.3782	11100	0.0891	-	-
14.5078	11200	0.0911	-	-
14.6373	11300	0.1128	-	-
14.7668	11400	0.1686	-	-
14.8964	11500	0.0874	0.5874	0.3945
15.0259	11600	0.0909	-	-
15.1554	11700	0.0778	-	-
15.2850	11800	0.1055	-	-
15.4145	11900	0.0872	-	-
15.5440	12000	0.0884	0.5894	0.3934
15.6736	12100	0.1101	-	-
15.8031	12200	0.1354	-	-
15.9326	12300	0.0762	-	-
16.0622	12400	0.0782	-	-
16.1917	12500	0.0936	0.5589	0.3919
16.3212	12600	0.072	-	-
16.4508	12700	0.0806	-	-
16.5803	12800	0.0929	-	-
16.7098	12900	0.1215	-	-
16.8394	13000	0.1039	0.6025	0.3926
16.9689	13100	0.0738	-	-
17.0984	13200	0.0651	-	-
17.2280	13300	0.0943	-	-
17.3575	13400	0.0678	-	-
17.4870	13500	0.077	0.6002	0.3941
17.6166	13600	0.0839	-	-
17.7461	13700	0.1268	-	-
17.8756	13800	0.0764	-	-
18.0052	13900	0.0686	-	-
18.1347	14000	0.0697	0.5898	0.3913
18.2642	14100	0.0871	-	-
18.3938	14200	0.0699	-	-
18.5233	14300	0.0611	-	-
18.6528	14400	0.0872	-	-
18.7824	14500	0.1281	0.6087	0.3927
18.9119	14600	0.0583	-	-
19.0415	14700	0.0658	-	-
19.1710	14800	0.0595	-	-
19.3005	14900	0.0816	-	-
19.4301	15000	0.0699	0.6078	0.3965
19.5596	15100	0.0729	-	-
19.6891	15200	0.0908	-	-
19.8187	15300	0.0978	-	-
19.9482	15400	0.0585	-	-
20.0777	15500	0.0557	0.5861	0.3925
20.2073	15600	0.0787	-	-
20.3368	15700	0.061	-	-
20.4663	15800	0.0638	-	-
20.5959	15900	0.0656	-	-
20.7254	16000	0.1003	0.6032	0.3923
20.8549	16100	0.0718	-	-
20.9845	16200	0.0625	-	-
21.1140	16300	0.0532	-	-
21.2435	16400	0.0739	-	-
21.3731	16500	0.0552	0.6080	0.3942
21.5026	16600	0.0588	-	-
21.6321	16700	0.0716	-	-
21.7617	16800	0.1078	-	-
21.8912	16900	0.0559	-	-
22.0207	17000	0.0596	0.6044	0.3922
22.1503	17100	0.0512	-	-
22.2798	17200	0.0716	-	-
22.4093	17300	0.0574	-	-
22.5389	17400	0.058	-	-
22.6684	17500	0.07	0.6117	0.3942
22.7979	17600	0.0965	-	-
22.9275	17700	0.0507	-	-
23.0570	17800	0.0498	-	-
23.1865	17900	0.0524	-	-
23.3161	18000	0.0656	0.5936	0.3936
23.4456	18100	0.057	-	-
23.5751	18200	0.0619	-	-
23.7047	18300	0.0785	-	-
23.8342	18400	0.0729	-	-
23.9637	18500	0.0541	0.6174	0.3979
24.0933	18600	0.0456	-	-
24.2228	18700	0.0696	-	-
24.3523	18800	0.048	-	-
24.4819	18900	0.0547	-	-
24.6114	19000	0.0553	0.6146	0.3962
24.7409	19100	0.0936	-	-
24.8705	19200	0.0579	-	-
25.0	19300	0.0498	0.5290	0.3963

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.11.9
Sentence Transformers: 3.0.1
Transformers: 4.43.3
PyTorch: 2.3.1+cu121
Accelerate: 0.30.1
Datasets: 2.19.2
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}