metadata

language:
  - en
license: apache-2.0
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:50000
  - loss:CachedGISTEmbedLoss
base_model: microsoft/mpnet-base
widget:
  - source_sentence: what does the accounts receivable turnover measure?
    sentences:
      - >-
        The accounts receivable turnover ratio is an accounting measure used to
        quantify a company's effectiveness in collecting its receivables or
        money owed by clients. The ratio shows how well a company uses and
        manages the credit it extends to customers and how quickly that
        short-term debt is collected or is paid.
      - >-
        Capital budgeting, and investment appraisal, is the planning process
        used to determine whether an organization's long term investments such
        as new machinery, replacement of machinery, new plants, new products,
        and research development projects are worth the funding of cash through
        the firm's capitalization structure ( ...
      - >-
        The accounts receivable turnover ratio is an accounting measure used to
        quantify a company's effectiveness in collecting its receivables or
        money owed by clients. The ratio shows how well a company uses and
        manages the credit it extends to customers and how quickly that
        short-term debt is collected or is paid.
  - source_sentence: does gabapentin cause liver problems?
    sentences:
      - >-
        Gabapentin has no appreciable liver metabolism, yet, suspected cases of
        gabapentin-induced hepatotoxicity have been reported. Per literature
        review, two cases of possible gabapentin-induced liver injury have been
        reported.
      - >-
        Strongholds are a type of story mission which only unlocks after enough
        progression through the game. There are three Stronghold's during the
        first section of progression through The Division 2. You'll need to
        complete the first two and have reached level 30 before being able to
        unlock the final Stronghold.
      - >-
        The most-common side effects attributed to Gabapentin include mild
        sedation, ataxia, and occasional diarrhea. Sedation can be minimized by
        tapering from a smaller starting dose to the desired dose. When treating
        seizures, it is ideal to wean off the drug to reduce the risk of
        withdrawal seizures.
  - source_sentence: how long should you wait to give blood after eating?
    sentences:
      - >-
        Until the bleeding has stopped it is natural to taste blood or to see
        traces of blood in your saliva. You may stop using gauze after the flow
        stops – usually around 8 hours after surgery.
      - >-
        Before donation The first and most important rule—never donate blood on
        an empty stomach. “Eat a wholesome meal about 2-3 hours before donating
        to keep your blood sugar stable," says Dr Chaturvedi. The timing of the
        meal is important too. You need to allow the food to be digested
        properly before the blood is drawn.
      - >-
        While grid computing involves virtualizing computing resources to store
        massive amounts of data, whereas cloud computing is where an application
        doesn't access resources directly, rather it accesses them through a
        service over the internet. ...
  - source_sentence: what is the difference between chicken francese and chicken marsala?
    sentences:
      - >-
        Chicken is the species name, equivalent to our “human.” Rooster is an
        adult male, equivalent to “man.” Hen is an adult female, equivalent to
        “woman.” Cockerel is a juvenile male, equivalent to “boy/young man.”
      - What is 99 kg in pounds? - 99 kg is equal to 218.26 pounds.
      - >-
        The difference between the two is for Francese, the chicken breast is
        first dipped in flour, then into a beaten egg mixture, before being
        cooked. For piccata, the chicken is first dipped in egg and then in
        flour. Both are then simmered in a lemony butter sauce, but the piccata
        sauce includes capers.”
  - source_sentence: what energy is released when coal is burned?
    sentences:
      - >-
        When coal is burned, it reacts with the oxygen in the air. This chemical
        reaction converts the stored solar energy into thermal energy, which is
        released as heat. But it also produces carbon dioxide and methane.
      - >-
        When coal is burned it releases a number of airborne toxins and
        pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides,
        particulates, and various other heavy metals.
      - >-
        Squad Building Challenges allow you to exchange sets of players for
        coins, packs, and special items in FUT 20. Each of these challenges come
        with specific requirements, such as including players from certain
        teams. ... Live SBCs are time-limited challenges which often give out
        unique, high-rated versions of players.
datasets:
  - tomaarsen/gooaq-hard-negatives
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
co2_eq_emissions:
  emissions: 40.54325678627484
  energy_consumed: 0.10430421450436282
  source: codecarbon
  training_type: fine-tuning
  on_cloud: false
  cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
  ram_total_size: 31.777088165283203
  hours_used: 0.301
  hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
  - name: MPNet base trained on Natural Questions pairs
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoClimateFEVER
          type: NanoClimateFEVER
        metrics:
          - type: cosine_accuracy@1
            value: 0.22
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.44
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.52
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.72
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.22
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16666666666666663
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.12
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09399999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.09333333333333332
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.195
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.2333333333333333
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.37233333333333335
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.2744024872493329
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3594365079365079
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.20181676147957636
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoDBPedia
          type: NanoDBPedia
        metrics:
          - type: cosine_accuracy@1
            value: 0.46
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.62
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.76
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.82
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.46
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.38666666666666666
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.38799999999999996
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.344
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.03065300183409328
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.07730098142643593
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.14588470319900892
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.22159653924772912
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3920743245484332
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.567
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.28153419189397744
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoFEVER
          type: NanoFEVER
        metrics:
          - type: cosine_accuracy@1
            value: 0.38
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.54
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.58
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.68
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.38
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.12
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.37
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.52
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.57
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.66
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5156585003907987
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4756666666666666
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.47620972127897226
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoFiQA2018
          type: NanoFiQA2018
        metrics:
          - type: cosine_accuracy@1
            value: 0.28
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.52
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.58
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.28
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.22
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.16399999999999998
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09799999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.1371904761904762
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.3226904761904762
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3682142857142857
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.43073809523809525
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3420135901424927
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.38405555555555554
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2826394452885763
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoHotpotQA
          type: NanoHotpotQA
        metrics:
          - type: cosine_accuracy@1
            value: 0.34
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.52
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.62
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.72
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.34
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.19333333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.14400000000000002
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09200000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.17
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.29
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.36
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.46
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3723049657456267
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4570793650793651
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2995175868330484
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoMSMARCO
          type: NanoMSMARCO
        metrics:
          - type: cosine_accuracy@1
            value: 0.1
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.28
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.52
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.68
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.1
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.09333333333333332
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.10400000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.068
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.1
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.28
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.52
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.68
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.36083481845261806
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.26157142857142857
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.27215692684924997
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoNFCorpus
          type: NanoNFCorpus
        metrics:
          - type: cosine_accuracy@1
            value: 0.26
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.38
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.44
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.5
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.26
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.21333333333333332
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19599999999999998
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.13799999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.01122167476431692
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.02047531859468654
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.03079316493603994
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.0422192068561938
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.1654539374427929
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3367460317460317
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.04901233559063261
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoNQ
          type: NanoNQ
        metrics:
          - type: cosine_accuracy@1
            value: 0.14
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.36
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.44
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.58
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.14
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.11999999999999998
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.08800000000000002
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.06000000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.13
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.34
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.41
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.55
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.33223439819785083
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.2734365079365079
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2764557370904448
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoQuoraRetrieval
          type: NanoQuoraRetrieval
        metrics:
          - type: cosine_accuracy@1
            value: 0.82
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.9
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.92
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.96
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.82
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.3666666666666666
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.244
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.13399999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7206666666666667
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8553333333333333
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8993333333333333
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9566666666666666
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8807317086981499
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8616666666666666
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8525831566094724
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoSCIDOCS
          type: NanoSCIDOCS
        metrics:
          - type: cosine_accuracy@1
            value: 0.34
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.48
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.54
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.66
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.34
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.24666666666666667
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.212
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.14800000000000002
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.07066666666666668
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.15366666666666667
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.21866666666666668
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.30466666666666664
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.28968259227673265
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4286349206349206
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.22985309744949503
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoArguAna
          type: NanoArguAna
        metrics:
          - type: cosine_accuracy@1
            value: 0.18
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.56
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.62
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.84
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.18
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18666666666666668
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.124
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08399999999999999
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.18
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.56
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.62
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.84
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.49726259302609505
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.389079365079365
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3967117258845785
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoSciFact
          type: NanoSciFact
        metrics:
          - type: cosine_accuracy@1
            value: 0.38
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.46
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.48
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.62
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.38
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16666666666666663
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.10400000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.068
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.345
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.44
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.46
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.605
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.47012843706683605
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4409285714285714
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.43840522432574647
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoTouche2020
          type: NanoTouche2020
        metrics:
          - type: cosine_accuracy@1
            value: 0.5306122448979592
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.7551020408163265
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8571428571428571
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9387755102040817
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5306122448979592
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.45578231292517
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.4040816326530612
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.336734693877551
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.03881638827876476
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.10008002766114979
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.13975964122053652
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.22966349775526734
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.39339080810676896
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6553206997084549
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.31344772891929434
            name: Cosine Map@100
      - task:
          type: nano-beir
          name: Nano BEIR
        dataset:
          name: NanoBEIR mean
          type: NanoBEIR_mean
        metrics:
          - type: cosine_accuracy@1
            value: 0.3408163265306122
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5227001569858712
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6013186813186814
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7152904238618524
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3408163265306122
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.23044479330193612
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1855447409733124
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.13344113029827318
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.18442678521033212
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.31958052337482684
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3827680868002465
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.4886833850587655
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4066287047188099
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4531247913084647
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.33618027996100497
            name: Cosine Map@100

MPNet base trained on Natural Questions pairs

This is a sentence-transformers model finetuned from microsoft/mpnet-base on the gooaq-hard-negatives dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: microsoft/mpnet-base
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- gooaq-hard-negatives
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("tomaarsen/mpnet-base-nq-cgist-triplet-neg-gte")
# Run inference
sentences = [
    'what energy is released when coal is burned?',
    'When coal is burned, it reacts with the oxygen in the air. This chemical reaction converts the stored solar energy into thermal energy, which is released as heat. But it also produces carbon dioxide and methane.',
    'When coal is burned it releases a number of airborne toxins and pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides, particulates, and various other heavy metals.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Datasets: NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoArguAna, NanoSciFact and NanoTouche2020
Evaluated with InformationRetrievalEvaluator

Metric	NanoClimateFEVER	NanoDBPedia	NanoFEVER	NanoFiQA2018	NanoHotpotQA	NanoMSMARCO	NanoNFCorpus	NanoNQ	NanoQuoraRetrieval	NanoSCIDOCS	NanoArguAna	NanoSciFact	NanoTouche2020
cosine_accuracy@1	0.22	0.46	0.38	0.28	0.34	0.1	0.26	0.14	0.82	0.34	0.18	0.38	0.5306
cosine_accuracy@3	0.44	0.62	0.54	0.5	0.52	0.28	0.38	0.36	0.9	0.48	0.56	0.46	0.7551
cosine_accuracy@5	0.52	0.76	0.58	0.52	0.62	0.52	0.44	0.44	0.92	0.54	0.62	0.48	0.8571
cosine_accuracy@10	0.72	0.82	0.68	0.58	0.72	0.68	0.5	0.58	0.96	0.66	0.84	0.62	0.9388
cosine_precision@1	0.22	0.46	0.38	0.28	0.34	0.1	0.26	0.14	0.82	0.34	0.18	0.38	0.5306
cosine_precision@3	0.1667	0.3867	0.18	0.22	0.1933	0.0933	0.2133	0.12	0.3667	0.2467	0.1867	0.1667	0.4558
cosine_precision@5	0.12	0.388	0.12	0.164	0.144	0.104	0.196	0.088	0.244	0.212	0.124	0.104	0.4041
cosine_precision@10	0.094	0.344	0.07	0.098	0.092	0.068	0.138	0.06	0.134	0.148	0.084	0.068	0.3367
cosine_recall@1	0.0933	0.0307	0.37	0.1372	0.17	0.1	0.0112	0.13	0.7207	0.0707	0.18	0.345	0.0388
cosine_recall@3	0.195	0.0773	0.52	0.3227	0.29	0.28	0.0205	0.34	0.8553	0.1537	0.56	0.44	0.1001
cosine_recall@5	0.2333	0.1459	0.57	0.3682	0.36	0.52	0.0308	0.41	0.8993	0.2187	0.62	0.46	0.1398
cosine_recall@10	0.3723	0.2216	0.66	0.4307	0.46	0.68	0.0422	0.55	0.9567	0.3047	0.84	0.605	0.2297
cosine_ndcg@10	0.2744	0.3921	0.5157	0.342	0.3723	0.3608	0.1655	0.3322	0.8807	0.2897	0.4973	0.4701	0.3934
cosine_mrr@10	0.3594	0.567	0.4757	0.3841	0.4571	0.2616	0.3367	0.2734	0.8617	0.4286	0.3891	0.4409	0.6553
cosine_map@100	0.2018	0.2815	0.4762	0.2826	0.2995	0.2722	0.049	0.2765	0.8526	0.2299	0.3967	0.4384	0.3134

Nano BEIR

Dataset: NanoBEIR_mean
Evaluated with NanoBEIREvaluator

Metric	Value
cosine_accuracy@1	0.3408
cosine_accuracy@3	0.5227
cosine_accuracy@5	0.6013
cosine_accuracy@10	0.7153
cosine_precision@1	0.3408
cosine_precision@3	0.2304
cosine_precision@5	0.1855
cosine_precision@10	0.1334
cosine_recall@1	0.1844
cosine_recall@3	0.3196
cosine_recall@5	0.3828
cosine_recall@10	0.4887
cosine_ndcg@10	0.4066
cosine_mrr@10	0.4531
cosine_map@100	0.3362

Training Details

Training Dataset

gooaq-hard-negatives

Dataset: gooaq-hard-negatives at 87594a1
Size: 50,000 training samples
Columns: question, answer, and negative

Approximate statistics based on the first 1000 samples:

	question	answer	negative
type	string	string	string
details	min: 8 tokens mean: 11.53 tokens max: 28 tokens	min: 14 tokens mean: 59.79 tokens max: 150 tokens	min: 15 tokens mean: 58.76 tokens max: 143 tokens

Samples:

question	answer	negative
`what is the difference between calories from fat and total fat?`	`Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories.`	`Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories.`
`what is the difference between return transcript and account transcript?`	`A tax return transcript usually meets the needs of lending institutions offering mortgages and student loans. ... Tax Account Transcript - shows basic data such as return type, marital status, adjusted gross income, taxable income and all payment types. It also shows changes made after you filed your original return.`	`Trial balance is not a financial statement whereas a balance sheet is a financial statement. Trial balance is solely used for internal purposes whereas a balance sheet is used for purposes other than internal i.e. external. In a trial balance, each and every account is divided into debit (dr.) and credit (cr.)`
`how long does my dog need to fast before sedation?`	`Now, guidelines are aimed towards 6-8 hours before surgery. This pre-op fasting time is much more beneficial for your pets because you have enough food in there to neutralize the stomach acid, preventing it from coming up the esophagus that causes regurgitation under anesthetic.`	`Try not to let your pooch rapidly wolf down his/her food! Do not let the dog play or exercise (e.g. go for a walk) for at least two hours after having a meal. Ensure continuous fresh water is available to avoid your pet gulping down a large amount after eating.`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.01}

Evaluation Dataset

gooaq-hard-negatives

Dataset: gooaq-hard-negatives at 87594a1
Size: 10,048,700 evaluation samples
Columns: question, answer, and negative

Approximate statistics based on the first 1000 samples:

	question	answer	negative
type	string	string	string
details	min: 8 tokens mean: 11.61 tokens max: 21 tokens	min: 16 tokens mean: 58.16 tokens max: 131 tokens	min: 14 tokens mean: 57.98 tokens max: 157 tokens

Samples:

question	answer	negative
`how is height width and length written?`	`The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width.`	`The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width. That's important.`
`what is the difference between pork shoulder and loin?`	`All the recipes I've found for pulled pork recommends a shoulder/butt. Shoulders take longer to cook than a loin, because they're tougher. Loins are lean, while shoulders have marbled fat inside.`	`They are extracted from the loin, which runs from the hip to the shoulder, and it has a small strip of meat called the tenderloin. Unlike other pork, this pork chop is cut from four major sections, which are the shoulder, also known as the blade chops, ribs chops, loin chops, and the last, which is the sirloin chops.`
`is the yin yang symbol religious?`	`The ubiquitous yin-yang symbol holds its roots in Taoism/Daoism, a Chinese religion and philosophy. The yin, the dark swirl, is associated with shadows, femininity, and the trough of a wave; the yang, the light swirl, represents brightness, passion and growth.`	`Yin energy is in the calm colors around you, in the soft music, in the soothing sound of a water fountain, or the relaxing images of water. Yang (active energy) is the feng shui energy expressed in strong, vibrant sounds and colors, bright lights, upward moving energy, tall plants, etc.`

Loss: CachedGISTEmbedLoss with these parameters:

{'guide': SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
), 'temperature': 0.01}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 2048
per_device_eval_batch_size: 2048
learning_rate: 2e-05
num_train_epochs: 1
warmup_ratio: 0.1
seed: 12
bf16: True

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 2048
per_device_eval_batch_size: 2048
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 12
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	NanoClimateFEVER_cosine_ndcg@10	NanoDBPedia_cosine_ndcg@10	NanoFEVER_cosine_ndcg@10	NanoFiQA2018_cosine_ndcg@10	NanoHotpotQA_cosine_ndcg@10	NanoMSMARCO_cosine_ndcg@10	NanoNFCorpus_cosine_ndcg@10	NanoNQ_cosine_ndcg@10	NanoQuoraRetrieval_cosine_ndcg@10	NanoSCIDOCS_cosine_ndcg@10	NanoArguAna_cosine_ndcg@10	NanoSciFact_cosine_ndcg@10	NanoTouche2020_cosine_ndcg@10	NanoBEIR_mean_cosine_ndcg@10
0.04	1	11.5141	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2	5	9.4407	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4	10	5.6005	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6	15	3.7323	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8	20	2.7976	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
1.0	25	2.1899	1.3429	0.2744	0.3921	0.5157	0.3420	0.3723	0.3608	0.1655	0.3322	0.8807	0.2897	0.4973	0.4701	0.3934	0.4066

Environmental Impact

Carbon emissions were measured using CodeCarbon.

Energy Consumed: 0.104 kWh
Carbon Emitted: 0.041 kg of CO2
Hours Used: 0.301 hours

Training Hardware

On Cloud: No
GPU Model: 1 x NVIDIA GeForce RTX 3090
CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
RAM Size: 31.78 GB

Framework Versions

Python: 3.11.6
Sentence Transformers: 3.4.0.dev0
Transformers: 4.46.2
PyTorch: 2.5.0+cu121
Accelerate: 0.35.0.dev0
Datasets: 2.20.0
Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}