README.md

metadata

base_model: BAAI/bge-base-en-v1.5
datasets: []
language:
  - en
library_name: sentence-transformers
license: apache-2.0
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:3305
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >

      Limitation of Liability


      CUSTOMER’S ENTIRE LIABILITY AND PACNET’S EXCLUSIVE REMEDIES AGAINST
      CUSTOMER FOR ANY DAMAGES ARISING

      FROM ANY ACT OR OMISSION RELATING TO THE SERVICES, REGARDLESS OF THE FORM
      OF ACTION, WHETHER IN CONTRACT,

      UNDER STATUTE, IN TORT OR OTHERWISE, INCLUDING NEGLIGENCE, WILL BE
      LIMITED, FOR EACH EVENT OR SERIES OF

      CONNECTED EVENTS, AS FOLLOWS:


      FOR PERSONAL INJURY OR DEATH, UNLIMITED, BUT SUBJECT TO PROVEN DIRECT
      DAMAGES; AND

      FOR ALL OTHER EVENTS, SUBJECT TO A MAXIMUM EQUAL TO THE AGGREGATE MONTHLY
      SERVICE CHARGES PAID OR

      PAYBALE BY THE CUSTOMER UNDER THE AGREEMENT.



      PACNET’S ENTIRE LIABILITY AND CUSTOMER’S EXCLUSIVE REMEDIES AGAINST PACNET
      OR ITS AFFILIATES FOR

      ANY DAMAGES ARISING FROM ANY ACT OR OMISSION RELATING TO THE AGREEMENT,
      REGARDLESS OF THE

      FORM OF ACTION, WHETHER IN CONTRACT, UNDER STATUTE, IN TORT OR OTHERWISE,
      INCLUDING

      NEGLIGENCE, WILL BE LIMITED, FOR EACH EVENT OR SERIES OF CONNECTED EVENTS,
      AS FOLLOWS:


      {i} | FOR PERSONAL INJURY OR DEATH, UNLIMITED, BUT SUBJECT TO PROVEN
      DIRECT DAMAGES;


      (ii) FOR FAILURE TO COMPLY WITH SERVICE LEVELS, TO THE AMOUNT OF CREDITS
      SET OUT IN THE

      RELEVANT SPECIFIC CONDITIONS OF THE RELEVANT SERVICE; AND


      (iii) FOR ALL OTHER EVENTS, SUBJECT TO A MAXIMUM EQUAL TO THE AGGREGATE
      MONTHLY SERVICE

      CHARGES PAID OR PAYABLE BY THE CUSTOMER UNDER THE AGREEMENT.

      .


      PACNET WILL IN NO CIRCUMSTANCES BE LIABLE FOR ANY DAMAGES (EXCEPT
      RESULTING IN PERSONAL INJURY

      OR DEATH) ATTRIBUTABLE TO ANY SERVICE, PRODUCT OR ACTIONS OF ANY PERSON
      OTHER THAN PACNET, ITS

      EMPLOYEES AND AGENTS.
    sentences:
      - Auto Renewal Cancellation Notice Period
      - Assignment
      - Absolute Maximum Amount of Liability
  - source_sentence: |

      Subcontracting

      (a) The Supplier must not subcontract any of its
      obligations under this Agreement, without the
      Company's prior written consent (which will not
      be unreasonably withheld).

      (b) The Supplier remains fully responsible for acts
      and omissions of its subcontractors and Supplier
      Personnel in connection with this Agreement or a
      Statement of Work as if they were its acts and
      omissions.


      Personnel

      (a) At the Company's reasonable request the
      Supplier must, at its cost, immediately (or by any
      date nominated by the Company) remove any
      person nominated by the Company from the
      performance of the Services and, if requested by
      the Company, provide an alternative person
      acceptable to the Company (acting reasonably).

      (b) The Supplier will not remove (temporarily or
      permanently) or replace a Key Personnel without
      the Company’s prior written consent (which must
      not be unreasonably withheld). Any substitute
      personnel must be at least equally qualified for
      the duties of the position as the person for whom
      they are substituted. The Supplier must use
      reasonable endeavours to provide uninterrupted
      transition between Key Personnel and their
      replacements.
    sentences:
      - Audit Rights
      - Severability
      - Subcontracting
  - source_sentence: >-
      All Intellectual Property shall be deemed to be owned by the Employer and
      Executive hereby relinquishes any right or claim to any such Intellectual
      Property except to the extent necessary to transfer the ownership of any
      such Intellectual Property to Employer. Executive shall promptly disclose
      to the Employer all Intellectual Property. Without royalty or separate
      consideration, Executive hereby assigns and agrees to assign to the
      Employer (or as otherwise directed by the Employer) Executive’s full
      right, title and interest in and to all Intellectual Property, including
      without limitation all copyright interests therein. Executive agrees to
      cooperate with Employer and to execute any and all applications for
      domestic and foreign patents, copyrights or other proprietary rights and
      to do such other acts (including, among other things, the execution and
      delivery of instruments of further assurance or confirmation) requested by
      the Employer to assign the Intellectual Property to the Employer and to
      permit the Employer to file, obtain and enforce any patents, copyrights or
      other proprietary rights in the Intellectual Property. Executive agrees
      that Executive’s obligation to cooperate and to execute, or cause to be
      executed, when it is in Executive’s power to do so, any such instrument or
      paper, will continue after termination of this Agreement. Executive agrees
      to make and maintain adequate and current written records of all
      Intellectual Property, in the form of notes, sketches, drawings, or
      reports relating hereto, which records shall be and remain the property of
      and available to the Employer at all times. The parties agree that the
      Intellectual Property does not include the items listed in the attached
      Exhibit A to this Agreement.
    sentences:
      - General Indemnities
      - Intellectual Property Ownership
      - Governing Law
  - source_sentence: >
      CBRE

      .


      HEVERTECH LTD

      .


      .
            
      .


      Preferred Supplier Light/Agreement

      .


      .
        
      .
       
      .


      Agreement; Number: NMS/16/050 |

      .


      QUALIFIED SERVICE LEVEL AGREEMENT

      BETWEEN

      CBRE MANAGED SERVICES LIMITED

      AND

      HEVERTECH LTDCBRE

      .


      HEVERTECH LTD

      .


      .
          
      .
       
      .


      .
       
      .
       
      .


      .
        
      .
          
      .


      Preferred!Supplier Light Agreement

      i]

      .


      W\olaclelealelaters Ulin elsiea Niky AeyAOkLY)

      .


      TABLE OF CONTENTS:

      .


      Qualified Service Level Agreement Pages 03 to 10 inclusive

      .


      Appendix 1 — Schedule of Rates Page

      .


      Appendix 2 — Key Contacts and Escalation Process Pages 8 to 9 inclusive

      .


      Appendix 3 - Working Capital Scheme Pages 10

      .


      .
       
      .


      CBRE Managed Services Lid

      February 2016 Page 2 of LOPreferred Supplier Light'A greement HEVERTECH
      LTD

      .


      Agreement Number: NMS/16/050

      .


      .
       


      THIS AGREEMENT is made on 1° June 2016

      BETWEEN


      (1) CBRE Managed Services Limited (Registered in England No. 1799580)
      whose registered

      office is at City Bridge House, 57 Southwark Street, London, SE1 1RU
      (“CBRE”); and


      (2) Hevertech Ltd (Registered in England No. 2803522) whose registered
      office is at: Unit 2

      Treefield Industrial Estate, Gildersome, Leeds, LS27 7JU (the “Supplier’).
    sentences:
      - Non Solicitation
      - Intellectual Property Infringement Indemnity
      - Title of Agreement
  - source_sentence: >

      The management of each individual entity within a suppliers organization
      is responsible for

      implementing the VAT Supplier Code of Conduct in their respective area of
      responsibility. They are

      obliged to take all appropriate action and provide the required structures
      and resources to ensure

      that all employees in the entity are familiar with the VAT Supplier Code
      of Conduct and that its

      principles are fully implemented.

      .


      All VAT suppliers are encouraged to direct any questions they might have
      with regard to the

      contents, interpretation or implementation of the VAT Supplier Code of
      Conduct to the VAT Strategic

      Procurement function.

      .


      .
       
      .


      Document created Release

      Name Index Date

      .


      .
       
      .


      File name

      .


      .
       
      .


      .
       
      .


      .
       
      .


      .
       
      .


      PMS Document BPO1FO30EA MEY A 18.11.2014

      .


      .
       
      .


      .
       
      .


      .
       
      .


      .
       
      .


      .
        
      .


      WAT Strategic Procurement BP01FO30E

      .


      Vakuumventile AG Supplier Code of Conduct Page 3 of 3

      .


      .
       
      .


      .
       
      .


      .
       
      .


      We, the undersigned, hereby confirm and declare in the name and on behalf
      of our company that

      .


      1. we have received the VAT Supplier Code of Condex;

      .


      2. by signing this declaration, we accept and commit to complying with all
      rules and requirements as

      laid out in the VAT Supplier Code of Conduct;

      .


      3. we accept that this declaration shall be exclusively governed by the
      material laws of Switzerland,

      excluding the UN Law of Sales (CISG).

      .


      PlacelDate —-Singagore. / tone 2077

      Company Kien Ann Engineering Pe ad

      Street 3c 500 kovo Cirle

      .


      Post codelcity Singapore 627035

      .


      Name of authorized signatory Jameson Low

      .


      L. Ze

      Signature << : Ly eA

      * fh

      20,

      Xn _A

      CETES

      .


      1. Please sign one (1) original c Of this document.

      2. Please note that only duly authorized personnel of your company may
      sign this document.

      3. Please send the duly signed original copy by conventional mail to:

      VAT VAKUUMVENTILE AG, SEELISTRASSE 1, STRATEGISCHER EINKAUF, CH-9469 HAAG

      .


      .
       
      .


      .
         
      .


      .
       
      .


      Document created Release

      .


      .
       
      .


      File name

      Name Index Date

      .


      .
       
      .


      .
       
      .


      .
       
      .


      .
       
      .


      .
       
      .


      .
       
      .


      .
       
      .


      PMS Document BPO1FO30EA MEY A 18.11.2014

       

       

      All business is conducted in compliance with governing national and
      international laws and

      regulations. As a matter of principle, we honor agreements and obligations
      we have entered into

      voluntarily. All suppliers are obliged to carefully study the rules and
      regulations pertinent to their

      area of responsibility and ensure full compliance. In case of doubt or
      queries, they are obliged to

      seek additional information and guidance from the appropriate channels or
      persons in charge. VAT

      has a zero tolerance policy with regard to violations of its Supplier Code
      of Conduct. Violations may

      lead to appropriate action being taken against the supplier.

      .


      2. Fair competition
    sentences:
      - Absolute Maximum Amount of Liability
      - Governing Law
      - Third Party Beneficiary
model-index:
  - name: BGE base Financial Matryoshka
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.007263922518159807
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.021791767554479417
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.03026634382566586
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.06174334140435835
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.007263922518159807
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.007263922518159807
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.006053268765133172
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.006174334140435836
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.007263922518159807
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.021791767554479417
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.03026634382566586
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.06174334140435835
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.028939379669254476
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.019243149237095962
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.029673742520760122
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.006053268765133172
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.021791767554479417
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.031476997578692496
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.06174334140435835
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.006053268765133172
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.007263922518159805
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.006295399515738498
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.006174334140435836
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.006053268765133172
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.021791767554479417
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.031476997578692496
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.06174334140435835
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.028312145815995213
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.018378876974518614
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.029262713498052723
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.007263922518159807
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.018159806295399514
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.02784503631961259
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.05811138014527845
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.007263922518159807
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.006053268765133171
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.0055690072639225175
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.0058111380145278455
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.007263922518159807
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.018159806295399514
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.02784503631961259
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.05811138014527845
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.026798615255571104
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.017617414197317337
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.029447278389058605
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.0036319612590799033
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.01694915254237288
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.02784503631961259
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.06295399515738499
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.0036319612590799033
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.005649717514124293
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.005569007263922519
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.006295399515738499
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.0036319612590799033
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.01694915254237288
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.02784503631961259
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.06295399515738499
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.027052121582546967
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.01650236365732733
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.028509723825826283
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.004842615012106538
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.025423728813559324
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.03753026634382567
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.06053268765133172
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.004842615012106538
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.008474576271186439
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.007506053268765135
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.006053268765133172
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.004842615012106538
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.025423728813559324
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.03753026634382567
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.06053268765133172
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.028532073992406013
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.018836715477151305
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.03024491886170751
            name: Cosine Map@100

BGE base Financial Matryoshka

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 tokens
• Similarity Function: Cosine Similarity
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("RishuD7/exigent-bge-base-financial-matryoshka")
# Run inference
sentences = [
    '\nThe management of each individual entity within a suppliers organization is responsible for\nimplementing the VAT Supplier Code of Conduct in their respective area of responsibility. They are\nobliged to take all appropriate action and provide the required structures and resources to ensure\nthat all employees in the entity are familiar with the VAT Supplier Code of Conduct and that its\nprinciples are fully implemented.\n.\n\nAll VAT suppliers are encouraged to direct any questions they might have with regard to the\ncontents, interpretation or implementation of the VAT Supplier Code of Conduct to the VAT Strategic\nProcurement function.\n.\n\n.\n \n.\n\nDocument created Release\nName Index Date\n.\n\n.\n \n.\n\nFile name\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\nPMS Document BPO1FO30EA MEY A 18.11.2014\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n  \n.\n\nWAT Strategic Procurement BP01FO30E\n.\n\nVakuumventile AG Supplier Code of Conduct Page 3 of 3\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\nWe, the undersigned, hereby confirm and declare in the name and on behalf of our company that\n.\n\n1. we have received the VAT Supplier Code of Condex;\n.\n\n2. by signing this declaration, we accept and commit to complying with all rules and requirements as\nlaid out in the VAT Supplier Code of Conduct;\n.\n\n3. we accept that this declaration shall be exclusively governed by the material laws of Switzerland,\nexcluding the UN Law of Sales (CISG).\n.\n\nPlacelDate —-Singagore. / tone 2077\nCompany Kien Ann Engineering Pe ad\nStreet 3c 500 kovo Cirle\n.\n\nPost codelcity Singapore 627035\n.\n\nName of authorized signatory Jameson Low\n.\n\nL. Ze\nSignature << : Ly eA\n* fh\n20,\nXn _A\nCETES\n.\n\n1. Please sign one (1) original c Of this document.\n2. Please note that only duly authorized personnel of your company may sign this document.\n3. Please send the duly signed original copy by conventional mail to:\nVAT VAKUUMVENTILE AG, SEELISTRASSE 1, STRATEGISCHER EINKAUF, CH-9469 HAAG\n.\n\n.\n \n.\n\n.\n   \n.\n\n.\n \n.\n\nDocument created Release\n.\n\n.\n \n.\n\nFile name\nName Index Date\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\nPMS Document BPO1FO30EA MEY A 18.11.2014\n\n \n\n \n\nAll business is conducted in compliance with governing national and international laws and\nregulations. As a matter of principle, we honor agreements and obligations we have entered into\nvoluntarily. All suppliers are obliged to carefully study the rules and regulations pertinent to their\narea of responsibility and ensure full compliance. In case of doubt or queries, they are obliged to\nseek additional information and guidance from the appropriate channels or persons in charge. VAT\nhas a zero tolerance policy with regard to violations of its Supplier Code of Conduct. Violations may\nlead to appropriate action being taken against the supplier.\n.\n\n2. Fair competition\n',
    'Governing Law',
    'Absolute Maximum Amount of Liability',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Dataset: dim_768
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.0073
cosine_accuracy@3	0.0218
cosine_accuracy@5	0.0303
cosine_accuracy@10	0.0617
cosine_precision@1	0.0073
cosine_precision@3	0.0073
cosine_precision@5	0.0061
cosine_precision@10	0.0062
cosine_recall@1	0.0073
cosine_recall@3	0.0218
cosine_recall@5	0.0303
cosine_recall@10	0.0617
cosine_ndcg@10	0.0289
cosine_mrr@10	0.0192
cosine_map@100	0.0297

Information Retrieval

• Dataset: dim_512
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.0061
cosine_accuracy@3	0.0218
cosine_accuracy@5	0.0315
cosine_accuracy@10	0.0617
cosine_precision@1	0.0061
cosine_precision@3	0.0073
cosine_precision@5	0.0063
cosine_precision@10	0.0062
cosine_recall@1	0.0061
cosine_recall@3	0.0218
cosine_recall@5	0.0315
cosine_recall@10	0.0617
cosine_ndcg@10	0.0283
cosine_mrr@10	0.0184
cosine_map@100	0.0293

Information Retrieval

• Dataset: dim_256
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.0073
cosine_accuracy@3	0.0182
cosine_accuracy@5	0.0278
cosine_accuracy@10	0.0581
cosine_precision@1	0.0073
cosine_precision@3	0.0061
cosine_precision@5	0.0056
cosine_precision@10	0.0058
cosine_recall@1	0.0073
cosine_recall@3	0.0182
cosine_recall@5	0.0278
cosine_recall@10	0.0581
cosine_ndcg@10	0.0268
cosine_mrr@10	0.0176
cosine_map@100	0.0294

Information Retrieval

• Dataset: dim_128
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.0036
cosine_accuracy@3	0.0169
cosine_accuracy@5	0.0278
cosine_accuracy@10	0.063
cosine_precision@1	0.0036
cosine_precision@3	0.0056
cosine_precision@5	0.0056
cosine_precision@10	0.0063
cosine_recall@1	0.0036
cosine_recall@3	0.0169
cosine_recall@5	0.0278
cosine_recall@10	0.063
cosine_ndcg@10	0.0271
cosine_mrr@10	0.0165
cosine_map@100	0.0285

Information Retrieval

• Dataset: dim_64
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.0048
cosine_accuracy@3	0.0254
cosine_accuracy@5	0.0375
cosine_accuracy@10	0.0605
cosine_precision@1	0.0048
cosine_precision@3	0.0085
cosine_precision@5	0.0075
cosine_precision@10	0.0061
cosine_recall@1	0.0048
cosine_recall@3	0.0254
cosine_recall@5	0.0375
cosine_recall@10	0.0605
cosine_ndcg@10	0.0285
cosine_mrr@10	0.0188
cosine_map@100	0.0302

Training Details

Training Dataset

Unnamed Dataset

• Size: 3,305 training samples
• Columns: positive and anchor
• Approximate statistics based on the first 1000 samples:

  	positive	anchor
  type	string	string
  details	min: 123 tokens mean: 353.07 tokens max: 512 tokens	min: 3 tokens mean: 5.37 tokens max: 8 tokens

• Samples:

  positive	anchor
  In no event shall CBRE, Client, or their respective affiliates incur liability under this agreement or otherwise relating to the Services beyond the insurance proceeds available with respect to the particular matter under the Insurance Policies required to be carried by CBRE AND Client under Article 6 above including, if applicable, proceeds of self-insurance. Each party shall and shall cause its affiliates to look solely to such insurance proceeds (and any such proceeds paid through self-insurance) to satisfy its claims against the released parties and agrees that it shall have no right of recovery beyond such proceeds; provided, however, that if insurance proceeds under such policies are not paid because a party has failed to maintain such policies, comply with policy requirements or, in the case of self-insurance, unreasonably denied a claim, such party shall be liable for the amounts that otherwise would have been payable under such policies had such party maintained such policies, complied with the policy requirement or not unreasonably denied such claim, as the case may be.	Absolute Maximum Amount of Liability
  4. Rent. 4.01 From and after the Commencement Date, Tenant shall pay Landlord, without any setoff or deduction, unless expressly set forth in this Lease, all Base Rent and Additional Rent due for the Term (collectively referred to as "Rent"). "Additional Rent" means all sums (exclusive of Base Rent) that Tenant is required to pay Landlord under this Lease. Tenant shall pay and be liable for all rental, sales and use taxes (but excluding income taxes), if any, imposed upon or measured by Rent. Base Rent and recurring monthly charges of Additional Rent shall be due and payable in advance on the first day of each calendar month without notice or demand, provided that the installment of Base Rent attributable to the first (1st) full calendar month of the Term following the Abatement Period shall be due concurrently with the execution of this Lease by Tenant. All other items of Rent shall be due and payable on or before thirty (30) days after billing by Landlord. Rent shall be made payable to the entity, and sent to the address, that Landlord designates and shall be made by good and sufficient check or by other means acceptable to Landlord. Landlord may return to Tenant, at any time within fifteen (15) days after receiving same, any payment of Rent (a) made following any Default (irrespective of whether Landlord has commenced the exercise of any remedy), or (b) that is less than the amount due. Each such returned payment (whether made by returning Tenant's actual check, or by issuing a refund in the event Tenant's check was deposited) shall be conclusively presumed not to have been received or approved by Landlord. If Tenant does not pay any Rent when due hereunder, Tenant shall pay Landlord an administration fee in the amount of five percent (5%) of the past due amount. In addition, past due Rent shall accrue interest at a rate equal to the lesser of (i) twelve percent (12%) per annum or (ii) the maximum legal rate, and Tenant shall pay Landlord a fee for any checks returned by Tenant's bank for any reason. Notwithstanding the foregoing, no such late charge or of interest shall be imposed with respect to the first (1st) late payment in any calendar year, but not with respect to more than three (3) such late payments during the initial Term of this Lease.	Late Payment Charges
  Term This Agreement shall come into force and shall last unlimited from such date. Either Party may however terminate this Agreement at any time by giving upon thirty (30) days' written notice to the other Party. The Receiving Party's obligations contained in this Agreement to keep confidential and restrict use of the Disclosing Party's Confidential Information shall sur- vive for a period of five (5) years from the date of its termination for any reason whatsoever. lX. Contractual penalty For the purposes of this Non-Disclosure Agreement, " Confidential Information" includes all technical and/or commercial and/or financial information in the field designated in section 1., which a contracting Party (hereinafter referred to as the "EQ€i1gPedy") makes, or has made, accessible to the other contracting Party (hereinafter referred to as the ".&eiyi!g Partv") in oral, written, tangible or other form (e.9. disk, data carrier) directly or indirectly, in- cluding but not limited to, drawings, models, components, and other material. Confidential In- formation is to be identified as such. Orally communicated or visually, information having been designated as confidential at the time of disclosure will be confirmed as such in writing by the Disclosing Party within 30 (thirty) days from such disclosure being understood thatlhe ./A information will be considered Confidential Information during that period of 30 (thirty) days. /L t'-4 PF 0233 (September 2016) page 1 of 5 ä =. PFEIFFER F . F . VACUUM	Termination for Convenience

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• tf32: False
• load_best_model_at_end: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_128_cosine_map@100	dim_256_cosine_map@100	dim_512_cosine_map@100	dim_64_cosine_map@100	dim_768_cosine_map@100
1.5385	10	7.981	-	-	-	-	-
3.0769	20	0.9258	-	-	-	-	-
4.6154	30	0.1708	0.0285	0.0294	0.0293	0.0302	0.0297

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.0.1
• Transformers: 4.41.2
• PyTorch: 2.1.2+cu121
• Accelerate: 0.32.1
• Datasets: 2.19.1
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}