RishuD7's picture
Add new SentenceTransformer model.
b54a891 verified
|
raw
history blame
No virus
49.7 kB
---
base_model: BAAI/bge-base-en-v1.5
datasets: []
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:3305
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: '
Limitation of Liability
CUSTOMER’S ENTIRE LIABILITY AND PACNET’S EXCLUSIVE REMEDIES AGAINST CUSTOMER FOR
ANY DAMAGES ARISING
FROM ANY ACT OR OMISSION RELATING TO THE SERVICES, REGARDLESS OF THE FORM OF ACTION,
WHETHER IN CONTRACT,
UNDER STATUTE, IN TORT OR OTHERWISE, INCLUDING NEGLIGENCE, WILL BE LIMITED, FOR
EACH EVENT OR SERIES OF
CONNECTED EVENTS, AS FOLLOWS:
FOR PERSONAL INJURY OR DEATH, UNLIMITED, BUT SUBJECT TO PROVEN DIRECT DAMAGES;
AND
FOR ALL OTHER EVENTS, SUBJECT TO A MAXIMUM EQUAL TO THE AGGREGATE MONTHLY SERVICE
CHARGES PAID OR
PAYBALE BY THE CUSTOMER UNDER THE AGREEMENT.
PACNET’S ENTIRE LIABILITY AND CUSTOMER’S EXCLUSIVE REMEDIES AGAINST PACNET OR
ITS AFFILIATES FOR
ANY DAMAGES ARISING FROM ANY ACT OR OMISSION RELATING TO THE AGREEMENT, REGARDLESS
OF THE
FORM OF ACTION, WHETHER IN CONTRACT, UNDER STATUTE, IN TORT OR OTHERWISE, INCLUDING
NEGLIGENCE, WILL BE LIMITED, FOR EACH EVENT OR SERIES OF CONNECTED EVENTS, AS
FOLLOWS:
{i} | FOR PERSONAL INJURY OR DEATH, UNLIMITED, BUT SUBJECT TO PROVEN DIRECT DAMAGES;
(ii) FOR FAILURE TO COMPLY WITH SERVICE LEVELS, TO THE AMOUNT OF CREDITS SET OUT
IN THE
RELEVANT SPECIFIC CONDITIONS OF THE RELEVANT SERVICE; AND
(iii) FOR ALL OTHER EVENTS, SUBJECT TO A MAXIMUM EQUAL TO THE AGGREGATE MONTHLY
SERVICE
CHARGES PAID OR PAYABLE BY THE CUSTOMER UNDER THE AGREEMENT.
.
PACNET WILL IN NO CIRCUMSTANCES BE LIABLE FOR ANY DAMAGES (EXCEPT RESULTING IN
PERSONAL INJURY
OR DEATH) ATTRIBUTABLE TO ANY SERVICE, PRODUCT OR ACTIONS OF ANY PERSON OTHER
THAN PACNET, ITS
EMPLOYEES AND AGENTS.
'
sentences:
- Auto Renewal Cancellation Notice Period
- Assignment
- Absolute Maximum Amount of Liability
- source_sentence: '
Subcontracting
(a) The Supplier must not subcontract any of its
obligations under this Agreement, without the
Company''s prior written consent (which will not
be unreasonably withheld).
(b) The Supplier remains fully responsible for acts
and omissions of its subcontractors and Supplier
Personnel in connection with this Agreement or a
Statement of Work as if they were its acts and
omissions.
Personnel
(a) At the Company''s reasonable request the
Supplier must, at its cost, immediately (or by any
date nominated by the Company) remove any
person nominated by the Company from the
performance of the Services and, if requested by
the Company, provide an alternative person
acceptable to the Company (acting reasonably).
(b) The Supplier will not remove (temporarily or
permanently) or replace a Key Personnel without
the Company’s prior written consent (which must
not be unreasonably withheld). Any substitute
personnel must be at least equally qualified for
the duties of the position as the person for whom
they are substituted. The Supplier must use
reasonable endeavours to provide uninterrupted
transition between Key Personnel and their
replacements.
'
sentences:
- Audit Rights
- Severability
- Subcontracting
- source_sentence: All Intellectual Property shall be deemed to be owned by the Employer
and Executive hereby relinquishes any right or claim to any such Intellectual
Property except to the extent necessary to transfer the ownership of any such
Intellectual Property to Employer. Executive shall promptly disclose to the Employer
all Intellectual Property. Without royalty or separate consideration, Executive
hereby assigns and agrees to assign to the Employer (or as otherwise directed
by the Employer) Executive’s full right, title and interest in and to all Intellectual
Property, including without limitation all copyright interests therein. Executive
agrees to cooperate with Employer and to execute any and all applications for
domestic and foreign patents, copyrights or other proprietary rights and to do
such other acts (including, among other things, the execution and delivery of
instruments of further assurance or confirmation) requested by the Employer to
assign the Intellectual Property to the Employer and to permit the Employer to
file, obtain and enforce any patents, copyrights or other proprietary rights in
the Intellectual Property. Executive agrees that Executive’s obligation to cooperate
and to execute, or cause to be executed, when it is in Executive’s power to do
so, any such instrument or paper, will continue after termination of this Agreement.
Executive agrees to make and maintain adequate and current written records of
all Intellectual Property, in the form of notes, sketches, drawings, or reports
relating hereto, which records shall be and remain the property of and available
to the Employer at all times. The parties agree that the Intellectual Property
does not include the items listed in the attached Exhibit A to this Agreement.
sentences:
- General Indemnities
- Intellectual Property Ownership
- Governing Law
- source_sentence: "CBRE\n.\n\nHEVERTECH LTD\n.\n\n.\n \n.\n\nPreferred Supplier\
\ Light/Agreement\n.\n\n.\n \n.\n \n.\n\nAgreement; Number: NMS/16/050 |\n.\n\
\nQUALIFIED SERVICE LEVEL AGREEMENT\nBETWEEN\nCBRE MANAGED SERVICES LIMITED\n\
AND\nHEVERTECH LTDCBRE\n.\n\nHEVERTECH LTD\n.\n\n.\n \n.\n \n.\n\n.\n \n.\n\
\ \n.\n\n.\n \n.\n \n.\n\nPreferred!Supplier Light Agreement\ni]\n.\n\nW\\\
olaclelealelaters Ulin elsiea Niky AeyAOkLY)\n.\n\nTABLE OF CONTENTS:\n.\n\nQualified\
\ Service Level Agreement Pages 03 to 10 inclusive\n.\n\nAppendix 1 — Schedule\
\ of Rates Page\n.\n\nAppendix 2 — Key Contacts and Escalation Process Pages 8\
\ to 9 inclusive\n.\n\nAppendix 3 - Working Capital Scheme Pages 10\n.\n\n.\n\
\ \n.\n\nCBRE Managed Services Lid\nFebruary 2016 Page 2 of LOPreferred Supplier\
\ Light'A greement HEVERTECH LTD\n.\n\nAgreement Number: NMS/16/050\n.\n\n.\n\
\ \n\n\nTHIS AGREEMENT is made on 1° June 2016\nBETWEEN\n\n(1) CBRE Managed Services\
\ Limited (Registered in England No. 1799580) whose registered\noffice is at City\
\ Bridge House, 57 Southwark Street, London, SE1 1RU (“CBRE”); and\n\n(2) Hevertech\
\ Ltd (Registered in England No. 2803522) whose registered office is at: Unit\
\ 2\nTreefield Industrial Estate, Gildersome, Leeds, LS27 7JU (the “Supplier’).\n"
sentences:
- Non Solicitation
- Intellectual Property Infringement Indemnity
- Title of Agreement
- source_sentence: "\nThe management of each individual entity within a suppliers\
\ organization is responsible for\nimplementing the VAT Supplier Code of Conduct\
\ in their respective area of responsibility. They are\nobliged to take all appropriate\
\ action and provide the required structures and resources to ensure\nthat all\
\ employees in the entity are familiar with the VAT Supplier Code of Conduct and\
\ that its\nprinciples are fully implemented.\n.\n\nAll VAT suppliers are encouraged\
\ to direct any questions they might have with regard to the\ncontents, interpretation\
\ or implementation of the VAT Supplier Code of Conduct to the VAT Strategic\n\
Procurement function.\n.\n\n.\n \n.\n\nDocument created Release\nName Index Date\n\
.\n\n.\n \n.\n\nFile name\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\nPMS\
\ Document BPO1FO30EA MEY A 18.11.2014\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n\
.\n \n.\n\n.\n \n.\n\nWAT Strategic Procurement BP01FO30E\n.\n\nVakuumventile\
\ AG Supplier Code of Conduct Page 3 of 3\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\
\nWe, the undersigned, hereby confirm and declare in the name and on behalf of\
\ our company that\n.\n\n1. we have received the VAT Supplier Code of Condex;\n\
.\n\n2. by signing this declaration, we accept and commit to complying with all\
\ rules and requirements as\nlaid out in the VAT Supplier Code of Conduct;\n.\n\
\n3. we accept that this declaration shall be exclusively governed by the material\
\ laws of Switzerland,\nexcluding the UN Law of Sales (CISG).\n.\n\nPlacelDate\
\ —-Singagore. / tone 2077\nCompany Kien Ann Engineering Pe ad\nStreet 3c 500\
\ kovo Cirle\n.\n\nPost codelcity Singapore 627035\n.\n\nName of authorized signatory\
\ Jameson Low\n.\n\nL. Ze\nSignature << : Ly eA\n* fh\n20,\nXn _A\nCETES\n.\n\n\
1. Please sign one (1) original c Of this document.\n2. Please note that only\
\ duly authorized personnel of your company may sign this document.\n3. Please\
\ send the duly signed original copy by conventional mail to:\nVAT VAKUUMVENTILE\
\ AG, SEELISTRASSE 1, STRATEGISCHER EINKAUF, CH-9469 HAAG\n.\n\n.\n \n.\n\n.\n\
\ \n.\n\n.\n \n.\n\nDocument created Release\n.\n\n.\n \n.\n\nFile name\nName\
\ Index Date\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n\
\ \n.\n\n.\n \n.\n\nPMS Document BPO1FO30EA MEY A 18.11.2014\n\n \n\n \n\nAll\
\ business is conducted in compliance with governing national and international\
\ laws and\nregulations. As a matter of principle, we honor agreements and obligations\
\ we have entered into\nvoluntarily. All suppliers are obliged to carefully study\
\ the rules and regulations pertinent to their\narea of responsibility and ensure\
\ full compliance. In case of doubt or queries, they are obliged to\nseek additional\
\ information and guidance from the appropriate channels or persons in charge.\
\ VAT\nhas a zero tolerance policy with regard to violations of its Supplier Code\
\ of Conduct. Violations may\nlead to appropriate action being taken against the\
\ supplier.\n.\n\n2. Fair competition\n"
sentences:
- Absolute Maximum Amount of Liability
- Governing Law
- Third Party Beneficiary
model-index:
- name: BGE base Financial Matryoshka
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 768
type: dim_768
metrics:
- type: cosine_accuracy@1
value: 0.005502063273727648
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.02063273727647868
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.0343878954607978
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.055020632737276476
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.005502063273727648
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.0068775790921595595
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.0068775790921595595
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.005502063273727648
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.005502063273727648
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.02063273727647868
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.0343878954607978
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.055020632737276476
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.026159571077284855
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.017443396432392302
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.028690755071459656
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 512
type: dim_512
metrics:
- type: cosine_accuracy@1
value: 0.005502063273727648
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.023383768913342505
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.030261348005502064
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.06327372764786796
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.005502063273727648
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.0077945896377808336
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.006052269601100413
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.006327372764786796
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.005502063273727648
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.023383768913342505
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.030261348005502064
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.06327372764786796
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.02862375490125008
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.01840243662802122
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.029291334425024487
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 0.0068775790921595595
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.02200825309491059
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.03576341127922971
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.06602475928473177
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.0068775790921595595
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.007336084364970197
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.007152682255845944
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.006602475928473177
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.0068775790921595595
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.02200825309491059
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.03576341127922971
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.06602475928473177
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.03032191705264531
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.019712451693194466
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.03142394630610729
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 0.001375515818431912
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.017881705639614855
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.02200825309491059
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.061898211829436035
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.001375515818431912
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.0059605685465382845
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.004401650618982119
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.0061898211829436054
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.001375515818431912
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.017881705639614855
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.02200825309491059
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.061898211829436035
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.02519373355892647
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.01444837448963996
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.027157891073425876
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.005502063273727648
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.02888583218707015
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.0453920220082531
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.0687757909215956
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.005502063273727648
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.009628610729023383
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.009078404401650619
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.0068775790921595595
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.005502063273727648
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.02888583218707015
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.0453920220082531
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.0687757909215956
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.033131660986050276
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.022215672146896348
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.033485590251952455
name: Cosine Map@100
---
# BGE base Financial Matryoshka
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("RishuD7/exigent-moreepoch-bge-base-financial-matryoshka")
# Run inference
sentences = [
'\nThe management of each individual entity within a suppliers organization is responsible for\nimplementing the VAT Supplier Code of Conduct in their respective area of responsibility. They are\nobliged to take all appropriate action and provide the required structures and resources to ensure\nthat all employees in the entity are familiar with the VAT Supplier Code of Conduct and that its\nprinciples are fully implemented.\n.\n\nAll VAT suppliers are encouraged to direct any questions they might have with regard to the\ncontents, interpretation or implementation of the VAT Supplier Code of Conduct to the VAT Strategic\nProcurement function.\n.\n\n.\n \n.\n\nDocument created Release\nName Index Date\n.\n\n.\n \n.\n\nFile name\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\nPMS Document BPO1FO30EA MEY A 18.11.2014\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\nWAT Strategic Procurement BP01FO30E\n.\n\nVakuumventile AG Supplier Code of Conduct Page 3 of 3\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\nWe, the undersigned, hereby confirm and declare in the name and on behalf of our company that\n.\n\n1. we have received the VAT Supplier Code of Condex;\n.\n\n2. by signing this declaration, we accept and commit to complying with all rules and requirements as\nlaid out in the VAT Supplier Code of Conduct;\n.\n\n3. we accept that this declaration shall be exclusively governed by the material laws of Switzerland,\nexcluding the UN Law of Sales (CISG).\n.\n\nPlacelDate —-Singagore. / tone 2077\nCompany Kien Ann Engineering Pe ad\nStreet 3c 500 kovo Cirle\n.\n\nPost codelcity Singapore 627035\n.\n\nName of authorized signatory Jameson Low\n.\n\nL. Ze\nSignature << : Ly eA\n* fh\n20,\nXn _A\nCETES\n.\n\n1. Please sign one (1) original c Of this document.\n2. Please note that only duly authorized personnel of your company may sign this document.\n3. Please send the duly signed original copy by conventional mail to:\nVAT VAKUUMVENTILE AG, SEELISTRASSE 1, STRATEGISCHER EINKAUF, CH-9469 HAAG\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\nDocument created Release\n.\n\n.\n \n.\n\nFile name\nName Index Date\n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\n.\n \n.\n\nPMS Document BPO1FO30EA MEY A 18.11.2014\n\n \n\n \n\nAll business is conducted in compliance with governing national and international laws and\nregulations. As a matter of principle, we honor agreements and obligations we have entered into\nvoluntarily. All suppliers are obliged to carefully study the rules and regulations pertinent to their\narea of responsibility and ensure full compliance. In case of doubt or queries, they are obliged to\nseek additional information and guidance from the appropriate channels or persons in charge. VAT\nhas a zero tolerance policy with regard to violations of its Supplier Code of Conduct. Violations may\nlead to appropriate action being taken against the supplier.\n.\n\n2. Fair competition\n',
'Governing Law',
'Absolute Maximum Amount of Liability',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
## Evaluation
### Metrics
#### Information Retrieval
* Dataset: `dim_768`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.0055 |
| cosine_accuracy@3 | 0.0206 |
| cosine_accuracy@5 | 0.0344 |
| cosine_accuracy@10 | 0.055 |
| cosine_precision@1 | 0.0055 |
| cosine_precision@3 | 0.0069 |
| cosine_precision@5 | 0.0069 |
| cosine_precision@10 | 0.0055 |
| cosine_recall@1 | 0.0055 |
| cosine_recall@3 | 0.0206 |
| cosine_recall@5 | 0.0344 |
| cosine_recall@10 | 0.055 |
| cosine_ndcg@10 | 0.0262 |
| cosine_mrr@10 | 0.0174 |
| **cosine_map@100** | **0.0287** |
#### Information Retrieval
* Dataset: `dim_512`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.0055 |
| cosine_accuracy@3 | 0.0234 |
| cosine_accuracy@5 | 0.0303 |
| cosine_accuracy@10 | 0.0633 |
| cosine_precision@1 | 0.0055 |
| cosine_precision@3 | 0.0078 |
| cosine_precision@5 | 0.0061 |
| cosine_precision@10 | 0.0063 |
| cosine_recall@1 | 0.0055 |
| cosine_recall@3 | 0.0234 |
| cosine_recall@5 | 0.0303 |
| cosine_recall@10 | 0.0633 |
| cosine_ndcg@10 | 0.0286 |
| cosine_mrr@10 | 0.0184 |
| **cosine_map@100** | **0.0293** |
#### Information Retrieval
* Dataset: `dim_256`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.0069 |
| cosine_accuracy@3 | 0.022 |
| cosine_accuracy@5 | 0.0358 |
| cosine_accuracy@10 | 0.066 |
| cosine_precision@1 | 0.0069 |
| cosine_precision@3 | 0.0073 |
| cosine_precision@5 | 0.0072 |
| cosine_precision@10 | 0.0066 |
| cosine_recall@1 | 0.0069 |
| cosine_recall@3 | 0.022 |
| cosine_recall@5 | 0.0358 |
| cosine_recall@10 | 0.066 |
| cosine_ndcg@10 | 0.0303 |
| cosine_mrr@10 | 0.0197 |
| **cosine_map@100** | **0.0314** |
#### Information Retrieval
* Dataset: `dim_128`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.0014 |
| cosine_accuracy@3 | 0.0179 |
| cosine_accuracy@5 | 0.022 |
| cosine_accuracy@10 | 0.0619 |
| cosine_precision@1 | 0.0014 |
| cosine_precision@3 | 0.006 |
| cosine_precision@5 | 0.0044 |
| cosine_precision@10 | 0.0062 |
| cosine_recall@1 | 0.0014 |
| cosine_recall@3 | 0.0179 |
| cosine_recall@5 | 0.022 |
| cosine_recall@10 | 0.0619 |
| cosine_ndcg@10 | 0.0252 |
| cosine_mrr@10 | 0.0144 |
| **cosine_map@100** | **0.0272** |
#### Information Retrieval
* Dataset: `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.0055 |
| cosine_accuracy@3 | 0.0289 |
| cosine_accuracy@5 | 0.0454 |
| cosine_accuracy@10 | 0.0688 |
| cosine_precision@1 | 0.0055 |
| cosine_precision@3 | 0.0096 |
| cosine_precision@5 | 0.0091 |
| cosine_precision@10 | 0.0069 |
| cosine_recall@1 | 0.0055 |
| cosine_recall@3 | 0.0289 |
| cosine_recall@5 | 0.0454 |
| cosine_recall@10 | 0.0688 |
| cosine_ndcg@10 | 0.0331 |
| cosine_mrr@10 | 0.0222 |
| **cosine_map@100** | **0.0335** |
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 3,305 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
| | positive | anchor |
|:--------|:--------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------|
| type | string | string |
| details | <ul><li>min: 123 tokens</li><li>mean: 353.07 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 5.37 tokens</li><li>max: 8 tokens</li></ul> |
* Samples:
| positive | anchor |
|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------|
| <code>In no event shall CBRE, Client, or their respective affiliates incur liability under this agreement or otherwise relating to the Services beyond the insurance proceeds available with respect to the particular matter under the Insurance Policies required to be carried by CBRE AND Client under Article 6 above including, if applicable, proceeds of self-insurance. Each party shall and shall cause its affiliates to look solely to such insurance proceeds (and any such proceeds paid through self-insurance) to satisfy its claims against the released parties and agrees that it shall have no right of recovery beyond such proceeds; provided, however, that if insurance proceeds under such policies are not paid because a party has failed to maintain such policies, comply with policy requirements or, in the case of self-insurance, unreasonably denied a claim, such party shall be liable for the amounts that otherwise would have been payable under such policies had such party maintained such policies, complied with the policy requirement or not unreasonably denied such claim, as the case may be.</code> | <code>Absolute Maximum Amount of Liability</code> |
| <code>4. Rent. <br>4.01 From and after the Commencement Date, Tenant shall pay Landlord, without any<br>setoff or deduction, unless expressly set forth in this Lease, all Base Rent and Additional Rent<br>due for the Term (collectively referred to as "Rent"). "Additional Rent" means all sums<br>(exclusive of Base Rent) that Tenant is required to pay Landlord under this Lease. Tenant shall<br>pay and be liable for all rental, sales and use taxes (but excluding income taxes), if any,<br>imposed upon or measured by Rent. Base Rent and recurring monthly charges of Additional<br>Rent shall be due and payable in advance on the first day of each calendar month without<br>notice or demand, provided that the installment of Base Rent attributable to the first (1st) full<br>calendar month of the Term following the Abatement Period shall be due concurrently with the<br>execution of this Lease by Tenant. All other items of Rent shall be due and payable on or<br>before thirty (30) days after billing by Landlord. Rent shall be made payable to the entity, and<br>sent to the address, that Landlord designates and shall be made by good and sufficient check or<br>by other means acceptable to Landlord. Landlord may return to Tenant, at any time within<br>fifteen (15) days after receiving same, any payment of Rent (a) made following any Default<br>(irrespective of whether Landlord has commenced the exercise of any remedy), or (b) that is<br>less than the amount due. Each such returned payment (whether made by returning Tenant's<br>actual check, or by issuing a refund in the event Tenant's check was deposited) shall be<br>conclusively presumed not to have been received or approved by Landlord. If Tenant does not<br>pay any Rent when due hereunder, Tenant shall pay Landlord an administration fee in the<br>amount of five percent (5%) of the past due amount. In addition, past due Rent shall accrue<br>interest at a rate equal to the lesser of (i) twelve percent (12%) per annum or (ii) the maximum<br>legal rate, and Tenant shall pay Landlord a fee for any checks returned by Tenant's bank for<br>any reason. Notwithstanding the foregoing, no such late charge or of interest shall be imposed<br>with respect to the first (1st) late payment in any calendar year, but not with respect to more<br>than three (3) such late payments during the initial Term of this Lease. </code> | <code>Late Payment Charges</code> |
| <code>Term This Agreement shall come into force and shall last unlimited from such date. Either Party may however terminate this Agreement at any time by giving upon thirty (30) days' written notice to the other Party. The Receiving Party's obligations contained in this Agreement to keep confidential and restrict use of the Disclosing Party's Confidential Information shall sur- vive for a period of five (5) years from the date of its termination for any reason whatsoever. lX. Contractual penalty<br>For the purposes of this Non-Disclosure Agreement, " Confidential Information" includes all technical and/or commercial and/or financial information in the field designated in section 1., which a contracting Party (hereinafter referred to as the "EQ€i1gPedy") makes, or has made, accessible to the other contracting Party (hereinafter referred to as the ".&eiyi!g Partv") in oral, written, tangible or other form (e.9. disk, data carrier) directly or indirectly, in- cluding but not limited to, drawings, models, components, and other material. Confidential In- formation is to be identified as such. Orally communicated or visually, information having been designated as confidential at the time of disclosure will be confirmed as such in writing by the Disclosing Party within 30 (thirty) days from such disclosure being understood thatlhe ./A information will be considered Confidential Information during that period of 30 (thirty) days. /L t'-4 PF 0233 (September 2016) page 1 of 5 ä =.<br> PFEIFFER F<br>.<br> F<br>.<br> VACUUM<br></code> | <code>Termination for Convenience</code> |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
```json
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
768,
512,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1,
1
],
"n_dims_per_step": -1
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 10
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 10
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
</details>
### Training Logs
| Epoch | Step | Training Loss | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_512_cosine_map@100 | dim_64_cosine_map@100 | dim_768_cosine_map@100 |
|:----------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|:----------------------:|
| 1.5385 | 10 | 8.1013 | - | - | - | - | - |
| 3.0769 | 20 | 0.87 | - | - | - | - | - |
| 4.6154 | 30 | 0.2172 | - | - | - | - | - |
| 6.1538 | 40 | 0.0 | - | - | - | - | - |
| **7.3846** | **48** | **-** | **0.0272** | **0.0313** | **0.0293** | **0.0333** | **0.0285** |
| 1.2692 | 50 | 1.4329 | - | - | - | - | - |
| 2.8077 | 60 | 2.9916 | 0.0272 | 0.0314 | 0.0293 | 0.0335 | 0.0287 |
* The bold row denotes the saved checkpoint.
### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.1.2+cu121
- Accelerate: 0.32.1
- Datasets: 2.19.1
- Tokenizers: 0.19.1
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
```
#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->